2024-02-16T22:36:17.374 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-16T22:36:17.374 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --name pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738 --verbose --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738/orig.config.yaml 2024-02-16T22:36:17.420 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T22:36:17.502 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738 branch: wip-batrick-testing-20240215.160715 description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7562738' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240215.160715 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f78a58c0ffd401d1493058a1022c35f011d65275 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f78a58c0ffd401d1493058a1022c35f011d65275 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: i64440 sha1: ed7ab725e1a43615009dfb74a42bf33d80081bc8 owner: scheduled_pdonnell@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2634 sha1: f78a58c0ffd401d1493058a1022c35f011d65275 sleep_before_teardown: 0 suite: fs:upgrade:mds_upgrade_sequence suite_branch: i64440 suite_path: /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: ed7ab725e1a43615009dfb74a42bf33d80081bc8 targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6QadFqDbWyDlJmW1q6PrCeWaGARL4XxNcaOu2UAL0qUc2ZoHq+JQYu5jBxZDSH9UlNBy1VtA6EfirNQ3lp4Dg= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYPHynn3iYPQupgar2Al02NX2cBEApdPgoCudCKznhRvaxCqANchDMEOC0mVh749UR7au1Gi4hVElHqEh4lyDE= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - full_sequential_finally: - duration: 3600 sleep: null - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/pdonnell/src/github.com_batrick_ceph_i64440/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-16_18:26:20 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-16T22:36:17.502 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa; will attempt to use it 2024-02-16T22:36:17.502 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks 2024-02-16T22:36:17.503 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-16T22:36:17.504 INFO:teuthology.task.internal:Checking packages... 2024-02-16T22:36:17.525 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f78a58c0ffd401d1493058a1022c35f011d65275' 2024-02-16T22:36:17.525 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T22:36:17.525 INFO:teuthology.packaging:ref: None 2024-02-16T22:36:17.525 INFO:teuthology.packaging:tag: None 2024-02-16T22:36:17.525 INFO:teuthology.packaging:branch: wip-batrick-testing-20240215.160715 2024-02-16T22:36:17.525 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:36:17.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240215.160715 2024-02-16T22:36:17.675 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1450.gf78a58c0 2024-02-16T22:36:17.677 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-16T22:36:17.683 INFO:teuthology.task.internal:no buildpackages task found 2024-02-16T22:36:17.684 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-16T22:36:17.723 INFO:teuthology.task.internal:Saving configuration 2024-02-16T22:36:17.739 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-16T22:36:17.746 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T22:36:17.771 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 22:31:26.667808', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6QadFqDbWyDlJmW1q6PrCeWaGARL4XxNcaOu2UAL0qUc2ZoHq+JQYu5jBxZDSH9UlNBy1VtA6EfirNQ3lp4Dg='} 2024-02-16T22:36:17.794 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 22:31:26.666021', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYPHynn3iYPQupgar2Al02NX2cBEApdPgoCudCKznhRvaxCqANchDMEOC0mVh749UR7au1Gi4hVElHqEh4lyDE='} 2024-02-16T22:36:17.794 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-16T22:36:17.800 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-16T22:36:17.800 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-16T22:36:17.800 INFO:teuthology.run_tasks:Running task console_log... 2024-02-16T22:36:17.881 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5ee569d0d0>, signals=[15]) 2024-02-16T22:36:17.882 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-16T22:36:17.888 INFO:teuthology.task.internal:Opening connections... 2024-02-16T22:36:17.888 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:36:17.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:36:17.994 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-02-16T22:36:17.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:36:18.083 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-16T22:36:18.090 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-02-16T22:36:18.108 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-02-16T22:36:18.108 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-02-16T22:36:18.165 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-02-16T22:36:18.165 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T22:36:18.166 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T22:36:18.167 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-02-16T22:36:18.283 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-02-16T22:36:18.301 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-02-16T22:36:18.301 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-02-16T22:36:18.355 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T22:36:18.356 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T22:36:18.357 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-02-16T22:36:18.383 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-16T22:36:18.392 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-16T22:36:18.431 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-16T22:36:18.431 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-02-16T22:36:18.434 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-02-16T22:36:18.450 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-16T22:36:18.455 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-16T22:36:18.455 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-02-16T22:36:18.491 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-02-16T22:36:18.516 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-16T22:36:18.574 INFO:teuthology.run_tasks:Running task kernel... 2024-02-16T22:36:18.589 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-16T22:36:18.589 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-16T22:36:18.589 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-16T22:36:18.590 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-16T22:36:18.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:36:18.606 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-02-16T22:36:18.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:36:18.607 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-02-16T22:36:18.663 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-361.el9.x86_64 2024-02-16T22:36:18.663 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.14.0-361.el9.x86_64 2024-02-16T22:36:18.663 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2024-02-16T22:36:18.665 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-361.el9.x86_64 2024-02-16T22:36:18.665 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.14.0-361.el9.x86_64 2024-02-16T22:36:18.665 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-02-16T22:36:20.774 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-02-16T22:36:20.828 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-02-16T22:36:22.218 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 7.9 MB/s | 2.0 MB 00:00 2024-02-16T22:36:22.228 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-02-16T22:36:24.109 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.0 MB 00:01 2024-02-16T22:36:24.243 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-02-16T22:36:25.994 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-02-16T22:36:26.444 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-02-16T22:36:30.103 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-02-16T22:36:31.703 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-02-16T22:36:32.849 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:01 2024-02-16T22:36:38.213 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 9.0 kB/s | 1.7 kB 00:00 2024-02-16T22:36:40.628 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-16T22:36:40.628 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-16T22:36:40.730 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:36:40.737 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:36:40.738 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:36:40.738 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-02-16T22:36:40.738 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:36:40.738 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 100 M 2024-02-16T22:36:40.738 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 123 M 2024-02-16T22:36:40.738 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:36:41.189 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 13 MB/s | 6.1 MB 00:00 2024-02-16T22:36:41.689 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 22 MB/s | 21 MB 00:00 2024-02-16T22:36:42.464 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 26 MB/s | 33 MB 00:01 2024-02-16T22:36:42.965 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 18 MB/s | 39 MB 00:02 2024-02-16T22:36:42.968 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:36:42.968 INFO:teuthology.orchestra.run.smithi064.stdout:Total 45 MB/s | 100 MB 00:02 2024-02-16T22:36:42.968 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:36:43.398 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:36:43.398 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:36:43.952 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 445 kB/s | 6.1 MB 00:13 2024-02-16T22:36:44.475 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:36:44.475 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:36:46.427 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 15 kB 00:01 2024-02-16T22:36:47.062 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:36:47.562 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:00 2024-02-16T22:36:47.765 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-16T22:36:47.898 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:36:50.836 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:36:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:36:52.554 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-16T22:36:54.853 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-16T22:36:54.853 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-16T22:36:54.980 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:36:54.992 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:36:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:36:54.992 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-02-16T22:36:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:36:54.994 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 100 M 2024-02-16T22:36:54.995 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 123 M 2024-02-16T22:36:54.995 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:36:55.380 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 16 MB/s | 6.1 MB 00:00 2024-02-16T22:36:55.939 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 22 MB/s | 21 MB 00:00 2024-02-16T22:36:56.088 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:36:56.187 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:36:56.664 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 26 MB/s | 33 MB 00:01 2024-02-16T22:36:57.114 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 18 MB/s | 39 MB 00:02 2024-02-16T22:36:57.117 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:36:57.118 INFO:teuthology.orchestra.run.smithi192.stdout:Total 47 MB/s | 100 MB 00:02 2024-02-16T22:36:57.118 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:36:57.481 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:36:57.481 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:36:58.547 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:36:58.548 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:37:01.116 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:37:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-16T22:37:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:37:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:37:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:37:09.626 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:37:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:14.589 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:31.908 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:31.909 INFO:teuthology.orchestra.run.smithi064.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-16T22:37:31.909 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:37:31.992 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-16T22:37:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:37:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:37:32.794 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:37:33.281 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T22:37:33.547 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T22:37:33.547 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T22:37:33.547 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-16T22:37:33.548 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-16T22:37:33.548 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-16T22:37:33.548 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-16T22:37:33.548 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-02-16T22:37:33.548 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T22:37:33.661 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T22:37:33.661 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T22:37:33.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 68.el9 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8264254 2024-02-16T22:37:33.702 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:37:33.703 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-02-16T22:37:33.704 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-16T22:37:33.704 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 5.14.0-421.el9.x86_64 2024-02-16T22:37:33.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-16T22:37:34.351 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-02-16T22:37:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-16T22:37:35.774 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-02-16T22:37:35.776 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-02-16T22:37:35.791 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.z9AfvTVQ95 2024-02-16T22:37:35.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.z9AfvTVQ95 2024-02-16T22:37:35.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.z9AfvTVQ95 2024-02-16T22:37:36.109 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.z9AfvTVQ95 is 6KB 2024-02-16T22:37:36.158 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.z9AfvTVQ95 2024-02-16T22:37:36.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-02-16T22:37:36.242 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-16T22:37:36.242 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-16T22:37:36.242 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-16T22:37:36.242 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-16T22:37:36.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-16T22:37:36.381 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-02-16T22:37:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:43.796 INFO:teuthology.orchestra.run.smithi192.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-16T22:37:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:37:43.868 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:44.331 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:44.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-16T22:37:44.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:37:44.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:37:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:37:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:37:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:37:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-16T22:37:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-16T22:37:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-16T22:37:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-16T22:37:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:37:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:37:45.330 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T22:37:45.460 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T22:37:45.460 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T22:37:45.460 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-16T22:37:45.460 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-16T22:37:45.460 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-16T22:37:45.460 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-16T22:37:45.461 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=None, version=distro) 2024-02-16T22:37:45.461 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T22:37:45.569 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T22:37:45.569 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T22:37:45.569 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-02-16T22:37:45.612 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 68.el9 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8264254 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-16T22:37:45.613 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-02-16T22:37:45.614 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-02-16T22:37:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:37:45.614 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-16T22:37:45.614 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-16T22:37:45.614 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-16T22:37:45.614 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-02-16T22:37:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:37:45.614 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-02-16T22:37:45.615 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-16T22:37:45.615 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 5.14.0-421.el9.x86_64 2024-02-16T22:37:45.615 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-16T22:37:46.184 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-02-16T22:37:47.529 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-16T22:37:47.573 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-02-16T22:37:47.576 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-02-16T22:37:47.592 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.neBIBQZmjc 2024-02-16T22:37:47.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.neBIBQZmjc 2024-02-16T22:37:47.659 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.neBIBQZmjc 2024-02-16T22:37:47.800 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.neBIBQZmjc is 6KB 2024-02-16T22:37:47.819 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.neBIBQZmjc 2024-02-16T22:37:47.833 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-02-16T22:37:47.904 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-16T22:37:47.904 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-16T22:37:47.904 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-16T22:37:47.904 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-16T22:37:47.905 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-16T22:37:48.057 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-02-16T22:37:48.132 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-02-16T22:38:06.396 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-16T22:38:06.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-16T22:38:06.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:38:18.071 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-16T22:38:18.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-02-16T22:38:18.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:38:40.956 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-02-16T22:38:49.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-16T22:38:49.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:39:08.348 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-02-16T22:39:18.079 DEBUG:teuthology.orchestra.remote:timed out 2024-02-16T22:39:20.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-16T22:39:20.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:39:20.859 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-02-16T22:39:21.212 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-16T22:39:21.212 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-421.el9.x86_64"... 2024-02-16T22:39:21.212 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-02-16T22:39:21.226 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-421.el9.x86_64 2024-02-16T22:39:21.227 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-16T22:39:21.227 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-16T22:39:21.227 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-16T22:39:22.228 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-16T22:39:22.228 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-16T22:39:22.320 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-02-16T22:39:22.359 DEBUG:teuthology.parallel:result is None 2024-02-16T22:39:27.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-02-16T22:39:27.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:39:27.430 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-02-16T22:39:27.893 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-02-16T22:39:27.893 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-421.el9.x86_64"... 2024-02-16T22:39:27.894 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-02-16T22:39:27.908 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-421.el9.x86_64 2024-02-16T22:39:27.908 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-16T22:39:27.908 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-16T22:39:27.908 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-16T22:39:28.909 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-16T22:39:28.909 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-16T22:39:28.976 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-02-16T22:39:29.028 DEBUG:teuthology.parallel:result is None 2024-02-16T22:39:29.029 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-16T22:39:29.035 INFO:teuthology.task.internal:Creating test directory... 2024-02-16T22:39:29.036 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-16T22:39:29.039 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-16T22:39:29.054 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-16T22:39:29.066 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-16T22:39:29.100 INFO:teuthology.task.internal:Creating archive directory... 2024-02-16T22:39:29.100 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-16T22:39:29.102 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-16T22:39:29.133 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-16T22:39:29.139 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-16T22:39:29.139 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-16T22:39:29.171 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-16T22:39:29.201 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T22:39:29.206 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T22:39:29.215 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T22:39:29.219 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T22:39:29.220 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-16T22:39:29.227 INFO:teuthology.task.internal:Configuring sudo... 2024-02-16T22:39:29.227 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-16T22:39:29.260 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-16T22:39:29.287 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-16T22:39:29.296 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-16T22:39:29.297 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-16T22:39:29.329 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-16T22:39:29.344 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T22:39:29.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T22:39:29.478 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T22:39:29.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T22:39:29.604 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:39:29.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-16T22:39:29.672 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T22:39:29.714 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T22:39:29.760 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T22:39:29.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T22:39:29.882 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:39:29.883 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-16T22:39:29.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-02-16T22:39:29.953 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-02-16T22:39:30.013 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T22:39:30.038 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T22:39:30.400 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-16T22:39:30.407 INFO:teuthology.task.internal:Starting timer... 2024-02-16T22:39:30.407 INFO:teuthology.run_tasks:Running task pcp... 2024-02-16T22:39:30.453 INFO:teuthology.run_tasks:Running task selinux... 2024-02-16T22:39:30.515 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-16T22:39:30.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-02-16T22:39:30.577 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-02-16T22:39:30.579 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-02-16T22:39:30.661 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-02-16T22:39:30.663 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-16T22:39:30.664 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-02-16T22:39:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-02-16T22:39:30.690 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-02-16T22:39:30.713 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-02-16T22:39:30.713 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-02-16T22:39:30.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T22:39:30.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:39:30.760 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T22:39:30.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:39:30.787 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-16T22:39:30.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-02-16T22:39:30.833 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-02-16T22:39:30.865 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-16T22:39:30.875 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-16T22:39:30.880 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-16T22:39:31.035 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-16T22:39:31.050 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-16T22:39:31.051 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-16T22:43:40.250 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-02-16T22:43:40.251 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-16T22:43:40.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:43:40.341 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-02-16T22:43:40.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-16T22:43:40.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-02-16T22:43:40.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:43:40.513 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-02-16T22:43:40.586 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-02-16T22:43:40.586 INFO:teuthology.run_tasks:Running task clock... 2024-02-16T22:43:40.602 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-16T22:43:40.602 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-16T22:43:40.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T22:43:40.605 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-16T22:43:40.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T22:43:40.635 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-16T22:43:40.652 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-16T22:43:40.658 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-16T22:43:40.675 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-16T22:43:40.682 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-02-16T22:43:40.695 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-02-16T22:43:40.703 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-02-16T22:43:40.712 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-16T22:43:40.717 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-02-16T22:43:40.728 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-16T22:43:40.733 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-16T22:43:40.750 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-16T22:43:40.779 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-02-16T22:43:40.783 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T22:43:40.783 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-02-16T22:43:40.783 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:43:40.783 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:43:40.783 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:43:40.783 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:43:40.802 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-02-16T22:43:40.805 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T22:43:40.805 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-02-16T22:43:40.805 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:43:40.805 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:43:40.805 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:43:40.805 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:43:40.806 INFO:teuthology.run_tasks:Running task install... 2024-02-16T22:43:40.815 DEBUG:teuthology.task.install:project ceph 2024-02-16T22:43:40.815 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'}} 2024-02-16T22:43:40.815 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-16T22:43:40.815 INFO:teuthology.task.install:Using flavor: default 2024-02-16T22:43:40.821 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-16T22:43:40.821 INFO:teuthology.task.install:extra packages: [] 2024-02-16T22:43:40.821 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': None, 'wait_for_package': False} 2024-02-16T22:43:40.821 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T22:43:40.822 INFO:teuthology.packaging:ref: None 2024-02-16T22:43:40.822 INFO:teuthology.packaging:tag: None 2024-02-16T22:43:40.822 INFO:teuthology.packaging:branch: quincy 2024-02-16T22:43:40.822 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:43:40.822 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-02-16T22:43:40.824 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': None, 'wait_for_package': False} 2024-02-16T22:43:40.824 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T22:43:40.824 INFO:teuthology.packaging:ref: None 2024-02-16T22:43:40.824 INFO:teuthology.packaging:tag: None 2024-02-16T22:43:40.825 INFO:teuthology.packaging:branch: quincy 2024-02-16T22:43:40.825 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:43:40.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-02-16T22:43:41.033 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/ 2024-02-16T22:43:41.033 INFO:teuthology.task.install.rpm:Package version is 17.2.7-513.gb88fae66 2024-02-16T22:43:41.052 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/ 2024-02-16T22:43:41.053 INFO:teuthology.task.install.rpm:Package version is 17.2.7-513.gb88fae66 2024-02-16T22:43:41.153 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-16T22:43:41.154 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:43:41.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-16T22:43:41.162 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-16T22:43:41.162 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:43:41.162 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-16T22:43:41.181 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-16T22:43:41.181 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T22:43:41.181 INFO:teuthology.packaging:ref: None 2024-02-16T22:43:41.181 INFO:teuthology.packaging:tag: None 2024-02-16T22:43:41.182 INFO:teuthology.packaging:branch: quincy 2024-02-16T22:43:41.182 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:43:41.182 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-16T22:43:41.190 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-16T22:43:41.190 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T22:43:41.191 INFO:teuthology.packaging:ref: None 2024-02-16T22:43:41.191 INFO:teuthology.packaging:tag: None 2024-02-16T22:43:41.191 INFO:teuthology.packaging:branch: quincy 2024-02-16T22:43:41.191 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:43:41.191 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-16T22:43:41.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-16T22:43:41.259 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-16T22:43:41.331 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-16T22:43:41.338 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-16T22:43:41.397 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-02-16T22:43:41.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-02-16T22:43:41.404 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-02-16T22:43:41.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-02-16T22:43:41.650 INFO:teuthology.orchestra.run.smithi064.stdout:74 files removed 2024-02-16T22:43:41.655 INFO:teuthology.orchestra.run.smithi192.stdout:74 files removed 2024-02-16T22:43:41.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw 2024-02-16T22:43:41.682 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw 2024-02-16T22:43:42.407 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 191 kB/s | 77 kB 00:00 2024-02-16T22:43:42.508 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 153 kB/s | 77 kB 00:00 2024-02-16T22:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 23 kB/s | 9.1 kB 00:00 2024-02-16T22:43:42.975 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 22 kB/s | 9.1 kB 00:00 2024-02-16T22:43:43.216 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 5.6 kB/s | 1.8 kB 00:00 2024-02-16T22:43:43.334 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 5.6 kB/s | 1.8 kB 00:00 2024-02-16T22:43:43.694 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-16T22:43:43.736 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-02-16T22:43:45.203 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-02-16T22:43:45.252 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-02-16T22:43:46.702 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-02-16T22:43:47.877 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-02-16T22:43:48.654 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-02-16T22:43:49.819 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-02-16T22:43:54.287 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.1 MB 00:00 2024-02-16T22:43:54.446 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.1 MB 00:02 2024-02-16T22:43:56.256 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 15 kB 00:00 2024-02-16T22:43:56.863 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 15 kB 00:01 2024-02-16T22:43:57.898 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-16T22:44:02.915 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-02-16T22:44:05.655 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:44:05.668 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:05.668 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:17.2.7-513.gb88fae66.el9 ceph 14 M 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 3.1 M 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.9 M 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:17.2.7-513.gb88fae66.el9 ceph 5.0 M 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:17.2.7-513.gb88fae66.el9 ceph 23 M 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:17.2.7-513.gb88fae66.el9 ceph 25 k 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 635 k 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-16T22:44:05.669 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 458 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 4.9 M 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 45 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:17.2.7-513.gb88fae66.el9 ceph 116 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:17.2.7-513.gb88fae66.el9 ceph 156 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:17.2.7-513.gb88fae66.el9 ceph 320 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 293 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:17.2.7-513.gb88fae66.el9 ceph 99 k 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:05.670 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:44:05.671 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:05.671 INFO:teuthology.orchestra.run.smithi064.stdout:Install 22 Packages 2024-02-16T22:44:05.671 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 2 Packages 2024-02-16T22:44:05.671 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:05.671 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 58 M 2024-02-16T22:44:05.671 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:44:06.582 INFO:teuthology.orchestra.run.smithi064.stdout:(1/24): ceph-base-17.2.7-513.gb88fae66.el9.x86_ 6.0 MB/s | 5.0 MB 00:00 2024-02-16T22:44:06.682 INFO:teuthology.orchestra.run.smithi064.stdout:(2/24): ceph-selinux-17.2.7-513.gb88fae66.el9.x 252 kB/s | 25 kB 00:00 2024-02-16T22:44:06.833 INFO:teuthology.orchestra.run.smithi064.stdout:(3/24): libcephfs2-17.2.7-513.gb88fae66.el9.x86 4.1 MB/s | 635 kB 00:00 2024-02-16T22:44:06.958 INFO:teuthology.orchestra.run.smithi064.stdout:(4/24): libradosstriper1-17.2.7-513.gb88fae66.e 3.6 MB/s | 458 kB 00:00 2024-02-16T22:44:07.650 INFO:teuthology.orchestra.run.smithi064.stdout:(5/24): librgw2-17.2.7-513.gb88fae66.el9.x86_64 7.1 MB/s | 4.9 MB 00:00 2024-02-16T22:44:07.792 INFO:teuthology.orchestra.run.smithi064.stdout:(6/24): ceph-radosgw-17.2.7-513.gb88fae66.el9.x 6.9 MB/s | 14 MB 00:02 2024-02-16T22:44:07.819 INFO:teuthology.orchestra.run.smithi064.stdout:(7/24): python3-ceph-argparse-17.2.7-513.gb88fa 267 kB/s | 45 kB 00:00 2024-02-16T22:44:07.869 INFO:teuthology.orchestra.run.smithi064.stdout:(8/24): python3-ceph-common-17.2.7-513.gb88fae6 1.5 MB/s | 116 kB 00:00 2024-02-16T22:44:07.895 INFO:teuthology.orchestra.run.smithi064.stdout:(9/24): python3-cephfs-17.2.7-513.gb88fae66.el9 2.0 MB/s | 156 kB 00:00 2024-02-16T22:44:08.137 INFO:teuthology.orchestra.run.smithi064.stdout:(10/24): ceph-common-17.2.7-513.gb88fae66.el9.x 9.4 MB/s | 23 MB 00:02 2024-02-16T22:44:08.162 INFO:teuthology.orchestra.run.smithi064.stdout:(11/24): python3-rbd-17.2.7-513.gb88fae66.el9.x 1.1 MB/s | 293 kB 00:00 2024-02-16T22:44:08.187 INFO:teuthology.orchestra.run.smithi064.stdout:(12/24): python3-rados-17.2.7-513.gb88fae66.el9 1.0 MB/s | 320 kB 00:00 2024-02-16T22:44:08.212 INFO:teuthology.orchestra.run.smithi064.stdout:(13/24): python3-rgw-17.2.7-513.gb88fae66.el9.x 1.3 MB/s | 99 kB 00:00 2024-02-16T22:44:08.346 INFO:teuthology.orchestra.run.smithi064.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 279 kB/s | 44 kB 00:00 2024-02-16T22:44:08.371 INFO:teuthology.orchestra.run.smithi064.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 761 kB/s | 159 kB 00:00 2024-02-16T22:44:08.430 INFO:teuthology.orchestra.run.smithi064.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-02-16T22:44:08.455 INFO:teuthology.orchestra.run.smithi064.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 493 kB/s | 41 kB 00:00 2024-02-16T22:44:08.505 INFO:teuthology.orchestra.run.smithi064.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 661 kB 00:00 2024-02-16T22:44:08.947 INFO:teuthology.orchestra.run.smithi064.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 99 kB/s | 49 kB 00:00 2024-02-16T22:44:08.997 INFO:teuthology.orchestra.run.smithi064.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 137 kB/s | 67 kB 00:00 2024-02-16T22:44:09.131 INFO:teuthology.orchestra.run.smithi064.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 441 kB/s | 309 kB 00:00 2024-02-16T22:44:09.306 INFO:teuthology.orchestra.run.smithi064.stdout:(22/24): librados2-17.2.7-513.gb88fae66.el9.x86 10 MB/s | 3.1 MB 00:00 2024-02-16T22:44:09.515 INFO:teuthology.orchestra.run.smithi064.stdout:(23/24): thrift-0.15.0-2.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-02-16T22:44:09.565 INFO:teuthology.orchestra.run.smithi064.stdout:(24/24): librbd1-17.2.7-513.gb88fae66.el9.x86_6 6.8 MB/s | 2.9 MB 00:00 2024-02-16T22:44:09.571 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:44:09.571 INFO:teuthology.orchestra.run.smithi064.stdout:Total 15 MB/s | 58 MB 00:03 2024-02-16T22:44:09.571 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:44:09.603 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:44:09.603 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:44:10.089 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:44:10.089 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:44:11.341 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:44:11.694 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-02-16T22:44:11.751 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T22:44:11.843 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T22:44:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 3/26 2024-02-16T22:44:11.952 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T22:44:12.041 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T22:44:12.086 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-02-16T22:44:12.115 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-02-16T22:44:12.575 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-02-16T22:44:12.605 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 8/26 2024-02-16T22:44:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 8/26 2024-02-16T22:44:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-02-16T22:44:12.757 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x 10/26 2024-02-16T22:44:12.793 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 11/26 2024-02-16T22:44:12.857 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 12/26 2024-02-16T22:44:12.883 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T22:44:12.940 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T22:44:12.989 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-02-16T22:44:13.027 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-02-16T22:44:13.057 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-02-16T22:44:13.367 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-02-16T22:44:13.395 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 18/26 2024-02-16T22:44:13.469 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 18/26 2024-02-16T22:44:13.499 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 1.2 MB/s | 20 MB 00:16 2024-02-16T22:44:13.545 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 19/26 2024-02-16T22:44:13.589 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86 20/26 2024-02-16T22:44:16.018 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T22:44:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T22:44:16.618 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T22:44:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 22/26 2024-02-16T22:44:16.737 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 22/26 2024-02-16T22:44:16.737 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-16T22:44:16.737 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-16T22:44:16.737 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:16.765 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T22:44:18.571 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 6.4 kB/s | 1.7 kB 00:00 2024-02-16T22:44:21.235 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:44:21.250 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:44:21.250 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:17.2.7-513.gb88fae66.el9 ceph 14 M 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 3.1 M 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.9 M 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:17.2.7-513.gb88fae66.el9 ceph 5.0 M 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:17.2.7-513.gb88fae66.el9 ceph 23 M 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:17.2.7-513.gb88fae66.el9 ceph 25 k 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-16T22:44:21.251 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 635 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 458 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 4.9 M 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 45 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:17.2.7-513.gb88fae66.el9 ceph 116 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:17.2.7-513.gb88fae66.el9 ceph 156 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:17.2.7-513.gb88fae66.el9 ceph 320 k 2024-02-16T22:44:21.252 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 293 k 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:17.2.7-513.gb88fae66.el9 ceph 99 k 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout:Install 22 Packages 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 2 Packages 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:21.253 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 58 M 2024-02-16T22:44:21.254 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:44:21.963 INFO:teuthology.orchestra.run.smithi192.stdout:(1/24): ceph-base-17.2.7-513.gb88fae66.el9.x86_ 8.1 MB/s | 5.0 MB 00:00 2024-02-16T22:44:22.014 INFO:teuthology.orchestra.run.smithi192.stdout:(2/24): ceph-selinux-17.2.7-513.gb88fae66.el9.x 504 kB/s | 25 kB 00:00 2024-02-16T22:44:22.198 INFO:teuthology.orchestra.run.smithi192.stdout:(3/24): libcephfs2-17.2.7-513.gb88fae66.el9.x86 3.4 MB/s | 635 kB 00:00 2024-02-16T22:44:22.290 INFO:teuthology.orchestra.run.smithi192.stdout:(4/24): libradosstriper1-17.2.7-513.gb88fae66.e 4.9 MB/s | 458 kB 00:00 2024-02-16T22:44:22.790 INFO:teuthology.orchestra.run.smithi192.stdout:(5/24): librgw2-17.2.7-513.gb88fae66.el9.x86_64 9.8 MB/s | 4.9 MB 00:00 2024-02-16T22:44:22.840 INFO:teuthology.orchestra.run.smithi192.stdout:(6/24): python3-ceph-argparse-17.2.7-513.gb88fa 907 kB/s | 45 kB 00:00 2024-02-16T22:44:22.907 INFO:teuthology.orchestra.run.smithi192.stdout:(7/24): python3-ceph-common-17.2.7-513.gb88fae6 1.7 MB/s | 116 kB 00:00 2024-02-16T22:44:22.982 INFO:teuthology.orchestra.run.smithi192.stdout:(8/24): python3-cephfs-17.2.7-513.gb88fae66.el9 2.0 MB/s | 156 kB 00:00 2024-02-16T22:44:23.058 INFO:teuthology.orchestra.run.smithi192.stdout:(9/24): python3-rados-17.2.7-513.gb88fae66.el9. 4.2 MB/s | 320 kB 00:00 2024-02-16T22:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout:(10/24): python3-rbd-17.2.7-513.gb88fae66.el9.x 3.8 MB/s | 293 kB 00:00 2024-02-16T22:44:23.200 INFO:teuthology.orchestra.run.smithi192.stdout:(11/24): python3-rgw-17.2.7-513.gb88fae66.el9.x 1.5 MB/s | 99 kB 00:00 2024-02-16T22:44:23.384 INFO:teuthology.orchestra.run.smithi192.stdout:(12/24): ceph-radosgw-17.2.7-513.gb88fae66.el9. 6.9 MB/s | 14 MB 00:02 2024-02-16T22:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout:(13/24): ceph-common-17.2.7-513.gb88fae66.el9.x 10 MB/s | 23 MB 00:02 2024-02-16T22:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 184 kB/s | 44 kB 00:00 2024-02-16T22:44:23.684 INFO:teuthology.orchestra.run.smithi192.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 329 kB/s | 159 kB 00:00 2024-02-16T22:44:23.793 INFO:teuthology.orchestra.run.smithi192.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 661 kB 00:00 2024-02-16T22:44:23.835 INFO:teuthology.orchestra.run.smithi192.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 282 kB 00:00 2024-02-16T22:44:23.860 INFO:teuthology.orchestra.run.smithi192.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 235 kB/s | 41 kB 00:00 2024-02-16T22:44:24.011 INFO:teuthology.orchestra.run.smithi192.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2024-02-16T22:44:24.044 INFO:teuthology.orchestra.run.smithi192.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-02-16T22:44:24.111 INFO:teuthology.orchestra.run.smithi192.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 973 kB/s | 309 kB 00:00 2024-02-16T22:44:24.270 INFO:teuthology.orchestra.run.smithi192.stdout:(22/24): librados2-17.2.7-513.gb88fae66.el9.x86 14 MB/s | 3.1 MB 00:00 2024-02-16T22:44:24.320 INFO:teuthology.orchestra.run.smithi192.stdout:(23/24): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-02-16T22:44:24.420 INFO:teuthology.orchestra.run.smithi192.stdout:(24/24): librbd1-17.2.7-513.gb88fae66.el9.x86_6 9.5 MB/s | 2.9 MB 00:00 2024-02-16T22:44:24.423 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:44:24.424 INFO:teuthology.orchestra.run.smithi192.stdout:Total 18 MB/s | 58 MB 00:03 2024-02-16T22:44:24.424 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:44:24.459 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:44:24.460 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:44:24.928 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:44:24.928 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:44:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:44:26.512 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-02-16T22:44:26.564 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-02-16T22:44:26.660 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:26.664 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T22:44:26.752 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 3/26 2024-02-16T22:44:26.782 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T22:44:26.879 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T22:44:26.924 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-02-16T22:44:26.954 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-02-16T22:44:27.388 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-02-16T22:44:27.410 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 8/26 2024-02-16T22:44:27.464 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 8/26 2024-02-16T22:44:27.485 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-02-16T22:44:27.519 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x 10/26 2024-02-16T22:44:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 11/26 2024-02-16T22:44:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 12/26 2024-02-16T22:44:27.647 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T22:44:27.712 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T22:44:27.752 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-02-16T22:44:27.788 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-02-16T22:44:27.821 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-02-16T22:44:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 24/26 2024-02-16T22:44:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 24/26 2024-02-16T22:44:28.050 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:44:28.050 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-16T22:44:28.050 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T22:44:28.050 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T22:44:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-02-16T22:44:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-02-16T22:44:28.142 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 18/26 2024-02-16T22:44:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-02-16T22:44:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T22:44:28.231 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 18/26 2024-02-16T22:44:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 19/26 2024-02-16T22:44:28.337 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86 20/26 2024-02-16T22:44:29.391 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T22:44:29.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 1/26 2024-02-16T22:44:29.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T22:44:29.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 3/26 2024-02-16T22:44:29.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T22:44:29.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 5/26 2024-02-16T22:44:29.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 6/26 2024-02-16T22:44:29.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 7/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x 8/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86 9/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 10/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 11/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 12/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-02-16T22:44:29.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-02-16T22:44:29.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-02-16T22:44:29.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-02-16T22:44:29.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-02-16T22:44:29.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T22:44:29.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-02-16T22:44:29.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 25/26 2024-02-16T22:44:30.292 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T22:44:30.292 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:30.292 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.293 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:30.294 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:44:30.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-test 2024-02-16T22:44:30.816 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T22:44:31.150 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T22:44:31.205 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:29 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:44:31.376 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:44:31.377 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:31.377 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:44:31.377 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:31.377 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:44:31.377 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:17.2.7-513.gb88fae66.el9 ceph 48 M 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:17.2.7-513.gb88fae66.el9 ceph 162 k 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-02-16T22:44:31.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:31.379 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 49 M 2024-02-16T22:44:31.379 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 196 M 2024-02-16T22:44:31.379 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:44:31.557 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 300 kB 00:00 2024-02-16T22:44:31.599 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-16T22:44:31.657 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): libcephsqlite-17.2.7-513.gb88fae66.el9.x 585 kB/s | 162 kB 00:00 2024-02-16T22:44:31.683 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 2.2 MB/s | 186 kB 00:00 2024-02-16T22:44:31.716 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-02-16T22:44:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T22:44:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 22/26 2024-02-16T22:44:31.835 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 22/26 2024-02-16T22:44:31.835 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-16T22:44:31.835 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-16T22:44:31.835 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:31.872 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T22:44:34.083 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): ceph-test-17.2.7-513.gb88fae66.el9.x86_6 18 MB/s | 48 MB 00:02 2024-02-16T22:44:34.086 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:44:34.086 INFO:teuthology.orchestra.run.smithi064.stdout:Total 18 MB/s | 49 MB 00:02 2024-02-16T22:44:34.087 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:44:34.107 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:44:34.108 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:44:34.509 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:44:34.510 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:44:35.143 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:44:35.204 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-16T22:44:35.241 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-16T22:44:35.286 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-16T22:44:35.326 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-16T22:44:35.353 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 5/6 2024-02-16T22:44:40.128 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 5/6 2024-02-16T22:44:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 6/6 2024-02-16T22:44:41.327 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T22:44:41.328 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-02-16T22:44:41.328 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-02-16T22:44:41.328 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-02-16T22:44:41.328 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-02-16T22:44:41.328 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-02-16T22:44:41.328 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-02-16T22:44:41.328 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-02-16T22:44:41.328 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 6/6 2024-02-16T22:44:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 1/6 2024-02-16T22:44:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 2/6 2024-02-16T22:44:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-16T22:44:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-16T22:44:42.404 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-16T22:44:42.631 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 24/26 2024-02-16T22:44:42.709 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 24/26 2024-02-16T22:44:42.709 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:44:42.709 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-16T22:44:42.709 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T22:44:42.709 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T22:44:42.709 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:42.710 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-02-16T22:44:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-16T22:44:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:42.792 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:44:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:42.792 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-16.el9.x86_64 2024-02-16T22:44:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-16T22:44:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-16T22:44:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-16T22:44:42.793 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:42.793 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:44:42.800 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-02-16T22:44:42.800 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T22:44:43.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph 2024-02-16T22:44:43.666 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:41 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:44:43.846 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:44:43.848 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================== 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================== 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:17.2.7-513.gb88fae66.el9 ceph 6.5 k 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.1 M 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:17.2.7-513.gb88fae66.el9 ceph 1.4 M 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 239 k 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:17.2.7-513.gb88fae66.el9 ceph 3.7 M 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 15 M 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-16T22:44:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-16T22:44:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================== 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout:Install 33 Packages 2024-02-16T22:44:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:43.852 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 27 M 2024-02-16T22:44:43.852 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 99 M 2024-02-16T22:44:43.852 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:44:44.068 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T22:44:44.113 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 1/26 2024-02-16T22:44:44.113 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T22:44:44.113 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 3/26 2024-02-16T22:44:44.113 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T22:44:44.113 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 5/26 2024-02-16T22:44:44.113 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 6/26 2024-02-16T22:44:44.113 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 7/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x 8/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86 9/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 10/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 11/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 12/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-02-16T22:44:44.114 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-02-16T22:44:44.115 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-02-16T22:44:44.115 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T22:44:44.115 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-02-16T22:44:44.115 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 25/26 2024-02-16T22:44:44.243 INFO:teuthology.orchestra.run.smithi064.stdout:(1/33): ceph-17.2.7-513.gb88fae66.el9.x86_64.rp 20 kB/s | 6.5 kB 00:00 2024-02-16T22:44:44.451 INFO:teuthology.orchestra.run.smithi064.stdout:(2/33): ceph-mds-17.2.7-513.gb88fae66.el9.x86_6 3.9 MB/s | 2.1 MB 00:00 2024-02-16T22:44:44.527 INFO:teuthology.orchestra.run.smithi064.stdout:(3/33): ceph-mgr-17.2.7-513.gb88fae66.el9.x86_6 2.3 MB/s | 1.4 MB 00:00 2024-02-16T22:44:44.627 INFO:teuthology.orchestra.run.smithi064.stdout:(4/33): ceph-mgr-modules-core-17.2.7-513.gb88fa 2.3 MB/s | 239 kB 00:00 2024-02-16T22:44:44.836 INFO:teuthology.orchestra.run.smithi064.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 692 kB/s | 144 kB 00:00 2024-02-16T22:44:44.869 INFO:teuthology.orchestra.run.smithi064.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 953 kB/s | 32 kB 00:00 2024-02-16T22:44:44.919 INFO:teuthology.orchestra.run.smithi064.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-02-16T22:44:44.953 INFO:teuthology.orchestra.run.smithi064.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-02-16T22:44:45.003 INFO:teuthology.orchestra.run.smithi064.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-16T22:44:45.083 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T22:44:45.083 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:45.083 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-02-16T22:44:45.083 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.083 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.083 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:44:45.083 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-16T22:44:45.084 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:45.085 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:44:45.087 INFO:teuthology.orchestra.run.smithi064.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 14 MB/s | 1.1 MB 00:00 2024-02-16T22:44:45.121 INFO:teuthology.orchestra.run.smithi064.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-02-16T22:44:45.271 INFO:teuthology.orchestra.run.smithi064.stdout:(12/33): ceph-osd-17.2.7-513.gb88fae66.el9.x86_ 19 MB/s | 15 MB 00:00 2024-02-16T22:44:45.296 INFO:teuthology.orchestra.run.smithi064.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 706 kB/s | 124 kB 00:00 2024-02-16T22:44:45.323 INFO:teuthology.orchestra.run.smithi064.stdout:(14/33): python3-requests-2.25.1-8.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-02-16T22:44:45.347 INFO:teuthology.orchestra.run.smithi064.stdout:(15/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-16T22:44:45.439 INFO:teuthology.orchestra.run.smithi064.stdout:(16/33): ceph-mon-17.2.7-513.gb88fae66.el9.x86_ 3.1 MB/s | 3.7 MB 00:01 2024-02-16T22:44:45.447 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-test 2024-02-16T22:44:45.456 INFO:teuthology.orchestra.run.smithi064.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 325 kB/s | 43 kB 00:00 2024-02-16T22:44:45.481 INFO:teuthology.orchestra.run.smithi064.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-02-16T22:44:45.506 INFO:teuthology.orchestra.run.smithi064.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 212 kB/s | 11 kB 00:00 2024-02-16T22:44:45.523 INFO:teuthology.orchestra.run.smithi064.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 422 kB/s | 18 kB 00:00 2024-02-16T22:44:45.540 INFO:teuthology.orchestra.run.smithi064.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 692 kB/s | 23 kB 00:00 2024-02-16T22:44:45.565 INFO:teuthology.orchestra.run.smithi064.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 466 kB/s | 19 kB 00:00 2024-02-16T22:44:45.591 INFO:teuthology.orchestra.run.smithi064.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.3 MB/s | 358 kB 00:00 2024-02-16T22:44:45.607 INFO:teuthology.orchestra.run.smithi064.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 293 kB/s | 20 kB 00:00 2024-02-16T22:44:45.624 INFO:teuthology.orchestra.run.smithi064.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 787 kB/s | 46 kB 00:00 2024-02-16T22:44:45.650 INFO:teuthology.orchestra.run.smithi064.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-02-16T22:44:45.667 INFO:teuthology.orchestra.run.smithi064.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 392 kB/s | 16 kB 00:00 2024-02-16T22:44:45.692 INFO:teuthology.orchestra.run.smithi064.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-02-16T22:44:45.717 INFO:teuthology.orchestra.run.smithi064.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-02-16T22:44:45.743 INFO:teuthology.orchestra.run.smithi064.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-02-16T22:44:45.768 INFO:teuthology.orchestra.run.smithi064.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-02-16T22:44:45.793 INFO:teuthology.orchestra.run.smithi064.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-02-16T22:44:45.818 INFO:teuthology.orchestra.run.smithi064.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-02-16T22:44:45.821 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:44:45.822 INFO:teuthology.orchestra.run.smithi064.stdout:Total 14 MB/s | 27 MB 00:01 2024-02-16T22:44:45.822 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:44:45.864 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:44:45.864 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:44:46.022 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:28 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:44:46.188 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:17.2.7-513.gb88fae66.el9 ceph 48 M 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:17.2.7-513.gb88fae66.el9 ceph 162 k 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-16T22:44:46.189 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:46.190 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:44:46.190 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:44:46.190 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-02-16T22:44:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:46.190 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 49 M 2024-02-16T22:44:46.190 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 196 M 2024-02-16T22:44:46.190 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:44:46.226 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:44:46.226 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:44:46.351 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 300 kB 00:00 2024-02-16T22:44:46.375 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.5 MB/s | 63 kB 00:00 2024-02-16T22:44:46.417 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-02-16T22:44:46.459 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-02-16T22:44:46.534 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): libcephsqlite-17.2.7-513.gb88fae66.el9.x 473 kB/s | 162 kB 00:00 2024-02-16T22:44:47.454 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:44:47.504 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-16T22:44:47.537 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-16T22:44:47.570 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-16T22:44:47.665 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-16T22:44:47.751 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-16T22:44:47.798 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-16T22:44:47.872 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-16T22:44:47.932 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-16T22:44:47.970 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-16T22:44:48.020 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-16T22:44:48.084 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-16T22:44:48.277 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-16T22:44:48.332 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-16T22:44:48.371 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-16T22:44:48.406 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-16T22:44:48.437 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-16T22:44:48.525 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-16T22:44:48.526 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): ceph-test-17.2.7-513.gb88fae66.el9.x86_6 21 MB/s | 48 MB 00:02 2024-02-16T22:44:48.529 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:44:48.530 INFO:teuthology.orchestra.run.smithi192.stdout:Total 21 MB/s | 49 MB 00:02 2024-02-16T22:44:48.530 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:44:48.549 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:44:48.550 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:44:48.556 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T22:44:48.589 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-16T22:44:48.630 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-16T22:44:48.674 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-16T22:44:48.708 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-16T22:44:48.799 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-16T22:44:48.843 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-16T22:44:48.922 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-16T22:44:48.936 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:44:48.936 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:44:49.061 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-16T22:44:49.185 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T22:44:49.354 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.n 28/33 2024-02-16T22:44:49.385 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 29/33 2024-02-16T22:44:49.410 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 29/33 2024-02-16T22:44:49.410 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:44:49.410 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-16T22:44:49.410 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T22:44:49.410 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T22:44:49.410 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:49.517 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:44:49.583 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-16T22:44:49.617 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-16T22:44:49.662 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-16T22:44:49.702 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-16T22:44:49.729 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 5/6 2024-02-16T22:44:50.819 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 30/33 2024-02-16T22:44:50.845 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 30/33 2024-02-16T22:44:50.845 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:44:50.846 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-16T22:44:50.846 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T22:44:50.846 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T22:44:50.846 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:51.245 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 31/33 2024-02-16T22:44:51.270 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 31/33 2024-02-16T22:44:51.270 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:44:51.270 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-16T22:44:51.270 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T22:44:51.270 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T22:44:51.270 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:51.479 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 32/33 2024-02-16T22:44:51.504 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 32/33 2024-02-16T22:44:51.504 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:44:51.504 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-16T22:44:51.504 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T22:44:51.504 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T22:44:51.504 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:51.571 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:17.2.7-513.gb88fae66.el9.x86_64 33/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-2:17.2.7-513.gb88fae66.el9.x86_64 33/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:17.2.7-513.gb88fae66.el9.x86_64 1/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 2/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 3/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 4/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 5/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.n 6/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-16T22:44:53.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-16T22:44:53.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-16T22:44:53.232 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-16T22:44:53.232 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-16T22:44:53.232 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-16T22:44:53.233 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-16T22:44:53.233 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T22:44:53.233 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-16T22:44:53.233 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-16T22:44:53.233 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-16T22:44:53.233 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-16T22:44:53.233 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-16T22:44:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-16T22:44:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:54.143 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:44:54.190 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 5/6 2024-02-16T22:44:54.269 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 6/6 2024-02-16T22:44:54.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-base 2024-02-16T22:44:55.071 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:53 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:44:55.189 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:44:55.243 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:44:55.244 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:44:55.244 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:44:55.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install cephadm 2024-02-16T22:44:55.882 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:53 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:44:56.054 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:44:56.055 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:56.055 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 80 k 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 80 k 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 364 k 2024-02-16T22:44:56.056 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:44:56.253 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm-17.2.7-513.gb88fae66.el9.noarch.rpm 409 kB/s | 80 kB 00:00 2024-02-16T22:44:56.254 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:44:56.254 INFO:teuthology.orchestra.run.smithi064.stdout:Total 406 kB/s | 80 kB 00:00 2024-02-16T22:44:56.254 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:44:56.259 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:44:56.259 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:44:56.261 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:44:56.261 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:44:56.339 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:44:56.553 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 6/6 2024-02-16T22:44:56.553 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 1/6 2024-02-16T22:44:56.553 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 2/6 2024-02-16T22:44:56.553 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-16T22:44:56.553 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-16T22:44:56.553 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-16T22:44:56.700 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T22:44:56.810 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T22:44:56.951 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-16T22:44:56.951 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:56.951 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:44:56.952 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:56.952 INFO:teuthology.orchestra.run.smithi192.stdout: jq-1.6-16.el9.x86_64 2024-02-16T22:44:56.952 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:44:56.952 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-16T22:44:56.952 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-16T22:44:56.952 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-16T22:44:56.952 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:56.952 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:44:57.198 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T22:44:57.235 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph 2024-02-16T22:44:57.503 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T22:44:57.503 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:57.503 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:44:57.503 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:44:57.503 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:57.504 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:44:57.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-immutable-object-cache 2024-02-16T22:44:57.823 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:39 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:44:57.997 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:44:57.999 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================== 2024-02-16T22:44:57.999 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:44:57.999 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================== 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:17.2.7-513.gb88fae66.el9 ceph 6.5 k 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.1 M 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:17.2.7-513.gb88fae66.el9 ceph 1.4 M 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 239 k 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:17.2.7-513.gb88fae66.el9 ceph 3.7 M 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 15 M 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-16T22:44:58.000 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-16T22:44:58.001 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-16T22:44:58.002 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-16T22:44:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-16T22:44:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:58.003 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:44:58.003 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================== 2024-02-16T22:44:58.003 INFO:teuthology.orchestra.run.smithi192.stdout:Install 33 Packages 2024-02-16T22:44:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:44:58.004 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 27 M 2024-02-16T22:44:58.004 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 99 M 2024-02-16T22:44:58.004 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:44:58.219 INFO:teuthology.orchestra.run.smithi192.stdout:(1/33): ceph-17.2.7-513.gb88fae66.el9.x86_64.rp 39 kB/s | 6.5 kB 00:00 2024-02-16T22:44:58.291 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:56 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:44:58.437 INFO:teuthology.orchestra.run.smithi192.stdout:(2/33): ceph-mds-17.2.7-513.gb88fae66.el9.x86_6 5.4 MB/s | 2.1 MB 00:00 2024-02-16T22:44:58.463 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-513.gb88fae66.el9 ceph 137 k 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-16T22:44:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:58.465 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 137 k 2024-02-16T22:44:58.465 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 379 k 2024-02-16T22:44:58.465 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:44:58.737 INFO:teuthology.orchestra.run.smithi192.stdout:(3/33): ceph-mon-17.2.7-513.gb88fae66.el9.x86_6 7.2 MB/s | 3.7 MB 00:00 2024-02-16T22:44:58.804 INFO:teuthology.orchestra.run.smithi192.stdout:(4/33): ceph-mgr-modules-core-17.2.7-513.gb88fa 3.5 MB/s | 239 kB 00:00 2024-02-16T22:44:58.837 INFO:teuthology.orchestra.run.smithi192.stdout:(5/33): ceph-mgr-17.2.7-513.gb88fae66.el9.x86_6 1.8 MB/s | 1.4 MB 00:00 2024-02-16T22:44:58.877 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-immutable-object-cache-17.2.7-513.gb88fae6 332 kB/s | 137 kB 00:00 2024-02-16T22:44:58.877 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:44:58.878 INFO:teuthology.orchestra.run.smithi064.stdout:Total 331 kB/s | 137 kB 00:00 2024-02-16T22:44:58.878 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:44:58.886 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:44:58.886 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:44:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:44:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:44:58.963 INFO:teuthology.orchestra.run.smithi192.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 314 kB/s | 32 kB 00:00 2024-02-16T22:44:58.988 INFO:teuthology.orchestra.run.smithi192.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 786 kB/s | 144 kB 00:00 2024-02-16T22:44:59.013 INFO:teuthology.orchestra.run.smithi192.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 946 kB/s | 47 kB 00:00 2024-02-16T22:44:59.038 INFO:teuthology.orchestra.run.smithi192.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 770 kB/s | 39 kB 00:00 2024-02-16T22:44:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:44:59.080 INFO:teuthology.orchestra.run.smithi192.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-02-16T22:44:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T22:44:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T22:44:59.152 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:44:59.152 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-16T22:44:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:44:59.264 INFO:teuthology.orchestra.run.smithi192.stdout:(11/33): ceph-osd-17.2.7-513.gb88fae66.el9.x86_ 19 MB/s | 15 MB 00:00 2024-02-16T22:44:59.298 INFO:teuthology.orchestra.run.smithi192.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 4.4 MB/s | 1.1 MB 00:00 2024-02-16T22:44:59.323 INFO:teuthology.orchestra.run.smithi192.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 426 kB/s | 103 kB 00:00 2024-02-16T22:44:59.348 INFO:teuthology.orchestra.run.smithi192.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 1.5 MB/s | 124 kB 00:00 2024-02-16T22:44:59.373 INFO:teuthology.orchestra.run.smithi192.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-02-16T22:44:59.399 INFO:teuthology.orchestra.run.smithi192.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.4 MB/s | 187 kB 00:00 2024-02-16T22:44:59.624 INFO:teuthology.orchestra.run.smithi192.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 158 kB/s | 43 kB 00:00 2024-02-16T22:44:59.715 INFO:teuthology.orchestra.run.smithi192.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 117 kB/s | 11 kB 00:00 2024-02-16T22:44:59.774 INFO:teuthology.orchestra.run.smithi192.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 488 kB/s | 172 kB 00:00 2024-02-16T22:44:59.799 INFO:teuthology.orchestra.run.smithi192.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 211 kB/s | 18 kB 00:00 2024-02-16T22:44:59.850 INFO:teuthology.orchestra.run.smithi192.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-02-16T22:44:59.883 INFO:teuthology.orchestra.run.smithi192.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 778 kB/s | 358 kB 00:00 2024-02-16T22:44:59.908 INFO:teuthology.orchestra.run.smithi192.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 178 kB/s | 19 kB 00:00 2024-02-16T22:44:59.934 INFO:teuthology.orchestra.run.smithi192.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 234 kB/s | 20 kB 00:00 2024-02-16T22:44:59.992 INFO:teuthology.orchestra.run.smithi192.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 425 kB/s | 46 kB 00:00 2024-02-16T22:45:00.018 INFO:teuthology.orchestra.run.smithi192.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 724 kB/s | 79 kB 00:00 2024-02-16T22:45:00.068 INFO:teuthology.orchestra.run.smithi192.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-02-16T22:45:00.093 INFO:teuthology.orchestra.run.smithi192.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2024-02-16T22:45:00.118 INFO:teuthology.orchestra.run.smithi192.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 895 kB/s | 90 kB 00:00 2024-02-16T22:45:00.135 INFO:teuthology.orchestra.run.smithi192.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 532 kB/s | 36 kB 00:00 2024-02-16T22:45:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T22:45:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:00.148 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:00.148 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:00.211 INFO:teuthology.orchestra.run.smithi192.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 265 kB/s | 20 kB 00:00 2024-02-16T22:45:00.244 INFO:teuthology.orchestra.run.smithi192.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-02-16T22:45:00.286 INFO:teuthology.orchestra.run.smithi192.stdout:(33/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-02-16T22:45:00.293 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:00.293 INFO:teuthology.orchestra.run.smithi192.stdout:Total 12 MB/s | 27 MB 00:02 2024-02-16T22:45:00.293 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:00.328 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:00.328 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:00.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr 2024-02-16T22:45:00.671 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:00.672 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:01.016 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:59 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:01.134 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:01.187 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:01.189 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:45:01.189 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:01.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-dashboard 2024-02-16T22:45:01.835 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:59 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-16T22:45:01.971 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-16T22:45:02.004 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-16T22:45:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 1.7 M 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 23 k 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 15 k 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-16T22:45:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-16T22:45:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-16T22:45:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-02-16T22:45:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-16T22:45:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:Install 7 Packages 2024-02-16T22:45:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 2.0 M 2024-02-16T22:45:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 8.1 M 2024-02-16T22:45:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-16T22:45:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-16T22:45:02.224 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-16T22:45:02.296 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-16T22:45:02.351 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-16T22:45:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-16T22:45:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-16T22:45:02.464 INFO:teuthology.orchestra.run.smithi064.stdout:(1/7): ceph-prometheus-alerts-17.2.7-513.gb88fa 74 kB/s | 15 kB 00:00 2024-02-16T22:45:02.491 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-16T22:45:02.556 INFO:teuthology.orchestra.run.smithi064.stdout:(2/7): ceph-grafana-dashboards-17.2.7-513.gb88f 81 kB/s | 23 kB 00:00 2024-02-16T22:45:02.614 INFO:teuthology.orchestra.run.smithi064.stdout:(3/7): ceph-mgr-dashboard-17.2.7-513.gb88fae66. 4.9 MB/s | 1.7 MB 00:00 2024-02-16T22:45:02.648 INFO:teuthology.orchestra.run.smithi064.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 49 kB/s | 9.0 kB 00:00 2024-02-16T22:45:02.675 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-16T22:45:02.733 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-16T22:45:02.763 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-16T22:45:02.806 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-16T22:45:02.836 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-16T22:45:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 123 kB/s | 31 kB 00:00 2024-02-16T22:45:02.882 INFO:teuthology.orchestra.run.smithi064.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 125 kB/s | 41 kB 00:00 2024-02-16T22:45:02.912 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-16T22:45:02.946 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T22:45:02.980 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-16T22:45:02.982 INFO:teuthology.orchestra.run.smithi064.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 565 kB/s | 188 kB 00:00 2024-02-16T22:45:02.984 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:02.984 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.0 MB/s | 2.0 MB 00:00 2024-02-16T22:45:02.984 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:02.993 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:02.993 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:03.020 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-16T22:45:03.055 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-16T22:45:03.065 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:03.065 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:03.115 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-16T22:45:03.203 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-16T22:45:03.256 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-16T22:45:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-16T22:45:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-16T22:45:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-16T22:45:03.469 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-16T22:45:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-16T22:45:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-16T22:45:03.588 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T22:45:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.no 5/7 2024-02-16T22:45:03.750 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.n 28/33 2024-02-16T22:45:03.775 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 29/33 2024-02-16T22:45:03.800 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 29/33 2024-02-16T22:45:03.800 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:45:03.801 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-16T22:45:03.801 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T22:45:03.801 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T22:45:03.801 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.n 6/7 2024-02-16T22:45:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 7/7 2024-02-16T22:45:04.389 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 7/7 2024-02-16T22:45:04.389 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.n 1/7 2024-02-16T22:45:04.389 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 2/7 2024-02-16T22:45:04.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.no 3/7 2024-02-16T22:45:04.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-16T22:45:04.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-16T22:45:04.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-16T22:45:04.700 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-16T22:45:04.700 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:04.700 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:04.700 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:04.700 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:04.701 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:04.701 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-16T22:45:04.701 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-16T22:45:04.701 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-16T22:45:04.701 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-16T22:45:04.701 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:04.701 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:04.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-16T22:45:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 30/33 2024-02-16T22:45:05.177 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 30/33 2024-02-16T22:45:05.177 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:45:05.177 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-16T22:45:05.177 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T22:45:05.177 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T22:45:05.177 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:05.515 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:03 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 31/33 2024-02-16T22:45:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 31/33 2024-02-16T22:45:05.592 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:45:05.592 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-16T22:45:05.592 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T22:45:05.592 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T22:45:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:05.689 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:05.690 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================== 2024-02-16T22:45:05.690 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:45:05.690 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================== 2024-02-16T22:45:05.690 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:05.690 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 7.4 M 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:05.691 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================== 2024-02-16T22:45:05.692 INFO:teuthology.orchestra.run.smithi064.stdout:Install 12 Packages 2024-02-16T22:45:05.692 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:05.692 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 37 M 2024-02-16T22:45:05.692 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 209 M 2024-02-16T22:45:05.693 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:05.794 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 32/33 2024-02-16T22:45:05.816 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 32/33 2024-02-16T22:45:05.816 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:45:05.816 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-16T22:45:05.816 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T22:45:05.817 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T22:45:05.817 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:17.2.7-513.gb88fae66.el9.x86_64 33/33 2024-02-16T22:45:06.130 INFO:teuthology.orchestra.run.smithi064.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 222 kB/s | 32 kB 00:00 2024-02-16T22:45:06.164 INFO:teuthology.orchestra.run.smithi064.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 521 kB/s | 17 kB 00:00 2024-02-16T22:45:06.206 INFO:teuthology.orchestra.run.smithi064.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 846 kB/s | 34 kB 00:00 2024-02-16T22:45:06.273 INFO:teuthology.orchestra.run.smithi064.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-02-16T22:45:06.465 INFO:teuthology.orchestra.run.smithi064.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2024-02-16T22:45:06.532 INFO:teuthology.orchestra.run.smithi064.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-02-16T22:45:06.557 INFO:teuthology.orchestra.run.smithi064.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 217 kB 00:00 2024-02-16T22:45:06.674 INFO:teuthology.orchestra.run.smithi064.stdout:(8/12): ceph-mgr-diskprediction-local-17.2.7-51 11 MB/s | 7.4 MB 00:00 2024-02-16T22:45:06.758 INFO:teuthology.orchestra.run.smithi064.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 802 kB 00:00 2024-02-16T22:45:06.916 INFO:teuthology.orchestra.run.smithi064.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-02-16T22:45:06.950 INFO:teuthology.orchestra.run.smithi064.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 711 kB/s | 194 kB 00:00 2024-02-16T22:45:07.033 INFO:teuthology.orchestra.run.smithi064.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 898 kB/s | 245 kB 00:00 2024-02-16T22:45:07.036 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:07.036 INFO:teuthology.orchestra.run.smithi064.stdout:Total 28 MB/s | 37 MB 00:01 2024-02-16T22:45:07.058 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:07.084 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:07.084 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:07.534 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:07.535 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-2:17.2.7-513.gb88fae66.el9.x86_64 33/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:17.2.7-513.gb88fae66.el9.x86_64 1/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 2/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 3/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 4/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 5/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.n 6/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-16T22:45:07.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-16T22:45:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T22:45:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-16T22:45:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-16T22:45:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-16T22:45:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-16T22:45:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-16T22:45:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:08.323 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-16T22:45:08.354 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-16T22:45:08.457 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-16T22:45:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-16T22:45:08.480 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-16T22:45:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:08.482 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:08.491 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-16T22:45:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T22:45:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T22:45:08.784 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-16T22:45:08.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-base 2024-02-16T22:45:09.374 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:51 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:09.513 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-16T22:45:09.538 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:09.539 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:09.539 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:09.608 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install cephadm 2024-02-16T22:45:09.643 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-16T22:45:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:52 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:10.343 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:10.344 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:10.344 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:45:10.344 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:10.344 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:10.344 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 80 k 2024-02-16T22:45:10.345 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:10.345 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:10.345 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:10.345 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-02-16T22:45:10.345 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:10.345 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 80 k 2024-02-16T22:45:10.345 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 364 k 2024-02-16T22:45:10.345 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:10.551 INFO:teuthology.orchestra.run.smithi192.stdout:cephadm-17.2.7-513.gb88fae66.el9.noarch.rpm 390 kB/s | 80 kB 00:00 2024-02-16T22:45:10.552 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:10.552 INFO:teuthology.orchestra.run.smithi192.stdout:Total 387 kB/s | 80 kB 00:00 2024-02-16T22:45:10.552 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:10.557 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:10.557 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:10.559 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:10.559 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:10.641 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:11.010 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-16T22:45:11.048 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T22:45:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T22:45:11.560 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T22:45:11.900 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T22:45:11.901 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:11.901 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:11.901 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:11.901 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:11.901 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:12.102 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-16T22:45:12.107 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-immutable-object-cache 2024-02-16T22:45:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 12/12 2024-02-16T22:45:12.697 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:54 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-02-16T22:45:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-513.gb88fae66.el9 ceph 137 k 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 137 k 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 379 k 2024-02-16T22:45:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:13.090 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-immutable-object-cache-17.2.7-513.gb88fae6 620 kB/s | 137 kB 00:00 2024-02-16T22:45:13.091 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:13.091 INFO:teuthology.orchestra.run.smithi192.stdout:Total 616 kB/s | 137 kB 00:00 2024-02-16T22:45:13.091 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:13.099 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:13.099 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:13.149 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:13.150 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:13.282 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T22:45:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T22:45:13.407 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:45:13.407 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-16T22:45:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:14.423 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T22:45:14.423 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:14.423 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:14.423 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:14.423 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:14.424 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 12/12 2024-02-16T22:45:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 1/12 2024-02-16T22:45:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-16T22:45:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-16T22:45:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-16T22:45:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T22:45:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T22:45:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-16T22:45:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-16T22:45:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-16T22:45:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-16T22:45:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-16T22:45:14.678 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-16T22:45:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-16T22:45:15.136 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-16T22:45:15.136 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-16T22:45:15.136 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:15.136 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:15.262 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:57 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:15.375 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:15.425 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:15.426 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:15.426 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:15.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-rook 2024-02-16T22:45:15.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-dashboard 2024-02-16T22:45:16.053 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:14 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:16.064 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:58 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:16.229 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:16.230 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:16.230 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:45:16.230 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:16.230 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:16.230 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 1.7 M 2024-02-16T22:45:16.230 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 23 k 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 15 k 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout:Install 7 Packages 2024-02-16T22:45:16.231 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 47 k 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-16T22:45:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:Install 13 Packages 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.0 M 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 8.1 M 2024-02-16T22:45:16.234 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:16.235 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 2.0 M 2024-02-16T22:45:16.235 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 26 M 2024-02-16T22:45:16.235 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:16.556 INFO:teuthology.orchestra.run.smithi064.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 199 kB/s | 24 kB 00:00 2024-02-16T22:45:16.573 INFO:teuthology.orchestra.run.smithi064.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 129 kB/s | 18 kB 00:00 2024-02-16T22:45:16.585 INFO:teuthology.orchestra.run.smithi192.stdout:(1/7): ceph-prometheus-alerts-17.2.7-513.gb88fa 77 kB/s | 15 kB 00:00 2024-02-16T22:45:16.623 INFO:teuthology.orchestra.run.smithi064.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-02-16T22:45:16.648 INFO:teuthology.orchestra.run.smithi064.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-16T22:45:16.674 INFO:teuthology.orchestra.run.smithi064.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-16T22:45:16.690 INFO:teuthology.orchestra.run.smithi064.stdout:(6/13): ceph-mgr-rook-17.2.7-513.gb88fae66.el9. 184 kB/s | 47 kB 00:00 2024-02-16T22:45:16.694 INFO:teuthology.orchestra.run.smithi192.stdout:(2/7): ceph-grafana-dashboards-17.2.7-513.gb88f 79 kB/s | 23 kB 00:00 2024-02-16T22:45:16.707 INFO:teuthology.orchestra.run.smithi064.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 730 kB/s | 43 kB 00:00 2024-02-16T22:45:16.744 INFO:teuthology.orchestra.run.smithi192.stdout:(3/7): ceph-mgr-dashboard-17.2.7-513.gb88fae66. 4.9 MB/s | 1.7 MB 00:00 2024-02-16T22:45:16.795 INFO:teuthology.orchestra.run.smithi192.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 43 kB/s | 9.0 kB 00:00 2024-02-16T22:45:16.895 INFO:teuthology.orchestra.run.smithi192.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 204 kB/s | 41 kB 00:00 2024-02-16T22:45:16.916 INFO:teuthology.orchestra.run.smithi064.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 72 kB/s | 15 kB 00:00 2024-02-16T22:45:16.920 INFO:teuthology.orchestra.run.smithi192.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 203 kB/s | 31 kB 00:00 2024-02-16T22:45:16.946 INFO:teuthology.orchestra.run.smithi192.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-02-16T22:45:16.950 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:16.950 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.8 MB/s | 2.0 MB 00:00 2024-02-16T22:45:16.950 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:16.957 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:16.957 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:17.017 INFO:teuthology.orchestra.run.smithi064.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 94 kB/s | 32 kB 00:00 2024-02-16T22:45:17.021 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:17.021 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:17.133 INFO:teuthology.orchestra.run.smithi064.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 476 kB/s | 203 kB 00:00 2024-02-16T22:45:17.175 INFO:teuthology.orchestra.run.smithi064.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 372 kB/s | 59 kB 00:00 2024-02-16T22:45:17.217 INFO:teuthology.orchestra.run.smithi064.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-02-16T22:45:17.309 INFO:teuthology.orchestra.run.smithi064.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.6 MB/s | 1.0 MB 00:00 2024-02-16T22:45:17.314 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:17.314 INFO:teuthology.orchestra.run.smithi064.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2024-02-16T22:45:17.314 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:17.319 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:17.327 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:17.327 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:17.388 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-16T22:45:17.422 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-16T22:45:17.432 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:17.432 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:17.469 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-16T22:45:17.501 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-16T22:45:17.529 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.no 5/7 2024-02-16T22:45:17.758 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.n 6/7 2024-02-16T22:45:17.791 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 7/7 2024-02-16T22:45:17.983 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-16T22:45:18.228 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-16T22:45:18.284 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-16T22:45:18.320 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-16T22:45:18.344 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 7/7 2024-02-16T22:45:18.344 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.n 1/7 2024-02-16T22:45:18.344 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 2/7 2024-02-16T22:45:18.345 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.no 3/7 2024-02-16T22:45:18.345 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-16T22:45:18.345 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-16T22:45:18.345 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-16T22:45:18.362 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-16T22:45:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-16T22:45:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-16T22:45:18.622 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:18.672 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:18.895 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-16T22:45:19.379 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-16T22:45:19.435 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-16T22:45:19.469 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-16T22:45:19.469 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:01 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:19.515 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-16T22:45:19.539 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 13/13 2024-02-16T22:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================== 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================== 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 7.4 M 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-16T22:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================== 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-02-16T22:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 37 M 2024-02-16T22:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 209 M 2024-02-16T22:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:19.927 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 220 kB/s | 32 kB 00:00 2024-02-16T22:45:19.952 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 729 kB/s | 17 kB 00:00 2024-02-16T22:45:19.994 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 828 kB/s | 34 kB 00:00 2024-02-16T22:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-02-16T22:45:20.278 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 13/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 1/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-16T22:45:20.302 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-16T22:45:20.345 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-02-16T22:45:20.370 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-02-16T22:45:20.629 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 802 kB 00:00 2024-02-16T22:45:20.707 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-16T22:45:20.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-16T22:45:20.709 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-16T22:45:20.709 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-16T22:45:20.709 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-16T22:45:20.709 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:20.709 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:20.788 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-02-16T22:45:20.813 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 194 kB 00:00 2024-02-16T22:45:20.955 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-02-16T22:45:20.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-cephadm 2024-02-16T22:45:21.541 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:19 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:21.713 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:21.714 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-02-16T22:45:21.714 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:45:21.714 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-02-16T22:45:21.714 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:21.714 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 121 k 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-02-16T22:45:21.715 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:21.716 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 6.8 M 2024-02-16T22:45:21.716 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 33 M 2024-02-16T22:45:21.716 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:21.764 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-5 3.7 MB/s | 7.4 MB 00:01 2024-02-16T22:45:21.767 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:21.767 INFO:teuthology.orchestra.run.smithi192.stdout:Total 17 MB/s | 37 MB 00:02 2024-02-16T22:45:21.791 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:21.818 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:21.818 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:22.137 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): ceph-mgr-cephadm-17.2.7-513.gb88fae66.el 473 kB/s | 121 kB 00:00 2024-02-16T22:45:22.162 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 806 kB/s | 227 kB 00:00 2024-02-16T22:45:22.270 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:22.271 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:22.621 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.8 MB/s | 5.8 MB 00:00 2024-02-16T22:45:22.688 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 110 kB/s | 58 kB 00:00 2024-02-16T22:45:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-16T22:45:23.067 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-16T22:45:23.138 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 99 kB/s | 51 kB 00:00 2024-02-16T22:45:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-16T22:45:23.204 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-16T22:45:23.382 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T22:45:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T22:45:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-16T22:45:24.150 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-16T22:45:24.264 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-16T22:45:24.788 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 207 kB/s | 548 kB 00:02 2024-02-16T22:45:24.790 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:24.790 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.2 MB/s | 6.8 MB 00:03 2024-02-16T22:45:24.790 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:24.801 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:24.801 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:24.899 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:24.900 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:25.205 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:25.376 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-16T22:45:25.423 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-16T22:45:25.511 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-16T22:45:25.974 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-16T22:45:26.053 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-16T22:45:26.115 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-16T22:45:26.143 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 6/6 2024-02-16T22:45:26.540 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-16T22:45:26.568 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 12/12 2024-02-16T22:45:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 6/6 2024-02-16T22:45:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/6 2024-02-16T22:45:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-16T22:45:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-16T22:45:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-16T22:45:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-16T22:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-16T22:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-16T22:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-16T22:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-16T22:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-16T22:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-16T22:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:27.568 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-fuse 2024-02-16T22:45:28.156 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:26 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:28.328 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:28.329 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-16T22:45:28.329 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:28.329 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 812 k 2024-02-16T22:45:28.329 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-16T22:45:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-16T22:45:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:28.330 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:28.330 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:28.330 INFO:teuthology.orchestra.run.smithi064.stdout:Install 2 Packages 2024-02-16T22:45:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:28.330 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 890 k 2024-02-16T22:45:28.330 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 2.6 M 2024-02-16T22:45:28.330 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:28.464 INFO:teuthology.orchestra.run.smithi064.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 588 kB/s | 79 kB 00:00 2024-02-16T22:45:28.664 INFO:teuthology.orchestra.run.smithi064.stdout:(2/2): ceph-fuse-17.2.7-513.gb88fae66.el9.x86_6 2.4 MB/s | 812 kB 00:00 2024-02-16T22:45:28.666 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:28.666 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.6 MB/s | 890 kB 00:00 2024-02-16T22:45:28.666 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:28.687 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:28.687 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:28.766 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:28.766 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:28.951 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:29.081 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 12/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 1/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-16T22:45:29.150 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-16T22:45:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-16T22:45:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-16T22:45:29.158 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2/2 2024-02-16T22:45:29.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-16T22:45:29.593 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:29.593 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:29.593 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:29.593 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-16T22:45:29.593 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-16T22:45:29.593 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-16T22:45:29.593 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:29.594 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:29.896 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2/2 2024-02-16T22:45:29.896 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/2 2024-02-16T22:45:29.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-rook 2024-02-16T22:45:30.199 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-16T22:45:30.199 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:30.199 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:30.199 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-16T22:45:30.200 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:30.200 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:30.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados-devel 2024-02-16T22:45:30.548 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:12 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:30.726 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:30.727 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-02-16T22:45:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:45:30.727 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-02-16T22:45:30.727 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 47 k 2024-02-16T22:45:30.727 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:45:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-16T22:45:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-02-16T22:45:30.728 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2024-02-16T22:45:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:30.729 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.0 M 2024-02-16T22:45:30.729 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 26 M 2024-02-16T22:45:30.729 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:30.902 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 142 kB/s | 18 kB 00:00 2024-02-16T22:45:30.927 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 160 kB/s | 24 kB 00:00 2024-02-16T22:45:30.977 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-02-16T22:45:31.002 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): ceph-mgr-rook-17.2.7-513.gb88fae66.el9. 208 kB/s | 47 kB 00:00 2024-02-16T22:45:31.028 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2024-02-16T22:45:31.059 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:29 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:31.061 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 211 kB 00:00 2024-02-16T22:45:31.087 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 731 kB/s | 43 kB 00:00 2024-02-16T22:45:31.236 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:31.236 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:31.236 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:17.2.7-513.gb88fae66.el9 ceph 126 k 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:31.237 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 126 k 2024-02-16T22:45:31.238 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 448 k 2024-02-16T22:45:31.238 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:31.329 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 107 kB/s | 32 kB 00:00 2024-02-16T22:45:31.354 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 51 kB/s | 15 kB 00:00 2024-02-16T22:45:31.421 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 607 kB/s | 203 kB 00:00 2024-02-16T22:45:31.469 INFO:teuthology.orchestra.run.smithi064.stdout:librados-devel-17.2.7-513.gb88fae66.el9.x86_64. 547 kB/s | 126 kB 00:00 2024-02-16T22:45:31.469 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:31.469 INFO:teuthology.orchestra.run.smithi064.stdout:Total 544 kB/s | 126 kB 00:00 2024-02-16T22:45:31.470 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:31.475 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:31.476 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:31.513 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 372 kB/s | 59 kB 00:00 2024-02-16T22:45:31.532 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:31.533 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:31.547 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 715 kB/s | 90 kB 00:00 2024-02-16T22:45:31.672 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.0 MB/s | 1.0 MB 00:00 2024-02-16T22:45:31.673 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:31.674 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.2 MB/s | 2.0 MB 00:00 2024-02-16T22:45:31.674 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:31.687 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:31.687 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:31.785 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:31.787 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:31.797 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:32.309 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:32.469 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-16T22:45:32.522 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-16T22:45:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:32.553 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:32.553 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-16T22:45:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-16T22:45:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-16T22:45:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-16T22:45:32.799 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs2 2024-02-16T22:45:32.866 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-16T22:45:32.925 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-16T22:45:33.386 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:31 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:33.504 INFO:teuthology.orchestra.run.smithi064.stdout:Package libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:33.560 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:33.561 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:45:33.561 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:33.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs-devel 2024-02-16T22:45:33.635 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-16T22:45:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-16T22:45:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-16T22:45:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-16T22:45:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 13/13 2024-02-16T22:45:34.221 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:32 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:34.395 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:17.2.7-513.gb88fae66.el9 ceph 25 k 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 25 k 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 110 k 2024-02-16T22:45:34.396 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:34.564 INFO:teuthology.orchestra.run.smithi064.stdout:libcephfs-devel-17.2.7-513.gb88fae66.el9.x86_64 148 kB/s | 25 kB 00:00 2024-02-16T22:45:34.565 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:34.565 INFO:teuthology.orchestra.run.smithi064.stdout:Total 147 kB/s | 25 kB 00:00 2024-02-16T22:45:34.565 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:34.569 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:34.569 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:34.570 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:34.570 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:34.576 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 13/13 2024-02-16T22:45:34.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 1/13 2024-02-16T22:45:34.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-16T22:45:34.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-16T22:45:34.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-16T22:45:34.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-16T22:45:34.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-16T22:45:34.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-16T22:45:34.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-16T22:45:34.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-16T22:45:34.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-16T22:45:34.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-16T22:45:34.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-16T22:45:34.653 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:34.764 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-16T22:45:34.962 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:34.963 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:35.168 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:35.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-cephadm 2024-02-16T22:45:35.427 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:35.427 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:35.427 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:35.427 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:35.427 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:35.427 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:35.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados2 2024-02-16T22:45:35.842 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:17 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:36.008 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 121 k 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:36.009 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:36.010 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-02-16T22:45:36.010 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-02-16T22:45:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:36.010 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 6.8 M 2024-02-16T22:45:36.010 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 33 M 2024-02-16T22:45:36.010 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:36.172 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:34 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:36.233 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 227 kB 00:00 2024-02-16T22:45:36.289 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:36.308 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): ceph-mgr-cephadm-17.2.7-513.gb88fae66.el 476 kB/s | 121 kB 00:00 2024-02-16T22:45:36.344 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:36.345 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:45:36.345 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:36.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librbd1 2024-02-16T22:45:36.417 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-02-16T22:45:36.692 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 161 kB/s | 58 kB 00:00 2024-02-16T22:45:36.734 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 174 kB/s | 51 kB 00:00 2024-02-16T22:45:36.759 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-02-16T22:45:36.763 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:36.763 INFO:teuthology.orchestra.run.smithi192.stdout:Total 9.0 MB/s | 6.8 MB 00:00 2024-02-16T22:45:36.764 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:36.772 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:36.772 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:36.861 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:36.862 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:36.993 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:34 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:37.112 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:37.167 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:37.168 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:45:37.168 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:37.207 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:37.237 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rados 2024-02-16T22:45:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-16T22:45:37.436 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-16T22:45:37.821 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:35 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:37.940 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:37.952 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-16T22:45:37.996 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:37.997 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:45:37.997 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:38.029 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-16T22:45:38.064 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rgw 2024-02-16T22:45:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-16T22:45:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 6/6 2024-02-16T22:45:38.646 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:36 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:38.763 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:38.818 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:38.819 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:45:38.819 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:38.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-cephfs 2024-02-16T22:45:38.918 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 6/6 2024-02-16T22:45:38.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/6 2024-02-16T22:45:38.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-16T22:45:38.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-16T22:45:38.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-16T22:45:38.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-16T22:45:39.244 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:39.245 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:39.470 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:37 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:39.497 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-fuse 2024-02-16T22:45:39.587 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:39.641 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:39.642 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:45:39.643 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:39.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rbd 2024-02-16T22:45:40.077 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:22 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:40.244 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:40.245 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:40.246 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-02-16T22:45:40.246 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:40.246 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:40.246 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 812 k 2024-02-16T22:45:40.246 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-02-16T22:45:40.246 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-16T22:45:40.246 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:40.247 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:40.247 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:40.247 INFO:teuthology.orchestra.run.smithi192.stdout:Install 2 Packages 2024-02-16T22:45:40.247 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:40.247 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 890 k 2024-02-16T22:45:40.247 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 2.6 M 2024-02-16T22:45:40.247 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:40.287 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:38 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:40.402 INFO:teuthology.orchestra.run.smithi192.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 503 kB/s | 79 kB 00:00 2024-02-16T22:45:40.405 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:40.459 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:40.461 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-16T22:45:40.461 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:40.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-fuse 2024-02-16T22:45:40.778 INFO:teuthology.orchestra.run.smithi192.stdout:(2/2): ceph-fuse-17.2.7-513.gb88fae66.el9.x86_6 1.5 MB/s | 812 kB 00:00 2024-02-16T22:45:40.779 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:40.779 INFO:teuthology.orchestra.run.smithi192.stdout:Total 1.6 MB/s | 890 kB 00:00 2024-02-16T22:45:40.779 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:40.799 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:40.799 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:40.874 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:40.875 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:41.064 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:41.105 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:39 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:41.191 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-16T22:45:41.252 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2/2 2024-02-16T22:45:41.279 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:41.279 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-16T22:45:41.280 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:41.280 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 81 k 2024-02-16T22:45:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:41.280 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:41.280 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:41.280 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-16T22:45:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:41.281 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 81 k 2024-02-16T22:45:41.281 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 222 k 2024-02-16T22:45:41.281 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:41.642 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-fuse-17.2.7-513.gb88fae66.el9.x86_64.rpm 225 kB/s | 81 kB 00:00 2024-02-16T22:45:41.643 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:41.643 INFO:teuthology.orchestra.run.smithi064.stdout:Total 224 kB/s | 81 kB 00:00 2024-02-16T22:45:41.643 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:41.650 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:41.651 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:41.708 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:41.708 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2/2 2024-02-16T22:45:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/2 2024-02-16T22:45:41.960 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:42.261 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-16T22:45:42.261 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:42.261 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:42.261 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-16T22:45:42.261 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:42.261 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:42.360 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:42.536 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados-devel 2024-02-16T22:45:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:42.684 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:42.684 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:42.685 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:42.685 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:42.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-mirror 2024-02-16T22:45:43.131 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:25 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:43.299 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:17.2.7-513.gb88fae66.el9 ceph 126 k 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-02-16T22:45:43.300 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:43.301 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 126 k 2024-02-16T22:45:43.301 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 448 k 2024-02-16T22:45:43.301 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:43.461 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:41 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:43.636 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:43.637 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:43.637 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-16T22:45:43.637 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:43.637 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.9 M 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 2.9 M 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 12 M 2024-02-16T22:45:43.638 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:43.673 INFO:teuthology.orchestra.run.smithi192.stdout:librados-devel-17.2.7-513.gb88fae66.el9.x86_64. 339 kB/s | 126 kB 00:00 2024-02-16T22:45:43.674 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:43.674 INFO:teuthology.orchestra.run.smithi192.stdout:Total 337 kB/s | 126 kB 00:00 2024-02-16T22:45:43.674 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:43.680 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:43.680 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:43.735 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:43.735 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:43.884 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:43.988 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:44.346 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-mirror-17.2.7-513.gb88fae66.el9.x86_64.rpm 4.2 MB/s | 2.9 MB 00:00 2024-02-16T22:45:44.346 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:44.347 INFO:teuthology.orchestra.run.smithi064.stdout:Total 4.1 MB/s | 2.9 MB 00:00 2024-02-16T22:45:44.347 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:44.358 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:44.358 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:44.436 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:44.437 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:44.474 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:44.812 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:44.812 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:44.863 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:44.945 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:45:44.945 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-16T22:45:44.945 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T22:45:44.945 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T22:45:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:45.085 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs2 2024-02-16T22:45:45.674 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:27 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:45.792 INFO:teuthology.orchestra.run.smithi192.stdout:Package libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:45.844 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:45.845 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:45.845 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:45.919 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs-devel 2024-02-16T22:45:46.173 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:46.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:46.174 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:46.174 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:46.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:46.174 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:46.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-nbd 2024-02-16T22:45:46.503 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:28 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:46.667 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:17.2.7-513.gb88fae66.el9 ceph 25 k 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-02-16T22:45:46.668 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:46.669 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 25 k 2024-02-16T22:45:46.669 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 110 k 2024-02-16T22:45:46.669 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:46.835 INFO:teuthology.orchestra.run.smithi192.stdout:libcephfs-devel-17.2.7-513.gb88fae66.el9.x86_64 150 kB/s | 25 kB 00:00 2024-02-16T22:45:46.836 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:46.836 INFO:teuthology.orchestra.run.smithi192.stdout:Total 149 kB/s | 25 kB 00:00 2024-02-16T22:45:46.836 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:46.839 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:46.839 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:46.841 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:46.841 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:47.015 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:45 ago on Fri 16 Feb 2024 10:44:02 PM UTC. 2024-02-16T22:45:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:47.186 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 165 k 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-16T22:45:47.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:47.188 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 165 k 2024-02-16T22:45:47.188 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 477 k 2024-02-16T22:45:47.188 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-16T22:45:47.400 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:47.574 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-nbd-17.2.7-513.gb88fae66.el9.x86_64.rpm 426 kB/s | 165 kB 00:00 2024-02-16T22:45:47.575 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:47.575 INFO:teuthology.orchestra.run.smithi064.stdout:Total 425 kB/s | 165 kB 00:00 2024-02-16T22:45:47.575 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-16T22:45:47.583 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-16T22:45:47.584 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-16T22:45:47.641 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-16T22:45:47.642 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-16T22:45:47.666 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:47.666 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:47.666 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:47.666 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:47.666 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:47.666 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:47.789 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-16T22:45:47.857 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados2 2024-02-16T22:45:47.900 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:48.387 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:48.434 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:30 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:48.550 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:48.603 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:48.604 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:48.605 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:48.674 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librbd1 2024-02-16T22:45:48.701 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:48.701 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:48.702 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-16T22:45:48.702 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:48.702 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:45:48.702 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-16T22:45:48.914 DEBUG:teuthology.parallel:result is None 2024-02-16T22:45:49.253 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:31 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:49.367 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:49.417 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:49.418 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:49.418 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:49.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rados 2024-02-16T22:45:50.065 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:32 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:50.179 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:50.230 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:50.231 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:50.232 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:50.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rgw 2024-02-16T22:45:50.878 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:32 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:50.993 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:51.047 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:51.048 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:51.048 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:51.116 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-cephfs 2024-02-16T22:45:51.695 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:33 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:51.809 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:51.860 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:51.861 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:51.861 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:51.930 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rbd 2024-02-16T22:45:52.504 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:34 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:52.619 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T22:45:52.671 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:52.672 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-02-16T22:45:52.672 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:52.744 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-fuse 2024-02-16T22:45:53.324 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:35 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:53.491 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:53.491 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:53.491 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-02-16T22:45:53.491 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 81 k 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 81 k 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 222 k 2024-02-16T22:45:53.492 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:53.696 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-fuse-17.2.7-513.gb88fae66.el9.x86_64.rpm 398 kB/s | 81 kB 00:00 2024-02-16T22:45:53.697 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:53.697 INFO:teuthology.orchestra.run.smithi192.stdout:Total 395 kB/s | 81 kB 00:00 2024-02-16T22:45:53.697 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:53.704 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:53.704 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:53.758 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:53.758 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:53.895 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:54.002 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:54.431 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:54.717 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:54.717 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:54.717 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:54.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-mirror 2024-02-16T22:45:55.557 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:37 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:55.729 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:55.730 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:55.730 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-02-16T22:45:55.730 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:55.730 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:55.730 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.9 M 2024-02-16T22:45:55.730 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-02-16T22:45:55.731 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.9 M 2024-02-16T22:45:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 12 M 2024-02-16T22:45:55.731 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:56.333 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-mirror-17.2.7-513.gb88fae66.el9.x86_64.rpm 4.9 MB/s | 2.9 MB 00:00 2024-02-16T22:45:56.334 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:56.334 INFO:teuthology.orchestra.run.smithi192.stdout:Total 4.9 MB/s | 2.9 MB 00:00 2024-02-16T22:45:56.334 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:56.343 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:56.343 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:56.415 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:56.415 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:56.884 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:56.907 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:56.907 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:45:56.907 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-16T22:45:56.908 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T22:45:56.908 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T22:45:56.908 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:45:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:58.042 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:45:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:45:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:58.042 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:45:58.280 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-nbd 2024-02-16T22:45:58.861 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:40 ago on Fri 16 Feb 2024 10:44:18 PM UTC. 2024-02-16T22:45:59.024 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 165 k 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-02-16T22:45:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:45:59.026 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 165 k 2024-02-16T22:45:59.026 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 477 k 2024-02-16T22:45:59.026 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-02-16T22:45:59.401 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-nbd-17.2.7-513.gb88fae66.el9.x86_64.rpm 439 kB/s | 165 kB 00:00 2024-02-16T22:45:59.402 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:45:59.402 INFO:teuthology.orchestra.run.smithi192.stdout:Total 437 kB/s | 165 kB 00:00 2024-02-16T22:45:59.402 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-02-16T22:45:59.410 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-02-16T22:45:59.410 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-02-16T22:45:59.466 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-02-16T22:45:59.466 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-02-16T22:45:59.608 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-02-16T22:45:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:46:00.186 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:46:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T22:46:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:46:00.524 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-02-16T22:46:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T22:46:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:46:00.525 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-02-16T22:46:00.765 DEBUG:teuthology.parallel:result is None 2024-02-16T22:46:00.766 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T22:46:00.766 INFO:teuthology.packaging:ref: None 2024-02-16T22:46:00.766 INFO:teuthology.packaging:tag: None 2024-02-16T22:46:00.766 INFO:teuthology.packaging:branch: quincy 2024-02-16T22:46:00.767 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:46:00.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-02-16T22:46:01.071 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-16T22:46:01.096 INFO:teuthology.orchestra.run.smithi064.stdout:17.2.7-513.gb88fae66.el9 2024-02-16T22:46:01.096 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-513.gb88fae66.el9 2024-02-16T22:46:01.096 INFO:teuthology.task.install:The correct ceph version 17.2.7-513.gb88fae66 is installed. 2024-02-16T22:46:01.098 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T22:46:01.098 INFO:teuthology.packaging:ref: None 2024-02-16T22:46:01.098 INFO:teuthology.packaging:tag: None 2024-02-16T22:46:01.098 INFO:teuthology.packaging:branch: quincy 2024-02-16T22:46:01.098 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:46:01.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-02-16T22:46:01.410 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-16T22:46:01.432 INFO:teuthology.orchestra.run.smithi192.stdout:17.2.7-513.gb88fae66.el9 2024-02-16T22:46:01.432 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-513.gb88fae66.el9 2024-02-16T22:46:01.432 INFO:teuthology.task.install:The correct ceph version 17.2.7-513.gb88fae66 is installed. 2024-02-16T22:46:01.434 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-16T22:46:01.434 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:46:01.434 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-16T22:46:01.467 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:46:01.467 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-16T22:46:01.500 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-16T22:46:01.501 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:46:01.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-02-16T22:46:01.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-16T22:46:01.606 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:46:01.606 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-02-16T22:46:01.633 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-16T22:46:01.700 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-16T22:46:01.700 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:46:01.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-16T22:46:01.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-16T22:46:01.797 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:46:01.798 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-16T22:46:01.824 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-16T22:46:01.890 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-16T22:46:01.890 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:46:01.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-02-16T22:46:01.918 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-16T22:46:01.986 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:46:01.987 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-02-16T22:46:02.013 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-16T22:46:02.080 INFO:teuthology.run_tasks:Running task print... 2024-02-16T22:46:02.089 INFO:teuthology.task.print:**** done install task... 2024-02-16T22:46:02.089 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-16T22:46:02.190 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-16T22:46:02.190 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-02-16T22:46:02.191 INFO:tasks.cephadm:Cluster fsid is 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:46:02.191 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-16T22:46:02.191 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-16T22:46:02.191 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi064': '172.21.15.64', 'mon.smithi192': '172.21.15.192'} 2024-02-16T22:46:02.191 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-16T22:46:02.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-02-16T22:46:02.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-02-16T22:46:02.247 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-02-16T22:46:02.247 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-16T22:46:02.474 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 367151 Feb 16 22:46 /home/ubuntu/cephtest/cephadm 2024-02-16T22:46:02.475 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-16T22:46:02.696 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-r--r--. 1 ubuntu ubuntu 367151 Feb 16 22:46 /home/ubuntu/cephtest/cephadm 2024-02-16T22:46:02.696 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-16T22:46:02.713 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-16T22:46:02.738 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-02-16T22:46:02.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-02-16T22:46:02.756 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-02-16T22:46:03.664 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-16T22:46:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-16T22:46:32.350 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:46:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)", 2024-02-16T22:46:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0", 2024-02-16T22:46:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-02-16T22:46:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c" 2024-02-16T22:46:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-16T22:46:32.351 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:46:36.620 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-02-16T22:46:36.620 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)", 2024-02-16T22:46:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0", 2024-02-16T22:46:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-02-16T22:46:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c" 2024-02-16T22:46:36.621 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-02-16T22:46:36.621 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-02-16T22:46:36.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-02-16T22:46:36.672 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-02-16T22:46:36.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-02-16T22:46:36.742 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-02-16T22:46:36.768 INFO:tasks.cephadm:Writing seed config... 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-16T22:46:36.770 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-16T22:46:36.771 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-16T22:46:36.772 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-16T22:46:36.773 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:46:36.773 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-16T22:46:36.798 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2912dc76-cd1d-11ee-95ba-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-16T22:46:36.798 DEBUG:teuthology.orchestra.run.smithi064:mon.smithi064> sudo journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service 2024-02-16T22:46:36.840 INFO:tasks.cephadm:Bootstrapping... 2024-02-16T22:46:36.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:46:37.052 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:46:37.053 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '2912dc76-cd1d-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2024-02-16T22:46:37.077 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T22:46:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-16T22:46:37.077 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying podman|docker is present... 2024-02-16T22:46:37.100 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T22:46:37.101 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying lvm2 is present... 2024-02-16T22:46:37.101 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying time synchronization is in place... 2024-02-16T22:46:37.109 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-16T22:46:37.109 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-16T22:46:37.115 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-16T22:46:37.115 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-02-16T22:46:37.122 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-02-16T22:46:37.128 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-02-16T22:46:37.128 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-02-16T22:46:37.128 INFO:teuthology.orchestra.run.smithi064.stdout:Repeating the final host check... 2024-02-16T22:46:37.151 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T22:46:37.151 INFO:teuthology.orchestra.run.smithi064.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-16T22:46:37.152 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl is present 2024-02-16T22:46:37.152 INFO:teuthology.orchestra.run.smithi064.stdout:lvcreate is present 2024-02-16T22:46:37.158 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-16T22:46:37.158 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-16T22:46:37.164 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-16T22:46:37.164 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-02-16T22:46:37.171 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-02-16T22:46:37.177 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-02-16T22:46:37.177 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-02-16T22:46:37.177 INFO:teuthology.orchestra.run.smithi064.stdout:Host looks OK 2024-02-16T22:46:37.177 INFO:teuthology.orchestra.run.smithi064.stdout:Cluster fsid: 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:46:37.177 INFO:teuthology.orchestra.run.smithi064.stdout:Acquiring lock 140201206662816 on /run/cephadm/2912dc76-cd1d-11ee-95ba-87774f69a715.lock 2024-02-16T22:46:37.177 INFO:teuthology.orchestra.run.smithi064.stdout:Lock 140201206662816 acquired on /run/cephadm/2912dc76-cd1d-11ee-95ba-87774f69a715.lock 2024-02-16T22:46:37.178 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 3300 ... 2024-02-16T22:46:37.178 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 6789 ... 2024-02-16T22:46:37.178 INFO:teuthology.orchestra.run.smithi064.stdout:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-02-16T22:46:37.182 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.64 metric 100 2024-02-16T22:46:37.182 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 metric 100 2024-02-16T22:46:37.185 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-16T22:46:37.185 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-16T22:46:37.185 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1604sec hoplimit 64 pref medium 2024-02-16T22:46:37.187 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-16T22:46:37.187 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-16T22:46:37.188 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-16T22:46:37.188 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-16T22:46:37.188 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-02-16T22:46:37.188 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-16T22:46:37.188 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-02-16T22:46:37.188 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-02-16T22:46:37.189 INFO:teuthology.orchestra.run.smithi064.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-16T22:46:37.189 INFO:teuthology.orchestra.run.smithi064.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-16T22:46:37.190 INFO:teuthology.orchestra.run.smithi064.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-16T22:46:37.690 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0 2024-02-16T22:46:37.690 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-16T22:46:37.691 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Getting image source signatures 2024-02-16T22:46:37.691 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:5dfd87a714924b39df09034baf76a9bd7605cb8f0d34cc8ca301fad8be327611 2024-02-16T22:46:37.691 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-16T22:46:37.691 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying config sha256:870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0 2024-02-16T22:46:37.691 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-16T22:46:38.712 INFO:teuthology.orchestra.run.smithi064.stdout:ceph: stdout ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable) 2024-02-16T22:46:38.712 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph version: ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable) 2024-02-16T22:46:38.712 INFO:teuthology.orchestra.run.smithi064.stdout:Extracting ceph user uid/gid from container image... 2024-02-16T22:46:41.392 INFO:teuthology.orchestra.run.smithi064.stdout:stat: stdout 167 167 2024-02-16T22:46:41.392 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial keys... 2024-02-16T22:46:50.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDT5c9lFxi4OhAACCMugUmGKgOXzz8ICBKmkQ== 2024-02-16T22:46:51.868 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDb5c9l9VoTChAAAsHRarns8Sv6Yu73wpvAUQ== 2024-02-16T22:46:52.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDc5c9luTXlCRAAhzfmILxcikdxNWKw07OQdA== 2024-02-16T22:46:52.760 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial monmap... 2024-02-16T22:46:53.672 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-16T22:46:53.672 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-02-16T22:46:53.672 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:46:53.672 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-16T22:46:53.672 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool for smithi064 [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-16T22:46:53.672 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = octopus 2024-02-16T22:46:53.672 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: set fsid to 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:46:53.673 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-16T22:46:53.673 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:46:53.673 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mon... 2024-02-16T22:46:54.618 INFO:teuthology.orchestra.run.smithi064.stdout:create mon.smithi064 on 2024-02-16T22:46:54.821 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-16T22:46:54.963 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-16T22:46:55.186 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715.target. 2024-02-16T22:46:55.187 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715.target. 2024-02-16T22:46:55.467 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064 2024-02-16T22:46:55.468 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service: Unit ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service not loaded. 2024-02-16T22:46:55.677 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715.target.wants/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service → /etc/systemd/system/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@.service. 2024-02-16T22:46:56.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:55 smithi064 podman[28838]: 2024-02-16 22:46:55.824741762 +0000 UTC m=+0.018507121 image pull 870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0 quay.ceph.io/ceph-ci/ceph:quincy 2024-02-16T22:46:56.717 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:56 smithi064 podman[28838]: 2024-02-16 22:46:56.464290033 +0000 UTC m=+0.658055387 container create 9b9d40fb2fa165c501466fb9b6673113786bf9f6dfc121adeb4f0415a4caead1 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=quincy-b88fae6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-16T22:46:56.848 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T22:46:56.848 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-02-16T22:46:56.855 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T22:46:56.855 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-02-16T22:46:56.855 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-02-16T22:46:56.855 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-02-16T22:46:56.855 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon to start... 2024-02-16T22:46:56.855 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon... 2024-02-16T22:46:56.979 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:56 smithi064 podman[28838]: 2024-02-16 22:46:56.717076459 +0000 UTC m=+0.910841811 container init 9b9d40fb2fa165c501466fb9b6673113786bf9f6dfc121adeb4f0415a4caead1 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=quincy-b88fae6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-16T22:46:56.979 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:56 smithi064 podman[28838]: 2024-02-16 22:46:56.71988393 +0000 UTC m=+0.913649283 container start 9b9d40fb2fa165c501466fb9b6673113786bf9f6dfc121adeb4f0415a4caead1 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, org.label-schema.vendor=CentOS, RELEASE=quincy-b88fae6, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-02-16T22:46:56.980 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:56 smithi064 ceph-mon[28860]: mkfs 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:46:56.980 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:56 smithi064 ceph-mon[28860]: mon.smithi064 is new leader, mons smithi064 in quorum (ranks 0) 2024-02-16T22:46:56.980 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:56 smithi064 bash[28838]: 9b9d40fb2fa165c501466fb9b6673113786bf9f6dfc121adeb4f0415a4caead1 2024-02-16T22:46:56.980 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:56 smithi064 systemd[1]: Started Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T22:46:58.049 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:57 smithi064 ceph-mon[28860]: mon.smithi064 is new leader, mons smithi064 in quorum (ranks 0) 2024-02-16T22:46:58.049 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:57 smithi064 ceph-mon[28860]: monmap e1: 1 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:46:58.050 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:57 smithi064 ceph-mon[28860]: fsmap 2024-02-16T22:46:58.050 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:57 smithi064 ceph-mon[28860]: osdmap e1: 0 total, 0 up, 0 in 2024-02-16T22:46:58.050 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:57 smithi064 ceph-mon[28860]: mgrmap e1: no daemons active 2024-02-16T22:46:58.050 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:57 smithi064 ceph-mon[28860]: from='client.? 172.21.15.64:0/606300396' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-16T22:46:58.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout cluster: 2024-02-16T22:46:58.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout id: 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:46:58.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-16T22:46:58.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:46:58.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout services: 2024-02-16T22:46:58.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi064 (age 0.701378s) 2024-02-16T22:46:58.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout data: 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pgs: 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.399+0000 7f2631b70700 1 Processor -- start 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.399+0000 7f2631b70700 1 -- start start 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.399+0000 7f2631b70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.399+0000 7f2631b70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c1030c0 con 0x7f262c102770 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.400+0000 7f262b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:46:58.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.400+0000 7f262b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37930/0 (socket says 172.21.15.64:37930) 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.400+0000 7f262b7fe700 1 -- 172.21.15.64:0/3297100133 learned_addr learned my addr 172.21.15.64:0/3297100133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.400+0000 7f262b7fe700 1 -- 172.21.15.64:0/3297100133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c103200 con 0x7f262c102770 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.400+0000 7f262b7fe700 1 --2- 172.21.15.64:0/3297100133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f261c009e50 tx=0x7f261c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8de8b255195b7c server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.400+0000 7f262a7fc700 1 -- 172.21.15.64:0/3297100133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f261c0092e0 con 0x7f262c102770 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.400+0000 7f262a7fc700 1 -- 172.21.15.64:0/3297100133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c00ba20 con 0x7f262c102770 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.400+0000 7f262a7fc700 1 -- 172.21.15.64:0/3297100133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f261c00bd30 con 0x7f262c102770 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.401+0000 7f2631b70700 1 -- 172.21.15.64:0/3297100133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 msgr2=0x7f262c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.401+0000 7f2631b70700 1 --2- 172.21.15.64:0/3297100133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f261c009e50 tx=0x7f261c009840 comp rx=0 tx=0).stop 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.401+0000 7f2631b70700 1 -- 172.21.15.64:0/3297100133 shutdown_connections 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.401+0000 7f2631b70700 1 --2- 172.21.15.64:0/3297100133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c102b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.401+0000 7f2631b70700 1 -- 172.21.15.64:0/3297100133 >> 172.21.15.64:0/3297100133 conn(0x7f262c0fded0 msgr2=0x7f262c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.401+0000 7f2631b70700 1 -- 172.21.15.64:0/3297100133 shutdown_connections 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.401+0000 7f2631b70700 1 -- 172.21.15.64:0/3297100133 wait complete. 2024-02-16T22:46:58.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.401+0000 7f2631b70700 1 Processor -- start 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f2631b70700 1 -- start start 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f2631b70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f2631b70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c077190 con 0x7f262c102770 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f262b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f262b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37934/0 (socket says 172.21.15.64:37934) 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f262b7fe700 1 -- 172.21.15.64:0/606300396 learned_addr learned my addr 172.21.15.64:0/606300396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f262b7fe700 1 -- 172.21.15.64:0/606300396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009ae0 con 0x7f262c102770 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f262b7fe700 1 --2- 172.21.15.64:0/606300396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c078d10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f262c1034b0 tx=0x7f261c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f2623fff700 1 -- 172.21.15.64:0/606300396 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f261c012b90 con 0x7f262c102770 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f2623fff700 1 -- 172.21.15.64:0/606300396 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c012d10 con 0x7f262c102770 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.402+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c077390 con 0x7f262c102770 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.403+0000 7f2623fff700 1 -- 172.21.15.64:0/606300396 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f261c01e400 con 0x7f262c102770 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.403+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c077880 con 0x7f262c102770 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.403+0000 7f2623fff700 1 -- 172.21.15.64:0/606300396 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f261c019030 con 0x7f262c102770 2024-02-16T22:46:58.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.403+0000 7f2623fff700 1 -- 172.21.15.64:0/606300396 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f261c01ed00 con 0x7f262c102770 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.403+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c10a6b0 con 0x7f262c102770 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.406+0000 7f2623fff700 1 -- 172.21.15.64:0/606300396 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f261c017070 con 0x7f262c102770 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.478+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f262c10a0d0 con 0x7f262c102770 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.478+0000 7f2623fff700 1 -- 172.21.15.64:0/606300396 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f261c01e6d0 con 0x7f262c102770 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.479+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 msgr2=0x7f262c078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.479+0000 7f2631b70700 1 --2- 172.21.15.64:0/606300396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c078d10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f262c1034b0 tx=0x7f261c017000 comp rx=0 tx=0).stop 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.479+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 shutdown_connections 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.479+0000 7f2631b70700 1 --2- 172.21.15.64:0/606300396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c102770 0x7f262c078d10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.479+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 >> 172.21.15.64:0/606300396 conn(0x7f262c0fded0 msgr2=0x7f262c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.479+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 shutdown_connections 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:57.479+0000 7f2631b70700 1 -- 172.21.15.64:0/606300396 wait complete. 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:mon is available 2024-02-16T22:46:58.062 INFO:teuthology.orchestra.run.smithi064.stdout:Assimilating anything we can from ceph.conf... 2024-02-16T22:46:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:58 smithi064 ceph-mon[28860]: from='client.? 172.21.15.64:0/3755665922' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-16T22:46:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:58 smithi064 ceph-mon[28860]: from='client.? 172.21.15.64:0/3755665922' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-16T22:46:59.165 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-16T22:46:59.166 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.481+0000 7f635c725700 1 Processor -- start 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.481+0000 7f635c725700 1 -- start start 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.482+0000 7f635c725700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.482+0000 7f635c725700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63541030c0 con 0x7f6354102770 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.482+0000 7f635a4c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.482+0000 7f635a4c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37938/0 (socket says 172.21.15.64:37938) 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.482+0000 7f635a4c1700 1 -- 172.21.15.64:0/750237389 learned_addr learned my addr 172.21.15.64:0/750237389 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.482+0000 7f635a4c1700 1 -- 172.21.15.64:0/750237389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354103200 con 0x7f6354102770 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.482+0000 7f635a4c1700 1 --2- 172.21.15.64:0/750237389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354102b80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f63440098e0 tx=0x7f6344009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec3e26b417ef462 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f63594bf700 1 -- 172.21.15.64:0/750237389 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f634400b860 con 0x7f6354102770 2024-02-16T22:46:59.167 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f63594bf700 1 -- 172.21.15.64:0/750237389 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634400b9c0 con 0x7f6354102770 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f63594bf700 1 -- 172.21.15.64:0/750237389 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f634400bc90 con 0x7f6354102770 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f635c725700 1 -- 172.21.15.64:0/750237389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 msgr2=0x7f6354102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f635c725700 1 --2- 172.21.15.64:0/750237389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354102b80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f63440098e0 tx=0x7f6344009250 comp rx=0 tx=0).stop 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f635c725700 1 -- 172.21.15.64:0/750237389 shutdown_connections 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f635c725700 1 --2- 172.21.15.64:0/750237389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354102b80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f635c725700 1 -- 172.21.15.64:0/750237389 >> 172.21.15.64:0/750237389 conn(0x7f63540fded0 msgr2=0x7f6354100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f635c725700 1 -- 172.21.15.64:0/750237389 shutdown_connections 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.483+0000 7f635c725700 1 -- 172.21.15.64:0/750237389 wait complete. 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635c725700 1 Processor -- start 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635c725700 1 -- start start 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635c725700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354110d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635c725700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63541112d0 con 0x7f6354102770 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635a4c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354110d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:46:59.168 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635a4c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354110d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37944/0 (socket says 172.21.15.64:37944) 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635a4c1700 1 -- 172.21.15.64:0/3755665922 learned_addr learned my addr 172.21.15.64:0/3755665922 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635a4c1700 1 -- 172.21.15.64:0/3755665922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63440094f0 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.484+0000 7f635a4c1700 1 --2- 172.21.15.64:0/3755665922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354110d90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6344003080 tx=0x7f6344016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.485+0000 7f634affd700 1 -- 172.21.15.64:0/3755665922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f634401c5c0 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.485+0000 7f634affd700 1 -- 172.21.15.64:0/3755665922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634401c720 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.485+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63541114d0 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.485+0000 7f634affd700 1 -- 172.21.15.64:0/3755665922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f634401ca30 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.485+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63541120c0 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.485+0000 7f634affd700 1 -- 172.21.15.64:0/3755665922 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f634401a450 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.485+0000 7f634affd700 1 -- 172.21.15.64:0/3755665922 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f634402e790 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.486+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635405f5c0 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.488+0000 7f634affd700 1 -- 172.21.15.64:0/3755665922 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f634401a890 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.557+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6354103b90 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.560+0000 7f634affd700 1 -- 172.21.15.64:0/3755665922 <== mon.0 v2:172.21.15.64:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f634401aa30 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.560+0000 7f634affd700 1 -- 172.21.15.64:0/3755665922 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f634401bd60 con 0x7f6354102770 2024-02-16T22:46:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.561+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 msgr2=0x7f6354110d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:46:59.170 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.561+0000 7f635c725700 1 --2- 172.21.15.64:0/3755665922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354110d90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6344003080 tx=0x7f6344016000 comp rx=0 tx=0).stop 2024-02-16T22:46:59.170 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.561+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 shutdown_connections 2024-02-16T22:46:59.170 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.561+0000 7f635c725700 1 --2- 172.21.15.64:0/3755665922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6354102770 0x7f6354110d90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:46:59.170 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.561+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 >> 172.21.15.64:0/3755665922 conn(0x7f63540fded0 msgr2=0x7f63541054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:46:59.170 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.561+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 shutdown_connections 2024-02-16T22:46:59.170 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:58.561+0000 7f635c725700 1 -- 172.21.15.64:0/3755665922 wait complete. 2024-02-16T22:46:59.170 INFO:teuthology.orchestra.run.smithi064.stdout:Generating new minimal ceph.conf... 2024-02-16T22:47:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:46:59 smithi064 ceph-mon[28860]: from='client.? 172.21.15.64:0/2511067157' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:47:00.262 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.537+0000 7fce6a4b2700 1 Processor -- start 2024-02-16T22:47:00.262 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.537+0000 7fce6a4b2700 1 -- start start 2024-02-16T22:47:00.262 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce6a4b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:00.262 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce6a4b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce641030c0 con 0x7fce64102770 2024-02-16T22:47:00.262 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37950/0 (socket says 172.21.15.64:37950) 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce63fff700 1 -- 172.21.15.64:0/293429329 learned_addr learned my addr 172.21.15.64:0/293429329 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce63fff700 1 -- 172.21.15.64:0/293429329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64103200 con 0x7fce64102770 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce63fff700 1 --2- 172.21.15.64:0/293429329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64102b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fce540098e0 tx=0x7fce54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c203158ba90443c server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce62ffd700 1 -- 172.21.15.64:0/293429329 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce5400b860 con 0x7fce64102770 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce62ffd700 1 -- 172.21.15.64:0/293429329 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fce5400be50 con 0x7fce64102770 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.538+0000 7fce62ffd700 1 -- 172.21.15.64:0/293429329 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce54010380 con 0x7fce64102770 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.539+0000 7fce6a4b2700 1 -- 172.21.15.64:0/293429329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 msgr2=0x7fce64102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.539+0000 7fce6a4b2700 1 --2- 172.21.15.64:0/293429329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64102b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fce540098e0 tx=0x7fce54009250 comp rx=0 tx=0).stop 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.539+0000 7fce6a4b2700 1 -- 172.21.15.64:0/293429329 shutdown_connections 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.539+0000 7fce6a4b2700 1 --2- 172.21.15.64:0/293429329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64102b80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.539+0000 7fce6a4b2700 1 -- 172.21.15.64:0/293429329 >> 172.21.15.64:0/293429329 conn(0x7fce640fded0 msgr2=0x7fce64100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:00.263 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.539+0000 7fce6a4b2700 1 -- 172.21.15.64:0/293429329 shutdown_connections 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.539+0000 7fce6a4b2700 1 -- 172.21.15.64:0/293429329 wait complete. 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.539+0000 7fce6a4b2700 1 Processor -- start 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce6a4b2700 1 -- start start 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce6a4b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce6a4b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce64077190 con 0x7fce64102770 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37952/0 (socket says 172.21.15.64:37952) 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce63fff700 1 -- 172.21.15.64:0/2511067157 learned_addr learned my addr 172.21.15.64:0/2511067157 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce63fff700 1 -- 172.21.15.64:0/2511067157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540094f0 con 0x7fce64102770 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce63fff700 1 --2- 172.21.15.64:0/2511067157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64078d10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce5400b440 tx=0x7fce54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.540+0000 7fce60ff9700 1 -- 172.21.15.64:0/2511067157 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce54010760 con 0x7fce64102770 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.541+0000 7fce60ff9700 1 -- 172.21.15.64:0/2511067157 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fce54010d50 con 0x7fce64102770 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.541+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce64077390 con 0x7fce64102770 2024-02-16T22:47:00.264 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.541+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce64077820 con 0x7fce64102770 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.541+0000 7fce60ff9700 1 -- 172.21.15.64:0/2511067157 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce5401ae20 con 0x7fce64102770 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.541+0000 7fce60ff9700 1 -- 172.21.15.64:0/2511067157 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fce5402e6c0 con 0x7fce64102770 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.541+0000 7fce60ff9700 1 -- 172.21.15.64:0/2511067157 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fce54024930 con 0x7fce64102770 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.542+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6404bed0 con 0x7fce64102770 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.544+0000 7fce60ff9700 1 -- 172.21.15.64:0/2511067157 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fce54029070 con 0x7fce64102770 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.613+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fce641045e0 con 0x7fce64102770 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.613+0000 7fce60ff9700 1 -- 172.21.15.64:0/2511067157 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fce5401a6a0 con 0x7fce64102770 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.614+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 msgr2=0x7fce64078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.614+0000 7fce6a4b2700 1 --2- 172.21.15.64:0/2511067157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64078d10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce5400b440 tx=0x7fce54009250 comp rx=0 tx=0).stop 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.614+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 shutdown_connections 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.614+0000 7fce6a4b2700 1 --2- 172.21.15.64:0/2511067157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce64102770 0x7fce64078d10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.614+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 >> 172.21.15.64:0/2511067157 conn(0x7fce640fded0 msgr2=0x7fce640ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.614+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 shutdown_connections 2024-02-16T22:47:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:46:59.614+0000 7fce6a4b2700 1 -- 172.21.15.64:0/2511067157 wait complete. 2024-02-16T22:47:00.266 INFO:teuthology.orchestra.run.smithi064.stdout:Restarting the monitor... 2024-02-16T22:47:00.519 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:00 smithi064 systemd[1]: Stopping Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T22:47:00.842 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:00 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[28856]: 2024-02-16T22:47:00.515+0000 7feed61da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi064 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:47:00.842 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:00 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[28856]: 2024-02-16T22:47:00.515+0000 7feed61da700 -1 mon.smithi064@0(leader) e1 *** Got Signal Terminated *** 2024-02-16T22:47:00.842 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:00 smithi064 podman[29101]: 2024-02-16 22:47:00.59807556 +0000 UTC m=+0.178388209 container died 9b9d40fb2fa165c501466fb9b6673113786bf9f6dfc121adeb4f0415a4caead1 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-b88fae6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-16T22:47:01.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:00 smithi064 podman[29101]: 2024-02-16 22:47:00.932505764 +0000 UTC m=+0.512818413 container cleanup 9b9d40fb2fa165c501466fb9b6673113786bf9f6dfc121adeb4f0415a4caead1 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=quincy-b88fae6, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:47:01.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:00 smithi064 bash[29101]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064 2024-02-16T22:47:01.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:01 smithi064 podman[29115]: 2024-02-16 22:47:01.425503159 +0000 UTC m=+0.824514030 container remove 9b9d40fb2fa165c501466fb9b6673113786bf9f6dfc121adeb4f0415a4caead1 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=quincy-b88fae6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True) 2024-02-16T22:47:01.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:01 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service: Deactivated successfully. 2024-02-16T22:47:01.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:01 smithi064 systemd[1]: Stopped Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T22:47:01.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:01 smithi064 systemd[1]: Starting Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T22:47:02.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:01 smithi064 podman[29207]: 2024-02-16 22:47:01.776098352 +0000 UTC m=+0.018446882 image pull 870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0 quay.ceph.io/ceph-ci/ceph:quincy 2024-02-16T22:47:02.788 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 podman[29207]: 2024-02-16 22:47:02.510490407 +0000 UTC m=+0.752838926 container create ad935b7a6a7f2d4d9e693d448a725c3ff820ce0c6324682bc03a860c8f225ecd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, ceph=True, org.label-schema.vendor=CentOS, RELEASE=quincy-b88fae6, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 podman[29207]: 2024-02-16 22:47:02.704401483 +0000 UTC m=+0.946750008 container init ad935b7a6a7f2d4d9e693d448a725c3ff820ce0c6324682bc03a860c8f225ecd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=quincy-b88fae6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 podman[29207]: 2024-02-16 22:47:02.707938651 +0000 UTC m=+0.950287167 container start ad935b7a6a7f2d4d9e693d448a725c3ff820ce0c6324682bc03a860c8f225ecd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, org.label-schema.build-date=20240214, RELEASE=quincy-b88fae6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable), process ceph-mon, pid 2 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: pidfile_write: ignore empty --pid-file 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: load: jerasure load: lrc 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: RocksDB version: 6.15.5 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Compile date Feb 15 2024 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: DB SUMMARY 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: DB Session ID: 981Y2HN0XEJV5AHNUF3R 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: CURRENT file: CURRENT 2024-02-16T22:47:02.789 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T22:47:02.790 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-02-16T22:47:02.790 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi064/store.db dir, Total Num: 1, files: 000008.sst 2024-02-16T22:47:02.790 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi064/store.db: 000010.log size: 97278 ; 2024-02-16T22:47:02.790 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.error_if_exists: 0 2024-02-16T22:47:02.790 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.create_if_missing: 0 2024-02-16T22:47:02.790 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.paranoid_checks: 1 2024-02-16T22:47:02.790 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T22:47:02.790 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.env: 0x5609dbe42a80 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.fs: Posix File System 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.info_log: 0x5609dd0e9cc0 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.statistics: (nil) 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.use_fsync: 0 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_log_file_size: 0 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.allow_fallocate: 1 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T22:47:02.791 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.use_direct_reads: 0 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.db_log_dir: 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi064/store.db 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T22:47:02.792 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.write_buffer_manager: 0x5609dde5eb40 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.unordered_write: 0 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.row_cache: None 2024-02-16T22:47:02.793 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.wal_filter: None 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.preserve_deletes: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.two_write_queues: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.atomic_flush: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.log_readahead_size: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T22:47:02.794 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_background_jobs: 2 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_background_compactions: -1 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_subcompactions: 1 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_open_files: -1 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T22:47:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_background_flushes: -1 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Compression algorithms supported: 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: kZSTD supported: 0 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: kXpressCompression supported: 0 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: kZlibCompression supported: 1 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: kSnappyCompression supported: 1 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: kLZ4Compression supported: 1 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: kBZip2Compression supported: 0 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi064/store.db/MANIFEST-000009 2024-02-16T22:47:02.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.merge_operator: 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_filter: None 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5609dd13ef58) 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: index_type: 0 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: data_block_index_type: 0 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: index_shortening: 1 2024-02-16T22:47:02.797 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: hash_index_allow_collision: 1 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: checksum: 1 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: no_block_cache: 0 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_cache: 0x5609dd171090 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_cache_options: 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: capacity : 536870912 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: num_shard_bits : 4 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: strict_capacity_limit : 0 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_cache_compressed: (nil) 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: persistent_cache: (nil) 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_size: 4096 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_size_deviation: 10 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_restart_interval: 16 2024-02-16T22:47:02.798 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: index_block_restart_interval: 1 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: metadata_block_size: 4096 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: partition_filters: 0 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: use_delta_encoding: 1 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: whole_key_filtering: 1 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: verify_compression: 0 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: format_version: 4 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: enable_index_compression: 1 2024-02-16T22:47:02.799 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_align: 0 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compression: NoCompression 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.num_levels: 7 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T22:47:02.800 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T22:47:02.801 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T22:47:02.802 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T22:47:02.802 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T22:47:02.802 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T22:47:02.802 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T22:47:02.802 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T22:47:02.802 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T22:47:02.802 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T22:47:02.802 INFO:teuthology.orchestra.run.smithi064.stdout:Setting mon public_network to 172.21.0.0/20 2024-02-16T22:47:02.803 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T22:47:02.803 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T22:47:02.803 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T22:47:02.803 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T22:47:02.803 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.arena_block_size: 4194304 2024-02-16T22:47:02.803 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T22:47:02.803 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.table_properties_collectors: 2024-02-16T22:47:02.804 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.inplace_update_support: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.bloom_locality: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.max_successive_merges: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.ttl: 2592000 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.enable_blob_files: false 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.min_blob_size: 0 2024-02-16T22:47:02.805 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi064/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708123622735637, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708123622736437, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92550, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90645, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83360, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708123622, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "d239cc3d-9382-4b73-a2cc-a577319bcd91", "db_session_id": "981Y2HN0XEJV5AHNUF3R"}} 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708123622736781, "job": 1, "event": "recovery_finished"} 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi064/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5609dd105180 2024-02-16T22:47:02.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: DB pointer 0x5609dd1da000 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** DB Stats ** 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** Compaction Stats [default] ** 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:47:02.807 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: L0 2/0 92.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.5 0.00 0.00 1 0.001 0 0 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Sum 2/0 92.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.5 0.00 0.00 1 0.001 0 0 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.5 0.00 0.00 1 0.001 0 0 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** Compaction Stats [default] ** 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.5 0.00 0.00 1 0.001 0 0 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative compaction: 0.00 GB write, 27.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:47:02.808 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval compaction: 0.00 GB write, 27.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** Compaction Stats [default] ** 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: L0 2/0 92.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.5 0.00 0.00 1 0.001 0 0 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Sum 2/0 92.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.5 0.00 0.00 1 0.001 0 0 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** Compaction Stats [default] ** 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:47:02.809 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.5 0.00 0.00 1 0.001 0 0 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative compaction: 0.00 GB write, 27.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: starting mon.smithi064 rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi064 fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064@-1(???) e1 preinit fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064@-1(???).mds e1 new map 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064@-1(???).mds e1 print_map 2024-02-16T22:47:02.810 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: e1 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: legacy client fscid: -1 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: No filesystems configured 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 bash[29207]: ad935b7a6a7f2d4d9e693d448a725c3ff820ce0c6324682bc03a860c8f225ecd 2024-02-16T22:47:02.811 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 systemd[1]: Started Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T22:47:03.074 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mon.smithi064 is new leader, mons smithi064 in quorum (ranks 0) 2024-02-16T22:47:03.075 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: monmap e1: 1 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:47:03.075 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: fsmap 2024-02-16T22:47:03.075 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: osdmap e1: 0 total, 0 up, 0 in 2024-02-16T22:47:03.075 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:02 smithi064 ceph-mon[29226]: mgrmap e1: no daemons active 2024-02-16T22:47:03.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.175+0000 7f1a47073700 1 Processor -- start 2024-02-16T22:47:03.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.175+0000 7f1a47073700 1 -- start start 2024-02-16T22:47:03.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.175+0000 7f1a47073700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.175+0000 7f1a47073700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401030c0 con 0x7f1a40102770 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.175+0000 7f1a44e0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.175+0000 7f1a44e0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48722/0 (socket says 172.21.15.64:48722) 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.175+0000 7f1a44e0f700 1 -- 172.21.15.64:0/1040879542 learned_addr learned my addr 172.21.15.64:0/1040879542 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.176+0000 7f1a44e0f700 1 -- 172.21.15.64:0/1040879542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a40103200 con 0x7f1a40102770 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.176+0000 7f1a44e0f700 1 --2- 172.21.15.64:0/1040879542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a340098e0 tx=0x7f1a34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1839da589c4ba7 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.176+0000 7f1a3f7fe700 1 -- 172.21.15.64:0/1040879542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a3400b860 con 0x7f1a40102770 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.176+0000 7f1a3f7fe700 1 -- 172.21.15.64:0/1040879542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1a3400be50 con 0x7f1a40102770 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.176+0000 7f1a3f7fe700 1 -- 172.21.15.64:0/1040879542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a34010380 con 0x7f1a40102770 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 -- 172.21.15.64:0/1040879542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 msgr2=0x7f1a40102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 --2- 172.21.15.64:0/1040879542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a340098e0 tx=0x7f1a34009250 comp rx=0 tx=0).stop 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 -- 172.21.15.64:0/1040879542 shutdown_connections 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 --2- 172.21.15.64:0/1040879542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40102b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 -- 172.21.15.64:0/1040879542 >> 172.21.15.64:0/1040879542 conn(0x7f1a400fded0 msgr2=0x7f1a40100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 -- 172.21.15.64:0/1040879542 shutdown_connections 2024-02-16T22:47:03.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 -- 172.21.15.64:0/1040879542 wait complete. 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 Processor -- start 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.177+0000 7f1a47073700 1 -- start start 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a47073700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a47073700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40111ad0 con 0x7f1a40102770 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a44e0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a44e0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48730/0 (socket says 172.21.15.64:48730) 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a44e0f700 1 -- 172.21.15.64:0/2454735735 learned_addr learned my addr 172.21.15.64:0/2454735735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a44e0f700 1 -- 172.21.15.64:0/2454735735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340094f0 con 0x7f1a40102770 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a44e0f700 1 --2- 172.21.15.64:0/2454735735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40111590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a34000c00 tx=0x7f1a34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a3d7fa700 1 -- 172.21.15.64:0/2454735735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a340107c0 con 0x7f1a40102770 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a3d7fa700 1 -- 172.21.15.64:0/2454735735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1a34010db0 con 0x7f1a40102770 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a40111cd0 con 0x7f1a40102770 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a3d7fa700 1 -- 172.21.15.64:0/2454735735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a3401ad00 con 0x7f1a40102770 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.178+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a40112970 con 0x7f1a40102770 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.179+0000 7f1a3d7fa700 1 -- 172.21.15.64:0/2454735735 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1a34024440 con 0x7f1a40102770 2024-02-16T22:47:03.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.179+0000 7f1a3d7fa700 1 -- 172.21.15.64:0/2454735735 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1a34019710 con 0x7f1a40102770 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.179+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4005f5c0 con 0x7f1a40102770 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.182+0000 7f1a3d7fa700 1 -- 172.21.15.64:0/2454735735 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1a3401a530 con 0x7f1a40102770 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.250+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f1a401045e0 con 0x7f1a40102770 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.256+0000 7f1a3d7fa700 1 -- 172.21.15.64:0/2454735735 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1a34010920 con 0x7f1a40102770 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.256+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 msgr2=0x7f1a40111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.256+0000 7f1a47073700 1 --2- 172.21.15.64:0/2454735735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40111590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a34000c00 tx=0x7f1a34009250 comp rx=0 tx=0).stop 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.256+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 shutdown_connections 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.256+0000 7f1a47073700 1 --2- 172.21.15.64:0/2454735735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a40102770 0x7f1a40111590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.256+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 >> 172.21.15.64:0/2454735735 conn(0x7f1a400fded0 msgr2=0x7f1a400ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.256+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 shutdown_connections 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:03.256+0000 7f1a47073700 1 -- 172.21.15.64:0/2454735735 wait complete. 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mgr... 2024-02-16T22:47:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 9283 ... 2024-02-16T22:47:04.036 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mgr.smithi064.kubmlk 2024-02-16T22:47:04.036 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mgr.smithi064.kubmlk.service: Unit ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mgr.smithi064.kubmlk.service not loaded. 2024-02-16T22:47:04.183 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715.target.wants/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mgr.smithi064.kubmlk.service → /etc/systemd/system/ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@.service. 2024-02-16T22:47:04.537 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:04 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2454735735' entity='client.admin' 2024-02-16T22:47:04.764 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T22:47:04.764 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-02-16T22:47:04.773 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T22:47:04.773 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-02-16T22:47:04.773 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-02-16T22:47:04.773 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-02-16T22:47:04.783 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T22:47:04.784 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-02-16T22:47:04.792 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T22:47:04.792 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-02-16T22:47:04.792 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-02-16T22:47:04.792 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-16T22:47:04.792 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr to start... 2024-02-16T22:47:04.792 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr... 2024-02-16T22:47:05.571 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:05 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/845526999' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T22:47:05.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:47:05.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:47:05.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "2912dc76-cd1d-11ee-95ba-87774f69a715", 2024-02-16T22:47:05.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "smithi064" 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T22:47:05.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T22:47:05.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T22:46:56.779629+0000", 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.197+0000 7f9dc7848700 1 Processor -- start 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.197+0000 7f9dc7848700 1 -- start start 2024-02-16T22:47:05.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.197+0000 7f9dc7848700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0102b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.197+0000 7f9dc7848700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc01030d0 con 0x7f9dc0102780 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.198+0000 7f9dc55e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.198+0000 7f9dc55e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48744/0 (socket says 172.21.15.64:48744) 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.198+0000 7f9dc55e4700 1 -- 172.21.15.64:0/295973250 learned_addr learned my addr 172.21.15.64:0/295973250 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.198+0000 7f9dc55e4700 1 -- 172.21.15.64:0/295973250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc0103210 con 0x7f9dc0102780 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.198+0000 7f9dc55e4700 1 --2- 172.21.15.64:0/295973250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0102b90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0098e0 tx=0x7f9dbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d66237e43c18c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.198+0000 7f9db7fff700 1 -- 172.21.15.64:0/295973250 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dbc00b860 con 0x7f9dc0102780 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.198+0000 7f9db7fff700 1 -- 172.21.15.64:0/295973250 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9dbc00be50 con 0x7f9dc0102780 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.198+0000 7f9db7fff700 1 -- 172.21.15.64:0/295973250 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dbc010380 con 0x7f9dc0102780 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.199+0000 7f9dc7848700 1 -- 172.21.15.64:0/295973250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 msgr2=0x7f9dc0102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.199+0000 7f9dc7848700 1 --2- 172.21.15.64:0/295973250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0102b90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0098e0 tx=0x7f9dbc009250 comp rx=0 tx=0).stop 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.199+0000 7f9dc7848700 1 -- 172.21.15.64:0/295973250 shutdown_connections 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.199+0000 7f9dc7848700 1 --2- 172.21.15.64:0/295973250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0102b90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.199+0000 7f9dc7848700 1 -- 172.21.15.64:0/295973250 >> 172.21.15.64:0/295973250 conn(0x7f9dc00fdf00 msgr2=0x7f9dc0100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:05.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc7848700 1 -- 172.21.15.64:0/295973250 shutdown_connections 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc7848700 1 -- 172.21.15.64:0/295973250 wait complete. 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc7848700 1 Processor -- start 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc7848700 1 -- start start 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc7848700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0111560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc7848700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0111aa0 con 0x7f9dc0102780 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc55e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0111560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc55e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0111560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48748/0 (socket says 172.21.15.64:48748) 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.200+0000 7f9dc55e4700 1 -- 172.21.15.64:0/845526999 learned_addr learned my addr 172.21.15.64:0/845526999 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.201+0000 7f9dc55e4700 1 -- 172.21.15.64:0/845526999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0094f0 con 0x7f9dc0102780 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.201+0000 7f9dc55e4700 1 --2- 172.21.15.64:0/845526999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0111560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc000c00 tx=0x7f9dbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.201+0000 7f9db5ffb700 1 -- 172.21.15.64:0/845526999 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dbc0107c0 con 0x7f9dc0102780 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.201+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc0111ca0 con 0x7f9dc0102780 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.201+0000 7f9db5ffb700 1 -- 172.21.15.64:0/845526999 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9dbc010db0 con 0x7f9dc0102780 2024-02-16T22:47:05.864 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.201+0000 7f9db5ffb700 1 -- 172.21.15.64:0/845526999 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dbc01ad00 con 0x7f9dc0102780 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.201+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc0112940 con 0x7f9dc0102780 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.201+0000 7f9db5ffb700 1 -- 172.21.15.64:0/845526999 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9dbc024440 con 0x7f9dc0102780 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.202+0000 7f9db5ffb700 1 -- 172.21.15.64:0/845526999 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9dbc019710 con 0x7f9dc0102780 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.202+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc005f5c0 con 0x7f9dc0102780 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.204+0000 7f9db5ffb700 1 -- 172.21.15.64:0/845526999 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9dbc01a530 con 0x7f9dc0102780 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.277+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9dc01045f0 con 0x7f9dc0102780 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.278+0000 7f9db5ffb700 1 -- 172.21.15.64:0/845526999 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f9dbc010920 con 0x7f9dc0102780 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.279+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 msgr2=0x7f9dc0111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.279+0000 7f9dc7848700 1 --2- 172.21.15.64:0/845526999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0111560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc000c00 tx=0x7f9dbc009250 comp rx=0 tx=0).stop 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.279+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 shutdown_connections 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.279+0000 7f9dc7848700 1 --2- 172.21.15.64:0/845526999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9dc0102780 0x7f9dc0111560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.279+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 >> 172.21.15.64:0/845526999 conn(0x7f9dc00fdf00 msgr2=0x7f9dc00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.279+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 shutdown_connections 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:05.279+0000 7f9dc7848700 1 -- 172.21.15.64:0/845526999 wait complete. 2024-02-16T22:47:05.865 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (1/15)... 2024-02-16T22:47:08.848 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:47:08.848 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "2912dc76-cd1d-11ee-95ba-87774f69a715", 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "smithi064" 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-16T22:47:08.849 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T22:47:08.850 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T22:47:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T22:46:56.779629+0000", 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:47:08.852 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.226+0000 7f9356c6f700 1 Processor -- start 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.226+0000 7f9356c6f700 1 -- start start 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.226+0000 7f9356c6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350105380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.226+0000 7f9356c6f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93501058c0 con 0x7f9350071480 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.226+0000 7f9354a0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.227+0000 7f9354a0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48758/0 (socket says 172.21.15.64:48758) 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.227+0000 7f9354a0b700 1 -- 172.21.15.64:0/3331954284 learned_addr learned my addr 172.21.15.64:0/3331954284 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.227+0000 7f9354a0b700 1 -- 172.21.15.64:0/3331954284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350105a00 con 0x7f9350071480 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.227+0000 7f9354a0b700 1 --2- 172.21.15.64:0/3331954284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350105380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f93440098e0 tx=0x7f9344009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdae03a78057ee8 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.227+0000 7f934f7fe700 1 -- 172.21.15.64:0/3331954284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f934400b860 con 0x7f9350071480 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.227+0000 7f934f7fe700 1 -- 172.21.15.64:0/3331954284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f934400be50 con 0x7f9350071480 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.227+0000 7f934f7fe700 1 -- 172.21.15.64:0/3331954284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9344010380 con 0x7f9350071480 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.228+0000 7f9356c6f700 1 -- 172.21.15.64:0/3331954284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 msgr2=0x7f9350105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.228+0000 7f9356c6f700 1 --2- 172.21.15.64:0/3331954284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350105380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f93440098e0 tx=0x7f9344009250 comp rx=0 tx=0).stop 2024-02-16T22:47:08.853 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.228+0000 7f9356c6f700 1 -- 172.21.15.64:0/3331954284 shutdown_connections 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.228+0000 7f9356c6f700 1 --2- 172.21.15.64:0/3331954284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350105380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.228+0000 7f9356c6f700 1 -- 172.21.15.64:0/3331954284 >> 172.21.15.64:0/3331954284 conn(0x7f93500fcdd0 msgr2=0x7f93500ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.228+0000 7f9356c6f700 1 -- 172.21.15.64:0/3331954284 shutdown_connections 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.228+0000 7f9356c6f700 1 -- 172.21.15.64:0/3331954284 wait complete. 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.229+0000 7f9356c6f700 1 Processor -- start 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.229+0000 7f9356c6f700 1 -- start start 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.229+0000 7f9356c6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.229+0000 7f9356c6f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9350101280 con 0x7f9350071480 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.229+0000 7f9354a0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.229+0000 7f9354a0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48762/0 (socket says 172.21.15.64:48762) 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.229+0000 7f9354a0b700 1 -- 172.21.15.64:0/598117057 learned_addr learned my addr 172.21.15.64:0/598117057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.229+0000 7f9354a0b700 1 -- 172.21.15.64:0/598117057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440094f0 con 0x7f9350071480 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.230+0000 7f9354a0b700 1 --2- 172.21.15.64:0/598117057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350102e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9344006b00 tx=0x7f9344009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.230+0000 7f934d7fa700 1 -- 172.21.15.64:0/598117057 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93440107c0 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.230+0000 7f934d7fa700 1 -- 172.21.15.64:0/598117057 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9344010db0 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.230+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9350101480 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.230+0000 7f934d7fa700 1 -- 172.21.15.64:0/598117057 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f934401acf0 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.230+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9350101910 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.230+0000 7f934d7fa700 1 -- 172.21.15.64:0/598117057 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9344024440 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.230+0000 7f934d7fa700 1 -- 172.21.15.64:0/598117057 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9344019710 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.231+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935004bed0 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.233+0000 7f934d7fa700 1 -- 172.21.15.64:0/598117057 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f93440426d0 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.305+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9350106810 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.305+0000 7f934d7fa700 1 -- 172.21.15.64:0/598117057 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f9344031020 con 0x7f9350071480 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.306+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 msgr2=0x7f9350102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.306+0000 7f9356c6f700 1 --2- 172.21.15.64:0/598117057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350102e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9344006b00 tx=0x7f9344009250 comp rx=0 tx=0).stop 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.306+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 shutdown_connections 2024-02-16T22:47:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.306+0000 7f9356c6f700 1 --2- 172.21.15.64:0/598117057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9350071480 0x7f9350102e00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:08.856 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.306+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 >> 172.21.15.64:0/598117057 conn(0x7f93500fcdd0 msgr2=0x7f93500fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:08.856 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.306+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 shutdown_connections 2024-02-16T22:47:08.856 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:08.306+0000 7f9356c6f700 1 -- 172.21.15.64:0/598117057 wait complete. 2024-02-16T22:47:08.856 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (2/15)... 2024-02-16T22:47:08.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:08 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/598117057' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T22:47:11.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/234226508' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T22:47:11.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: Activating manager daemon smithi064.kubmlk 2024-02-16T22:47:11.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: mgrmap e2: smithi064.kubmlk(active, starting, since 0.00279928s) 2024-02-16T22:47:11.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: Manager daemon smithi064.kubmlk is now available 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:11.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:11 smithi064 ceph-mon[29226]: from='mgr.14100 172.21.15.64:0/3190917438' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "2912dc76-cd1d-11ee-95ba-87774f69a715", 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T22:47:11.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "smithi064" 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T22:47:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T22:47:11.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T22:47:11.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T22:46:56.779629+0000", 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.240+0000 7fc04759e700 1 Processor -- start 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.240+0000 7fc04759e700 1 -- start start 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.241+0000 7fc04759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048102990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.241+0000 7fc04759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc048102ed0 con 0x7fc048102580 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.241+0000 7fc04659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.241+0000 7fc04659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50850/0 (socket says 172.21.15.64:50850) 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.241+0000 7fc04659c700 1 -- 172.21.15.64:0/3172496209 learned_addr learned my addr 172.21.15.64:0/3172496209 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.241+0000 7fc04659c700 1 -- 172.21.15.64:0/3172496209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc048103010 con 0x7fc048102580 2024-02-16T22:47:11.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.241+0000 7fc04659c700 1 --2- 172.21.15.64:0/3172496209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048102990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0380098e0 tx=0x7fc038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86d8ad55dbb821 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04559a700 1 -- 172.21.15.64:0/3172496209 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc03800b860 con 0x7fc048102580 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04559a700 1 -- 172.21.15.64:0/3172496209 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc03800be50 con 0x7fc048102580 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04559a700 1 -- 172.21.15.64:0/3172496209 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc038010380 con 0x7fc048102580 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04759e700 1 -- 172.21.15.64:0/3172496209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 msgr2=0x7fc048102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04759e700 1 --2- 172.21.15.64:0/3172496209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048102990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0380098e0 tx=0x7fc038009250 comp rx=0 tx=0).stop 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04759e700 1 -- 172.21.15.64:0/3172496209 shutdown_connections 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04759e700 1 --2- 172.21.15.64:0/3172496209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048102990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04759e700 1 -- 172.21.15.64:0/3172496209 >> 172.21.15.64:0/3172496209 conn(0x7fc0480fdd20 msgr2=0x7fc048100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04759e700 1 -- 172.21.15.64:0/3172496209 shutdown_connections 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.242+0000 7fc04759e700 1 -- 172.21.15.64:0/3172496209 wait complete. 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.243+0000 7fc04759e700 1 Processor -- start 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.243+0000 7fc04759e700 1 -- start start 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.243+0000 7fc04759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048111700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.243+0000 7fc04759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc048111c40 con 0x7fc048102580 2024-02-16T22:47:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.243+0000 7fc04659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048111700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.243+0000 7fc04659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048111700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50860/0 (socket says 172.21.15.64:50860) 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.243+0000 7fc04659c700 1 -- 172.21.15.64:0/234226508 learned_addr learned my addr 172.21.15.64:0/234226508 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.243+0000 7fc04659c700 1 -- 172.21.15.64:0/234226508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380094f0 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.244+0000 7fc04659c700 1 --2- 172.21.15.64:0/234226508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048111700 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc038013030 tx=0x7fc038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.244+0000 7fc036ffd700 1 -- 172.21.15.64:0/234226508 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc038010760 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.244+0000 7fc036ffd700 1 -- 172.21.15.64:0/234226508 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc038010d50 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.244+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc048111e40 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.244+0000 7fc036ffd700 1 -- 172.21.15.64:0/234226508 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc03801ae60 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.244+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc048112ae0 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.244+0000 7fc036ffd700 1 -- 172.21.15.64:0/234226508 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc0380198e0 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.244+0000 7fc036ffd700 1 -- 172.21.15.64:0/234226508 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc038024b30 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.245+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04805f5c0 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.247+0000 7fc036ffd700 1 -- 172.21.15.64:0/234226508 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc038029060 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.319+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc048064ad0 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.320+0000 7fc036ffd700 1 -- 172.21.15.64:0/234226508 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fc03801a460 con 0x7fc048102580 2024-02-16T22:47:11.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.320+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 msgr2=0x7fc048111700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:11.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.320+0000 7fc04759e700 1 --2- 172.21.15.64:0/234226508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048111700 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc038013030 tx=0x7fc038009250 comp rx=0 tx=0).stop 2024-02-16T22:47:11.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.321+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 shutdown_connections 2024-02-16T22:47:11.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.321+0000 7fc04759e700 1 --2- 172.21.15.64:0/234226508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc048102580 0x7fc048111700 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:11.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.321+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 >> 172.21.15.64:0/234226508 conn(0x7fc0480fdd20 msgr2=0x7fc048100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:11.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.321+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 shutdown_connections 2024-02-16T22:47:11.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:11.321+0000 7fc04759e700 1 -- 172.21.15.64:0/234226508 wait complete. 2024-02-16T22:47:11.947 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (3/15)... 2024-02-16T22:47:13.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:13 smithi064 ceph-mon[29226]: mgrmap e3: smithi064.kubmlk(active, since 1.00726s) 2024-02-16T22:47:15.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "2912dc76-cd1d-11ee-95ba-87774f69a715", 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "smithi064" 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T22:47:15.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:15.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-16T22:47:15.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T22:47:15.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:15.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T22:47:15.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:15.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T22:47:15.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T22:47:15.088 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T22:47:15.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T22:47:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T22:46:56.779629+0000", 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.328+0000 7fee613dc700 1 Processor -- start 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.328+0000 7fee613dc700 1 -- start start 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.328+0000 7fee613dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.328+0000 7fee613dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c1030c0 con 0x7fee5c102770 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.329+0000 7fee5affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.329+0000 7fee5affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50962/0 (socket says 172.21.15.64:50962) 2024-02-16T22:47:15.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.329+0000 7fee5affd700 1 -- 172.21.15.64:0/806657199 learned_addr learned my addr 172.21.15.64:0/806657199 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.329+0000 7fee5affd700 1 -- 172.21.15.64:0/806657199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c103200 con 0x7fee5c102770 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.329+0000 7fee5affd700 1 --2- 172.21.15.64:0/806657199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c102b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fee500098e0 tx=0x7fee50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f42c08b321283 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.329+0000 7fee59ffb700 1 -- 172.21.15.64:0/806657199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee5000b860 con 0x7fee5c102770 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.329+0000 7fee59ffb700 1 -- 172.21.15.64:0/806657199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fee5000be50 con 0x7fee5c102770 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.329+0000 7fee59ffb700 1 -- 172.21.15.64:0/806657199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee50010380 con 0x7fee5c102770 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.330+0000 7fee613dc700 1 -- 172.21.15.64:0/806657199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 msgr2=0x7fee5c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.330+0000 7fee613dc700 1 --2- 172.21.15.64:0/806657199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c102b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fee500098e0 tx=0x7fee50009250 comp rx=0 tx=0).stop 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.330+0000 7fee613dc700 1 -- 172.21.15.64:0/806657199 shutdown_connections 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.330+0000 7fee613dc700 1 --2- 172.21.15.64:0/806657199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c102b80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.330+0000 7fee613dc700 1 -- 172.21.15.64:0/806657199 >> 172.21.15.64:0/806657199 conn(0x7fee5c0fded0 msgr2=0x7fee5c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.330+0000 7fee613dc700 1 -- 172.21.15.64:0/806657199 shutdown_connections 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.330+0000 7fee613dc700 1 -- 172.21.15.64:0/806657199 wait complete. 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.331+0000 7fee613dc700 1 Processor -- start 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.331+0000 7fee613dc700 1 -- start start 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.331+0000 7fee613dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:15.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.331+0000 7fee613dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c077190 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.331+0000 7fee5affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.331+0000 7fee5affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50978/0 (socket says 172.21.15.64:50978) 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.331+0000 7fee5affd700 1 -- 172.21.15.64:0/467520302 learned_addr learned my addr 172.21.15.64:0/467520302 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.331+0000 7fee5affd700 1 -- 172.21.15.64:0/467520302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500094f0 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.332+0000 7fee5affd700 1 --2- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c078d10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee50000c00 tx=0x7fee50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.333+0000 7fee437fe700 1 -- 172.21.15.64:0/467520302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee500107c0 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.333+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c077390 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.333+0000 7fee437fe700 1 -- 172.21.15.64:0/467520302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fee50010db0 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.333+0000 7fee437fe700 1 -- 172.21.15.64:0/467520302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee5001ad00 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.333+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c077820 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.334+0000 7fee437fe700 1 -- 172.21.15.64:0/467520302 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fee500193c0 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.334+0000 7fee437fe700 1 --2- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fee44036740 0x7fee44038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.334+0000 7fee437fe700 1 -- 172.21.15.64:0/467520302 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fee5004b4d0 con 0x7fee5c102770 2024-02-16T22:47:15.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.334+0000 7fee5a7fc700 1 --2- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fee44036740 0x7fee44038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.334+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c05f5c0 con 0x7fee5c102770 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.335+0000 7fee5a7fc700 1 --2- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fee44036740 0x7fee44038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009990 tx=0x7fee4c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.340+0000 7fee437fe700 1 -- 172.21.15.64:0/467520302 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fee50019670 con 0x7fee5c102770 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.610+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fee5c077be0 con 0x7fee5c102770 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.610+0000 7fee437fe700 1 -- 172.21.15.64:0/467520302 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fee50022340 con 0x7fee5c102770 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fee44036740 msgr2=0x7fee44038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 --2- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fee44036740 0x7fee44038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009990 tx=0x7fee4c006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 msgr2=0x7fee5c078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 --2- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c078d10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee50000c00 tx=0x7fee50009250 comp rx=0 tx=0).stop 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 shutdown_connections 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 --2- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fee44036740 0x7fee44038bf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 --2- 172.21.15.64:0/467520302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee5c102770 0x7fee5c078d10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 >> 172.21.15.64:0/467520302 conn(0x7fee5c0fded0 msgr2=0x7fee5c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 shutdown_connections 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:14.612+0000 7fee613dc700 1 -- 172.21.15.64:0/467520302 wait complete. 2024-02-16T22:47:15.101 INFO:teuthology.orchestra.run.smithi064.stdout:mgr is available 2024-02-16T22:47:15.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:15 smithi064 ceph-mon[29226]: mgrmap e4: smithi064.kubmlk(active, since 3s) 2024-02-16T22:47:15.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:15 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/467520302' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T22:47:16.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:47:16.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-02-16T22:47:16.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:47:16.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-16T22:47:16.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-16T22:47:16.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.514+0000 7f4f4338a700 1 Processor -- start 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.514+0000 7f4f4338a700 1 -- start start 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.514+0000 7f4f4338a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c0771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.514+0000 7f4f4338a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c0790f0 con 0x7f4f3c078ce0 2024-02-16T22:47:16.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.514+0000 7f4f41126700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c0771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.514+0000 7f4f41126700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c0771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50992/0 (socket says 172.21.15.64:50992) 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.514+0000 7f4f41126700 1 -- 172.21.15.64:0/69893947 learned_addr learned my addr 172.21.15.64:0/69893947 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.514+0000 7f4f41126700 1 -- 172.21.15.64:0/69893947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c077730 con 0x7f4f3c078ce0 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f41126700 1 --2- 172.21.15.64:0/69893947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c0771f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4f380098e0 tx=0x7f4f38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a54894dfeb44fdc server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f33fff700 1 -- 172.21.15.64:0/69893947 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f3800b860 con 0x7f4f3c078ce0 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f33fff700 1 -- 172.21.15.64:0/69893947 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4f3800be50 con 0x7f4f3c078ce0 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f33fff700 1 -- 172.21.15.64:0/69893947 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f38010380 con 0x7f4f3c078ce0 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f4338a700 1 -- 172.21.15.64:0/69893947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 msgr2=0x7f4f3c0771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f4338a700 1 --2- 172.21.15.64:0/69893947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c0771f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4f380098e0 tx=0x7f4f38009250 comp rx=0 tx=0).stop 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f4338a700 1 -- 172.21.15.64:0/69893947 shutdown_connections 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f4338a700 1 --2- 172.21.15.64:0/69893947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c0771f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.515+0000 7f4f4338a700 1 -- 172.21.15.64:0/69893947 >> 172.21.15.64:0/69893947 conn(0x7f4f3c0fded0 msgr2=0x7f4f3c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f4338a700 1 -- 172.21.15.64:0/69893947 shutdown_connections 2024-02-16T22:47:16.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f4338a700 1 -- 172.21.15.64:0/69893947 wait complete. 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f4338a700 1 Processor -- start 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f4338a700 1 -- start start 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f4338a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c1137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f4338a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c113d20 con 0x7f4f3c078ce0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f41126700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c1137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f41126700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c1137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51000/0 (socket says 172.21.15.64:51000) 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f41126700 1 -- 172.21.15.64:0/3932486955 learned_addr learned my addr 172.21.15.64:0/3932486955 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.516+0000 7f4f41126700 1 -- 172.21.15.64:0/3932486955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f380094f0 con 0x7f4f3c078ce0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.517+0000 7f4f41126700 1 --2- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c1137e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800b440 tx=0x7f4f38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.517+0000 7f4f31ffb700 1 -- 172.21.15.64:0/3932486955 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f38010760 con 0x7f4f3c078ce0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.517+0000 7f4f31ffb700 1 -- 172.21.15.64:0/3932486955 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4f38010d50 con 0x7f4f3c078ce0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.517+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c113f20 con 0x7f4f3c078ce0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.517+0000 7f4f31ffb700 1 -- 172.21.15.64:0/3932486955 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f3801adc0 con 0x7f4f3c078ce0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.517+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c114bc0 con 0x7f4f3c078ce0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.518+0000 7f4f31ffb700 1 -- 172.21.15.64:0/3932486955 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f4f38019410 con 0x7f4f3c078ce0 2024-02-16T22:47:16.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.518+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3c05f5c0 con 0x7f4f3c078ce0 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.518+0000 7f4f31ffb700 1 --2- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f4f28036720 0x7f4f28038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.518+0000 7f4f31ffb700 1 -- 172.21.15.64:0/3932486955 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4f38049ff0 con 0x7f4f3c078ce0 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.518+0000 7f4f40925700 1 --2- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f4f28036720 0x7f4f28038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.519+0000 7f4f40925700 1 --2- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f4f28036720 0x7f4f28038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009990 tx=0x7f4f2c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.523+0000 7f4f31ffb700 1 -- 172.21.15.64:0/3932486955 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4f3807bbd0 con 0x7f4f3c078ce0 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.713+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4f3c04bed0 con 0x7f4f3c078ce0 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.714+0000 7f4f31ffb700 1 -- 172.21.15.64:0/3932486955 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f4f38023940 con 0x7f4f3c078ce0 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f4f28036720 msgr2=0x7f4f28038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 --2- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f4f28036720 0x7f4f28038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009990 tx=0x7f4f2c006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 msgr2=0x7f4f3c1137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 --2- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c1137e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800b440 tx=0x7f4f38009250 comp rx=0 tx=0).stop 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 shutdown_connections 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 --2- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f4f28036720 0x7f4f28038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 --2- 172.21.15.64:0/3932486955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f3c078ce0 0x7f4f3c1137e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:16.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 >> 172.21.15.64:0/3932486955 conn(0x7f4f3c0fded0 msgr2=0x7f4f3c0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:16.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 shutdown_connections 2024-02-16T22:47:16.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:15.716+0000 7f4f4338a700 1 -- 172.21.15.64:0/3932486955 wait complete. 2024-02-16T22:47:16.235 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling cephadm module... 2024-02-16T22:47:16.616 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:16 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3932486955' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-16T22:47:17.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:17 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/76043451' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-16T22:47:17.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.637+0000 7f09c1ee5700 1 Processor -- start 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.637+0000 7f09c1ee5700 1 -- start start 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.637+0000 7f09c1ee5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc105400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.637+0000 7f09c1ee5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc105940 con 0x7f09bc071400 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.637+0000 7f09bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.637+0000 7f09bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51008/0 (socket says 172.21.15.64:51008) 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.637+0000 7f09bb7fe700 1 -- 172.21.15.64:0/2298284175 learned_addr learned my addr 172.21.15.64:0/2298284175 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09bb7fe700 1 -- 172.21.15.64:0/2298284175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc105a80 con 0x7f09bc071400 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09bb7fe700 1 --2- 172.21.15.64:0/2298284175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc105400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0098e0 tx=0x7f09ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab60ae26034bf8 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09ba7fc700 1 -- 172.21.15.64:0/2298284175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09ac00b8f0 con 0x7f09bc071400 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09ba7fc700 1 -- 172.21.15.64:0/2298284175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f09ac011420 con 0x7f09bc071400 2024-02-16T22:47:17.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09ba7fc700 1 -- 172.21.15.64:0/2298284175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09ac010540 con 0x7f09bc071400 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09c1ee5700 1 -- 172.21.15.64:0/2298284175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 msgr2=0x7f09bc105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09c1ee5700 1 --2- 172.21.15.64:0/2298284175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc105400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0098e0 tx=0x7f09ac009250 comp rx=0 tx=0).stop 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09c1ee5700 1 -- 172.21.15.64:0/2298284175 shutdown_connections 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09c1ee5700 1 --2- 172.21.15.64:0/2298284175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc105400 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09c1ee5700 1 -- 172.21.15.64:0/2298284175 >> 172.21.15.64:0/2298284175 conn(0x7f09bc0fcdd0 msgr2=0x7f09bc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09c1ee5700 1 -- 172.21.15.64:0/2298284175 shutdown_connections 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.638+0000 7f09c1ee5700 1 -- 172.21.15.64:0/2298284175 wait complete. 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.639+0000 7f09c1ee5700 1 Processor -- start 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.639+0000 7f09c1ee5700 1 -- start start 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.639+0000 7f09c1ee5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.639+0000 7f09c1ee5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc111bb0 con 0x7f09bc071400 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.639+0000 7f09bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc111670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51022/0 (socket says 172.21.15.64:51022) 2024-02-16T22:47:17.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09bb7fe700 1 -- 172.21.15.64:0/76043451 learned_addr learned my addr 172.21.15.64:0/76043451 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09bb7fe700 1 -- 172.21.15.64:0/76043451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0094f0 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09bb7fe700 1 --2- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc111670 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09ac000c00 tx=0x7f09ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09a3fff700 1 -- 172.21.15.64:0/76043451 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09ac0107b0 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09a3fff700 1 -- 172.21.15.64:0/76043451 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f09ac010da0 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc111db0 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09a3fff700 1 -- 172.21.15.64:0/76043451 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09ac01add0 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.640+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc112a00 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.641+0000 7f09a3fff700 1 -- 172.21.15.64:0/76043451 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f09ac02e440 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.641+0000 7f09a3fff700 1 --2- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f09a4036740 0x7f09a4038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.641+0000 7f09a3fff700 1 -- 172.21.15.64:0/76043451 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f09ac04afc0 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.641+0000 7f09baffd700 1 --2- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f09a4036740 0x7f09a4038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.641+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc112240 con 0x7f09bc071400 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.642+0000 7f09baffd700 1 --2- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f09a4036740 0x7f09a4038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009990 tx=0x7f09b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:17.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.646+0000 7f09a3fff700 1 -- 172.21.15.64:0/76043451 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f09ac024490 con 0x7f09bc071400 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:16.890+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f09bc112240 con 0x7f09bc071400 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.336+0000 7f09a3fff700 1 -- 172.21.15.64:0/76043451 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f09ac02e920 con 0x7f09bc071400 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.337+0000 7f09a3fff700 1 -- 172.21.15.64:0/76043451 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f09ac02ad70 con 0x7f09bc071400 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f09a4036740 msgr2=0x7f09a4038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 --2- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f09a4036740 0x7f09a4038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009990 tx=0x7f09b0006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 msgr2=0x7f09bc111670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 --2- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc111670 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09ac000c00 tx=0x7f09ac009250 comp rx=0 tx=0).stop 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 shutdown_connections 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 --2- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7f09a4036740 0x7f09a4038bf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 --2- 172.21.15.64:0/76043451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09bc071400 0x7f09bc111670 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 >> 172.21.15.64:0/76043451 conn(0x7f09bc0fcdd0 msgr2=0x7f09bc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 shutdown_connections 2024-02-16T22:47:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:17.339+0000 7f09c1ee5700 1 -- 172.21.15.64:0/76043451 wait complete. 2024-02-16T22:47:18.571 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:18 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/76043451' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-16T22:47:18.571 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:18 smithi064 ceph-mon[29226]: mgrmap e5: smithi064.kubmlk(active, since 6s) 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "smithi064.kubmlk", 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.300+0000 7fdb3cf28700 1 Processor -- start 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.300+0000 7fdb3cf28700 1 -- start start 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3cf28700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38104250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3cf28700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38104790 con 0x7fdb38103e40 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:19.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51044/0 (socket says 172.21.15.64:51044) 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3659c700 1 -- 172.21.15.64:0/211824966 learned_addr learned my addr 172.21.15.64:0/211824966 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3659c700 1 -- 172.21.15.64:0/211824966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb381048d0 con 0x7fdb38103e40 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3659c700 1 --2- 172.21.15.64:0/211824966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38104250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb280098e0 tx=0x7fdb28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9b1b6157da8862 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3559a700 1 -- 172.21.15.64:0/211824966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb2800b860 con 0x7fdb38103e40 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3559a700 1 -- 172.21.15.64:0/211824966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdb2800be50 con 0x7fdb38103e40 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.301+0000 7fdb3559a700 1 -- 172.21.15.64:0/211824966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb28010380 con 0x7fdb38103e40 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.302+0000 7fdb3cf28700 1 -- 172.21.15.64:0/211824966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 msgr2=0x7fdb38104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.302+0000 7fdb3cf28700 1 --2- 172.21.15.64:0/211824966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38104250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb280098e0 tx=0x7fdb28009250 comp rx=0 tx=0).stop 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.302+0000 7fdb3cf28700 1 -- 172.21.15.64:0/211824966 shutdown_connections 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.302+0000 7fdb3cf28700 1 --2- 172.21.15.64:0/211824966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38104250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.302+0000 7fdb3cf28700 1 -- 172.21.15.64:0/211824966 >> 172.21.15.64:0/211824966 conn(0x7fdb380fc580 msgr2=0x7fdb380fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.302+0000 7fdb3cf28700 1 -- 172.21.15.64:0/211824966 shutdown_connections 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.302+0000 7fdb3cf28700 1 -- 172.21.15.64:0/211824966 wait complete. 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.303+0000 7fdb3cf28700 1 Processor -- start 2024-02-16T22:47:19.119 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.303+0000 7fdb3cf28700 1 -- start start 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.303+0000 7fdb3cf28700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38111800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.303+0000 7fdb3cf28700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38111d40 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.303+0000 7fdb3659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38111800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.303+0000 7fdb3659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38111800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51052/0 (socket says 172.21.15.64:51052) 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.303+0000 7fdb3659c700 1 -- 172.21.15.64:0/2517218739 learned_addr learned my addr 172.21.15.64:0/2517218739 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.304+0000 7fdb3659c700 1 -- 172.21.15.64:0/2517218739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280094f0 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.304+0000 7fdb3659c700 1 --2- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38111800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b440 tx=0x7fdb28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.304+0000 7fdb26ffd700 1 -- 172.21.15.64:0/2517218739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb28010760 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.304+0000 7fdb26ffd700 1 -- 172.21.15.64:0/2517218739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdb28010d50 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.304+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb38111f40 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.304+0000 7fdb26ffd700 1 -- 172.21.15.64:0/2517218739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb28024d80 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.304+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb38112be0 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.305+0000 7fdb26ffd700 1 -- 172.21.15.64:0/2517218739 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fdb280108c0 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.305+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3805f5c0 con 0x7fdb38103e40 2024-02-16T22:47:19.120 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.305+0000 7fdb26ffd700 1 --2- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fdb200367e0 0x7fdb20038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.305+0000 7fdb35d9b700 1 -- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fdb200367e0 msgr2=0x7fdb20038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3542090188 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.305+0000 7fdb26ffd700 1 -- 172.21.15.64:0/2517218739 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdb2804a550 con 0x7fdb38103e40 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.305+0000 7fdb35d9b700 1 --2- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fdb200367e0 0x7fdb20038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.310+0000 7fdb26ffd700 1 -- 172.21.15.64:0/2517218739 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fdb280193c0 con 0x7fdb38103e40 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.506+0000 7fdb35d9b700 1 -- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fdb200367e0 msgr2=0x7fdb20038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3542090188 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.506+0000 7fdb35d9b700 1 --2- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fdb200367e0 0x7fdb20038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.570+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdb38101410 con 0x7fdb38103e40 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.570+0000 7fdb26ffd700 1 -- 172.21.15.64:0/2517218739 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fdb28022340 con 0x7fdb38103e40 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fdb200367e0 msgr2=0x7fdb20038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 --2- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fdb200367e0 0x7fdb20038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 msgr2=0x7fdb38111800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 --2- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38111800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b440 tx=0x7fdb28009250 comp rx=0 tx=0).stop 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 shutdown_connections 2024-02-16T22:47:19.121 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 --2- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fdb200367e0 0x7fdb20038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:19.122 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 --2- 172.21.15.64:0/2517218739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb38103e40 0x7fdb38111800 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:19.122 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 >> 172.21.15.64:0/2517218739 conn(0x7fdb380fc580 msgr2=0x7fdb380fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:19.122 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 shutdown_connections 2024-02-16T22:47:19.122 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:18.573+0000 7fdb3cf28700 1 -- 172.21.15.64:0/2517218739 wait complete. 2024-02-16T22:47:19.122 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-02-16T22:47:19.122 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 5... 2024-02-16T22:47:19.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:19 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2517218739' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: Active manager daemon smithi064.kubmlk restarted 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: Activating manager daemon smithi064.kubmlk 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: osdmap e2: 0 total, 0 up, 0 in 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: mgrmap e6: smithi064.kubmlk(active, starting, since 0.00334839s) 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:47:24.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: Manager daemon smithi064.kubmlk is now available 2024-02-16T22:47:24.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:24.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:24.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:47:24.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:47:24.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:24 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:47:25.328 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:25 smithi064 ceph-mon[29226]: Found migration_current of "None". Setting to last migration. 2024-02-16T22:47:25.328 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:25 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:47:25.328 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:25 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:47:25.328 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:25 smithi064 ceph-mon[29226]: mgrmap e7: smithi064.kubmlk(active, since 1.00515s) 2024-02-16T22:47:25.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:47:25.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-16T22:47:25.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-16T22:47:25.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:47:25.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.559+0000 7fb9272bd700 1 Processor -- start 2024-02-16T22:47:25.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.559+0000 7fb9272bd700 1 -- start start 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.559+0000 7fb9272bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920105380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.559+0000 7fb9272bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9201058c0 con 0x7fb920076c20 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.559+0000 7fb925059700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.559+0000 7fb925059700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51066/0 (socket says 172.21.15.64:51066) 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.559+0000 7fb925059700 1 -- 172.21.15.64:0/4001306120 learned_addr learned my addr 172.21.15.64:0/4001306120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.560+0000 7fb925059700 1 -- 172.21.15.64:0/4001306120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb920105a00 con 0x7fb920076c20 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.560+0000 7fb925059700 1 --2- 172.21.15.64:0/4001306120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920105380 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098e0 tx=0x7fb91c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890144d1c0977122 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.560+0000 7fb917fff700 1 -- 172.21.15.64:0/4001306120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb91c00b860 con 0x7fb920076c20 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.560+0000 7fb917fff700 1 -- 172.21.15.64:0/4001306120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb91c00be50 con 0x7fb920076c20 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.560+0000 7fb917fff700 1 -- 172.21.15.64:0/4001306120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb91c010380 con 0x7fb920076c20 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 -- 172.21.15.64:0/4001306120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 msgr2=0x7fb920105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 --2- 172.21.15.64:0/4001306120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920105380 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098e0 tx=0x7fb91c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 -- 172.21.15.64:0/4001306120 shutdown_connections 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 --2- 172.21.15.64:0/4001306120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920105380 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:25.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 -- 172.21.15.64:0/4001306120 >> 172.21.15.64:0/4001306120 conn(0x7fb9200fcdd0 msgr2=0x7fb9200ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 -- 172.21.15.64:0/4001306120 shutdown_connections 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 -- 172.21.15.64:0/4001306120 wait complete. 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 Processor -- start 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.561+0000 7fb9272bd700 1 -- start start 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb9272bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb9272bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb920101280 con 0x7fb920076c20 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb925059700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb925059700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51078/0 (socket says 172.21.15.64:51078) 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb925059700 1 -- 172.21.15.64:0/2010676968 learned_addr learned my addr 172.21.15.64:0/2010676968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb925059700 1 -- 172.21.15.64:0/2010676968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c0094f0 con 0x7fb920076c20 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb925059700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920102e00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb91c003050 tx=0x7fb91c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb91c010760 con 0x7fb920076c20 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb91c010d50 con 0x7fb920076c20 2024-02-16T22:47:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb920101480 con 0x7fb920076c20 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb91c01ae20 con 0x7fb920076c20 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.562+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb920101910 con 0x7fb920076c20 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.563+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb91c02e440 con 0x7fb920076c20 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.564+0000 7fb915ffb700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 0x7fb90c038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.564+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 --> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb90c039330 con 0x7fb90c036770 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.564+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb91c04a690 con 0x7fb920076c20 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.564+0000 7fb924858700 1 -- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 msgr2=0x7fb90c038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3542090188 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.564+0000 7fb924858700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 0x7fb90c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.765+0000 7fb924858700 1 -- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 msgr2=0x7fb90c038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3542090188 2024-02-16T22:47:25.612 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:19.765+0000 7fb924858700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 0x7fb90c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:20.166+0000 7fb924858700 1 -- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 msgr2=0x7fb90c038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3542090188 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:20.166+0000 7fb924858700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 0x7fb90c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:20.967+0000 7fb924858700 1 -- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 msgr2=0x7fb90c038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3542090188 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:20.967+0000 7fb924858700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 0x7fb90c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:22.568+0000 7fb924858700 1 -- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 msgr2=0x7fb90c038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3542090188 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:22.568+0000 7fb924858700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 0x7fb90c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:24.034+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7fb91c02e880 con 0x7fb920076c20 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:24.034+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 msgr2=0x7fb90c038c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:24.034+0000 7fb915ffb700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 0x7fb90c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.035+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7fb91c012000 con 0x7fb920076c20 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.035+0000 7fb915ffb700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fb90c039ff0 0x7fb90c03c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.035+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb90c039330 con 0x7fb90c039ff0 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.036+0000 7fb924858700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fb90c039ff0 0x7fb90c03c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.036+0000 7fb924858700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fb90c039ff0 0x7fb90c03c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9100039d0 tx=0x7fb910009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:25.613 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.037+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fb90c039330 con 0x7fb90c039ff0 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb92005f5c0 con 0x7fb90c039ff0 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb915ffb700 1 -- 172.21.15.64:0/2010676968 <== mgr.14120 v2:172.21.15.64:6800/3482665154 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb92005f5c0 con 0x7fb90c039ff0 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fb90c039ff0 msgr2=0x7fb90c03c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fb90c039ff0 0x7fb90c03c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9100039d0 tx=0x7fb910009350 comp rx=0 tx=0).stop 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 msgr2=0x7fb920102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920102e00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb91c003050 tx=0x7fb91c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 shutdown_connections 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fb90c039ff0 0x7fb90c03c3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb920076c20 0x7fb920102e00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 --2- 172.21.15.64:0/2010676968 >> [v2:172.21.15.64:6800/3542090188,v1:172.21.15.64:6801/3542090188] conn(0x7fb90c036770 0x7fb90c038c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 >> 172.21.15.64:0/2010676968 conn(0x7fb9200fcdd0 msgr2=0x7fb9200fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 shutdown_connections 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:25.043+0000 7fb9272bd700 1 -- 172.21.15.64:0/2010676968 wait complete. 2024-02-16T22:47:25.614 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 5 is available 2024-02-16T22:47:25.615 INFO:teuthology.orchestra.run.smithi064.stdout:Setting orchestrator backend to cephadm... 2024-02-16T22:47:26.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:26 smithi064 ceph-mon[29226]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-16T22:47:26.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:26 smithi064 ceph-mon[29226]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-16T22:47:26.816 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.049+0000 7f70c16e0700 1 Processor -- start 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.049+0000 7f70c16e0700 1 -- start start 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.049+0000 7f70c16e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.049+0000 7f70c16e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc105960 con 0x7f70bc0713e0 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.049+0000 7f70baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.049+0000 7f70baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58858/0 (socket says 172.21.15.64:58858) 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.049+0000 7f70baffd700 1 -- 172.21.15.64:0/2875289786 learned_addr learned my addr 172.21.15.64:0/2875289786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.050+0000 7f70baffd700 1 -- 172.21.15.64:0/2875289786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc105aa0 con 0x7f70bc0713e0 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.050+0000 7f70baffd700 1 --2- 172.21.15.64:0/2875289786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc105420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0098e0 tx=0x7f70ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ed299f6adf465 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.050+0000 7f70b9ffb700 1 -- 172.21.15.64:0/2875289786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70ac00b860 con 0x7f70bc0713e0 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.050+0000 7f70b9ffb700 1 -- 172.21.15.64:0/2875289786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f70ac00be50 con 0x7f70bc0713e0 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.050+0000 7f70b9ffb700 1 -- 172.21.15.64:0/2875289786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70ac010380 con 0x7f70bc0713e0 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.051+0000 7f70c16e0700 1 -- 172.21.15.64:0/2875289786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 msgr2=0x7f70bc105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:26.822 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.051+0000 7f70c16e0700 1 --2- 172.21.15.64:0/2875289786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc105420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0098e0 tx=0x7f70ac009250 comp rx=0 tx=0).stop 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.051+0000 7f70c16e0700 1 -- 172.21.15.64:0/2875289786 shutdown_connections 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.051+0000 7f70c16e0700 1 --2- 172.21.15.64:0/2875289786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc105420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.051+0000 7f70c16e0700 1 -- 172.21.15.64:0/2875289786 >> 172.21.15.64:0/2875289786 conn(0x7f70bc0fcdd0 msgr2=0x7f70bc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.051+0000 7f70c16e0700 1 -- 172.21.15.64:0/2875289786 shutdown_connections 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.051+0000 7f70c16e0700 1 -- 172.21.15.64:0/2875289786 wait complete. 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70c16e0700 1 Processor -- start 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70c16e0700 1 -- start start 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70c16e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc1115a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70c16e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc111ae0 con 0x7f70bc0713e0 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc1115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc1115a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58872/0 (socket says 172.21.15.64:58872) 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70baffd700 1 -- 172.21.15.64:0/1237259032 learned_addr learned my addr 172.21.15.64:0/1237259032 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70baffd700 1 -- 172.21.15.64:0/1237259032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac0094f0 con 0x7f70bc0713e0 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.052+0000 7f70baffd700 1 --2- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc1115a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f70ac003050 tx=0x7f70ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.053+0000 7f70a37fe700 1 -- 172.21.15.64:0/1237259032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70ac010760 con 0x7f70bc0713e0 2024-02-16T22:47:26.823 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.053+0000 7f70a37fe700 1 -- 172.21.15.64:0/1237259032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f70ac010d50 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.053+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc111ce0 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.053+0000 7f70a37fe700 1 -- 172.21.15.64:0/1237259032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70ac01ae20 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.053+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc112980 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.054+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc05f5c0 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.054+0000 7f70a37fe700 1 -- 172.21.15.64:0/1237259032 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f70ac02e440 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.054+0000 7f70a37fe700 1 --2- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f70a4036660 0x7f70a4038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.054+0000 7f70a37fe700 1 -- 172.21.15.64:0/1237259032 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f70ac04a470 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.054+0000 7f70ba7fc700 1 --2- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f70a4036660 0x7f70a4038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.055+0000 7f70ba7fc700 1 --2- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f70a4036660 0x7f70a4038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009990 tx=0x7f70b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.063+0000 7f70a37fe700 1 -- 172.21.15.64:0/1237259032 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f70ac018530 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.086+0000 7f70a37fe700 1 -- 172.21.15.64:0/1237259032 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f70ac0243f0 con 0x7f70bc0713e0 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.286+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f70bc0ff100 con 0x7f70a4036660 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.293+0000 7f70a37fe700 1 -- 172.21.15.64:0/1237259032 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f70bc0ff100 con 0x7f70a4036660 2024-02-16T22:47:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.296+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f70a4036660 msgr2=0x7f70a4038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.296+0000 7f70c16e0700 1 --2- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f70a4036660 0x7f70a4038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009990 tx=0x7f70b0006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.296+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 msgr2=0x7f70bc1115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.296+0000 7f70c16e0700 1 --2- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc1115a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f70ac003050 tx=0x7f70ac009250 comp rx=0 tx=0).stop 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.297+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 shutdown_connections 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.297+0000 7f70c16e0700 1 --2- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f70a4036660 0x7f70a4038b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.297+0000 7f70c16e0700 1 --2- 172.21.15.64:0/1237259032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70bc0713e0 0x7f70bc1115a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.297+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 >> 172.21.15.64:0/1237259032 conn(0x7f70bc0fcdd0 msgr2=0x7f70bc0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.297+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 shutdown_connections 2024-02-16T22:47:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:26.297+0000 7f70c16e0700 1 -- 172.21.15.64:0/1237259032 wait complete. 2024-02-16T22:47:27.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:27 smithi064 ceph-mon[29226]: mgrmap e8: smithi064.kubmlk(active, since 2s) 2024-02-16T22:47:27.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:27 smithi064 ceph-mon[29226]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:27.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:27 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:27.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:27 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:47:27.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:27 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:27.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:27 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.203+0000 7f45c33fc700 1 Processor -- start 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.203+0000 7f45c33fc700 1 -- start start 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.203+0000 7f45c33fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.203+0000 7f45c33fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc1030c0 con 0x7f45bc102770 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.204+0000 7f45c1198700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.204+0000 7f45c1198700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58878/0 (socket says 172.21.15.64:58878) 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.204+0000 7f45c1198700 1 -- 172.21.15.64:0/3462294374 learned_addr learned my addr 172.21.15.64:0/3462294374 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.204+0000 7f45c1198700 1 -- 172.21.15.64:0/3462294374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc103200 con 0x7f45bc102770 2024-02-16T22:47:27.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.204+0000 7f45c1198700 1 --2- 172.21.15.64:0/3462294374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc102b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009e50 tx=0x7f45b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce0105b8125f968 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.204+0000 7f45bbfff700 1 -- 172.21.15.64:0/3462294374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b40092e0 con 0x7f45bc102770 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.204+0000 7f45bbfff700 1 -- 172.21.15.64:0/3462294374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f45b4012430 con 0x7f45bc102770 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.204+0000 7f45bbfff700 1 -- 172.21.15.64:0/3462294374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b40075c0 con 0x7f45bc102770 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.205+0000 7f45c33fc700 1 -- 172.21.15.64:0/3462294374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 msgr2=0x7f45bc102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.205+0000 7f45c33fc700 1 --2- 172.21.15.64:0/3462294374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc102b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009e50 tx=0x7f45b4009840 comp rx=0 tx=0).stop 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.205+0000 7f45c33fc700 1 -- 172.21.15.64:0/3462294374 shutdown_connections 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.205+0000 7f45c33fc700 1 --2- 172.21.15.64:0/3462294374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc102b80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.205+0000 7f45c33fc700 1 -- 172.21.15.64:0/3462294374 >> 172.21.15.64:0/3462294374 conn(0x7f45bc0fded0 msgr2=0x7f45bc100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.205+0000 7f45c33fc700 1 -- 172.21.15.64:0/3462294374 shutdown_connections 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.205+0000 7f45c33fc700 1 -- 172.21.15.64:0/3462294374 wait complete. 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c33fc700 1 Processor -- start 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c33fc700 1 -- start start 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c33fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc077c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c33fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc076080 con 0x7f45bc102770 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c1198700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c1198700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc077c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58884/0 (socket says 172.21.15.64:58884) 2024-02-16T22:47:27.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c1198700 1 -- 172.21.15.64:0/1575189794 learned_addr learned my addr 172.21.15.64:0/1575189794 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c1198700 1 -- 172.21.15.64:0/1575189794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009ae0 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45c1198700 1 --2- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc077c00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45b4000f80 tx=0x7f45b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.206+0000 7f45b9ffb700 1 -- 172.21.15.64:0/1575189794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b40092e0 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.207+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc076280 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.207+0000 7f45b9ffb700 1 -- 172.21.15.64:0/1575189794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f45b4007cc0 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.207+0000 7f45b9ffb700 1 -- 172.21.15.64:0/1575189794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b401c990 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.207+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc076770 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.208+0000 7f45b9ffb700 1 -- 172.21.15.64:0/1575189794 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f45b4019070 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.208+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc10b670 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.208+0000 7f45b9ffb700 1 --2- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f45a4036790 0x7f45a4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.208+0000 7f45b9ffb700 1 -- 172.21.15.64:0/1575189794 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f45b4049e80 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.208+0000 7f45c0997700 1 --2- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f45a4036790 0x7f45a4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.208+0000 7f45c0997700 1 --2- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f45a4036790 0x7f45a4038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009990 tx=0x7f45b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.213+0000 7f45b9ffb700 1 -- 172.21.15.64:0/1575189794 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f45b4019cf0 con 0x7f45bc102770 2024-02-16T22:47:27.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.407+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f45bc104650 con 0x7f45a4036790 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.408+0000 7f45b9ffb700 1 -- 172.21.15.64:0/1575189794 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f45bc104650 con 0x7f45a4036790 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.409+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f45a4036790 msgr2=0x7f45a4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.409+0000 7f45c33fc700 1 --2- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f45a4036790 0x7f45a4038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009990 tx=0x7f45b0006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.409+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 msgr2=0x7f45bc077c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.409+0000 7f45c33fc700 1 --2- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc077c00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45b4000f80 tx=0x7f45b4009840 comp rx=0 tx=0).stop 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.410+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 shutdown_connections 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.410+0000 7f45c33fc700 1 --2- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f45a4036790 0x7f45a4038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.410+0000 7f45c33fc700 1 --2- 172.21.15.64:0/1575189794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45bc102770 0x7f45bc077c00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.410+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 >> 172.21.15.64:0/1575189794 conn(0x7f45bc0fded0 msgr2=0x7f45bc0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.410+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 shutdown_connections 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:27.410+0000 7f45c33fc700 1 -- 172.21.15.64:0/1575189794 wait complete. 2024-02-16T22:47:27.948 INFO:teuthology.orchestra.run.smithi064.stdout:Generating ssh key... 2024-02-16T22:47:28.322 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:28 smithi064 ceph-mon[29226]: [16/Feb/2024:22:47:26] ENGINE Bus STARTING 2024-02-16T22:47:28.323 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:28 smithi064 ceph-mon[29226]: [16/Feb/2024:22:47:26] ENGINE Serving on https://172.21.15.64:7150 2024-02-16T22:47:28.323 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:28 smithi064 ceph-mon[29226]: [16/Feb/2024:22:47:26] ENGINE Bus STARTED 2024-02-16T22:47:29.301 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:29 smithi064 ceph-mon[29226]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:29.302 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:29 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:29.302 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:29 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:29.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.421+0000 7f9bd4866700 1 Processor -- start 2024-02-16T22:47:29.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.422+0000 7f9bd4866700 1 -- start start 2024-02-16T22:47:29.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.422+0000 7f9bd4866700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd0101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:29.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.422+0000 7f9bd4866700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd0101fc0 con 0x7f9bd0101670 2024-02-16T22:47:29.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.422+0000 7f9bce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd0101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:29.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.422+0000 7f9bce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd0101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58890/0 (socket says 172.21.15.64:58890) 2024-02-16T22:47:29.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.422+0000 7f9bce59c700 1 -- 172.21.15.64:0/3312327977 learned_addr learned my addr 172.21.15.64:0/3312327977 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:29.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.422+0000 7f9bce59c700 1 -- 172.21.15.64:0/3312327977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd0102100 con 0x7f9bd0101670 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.422+0000 7f9bce59c700 1 --2- 172.21.15.64:0/3312327977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd0101a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40098e0 tx=0x7f9bc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56ba56966c648d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bcd59a700 1 -- 172.21.15.64:0/3312327977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc400b860 con 0x7f9bd0101670 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bcd59a700 1 -- 172.21.15.64:0/3312327977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9bc400be50 con 0x7f9bd0101670 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bcd59a700 1 -- 172.21.15.64:0/3312327977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc4010380 con 0x7f9bd0101670 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bd4866700 1 -- 172.21.15.64:0/3312327977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 msgr2=0x7f9bd0101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bd4866700 1 --2- 172.21.15.64:0/3312327977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd0101a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40098e0 tx=0x7f9bc4009250 comp rx=0 tx=0).stop 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bd4866700 1 -- 172.21.15.64:0/3312327977 shutdown_connections 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bd4866700 1 --2- 172.21.15.64:0/3312327977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd0101a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bd4866700 1 -- 172.21.15.64:0/3312327977 >> 172.21.15.64:0/3312327977 conn(0x7f9bd00fcdd0 msgr2=0x7f9bd00ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bd4866700 1 -- 172.21.15.64:0/3312327977 shutdown_connections 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.423+0000 7f9bd4866700 1 -- 172.21.15.64:0/3312327977 wait complete. 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.424+0000 7f9bd4866700 1 Processor -- start 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.424+0000 7f9bd4866700 1 -- start start 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.424+0000 7f9bd4866700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd01144a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.424+0000 7f9bd4866700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd0076c10 con 0x7f9bd0101670 2024-02-16T22:47:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.424+0000 7f9bce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd01144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.424+0000 7f9bce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd01144a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58894/0 (socket says 172.21.15.64:58894) 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.424+0000 7f9bce59c700 1 -- 172.21.15.64:0/4268060090 learned_addr learned my addr 172.21.15.64:0/4268060090 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.424+0000 7f9bce59c700 1 -- 172.21.15.64:0/4268060090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40094f0 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.425+0000 7f9bce59c700 1 --2- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd01144a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400b440 tx=0x7f9bc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.425+0000 7f9bbeffd700 1 -- 172.21.15.64:0/4268060090 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc40107c0 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.425+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd0076e10 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.425+0000 7f9bbeffd700 1 -- 172.21.15.64:0/4268060090 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9bc4010db0 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.425+0000 7f9bbeffd700 1 -- 172.21.15.64:0/4268060090 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc401ada0 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.425+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd0114ba0 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.426+0000 7f9bbeffd700 1 -- 172.21.15.64:0/4268060090 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f9bc4010920 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.426+0000 7f9bbeffd700 1 --2- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f9bb8036790 0x7f9bb8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.426+0000 7f9bbeffd700 1 -- 172.21.15.64:0/4268060090 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9bc404acc0 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.426+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd010a570 con 0x7f9bd0101670 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.426+0000 7f9bcdd9b700 1 --2- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f9bb8036790 0x7f9bb8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:29.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.426+0000 7f9bcdd9b700 1 --2- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f9bb8036790 0x7f9bb8038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009990 tx=0x7f9bc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.431+0000 7f9bbeffd700 1 -- 172.21.15.64:0/4268060090 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9bc4018940 con 0x7f9bd0101670 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.624+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bd0103650 con 0x7f9bb8036790 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.733+0000 7f9bbeffd700 1 -- 172.21.15.64:0/4268060090 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9bd0103650 con 0x7f9bb8036790 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.735+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f9bb8036790 msgr2=0x7f9bb8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.735+0000 7f9bd4866700 1 --2- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f9bb8036790 0x7f9bb8038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009990 tx=0x7f9bc0006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.735+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 msgr2=0x7f9bd01144a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.735+0000 7f9bd4866700 1 --2- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd01144a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400b440 tx=0x7f9bc4009250 comp rx=0 tx=0).stop 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.735+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 shutdown_connections 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.735+0000 7f9bd4866700 1 --2- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f9bb8036790 0x7f9bb8038c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.735+0000 7f9bd4866700 1 --2- 172.21.15.64:0/4268060090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd0101670 0x7f9bd01144a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.735+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 >> 172.21.15.64:0/4268060090 conn(0x7f9bd00fcdd0 msgr2=0x7f9bd00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.736+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 shutdown_connections 2024-02-16T22:47:29.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:28.736+0000 7f9bd4866700 1 -- 172.21.15.64:0/4268060090 wait complete. 2024-02-16T22:47:30.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:30 smithi064 ceph-mon[29226]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:30.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:30 smithi064 ceph-mon[29226]: Generating ssh key... 2024-02-16T22:47:30.466 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3CnEaDMje6txJ1TRToJpnjbXSpi7SgKfCr4jvuI1gF5sSZjBgq6DS5/cmKN+t9+N4N2Kgv4yrFgWXVcbLktk5nslubJBNjKNx1zB5328G73/zbwY3HKsfLQBkfmaaDKsLR86GCvNJQ0ekA6EJlRvuocMk1N8A+nbPNwt2PsEC0Ap+DcPX16ut2Hwg3Ioe5JT1uOCp6Qcl8vDZP0ZmgcFsGVjjTv1YSz1DrOKtHDUlatgE7shYsxeycmxUlzfaZC5XeL+Z0mtuu7UHP8sMDwc0gkmhT0swZjNxi1daTGT4hd0ixfb1ltqqW6G5NPElCSiB33RW8l2qJY73bKuZ6LVwZyml/DaIAvwtdXLzfCC0F2ZN/jAT4+37T0GrPZnB0DQN/W57NTF/O7APukYrSSifN0v4y+6pIqWXWCLX/mwQBd86mxYdIlLjT0jWVU7n4olDBGvlOBBBUZbl8dDrGEYPomDv5s5dkWImJulWR3CWUxzJChLk48mJ7bsVUOwskqU= ceph-2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:47:30.466 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.675+0000 7fd785ad5700 1 Processor -- start 2024-02-16T22:47:30.466 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.675+0000 7fd785ad5700 1 -- start start 2024-02-16T22:47:30.466 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.675+0000 7fd785ad5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd780102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:30.466 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.675+0000 7fd785ad5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7801030c0 con 0x7fd780102770 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.675+0000 7fd77f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd780102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.675+0000 7fd77f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd780102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58904/0 (socket says 172.21.15.64:58904) 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.675+0000 7fd77f7fe700 1 -- 172.21.15.64:0/4197106639 learned_addr learned my addr 172.21.15.64:0/4197106639 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.676+0000 7fd77f7fe700 1 -- 172.21.15.64:0/4197106639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780103200 con 0x7fd780102770 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.676+0000 7fd77f7fe700 1 --2- 172.21.15.64:0/4197106639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd780102b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd770009e50 tx=0x7fd770009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5687c30e7c4c707 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.676+0000 7fd77e7fc700 1 -- 172.21.15.64:0/4197106639 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7700092e0 con 0x7fd780102770 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.676+0000 7fd77e7fc700 1 -- 172.21.15.64:0/4197106639 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd770012430 con 0x7fd780102770 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.676+0000 7fd77e7fc700 1 -- 172.21.15.64:0/4197106639 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7700075c0 con 0x7fd780102770 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.676+0000 7fd785ad5700 1 -- 172.21.15.64:0/4197106639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 msgr2=0x7fd780102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.676+0000 7fd785ad5700 1 --2- 172.21.15.64:0/4197106639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd780102b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd770009e50 tx=0x7fd770009840 comp rx=0 tx=0).stop 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 -- 172.21.15.64:0/4197106639 shutdown_connections 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 --2- 172.21.15.64:0/4197106639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd780102b80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 -- 172.21.15.64:0/4197106639 >> 172.21.15.64:0/4197106639 conn(0x7fd7800fded0 msgr2=0x7fd780100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 -- 172.21.15.64:0/4197106639 shutdown_connections 2024-02-16T22:47:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 -- 172.21.15.64:0/4197106639 wait complete. 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 Processor -- start 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 -- start start 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd7801115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.677+0000 7fd785ad5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd780111af0 con 0x7fd780102770 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd77f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd7801115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd77f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd7801115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58916/0 (socket says 172.21.15.64:58916) 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd77f7fe700 1 -- 172.21.15.64:0/2488553146 learned_addr learned my addr 172.21.15.64:0/2488553146 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd77f7fe700 1 -- 172.21.15.64:0/2488553146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770009ae0 con 0x7fd780102770 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd77f7fe700 1 --2- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd7801115b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd770009ab0 tx=0x7fd770009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd777fff700 1 -- 172.21.15.64:0/2488553146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7700092e0 con 0x7fd780102770 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd777fff700 1 -- 172.21.15.64:0/2488553146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd770007d10 con 0x7fd780102770 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd780111cf0 con 0x7fd780102770 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd777fff700 1 -- 172.21.15.64:0/2488553146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd77001c990 con 0x7fd780102770 2024-02-16T22:47:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.678+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd780112990 con 0x7fd780102770 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.679+0000 7fd777fff700 1 -- 172.21.15.64:0/2488553146 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd770019070 con 0x7fd780102770 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.679+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78005f5c0 con 0x7fd780102770 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.679+0000 7fd777fff700 1 --2- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fd76c0367e0 0x7fd76c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.679+0000 7fd777fff700 1 -- 172.21.15.64:0/2488553146 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd770049e20 con 0x7fd780102770 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.679+0000 7fd77effd700 1 --2- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fd76c0367e0 0x7fd76c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.680+0000 7fd77effd700 1 --2- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fd76c0367e0 0x7fd76c038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd768009990 tx=0x7fd768006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.684+0000 7fd777fff700 1 -- 172.21.15.64:0/2488553146 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd770019910 con 0x7fd780102770 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.875+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd780113550 con 0x7fd76c0367e0 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.875+0000 7fd777fff700 1 -- 172.21.15.64:0/2488553146 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd780113550 con 0x7fd76c0367e0 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fd76c0367e0 msgr2=0x7fd76c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 --2- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fd76c0367e0 0x7fd76c038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd768009990 tx=0x7fd768006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 msgr2=0x7fd7801115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 --2- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd7801115b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd770009ab0 tx=0x7fd770009840 comp rx=0 tx=0).stop 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 shutdown_connections 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 --2- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fd76c0367e0 0x7fd76c038c90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 --2- 172.21.15.64:0/2488553146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd780102770 0x7fd7801115b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 >> 172.21.15.64:0/2488553146 conn(0x7fd7800fded0 msgr2=0x7fd7800ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 shutdown_connections 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:29.877+0000 7fd785ad5700 1 -- 172.21.15.64:0/2488553146 wait complete. 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:Adding key to root@localhost authorized_keys... 2024-02-16T22:47:30.470 INFO:teuthology.orchestra.run.smithi064.stdout:Adding host smithi064... 2024-02-16T22:47:31.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:31 smithi064 ceph-mon[29226]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:32.241 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:32 smithi064 ceph-mon[29226]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:33 smithi064 ceph-mon[29226]: Deploying cephadm binary to smithi064 2024-02-16T22:47:33.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Added host 'smithi064' with addr '172.21.15.64' 2024-02-16T22:47:33.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.852+0000 7f34793ce700 1 Processor -- start 2024-02-16T22:47:33.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.852+0000 7f34793ce700 1 -- start start 2024-02-16T22:47:33.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.852+0000 7f34793ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474105360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:33.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.852+0000 7f34793ce700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34741058a0 con 0x7f34740714a0 2024-02-16T22:47:33.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.853+0000 7f3472ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474105360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:33.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.853+0000 7f3472ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474105360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47834/0 (socket says 172.21.15.64:47834) 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.853+0000 7f3472ffd700 1 -- 172.21.15.64:0/3561523537 learned_addr learned my addr 172.21.15.64:0/3561523537 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.853+0000 7f3472ffd700 1 -- 172.21.15.64:0/3561523537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34741059e0 con 0x7f34740714a0 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.853+0000 7f3472ffd700 1 --2- 172.21.15.64:0/3561523537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474105360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f346c009e50 tx=0x7f346c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5f9d8a97453d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.853+0000 7f3471ffb700 1 -- 172.21.15.64:0/3561523537 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f346c0092e0 con 0x7f34740714a0 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.853+0000 7f3471ffb700 1 -- 172.21.15.64:0/3561523537 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f346c012430 con 0x7f34740714a0 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.853+0000 7f3471ffb700 1 -- 172.21.15.64:0/3561523537 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f346c0075c0 con 0x7f34740714a0 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.854+0000 7f34793ce700 1 -- 172.21.15.64:0/3561523537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 msgr2=0x7f3474105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.854+0000 7f34793ce700 1 --2- 172.21.15.64:0/3561523537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474105360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f346c009e50 tx=0x7f346c009840 comp rx=0 tx=0).stop 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.854+0000 7f34793ce700 1 -- 172.21.15.64:0/3561523537 shutdown_connections 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.854+0000 7f34793ce700 1 --2- 172.21.15.64:0/3561523537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474105360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.854+0000 7f34793ce700 1 -- 172.21.15.64:0/3561523537 >> 172.21.15.64:0/3561523537 conn(0x7f34740fcdd0 msgr2=0x7f34740ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.854+0000 7f34793ce700 1 -- 172.21.15.64:0/3561523537 shutdown_connections 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.854+0000 7f34793ce700 1 -- 172.21.15.64:0/3561523537 wait complete. 2024-02-16T22:47:33.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.855+0000 7f34793ce700 1 Processor -- start 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.855+0000 7f34793ce700 1 -- start start 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.855+0000 7f34793ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.855+0000 7f34793ce700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3474101260 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.855+0000 7f3472ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.855+0000 7f3472ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474102de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47848/0 (socket says 172.21.15.64:47848) 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.855+0000 7f3472ffd700 1 -- 172.21.15.64:0/3034959992 learned_addr learned my addr 172.21.15.64:0/3034959992 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.855+0000 7f3472ffd700 1 -- 172.21.15.64:0/3034959992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c009ae0 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.856+0000 7f3472ffd700 1 --2- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474102de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f346c009fd0 tx=0x7f346c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.856+0000 7f346b7fe700 1 -- 172.21.15.64:0/3034959992 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f346c0092e0 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.856+0000 7f346b7fe700 1 -- 172.21.15.64:0/3034959992 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f346c007d10 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.856+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3474101460 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.856+0000 7f346b7fe700 1 -- 172.21.15.64:0/3034959992 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f346c01c880 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.856+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3474101950 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.856+0000 7f346b7fe700 1 -- 172.21.15.64:0/3034959992 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f346c019070 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.857+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347410b640 con 0x7f34740714a0 2024-02-16T22:47:33.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.857+0000 7f346b7fe700 1 --2- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34600367e0 0x7f3460038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.857+0000 7f346b7fe700 1 -- 172.21.15.64:0/3034959992 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f346c049e10 con 0x7f34740714a0 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.857+0000 7f34727fc700 1 --2- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34600367e0 0x7f3460038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.857+0000 7f34727fc700 1 --2- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34600367e0 0x7f3460038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f345c009990 tx=0x7f345c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:30.862+0000 7f346b7fe700 1 -- 172.21.15.64:0/3034959992 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f346c0243b0 con 0x7f34740714a0 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:31.055+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}) v1 -- 0x7f347402b410 con 0x7f34600367e0 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.141+0000 7f346b7fe700 1 -- 172.21.15.64:0/3034959992 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f347402b410 con 0x7f34600367e0 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34600367e0 msgr2=0x7f3460038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 --2- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34600367e0 0x7f3460038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f345c009990 tx=0x7f345c006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 msgr2=0x7f3474102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 --2- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474102de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f346c009fd0 tx=0x7f346c009840 comp rx=0 tx=0).stop 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 shutdown_connections 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 --2- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34600367e0 0x7f3460038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 --2- 172.21.15.64:0/3034959992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34740714a0 0x7f3474102de0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:33.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 >> 172.21.15.64:0/3034959992 conn(0x7f34740fcdd0 msgr2=0x7f34740fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:33.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 shutdown_connections 2024-02-16T22:47:33.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:33.144+0000 7f34793ce700 1 -- 172.21.15.64:0/3034959992 wait complete. 2024-02-16T22:47:33.679 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying mon service with default placement... 2024-02-16T22:47:34.299 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:34 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:34.299 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:34 smithi064 ceph-mon[29226]: Added host smithi064 2024-02-16T22:47:34.299 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:34 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:47:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:35 smithi064 ceph-mon[29226]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:35 smithi064 ceph-mon[29226]: Saving service mon spec with placement count:5 2024-02-16T22:47:35.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:35 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:35.798 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f3426266700 1 Processor -- start 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f3426266700 1 -- start start 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f3426266700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f3426266700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34201030c0 con 0x7f3420102770 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f341ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f341ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47858/0 (socket says 172.21.15.64:47858) 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f341ffff700 1 -- 172.21.15.64:0/1231399148 learned_addr learned my addr 172.21.15.64:0/1231399148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f341ffff700 1 -- 172.21.15.64:0/1231399148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3420103200 con 0x7f3420102770 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.134+0000 7f341ffff700 1 --2- 172.21.15.64:0/1231399148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420102b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f34100098e0 tx=0x7f3410009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed178c7b95dc0ad server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f341effd700 1 -- 172.21.15.64:0/1231399148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f341000b860 con 0x7f3420102770 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f341effd700 1 -- 172.21.15.64:0/1231399148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f341000be50 con 0x7f3420102770 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f341effd700 1 -- 172.21.15.64:0/1231399148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3410010380 con 0x7f3420102770 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f3426266700 1 -- 172.21.15.64:0/1231399148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 msgr2=0x7f3420102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f3426266700 1 --2- 172.21.15.64:0/1231399148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420102b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f34100098e0 tx=0x7f3410009250 comp rx=0 tx=0).stop 2024-02-16T22:47:35.799 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f3426266700 1 -- 172.21.15.64:0/1231399148 shutdown_connections 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f3426266700 1 --2- 172.21.15.64:0/1231399148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420102b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f3426266700 1 -- 172.21.15.64:0/1231399148 >> 172.21.15.64:0/1231399148 conn(0x7f34200fded0 msgr2=0x7f3420100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f3426266700 1 -- 172.21.15.64:0/1231399148 shutdown_connections 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.135+0000 7f3426266700 1 -- 172.21.15.64:0/1231399148 wait complete. 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.136+0000 7f3426266700 1 Processor -- start 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.136+0000 7f3426266700 1 -- start start 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.136+0000 7f3426266700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.136+0000 7f3426266700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420111a40 con 0x7f3420102770 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.136+0000 7f341ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.136+0000 7f341ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420111500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47866/0 (socket says 172.21.15.64:47866) 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.136+0000 7f341ffff700 1 -- 172.21.15.64:0/2522824075 learned_addr learned my addr 172.21.15.64:0/2522824075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.136+0000 7f341ffff700 1 -- 172.21.15.64:0/2522824075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34100094f0 con 0x7f3420102770 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.137+0000 7f341ffff700 1 --2- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420111500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f341000b440 tx=0x7f3410009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.137+0000 7f341cff9700 1 -- 172.21.15.64:0/2522824075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3410010760 con 0x7f3420102770 2024-02-16T22:47:35.800 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.137+0000 7f341cff9700 1 -- 172.21.15.64:0/2522824075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3410010d50 con 0x7f3420102770 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.137+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3420111c40 con 0x7f3420102770 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.137+0000 7f341cff9700 1 -- 172.21.15.64:0/2522824075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3410024d80 con 0x7f3420102770 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.137+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34201128e0 con 0x7f3420102770 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.138+0000 7f341cff9700 1 -- 172.21.15.64:0/2522824075 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f341001a450 con 0x7f3420102770 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.138+0000 7f341cff9700 1 --2- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34080367e0 0x7f3408038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.138+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342005f5c0 con 0x7f3420102770 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.138+0000 7f341cff9700 1 -- 172.21.15.64:0/2522824075 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f341004a6b0 con 0x7f3420102770 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.138+0000 7f341f7fe700 1 --2- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34080367e0 0x7f3408038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.138+0000 7f341f7fe700 1 --2- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34080367e0 0x7f3408038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3414009990 tx=0x7f3414006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.143+0000 7f341cff9700 1 -- 172.21.15.64:0/2522824075 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f341002d3f0 con 0x7f3420102770 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.353+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f34200ff380 con 0x7f34080367e0 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.360+0000 7f341cff9700 1 -- 172.21.15.64:0/2522824075 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f34200ff380 con 0x7f34080367e0 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34080367e0 msgr2=0x7f3408038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:35.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 --2- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34080367e0 0x7f3408038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3414009990 tx=0x7f3414006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 msgr2=0x7f3420111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 --2- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420111500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f341000b440 tx=0x7f3410009250 comp rx=0 tx=0).stop 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 shutdown_connections 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 --2- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f34080367e0 0x7f3408038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 --2- 172.21.15.64:0/2522824075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3420102770 0x7f3420111500 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 >> 172.21.15.64:0/2522824075 conn(0x7f34200fded0 msgr2=0x7f34200fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 shutdown_connections 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:34.362+0000 7f3426266700 1 -- 172.21.15.64:0/2522824075 wait complete. 2024-02-16T22:47:35.802 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying mgr service with default placement... 2024-02-16T22:47:36.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:36 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:36.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:36 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:36.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:36 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:37.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-16T22:47:37.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.227+0000 7efc92336700 1 Processor -- start 2024-02-16T22:47:37.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.228+0000 7efc92336700 1 -- start start 2024-02-16T22:47:37.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.228+0000 7efc92336700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:37.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.228+0000 7efc92336700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c1030c0 con 0x7efc8c102770 2024-02-16T22:47:37.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.228+0000 7efc8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:37.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.228+0000 7efc8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47876/0 (socket says 172.21.15.64:47876) 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.228+0000 7efc8bfff700 1 -- 172.21.15.64:0/549556161 learned_addr learned my addr 172.21.15.64:0/549556161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.228+0000 7efc8bfff700 1 -- 172.21.15.64:0/549556161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c103200 con 0x7efc8c102770 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.229+0000 7efc8bfff700 1 --2- 172.21.15.64:0/549556161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c102b80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0098e0 tx=0x7efc7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa33e8f2e35f346 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.229+0000 7efc8affd700 1 -- 172.21.15.64:0/549556161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc7c00b860 con 0x7efc8c102770 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.229+0000 7efc8affd700 1 -- 172.21.15.64:0/549556161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc7c00be90 con 0x7efc8c102770 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.229+0000 7efc8affd700 1 -- 172.21.15.64:0/549556161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc7c0103c0 con 0x7efc8c102770 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.230+0000 7efc92336700 1 -- 172.21.15.64:0/549556161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 msgr2=0x7efc8c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.230+0000 7efc92336700 1 --2- 172.21.15.64:0/549556161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c102b80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0098e0 tx=0x7efc7c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.230+0000 7efc92336700 1 -- 172.21.15.64:0/549556161 shutdown_connections 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.230+0000 7efc92336700 1 --2- 172.21.15.64:0/549556161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c102b80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.230+0000 7efc92336700 1 -- 172.21.15.64:0/549556161 >> 172.21.15.64:0/549556161 conn(0x7efc8c0fded0 msgr2=0x7efc8c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.230+0000 7efc92336700 1 -- 172.21.15.64:0/549556161 shutdown_connections 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.230+0000 7efc92336700 1 -- 172.21.15.64:0/549556161 wait complete. 2024-02-16T22:47:37.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.230+0000 7efc92336700 1 Processor -- start 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc92336700 1 -- start start 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc92336700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc92336700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c111a80 con 0x7efc8c102770 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c111540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47892/0 (socket says 172.21.15.64:47892) 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc8bfff700 1 -- 172.21.15.64:0/1782099161 learned_addr learned my addr 172.21.15.64:0/1782099161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc8bfff700 1 -- 172.21.15.64:0/1782099161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c0094f0 con 0x7efc8c102770 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc8bfff700 1 --2- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c111540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc7c011650 tx=0x7efc7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc88ff9700 1 -- 172.21.15.64:0/1782099161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc7c010890 con 0x7efc8c102770 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc88ff9700 1 -- 172.21.15.64:0/1782099161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc7c010ec0 con 0x7efc8c102770 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c111c80 con 0x7efc8c102770 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.231+0000 7efc88ff9700 1 -- 172.21.15.64:0/1782099161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc7c01a430 con 0x7efc8c102770 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.232+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8c112920 con 0x7efc8c102770 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.232+0000 7efc88ff9700 1 -- 172.21.15.64:0/1782099161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7efc7c01a6b0 con 0x7efc8c102770 2024-02-16T22:47:37.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.233+0000 7efc88ff9700 1 --2- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7efc74036790 0x7efc74038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.233+0000 7efc88ff9700 1 -- 172.21.15.64:0/1782099161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7efc7c04a4c0 con 0x7efc8c102770 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.233+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8c05f5c0 con 0x7efc8c102770 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.233+0000 7efc8b7fe700 1 --2- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7efc74036790 0x7efc74038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.233+0000 7efc8b7fe700 1 --2- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7efc74036790 0x7efc74038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efc80009990 tx=0x7efc80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.238+0000 7efc88ff9700 1 -- 172.21.15.64:0/1782099161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7efc7c02d3f0 con 0x7efc8c102770 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.453+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7efc8c0ff3a0 con 0x7efc74036790 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.457+0000 7efc88ff9700 1 -- 172.21.15.64:0/1782099161 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efc8c0ff3a0 con 0x7efc74036790 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7efc74036790 msgr2=0x7efc74038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 --2- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7efc74036790 0x7efc74038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efc80009990 tx=0x7efc80006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 msgr2=0x7efc8c111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 --2- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c111540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc7c011650 tx=0x7efc7c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 shutdown_connections 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 --2- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7efc74036790 0x7efc74038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:37.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 --2- 172.21.15.64:0/1782099161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c102770 0x7efc8c111540 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:37.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 >> 172.21.15.64:0/1782099161 conn(0x7efc8c0fded0 msgr2=0x7efc8c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:37.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 shutdown_connections 2024-02-16T22:47:37.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:36.459+0000 7efc92336700 1 -- 172.21.15.64:0/1782099161 wait complete. 2024-02-16T22:47:37.404 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying crash service with default placement... 2024-02-16T22:47:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:37 smithi064 ceph-mon[29226]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:38.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:37 smithi064 ceph-mon[29226]: Saving service mgr spec with placement count:2 2024-02-16T22:47:38.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:37 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:38.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:37 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-02-16T22:47:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.794+0000 7fbdda5d4700 1 Processor -- start 2024-02-16T22:47:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.794+0000 7fbdda5d4700 1 -- start start 2024-02-16T22:47:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.794+0000 7fbdda5d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.794+0000 7fbdda5d4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4079110 con 0x7fbdd4078d00 2024-02-16T22:47:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.794+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.794+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47898/0 (socket says 172.21.15.64:47898) 2024-02-16T22:47:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.794+0000 7fbdd3fff700 1 -- 172.21.15.64:0/2719823513 learned_addr learned my addr 172.21.15.64:0/2719823513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.794+0000 7fbdd3fff700 1 -- 172.21.15.64:0/2719823513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4077750 con 0x7fbdd4078d00 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.795+0000 7fbdd3fff700 1 --2- 172.21.15.64:0/2719823513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4077210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40098e0 tx=0x7fbdc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69ed63df18f669e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.795+0000 7fbdd2ffd700 1 -- 172.21.15.64:0/2719823513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc400b860 con 0x7fbdd4078d00 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.795+0000 7fbdd2ffd700 1 -- 172.21.15.64:0/2719823513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdc400be90 con 0x7fbdd4078d00 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.795+0000 7fbdd2ffd700 1 -- 172.21.15.64:0/2719823513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc40103c0 con 0x7fbdd4078d00 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.795+0000 7fbdda5d4700 1 -- 172.21.15.64:0/2719823513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 msgr2=0x7fbdd4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.795+0000 7fbdda5d4700 1 --2- 172.21.15.64:0/2719823513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4077210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40098e0 tx=0x7fbdc4009250 comp rx=0 tx=0).stop 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.796+0000 7fbdda5d4700 1 -- 172.21.15.64:0/2719823513 shutdown_connections 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.796+0000 7fbdda5d4700 1 --2- 172.21.15.64:0/2719823513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4077210 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.796+0000 7fbdda5d4700 1 -- 172.21.15.64:0/2719823513 >> 172.21.15.64:0/2719823513 conn(0x7fbdd40fded0 msgr2=0x7fbdd4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.796+0000 7fbdda5d4700 1 -- 172.21.15.64:0/2719823513 shutdown_connections 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.796+0000 7fbdda5d4700 1 -- 172.21.15.64:0/2719823513 wait complete. 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.796+0000 7fbdda5d4700 1 Processor -- start 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.797+0000 7fbdda5d4700 1 -- start start 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.797+0000 7fbdda5d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:38.532 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.797+0000 7fbdda5d4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4113d40 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.797+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.797+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47908/0 (socket says 172.21.15.64:47908) 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.797+0000 7fbdd3fff700 1 -- 172.21.15.64:0/3026301618 learned_addr learned my addr 172.21.15.64:0/3026301618 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.797+0000 7fbdd3fff700 1 -- 172.21.15.64:0/3026301618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40094f0 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.797+0000 7fbdd3fff700 1 --2- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4113800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40094c0 tx=0x7fbdc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.798+0000 7fbdd0ff9700 1 -- 172.21.15.64:0/3026301618 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc4010890 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.798+0000 7fbdd0ff9700 1 -- 172.21.15.64:0/3026301618 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdc4010ec0 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.798+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd4113f40 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.798+0000 7fbdd0ff9700 1 -- 172.21.15.64:0/3026301618 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc40243f0 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.798+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd4114be0 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.799+0000 7fbdd0ff9700 1 -- 172.21.15.64:0/3026301618 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbdc402d3f0 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.799+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd405f5c0 con 0x7fbdd4078d00 2024-02-16T22:47:38.533 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.799+0000 7fbdd0ff9700 1 --2- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fbdbc036790 0x7fbdbc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.799+0000 7fbdd0ff9700 1 -- 172.21.15.64:0/3026301618 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbdc404a2c0 con 0x7fbdd4078d00 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.799+0000 7fbdd37fe700 1 --2- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fbdbc036790 0x7fbdbc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.799+0000 7fbdd37fe700 1 --2- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fbdbc036790 0x7fbdbc038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009990 tx=0x7fbdc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:37.804+0000 7fbdd0ff9700 1 -- 172.21.15.64:0/3026301618 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbdc402b8e0 con 0x7fbdd4078d00 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.008+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdd4100210 con 0x7fbdbc036790 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.013+0000 7fbdd0ff9700 1 -- 172.21.15.64:0/3026301618 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fbdd4100210 con 0x7fbdbc036790 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fbdbc036790 msgr2=0x7fbdbc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 --2- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fbdbc036790 0x7fbdbc038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009990 tx=0x7fbdc8006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 msgr2=0x7fbdd4113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 --2- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4113800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40094c0 tx=0x7fbdc4009250 comp rx=0 tx=0).stop 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 shutdown_connections 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 --2- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fbdbc036790 0x7fbdbc038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 --2- 172.21.15.64:0/3026301618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd4078d00 0x7fbdd4113800 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:38.534 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 >> 172.21.15.64:0/3026301618 conn(0x7fbdd40fded0 msgr2=0x7fbdd40ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:38.535 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 shutdown_connections 2024-02-16T22:47:38.535 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.015+0000 7fbdda5d4700 1 -- 172.21.15.64:0/3026301618 wait complete. 2024-02-16T22:47:38.535 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying prometheus service with default placement... 2024-02-16T22:47:39.052 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:39 smithi064 ceph-mon[29226]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:39.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:39 smithi064 ceph-mon[29226]: Saving service crash spec with placement * 2024-02-16T22:47:39.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:39 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:40.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-02-16T22:47:40.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.890+0000 7fa682e62700 1 Processor -- start 2024-02-16T22:47:40.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.890+0000 7fa682e62700 1 -- start start 2024-02-16T22:47:40.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.890+0000 7fa682e62700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c105400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:40.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.890+0000 7fa682e62700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c105940 con 0x7fa67c076ba0 2024-02-16T22:47:40.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.890+0000 7fa680bfe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.890+0000 7fa680bfe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47918/0 (socket says 172.21.15.64:47918) 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.891+0000 7fa680bfe700 1 -- 172.21.15.64:0/1674074051 learned_addr learned my addr 172.21.15.64:0/1674074051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.891+0000 7fa680bfe700 1 -- 172.21.15.64:0/1674074051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c105a80 con 0x7fa67c076ba0 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.891+0000 7fa680bfe700 1 --2- 172.21.15.64:0/1674074051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c105400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa6700098e0 tx=0x7fa670009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0b9cb3ac8e25c server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.891+0000 7fa67b7fe700 1 -- 172.21.15.64:0/1674074051 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa67000b8f0 con 0x7fa67c076ba0 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.891+0000 7fa67b7fe700 1 -- 172.21.15.64:0/1674074051 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa670011420 con 0x7fa67c076ba0 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.891+0000 7fa67b7fe700 1 -- 172.21.15.64:0/1674074051 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa670010540 con 0x7fa67c076ba0 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 -- 172.21.15.64:0/1674074051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 msgr2=0x7fa67c105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 --2- 172.21.15.64:0/1674074051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c105400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa6700098e0 tx=0x7fa670009250 comp rx=0 tx=0).stop 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 -- 172.21.15.64:0/1674074051 shutdown_connections 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 --2- 172.21.15.64:0/1674074051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c105400 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 -- 172.21.15.64:0/1674074051 >> 172.21.15.64:0/1674074051 conn(0x7fa67c0fcdd0 msgr2=0x7fa67c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 -- 172.21.15.64:0/1674074051 shutdown_connections 2024-02-16T22:47:40.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 -- 172.21.15.64:0/1674074051 wait complete. 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 Processor -- start 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.892+0000 7fa682e62700 1 -- start start 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa682e62700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa682e62700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c101300 con 0x7fa67c076ba0 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa680bfe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa680bfe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c102e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47926/0 (socket says 172.21.15.64:47926) 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa680bfe700 1 -- 172.21.15.64:0/382392676 learned_addr learned my addr 172.21.15.64:0/382392676 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa680bfe700 1 -- 172.21.15.64:0/382392676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700094f0 con 0x7fa67c076ba0 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa680bfe700 1 --2- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c102e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa670011cf0 tx=0x7fa670009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa6797fa700 1 -- 172.21.15.64:0/382392676 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa670010800 con 0x7fa67c076ba0 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa6797fa700 1 -- 172.21.15.64:0/382392676 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa670010e30 con 0x7fa67c076ba0 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.893+0000 7fa6797fa700 1 -- 172.21.15.64:0/382392676 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa67001ae80 con 0x7fa67c076ba0 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.894+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c101500 con 0x7fa67c076ba0 2024-02-16T22:47:40.044 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.894+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c101990 con 0x7fa67c076ba0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.895+0000 7fa6797fa700 1 -- 172.21.15.64:0/382392676 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa670010960 con 0x7fa67c076ba0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.895+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c04bed0 con 0x7fa67c076ba0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.895+0000 7fa6797fa700 1 --2- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fa66c0367e0 0x7fa66c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.895+0000 7fa6797fa700 1 -- 172.21.15.64:0/382392676 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa67004a290 con 0x7fa67c076ba0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.895+0000 7fa67bfff700 1 --2- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fa66c0367e0 0x7fa66c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.895+0000 7fa67bfff700 1 --2- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fa66c0367e0 0x7fa66c038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa668009990 tx=0x7fa668006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:38.900+0000 7fa6797fa700 1 -- 172.21.15.64:0/382392676 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa67002b840 con 0x7fa67c076ba0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.107+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fa67c0fe2c0 con 0x7fa66c0367e0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.111+0000 7fa6797fa700 1 -- 172.21.15.64:0/382392676 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fa67c0fe2c0 con 0x7fa66c0367e0 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fa66c0367e0 msgr2=0x7fa66c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 --2- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fa66c0367e0 0x7fa66c038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa668009990 tx=0x7fa668006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 msgr2=0x7fa67c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 --2- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c102e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa670011cf0 tx=0x7fa670009250 comp rx=0 tx=0).stop 2024-02-16T22:47:40.045 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 shutdown_connections 2024-02-16T22:47:40.046 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 --2- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fa66c0367e0 0x7fa66c038c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:40.046 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 --2- 172.21.15.64:0/382392676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa67c076ba0 0x7fa67c102e80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:40.046 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 >> 172.21.15.64:0/382392676 conn(0x7fa67c0fcdd0 msgr2=0x7fa67c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:40.046 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 shutdown_connections 2024-02-16T22:47:40.046 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:39.113+0000 7fa682e62700 1 -- 172.21.15.64:0/382392676 wait complete. 2024-02-16T22:47:40.046 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying grafana service with default placement... 2024-02-16T22:47:40.288 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:40 smithi064 ceph-mon[29226]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:40.288 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:40 smithi064 ceph-mon[29226]: Saving service prometheus spec with placement count:1 2024-02-16T22:47:40.288 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:40 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:41.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.484+0000 7fe501154700 1 Processor -- start 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.485+0000 7fe501154700 1 -- start start 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.485+0000 7fe501154700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.485+0000 7fe501154700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1030c0 con 0x7fe4fc102770 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.485+0000 7fe4fad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.485+0000 7fe4fad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53636/0 (socket says 172.21.15.64:53636) 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.485+0000 7fe4fad9d700 1 -- 172.21.15.64:0/4008748187 learned_addr learned my addr 172.21.15.64:0/4008748187 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.485+0000 7fe4fad9d700 1 -- 172.21.15.64:0/4008748187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc103200 con 0x7fe4fc102770 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.485+0000 7fe4fad9d700 1 --2- 172.21.15.64:0/4008748187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc102b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0098e0 tx=0x7fe4ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a5d72bfd0f256f server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe4f9d9b700 1 -- 172.21.15.64:0/4008748187 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec00b860 con 0x7fe4fc102770 2024-02-16T22:47:41.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe4f9d9b700 1 -- 172.21.15.64:0/4008748187 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4ec00be90 con 0x7fe4fc102770 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe4f9d9b700 1 -- 172.21.15.64:0/4008748187 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec0103c0 con 0x7fe4fc102770 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe501154700 1 -- 172.21.15.64:0/4008748187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 msgr2=0x7fe4fc102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe501154700 1 --2- 172.21.15.64:0/4008748187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc102b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0098e0 tx=0x7fe4ec009250 comp rx=0 tx=0).stop 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe501154700 1 -- 172.21.15.64:0/4008748187 shutdown_connections 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe501154700 1 --2- 172.21.15.64:0/4008748187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc102b80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe501154700 1 -- 172.21.15.64:0/4008748187 >> 172.21.15.64:0/4008748187 conn(0x7fe4fc0fded0 msgr2=0x7fe4fc100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe501154700 1 -- 172.21.15.64:0/4008748187 shutdown_connections 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.486+0000 7fe501154700 1 -- 172.21.15.64:0/4008748187 wait complete. 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.487+0000 7fe501154700 1 Processor -- start 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.487+0000 7fe501154700 1 -- start start 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.487+0000 7fe501154700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.487+0000 7fe501154700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc077190 con 0x7fe4fc102770 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.487+0000 7fe4fad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:41.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.487+0000 7fe4fad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53642/0 (socket says 172.21.15.64:53642) 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.487+0000 7fe4fad9d700 1 -- 172.21.15.64:0/841916104 learned_addr learned my addr 172.21.15.64:0/841916104 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.488+0000 7fe4fad9d700 1 -- 172.21.15.64:0/841916104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec0094f0 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.488+0000 7fe4fad9d700 1 --2- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc078d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec011680 tx=0x7fe4ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.488+0000 7fe4e37fe700 1 -- 172.21.15.64:0/841916104 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec010890 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.488+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc077390 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.488+0000 7fe4e37fe700 1 -- 172.21.15.64:0/841916104 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4ec010ec0 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.488+0000 7fe4e37fe700 1 -- 172.21.15.64:0/841916104 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec0243f0 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.488+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc077820 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.489+0000 7fe4e37fe700 1 -- 172.21.15.64:0/841916104 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fe4ec01a8c0 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.489+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc04bed0 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.489+0000 7fe4e37fe700 1 --2- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fe4e4036790 0x7fe4e4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.489+0000 7fe4e37fe700 1 -- 172.21.15.64:0/841916104 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe4ec04a260 con 0x7fe4fc102770 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.489+0000 7fe4fa59c700 1 --2- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fe4e4036790 0x7fe4e4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.490+0000 7fe4fa59c700 1 --2- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fe4e4036790 0x7fe4e4038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009990 tx=0x7fe4f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:41.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.494+0000 7fe4e37fe700 1 -- 172.21.15.64:0/841916104 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe4ec02b8e0 con 0x7fe4fc102770 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.698+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4fc078060 con 0x7fe4e4036790 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.702+0000 7fe4e37fe700 1 -- 172.21.15.64:0/841916104 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe4fc078060 con 0x7fe4e4036790 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fe4e4036790 msgr2=0x7fe4e4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 --2- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fe4e4036790 0x7fe4e4038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009990 tx=0x7fe4f0006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 msgr2=0x7fe4fc078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 --2- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc078d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec011680 tx=0x7fe4ec009250 comp rx=0 tx=0).stop 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 shutdown_connections 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 --2- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7fe4e4036790 0x7fe4e4038c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 --2- 172.21.15.64:0/841916104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc102770 0x7fe4fc078d10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 >> 172.21.15.64:0/841916104 conn(0x7fe4fc0fded0 msgr2=0x7fe4fc0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 shutdown_connections 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:40.704+0000 7fe501154700 1 -- 172.21.15.64:0/841916104 wait complete. 2024-02-16T22:47:41.388 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying node-exporter service with default placement... 2024-02-16T22:47:42.008 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:41 smithi064 ceph-mon[29226]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:42.008 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:41 smithi064 ceph-mon[29226]: Saving service grafana spec with placement count:1 2024-02-16T22:47:42.009 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:41 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:42.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.793+0000 7f0677eab700 1 Processor -- start 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.793+0000 7f0677eab700 1 -- start start 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0677eab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670104230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0677eab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0670104770 con 0x7f0670103e20 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0675c47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670104230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0675c47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670104230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53644/0 (socket says 172.21.15.64:53644) 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0675c47700 1 -- 172.21.15.64:0/924306810 learned_addr learned my addr 172.21.15.64:0/924306810 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0675c47700 1 -- 172.21.15.64:0/924306810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06701048b0 con 0x7f0670103e20 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0675c47700 1 --2- 172.21.15.64:0/924306810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670104230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f06600098e0 tx=0x7f0660009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d10327410b9f38 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0674c45700 1 -- 172.21.15.64:0/924306810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f066000b860 con 0x7f0670103e20 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0674c45700 1 -- 172.21.15.64:0/924306810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f066000be90 con 0x7f0670103e20 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.794+0000 7f0674c45700 1 -- 172.21.15.64:0/924306810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f06600103c0 con 0x7f0670103e20 2024-02-16T22:47:42.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.795+0000 7f0677eab700 1 -- 172.21.15.64:0/924306810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 msgr2=0x7f0670104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.795+0000 7f0677eab700 1 --2- 172.21.15.64:0/924306810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670104230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f06600098e0 tx=0x7f0660009250 comp rx=0 tx=0).stop 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.795+0000 7f0677eab700 1 -- 172.21.15.64:0/924306810 shutdown_connections 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.795+0000 7f0677eab700 1 --2- 172.21.15.64:0/924306810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670104230 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.795+0000 7f0677eab700 1 -- 172.21.15.64:0/924306810 >> 172.21.15.64:0/924306810 conn(0x7f06700fc580 msgr2=0x7f06700fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.795+0000 7f0677eab700 1 -- 172.21.15.64:0/924306810 shutdown_connections 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.795+0000 7f0677eab700 1 -- 172.21.15.64:0/924306810 wait complete. 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0677eab700 1 Processor -- start 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0677eab700 1 -- start start 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0677eab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670111850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0677eab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0670111d90 con 0x7f0670103e20 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0675c47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670111850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0675c47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670111850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53656/0 (socket says 172.21.15.64:53656) 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0675c47700 1 -- 172.21.15.64:0/3051703420 learned_addr learned my addr 172.21.15.64:0/3051703420 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0675c47700 1 -- 172.21.15.64:0/3051703420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06600094f0 con 0x7f0670103e20 2024-02-16T22:47:42.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f0675c47700 1 --2- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670111850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0660006b00 tx=0x7f0660009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f06667fc700 1 -- 172.21.15.64:0/3051703420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f06600109a0 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.796+0000 7f06667fc700 1 -- 172.21.15.64:0/3051703420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f066001a430 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.797+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0670111f90 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.797+0000 7f06667fc700 1 -- 172.21.15.64:0/3051703420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f06600245b0 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.797+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0670112c30 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.798+0000 7f06667fc700 1 -- 172.21.15.64:0/3051703420 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f066001aaa0 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.798+0000 7f06667fc700 1 --2- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f065c036790 0x7f065c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.798+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067005f5c0 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.798+0000 7f06667fc700 1 -- 172.21.15.64:0/3051703420 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f066004a680 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.798+0000 7f0675446700 1 --2- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f065c036790 0x7f065c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.798+0000 7f0675446700 1 --2- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f065c036790 0x7f065c038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f066c009990 tx=0x7f066c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:41.803+0000 7f06667fc700 1 -- 172.21.15.64:0/3051703420 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0660029070 con 0x7f0670103e20 2024-02-16T22:47:42.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.007+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f06701013e0 con 0x7f065c036790 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.011+0000 7f06667fc700 1 -- 172.21.15.64:0/3051703420 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f06701013e0 con 0x7f065c036790 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f065c036790 msgr2=0x7f065c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 --2- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f065c036790 0x7f065c038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f066c009990 tx=0x7f066c006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 msgr2=0x7f0670111850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 --2- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670111850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0660006b00 tx=0x7f0660009250 comp rx=0 tx=0).stop 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 shutdown_connections 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 --2- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f065c036790 0x7f065c038c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 --2- 172.21.15.64:0/3051703420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0670103e20 0x7f0670111850 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 >> 172.21.15.64:0/3051703420 conn(0x7f06700fc580 msgr2=0x7f06700fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 shutdown_connections 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.013+0000 7f0677eab700 1 -- 172.21.15.64:0/3051703420 wait complete. 2024-02-16T22:47:42.577 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying alertmanager service with default placement... 2024-02-16T22:47:43.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:43 smithi064 ceph-mon[29226]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:43.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:43 smithi064 ceph-mon[29226]: Saving service node-exporter spec with placement * 2024-02-16T22:47:43.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:43 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:43.747 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-02-16T22:47:43.747 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.966+0000 7f0a182fa700 1 Processor -- start 2024-02-16T22:47:43.747 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.966+0000 7f0a182fa700 1 -- start start 2024-02-16T22:47:43.747 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a182fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:43.747 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a182fa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101030c0 con 0x7f0a10102770 2024-02-16T22:47:43.747 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a16096700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a16096700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53670/0 (socket says 172.21.15.64:53670) 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a16096700 1 -- 172.21.15.64:0/4111667151 learned_addr learned my addr 172.21.15.64:0/4111667151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a16096700 1 -- 172.21.15.64:0/4111667151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a10103200 con 0x7f0a10102770 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a16096700 1 --2- 172.21.15.64:0/4111667151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10102b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0098e0 tx=0x7f0a0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b94ac3fad1fff73 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a15094700 1 -- 172.21.15.64:0/4111667151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a0c00b860 con 0x7f0a10102770 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a15094700 1 -- 172.21.15.64:0/4111667151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a0c00be90 con 0x7f0a10102770 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.967+0000 7f0a15094700 1 -- 172.21.15.64:0/4111667151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a0c0103c0 con 0x7f0a10102770 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 -- 172.21.15.64:0/4111667151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 msgr2=0x7f0a10102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 --2- 172.21.15.64:0/4111667151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10102b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0098e0 tx=0x7f0a0c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 -- 172.21.15.64:0/4111667151 shutdown_connections 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 --2- 172.21.15.64:0/4111667151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10102b80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 -- 172.21.15.64:0/4111667151 >> 172.21.15.64:0/4111667151 conn(0x7f0a100fded0 msgr2=0x7f0a10100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 -- 172.21.15.64:0/4111667151 shutdown_connections 2024-02-16T22:47:43.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 -- 172.21.15.64:0/4111667151 wait complete. 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 Processor -- start 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.968+0000 7f0a182fa700 1 -- start start 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a182fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a182fa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10077190 con 0x7f0a10102770 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a16096700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a16096700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53686/0 (socket says 172.21.15.64:53686) 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a16096700 1 -- 172.21.15.64:0/2937044775 learned_addr learned my addr 172.21.15.64:0/2937044775 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a16096700 1 -- 172.21.15.64:0/2937044775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0094f0 con 0x7f0a10102770 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a16096700 1 --2- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10078d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c011680 tx=0x7f0a0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a02ffd700 1 -- 172.21.15.64:0/2937044775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a0c010890 con 0x7f0a10102770 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a02ffd700 1 -- 172.21.15.64:0/2937044775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a0c010ec0 con 0x7f0a10102770 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a10077390 con 0x7f0a10102770 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a02ffd700 1 -- 172.21.15.64:0/2937044775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a0c0243f0 con 0x7f0a10102770 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.969+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a10077820 con 0x7f0a10102770 2024-02-16T22:47:43.749 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.970+0000 7f0a02ffd700 1 -- 172.21.15.64:0/2937044775 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f0a0c0109f0 con 0x7f0a10102770 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.970+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1004bed0 con 0x7f0a10102770 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.970+0000 7f0a02ffd700 1 --2- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f09fc0367e0 0x7f09fc038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.970+0000 7f0a02ffd700 1 -- 172.21.15.64:0/2937044775 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0a0c04a210 con 0x7f0a10102770 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.970+0000 7f0a15895700 1 --2- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f09fc0367e0 0x7f09fc038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.971+0000 7f0a15895700 1 --2- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f09fc0367e0 0x7f09fc038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009990 tx=0x7f0a04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:42.975+0000 7f0a02ffd700 1 -- 172.21.15.64:0/2937044775 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0a0c02b4d0 con 0x7f0a10102770 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.176+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a10078060 con 0x7f09fc0367e0 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.183+0000 7f0a02ffd700 1 -- 172.21.15.64:0/2937044775 <== mgr.14120 v2:172.21.15.64:6800/3482665154 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f0a10078060 con 0x7f09fc0367e0 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f09fc0367e0 msgr2=0x7f09fc038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 --2- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f09fc0367e0 0x7f09fc038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009990 tx=0x7f0a04006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 msgr2=0x7f0a10078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 --2- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10078d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c011680 tx=0x7f0a0c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 shutdown_connections 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 --2- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f09fc0367e0 0x7f09fc038c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 --2- 172.21.15.64:0/2937044775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a10102770 0x7f0a10078d10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:43.750 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 >> 172.21.15.64:0/2937044775 conn(0x7f0a100fded0 msgr2=0x7f0a100ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:43.751 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 shutdown_connections 2024-02-16T22:47:43.751 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:43.185+0000 7f0a182fa700 1 -- 172.21.15.64:0/2937044775 wait complete. 2024-02-16T22:47:44.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:44 smithi064 ceph-mon[29226]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:44.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:44 smithi064 ceph-mon[29226]: Saving service alertmanager spec with placement count:1 2024-02-16T22:47:44.360 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:44 smithi064 ceph-mon[29226]: from='mgr.14120 172.21.15.64:0/583626597' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.161+0000 7f800fa26700 1 Processor -- start 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.161+0000 7f800fa26700 1 -- start start 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.161+0000 7f800fa26700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f8008101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.161+0000 7f800fa26700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008101fc0 con 0x7f8008101670 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.161+0000 7f800d7c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f8008101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.161+0000 7f800d7c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f8008101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53690/0 (socket says 172.21.15.64:53690) 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.161+0000 7f800d7c2700 1 -- 172.21.15.64:0/4067181374 learned_addr learned my addr 172.21.15.64:0/4067181374 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.161+0000 7f800d7c2700 1 -- 172.21.15.64:0/4067181374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8008102100 con 0x7f8008101670 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f800d7c2700 1 --2- 172.21.15.64:0/4067181374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f8008101a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f800400bb80 tx=0x7f800400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fb052238329a4c server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f7ffffff700 1 -- 172.21.15.64:0/4067181374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f800400dd00 con 0x7f8008101670 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f7ffffff700 1 -- 172.21.15.64:0/4067181374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f800400de60 con 0x7f8008101670 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f7ffffff700 1 -- 172.21.15.64:0/4067181374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8004018920 con 0x7f8008101670 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f800fa26700 1 -- 172.21.15.64:0/4067181374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 msgr2=0x7f8008101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f800fa26700 1 --2- 172.21.15.64:0/4067181374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f8008101a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f800400bb80 tx=0x7f800400b4f0 comp rx=0 tx=0).stop 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f800fa26700 1 -- 172.21.15.64:0/4067181374 shutdown_connections 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f800fa26700 1 --2- 172.21.15.64:0/4067181374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f8008101a80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.162+0000 7f800fa26700 1 -- 172.21.15.64:0/4067181374 >> 172.21.15.64:0/4067181374 conn(0x7f80080fcdd0 msgr2=0x7f80080ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800fa26700 1 -- 172.21.15.64:0/4067181374 shutdown_connections 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800fa26700 1 -- 172.21.15.64:0/4067181374 wait complete. 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800fa26700 1 Processor -- start 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800fa26700 1 -- start start 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800fa26700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f800810f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800fa26700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800810f7f0 con 0x7f8008101670 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800d7c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f800810f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800d7c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f800810f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53698/0 (socket says 172.21.15.64:53698) 2024-02-16T22:47:44.921 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.163+0000 7f800d7c2700 1 -- 172.21.15.64:0/2657877551 learned_addr learned my addr 172.21.15.64:0/2657877551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.164+0000 7f800d7c2700 1 -- 172.21.15.64:0/2657877551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800400b790 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.164+0000 7f800d7c2700 1 --2- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f800810f2b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80040106d0 tx=0x7f800400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.164+0000 7f7ffdffb700 1 -- 172.21.15.64:0/2657877551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8004010c60 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.164+0000 7f7ffdffb700 1 -- 172.21.15.64:0/2657877551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f800401d050 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.164+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800810f9f0 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.164+0000 7f7ffdffb700 1 -- 172.21.15.64:0/2657877551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8004022620 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.164+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8008110690 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.165+0000 7f7ffdffb700 1 -- 172.21.15.64:0/2657877551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8004022850 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.165+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800805f5c0 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.165+0000 7f7ffdffb700 1 --2- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f7ff40367e0 0x7f7ff4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.165+0000 7f7ffdffb700 1 -- 172.21.15.64:0/2657877551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8004014070 con 0x7f8008101670 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.165+0000 7f800cfc1700 1 --2- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f7ff40367e0 0x7f7ff4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.166+0000 7f800cfc1700 1 --2- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f7ff40367e0 0x7f7ff4038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009990 tx=0x7f7ff8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:44.922 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.170+0000 7f7ffdffb700 1 -- 172.21.15.64:0/2657877551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8004021330 con 0x7f8008101670 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.358+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f800804bed0 con 0x7f8008101670 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.362+0000 7f7ffdffb700 1 -- 172.21.15.64:0/2657877551 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8004021330 con 0x7f8008101670 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f7ff40367e0 msgr2=0x7f7ff4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 --2- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f7ff40367e0 0x7f7ff4038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009990 tx=0x7f7ff8006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 msgr2=0x7f800810f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 --2- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f800810f2b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80040106d0 tx=0x7f800400b4f0 comp rx=0 tx=0).stop 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 shutdown_connections 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 --2- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f7ff40367e0 0x7f7ff4038c90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 --2- 172.21.15.64:0/2657877551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8008101670 0x7f800810f2b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 >> 172.21.15.64:0/2657877551 conn(0x7f80080fcdd0 msgr2=0x7f80081063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 shutdown_connections 2024-02-16T22:47:44.923 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:44.364+0000 7f800fa26700 1 -- 172.21.15.64:0/2657877551 wait complete. 2024-02-16T22:47:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:45 smithi064 ceph-mon[29226]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:47:45.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:45 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2657877551' entity='client.admin' 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.355+0000 7f652a053700 1 Processor -- start 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.355+0000 7f652a053700 1 -- start start 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.356+0000 7f652a053700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f6524101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.356+0000 7f652a053700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6524101fc0 con 0x7f6524101670 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.356+0000 7f65237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f6524101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.356+0000 7f65237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f6524101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53706/0 (socket says 172.21.15.64:53706) 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.356+0000 7f65237fe700 1 -- 172.21.15.64:0/3793647328 learned_addr learned my addr 172.21.15.64:0/3793647328 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.356+0000 7f65237fe700 1 -- 172.21.15.64:0/3793647328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6524102100 con 0x7f6524101670 2024-02-16T22:47:46.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.356+0000 7f65237fe700 1 --2- 172.21.15.64:0/3793647328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f6524101a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f650c0098e0 tx=0x7f650c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5eabbaf13e88b8f server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.356+0000 7f65227fc700 1 -- 172.21.15.64:0/3793647328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f650c00b860 con 0x7f6524101670 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f65227fc700 1 -- 172.21.15.64:0/3793647328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f650c00be90 con 0x7f6524101670 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f65227fc700 1 -- 172.21.15.64:0/3793647328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f650c0103c0 con 0x7f6524101670 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f652a053700 1 -- 172.21.15.64:0/3793647328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 msgr2=0x7f6524101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f652a053700 1 --2- 172.21.15.64:0/3793647328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f6524101a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f650c0098e0 tx=0x7f650c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f652a053700 1 -- 172.21.15.64:0/3793647328 shutdown_connections 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f652a053700 1 --2- 172.21.15.64:0/3793647328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f6524101a80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f652a053700 1 -- 172.21.15.64:0/3793647328 >> 172.21.15.64:0/3793647328 conn(0x7f65240fcdd0 msgr2=0x7f65240ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f652a053700 1 -- 172.21.15.64:0/3793647328 shutdown_connections 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.357+0000 7f652a053700 1 -- 172.21.15.64:0/3793647328 wait complete. 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.358+0000 7f652a053700 1 Processor -- start 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.358+0000 7f652a053700 1 -- start start 2024-02-16T22:47:46.195 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.358+0000 7f652a053700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f652410f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.358+0000 7f652a053700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652410fbb0 con 0x7f6524101670 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.358+0000 7f65237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f652410f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.358+0000 7f65237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f652410f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53710/0 (socket says 172.21.15.64:53710) 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.358+0000 7f65237fe700 1 -- 172.21.15.64:0/1124311745 learned_addr learned my addr 172.21.15.64:0/1124311745 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.358+0000 7f65237fe700 1 -- 172.21.15.64:0/1124311745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c0094f0 con 0x7f6524101670 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.359+0000 7f65237fe700 1 --2- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f652410f670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f650c006b00 tx=0x7f650c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.359+0000 7f651bfff700 1 -- 172.21.15.64:0/1124311745 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f650c0109a0 con 0x7f6524101670 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.359+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652410fdb0 con 0x7f6524101670 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.359+0000 7f651bfff700 1 -- 172.21.15.64:0/1124311745 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f650c01a430 con 0x7f6524101670 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.359+0000 7f651bfff700 1 -- 172.21.15.64:0/1124311745 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f650c0245b0 con 0x7f6524101670 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.359+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6524110a50 con 0x7f6524101670 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.360+0000 7f651bfff700 1 -- 172.21.15.64:0/1124311745 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f650c0247f0 con 0x7f6524101670 2024-02-16T22:47:46.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.360+0000 7f651bfff700 1 --2- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f65100389d0 0x7f651003ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.360+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652405f5c0 con 0x7f6524101670 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.360+0000 7f651bfff700 1 -- 172.21.15.64:0/1124311745 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f650c04a790 con 0x7f6524101670 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.360+0000 7f6522ffd700 1 --2- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f65100389d0 0x7f651003ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.360+0000 7f6522ffd700 1 --2- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f65100389d0 0x7f651003ae80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6514009990 tx=0x7f6514006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.367+0000 7f651bfff700 1 -- 172.21.15.64:0/1124311745 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f650c0189a0 con 0x7f6524101670 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.558+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f6524103620 con 0x7f6524101670 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.562+0000 7f651bfff700 1 -- 172.21.15.64:0/1124311745 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f650c018350 con 0x7f6524101670 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.564+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f65100389d0 msgr2=0x7f651003ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.564+0000 7f652a053700 1 --2- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f65100389d0 0x7f651003ae80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6514009990 tx=0x7f6514006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.564+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 msgr2=0x7f652410f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.564+0000 7f652a053700 1 --2- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f652410f670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f650c006b00 tx=0x7f650c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:46.197 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.564+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 shutdown_connections 2024-02-16T22:47:46.198 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.564+0000 7f652a053700 1 --2- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f65100389d0 0x7f651003ae80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:46.198 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.564+0000 7f652a053700 1 --2- 172.21.15.64:0/1124311745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6524101670 0x7f652410f670 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:46.198 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.564+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 >> 172.21.15.64:0/1124311745 conn(0x7f65240fcdd0 msgr2=0x7f65241063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:46.198 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.565+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 shutdown_connections 2024-02-16T22:47:46.198 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:45.565+0000 7f652a053700 1 -- 172.21.15.64:0/1124311745 wait complete. 2024-02-16T22:47:46.198 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling the dashboard module... 2024-02-16T22:47:46.849 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:46 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1124311745' entity='client.admin' 2024-02-16T22:47:46.850 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:46 smithi064 ceph-mon[29226]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:47:47.832 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:47 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1484395494' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-16T22:47:48.083 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.591+0000 7f2a68821700 1 Processor -- start 2024-02-16T22:47:48.083 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.591+0000 7f2a68821700 1 -- start start 2024-02-16T22:47:48.083 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.592+0000 7f2a68821700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a64105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:48.083 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.592+0000 7f2a68821700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a64105960 con 0x7f2a64076b80 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.592+0000 7f2a6259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a64105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.592+0000 7f2a6259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a64105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53724/0 (socket says 172.21.15.64:53724) 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.592+0000 7f2a6259c700 1 -- 172.21.15.64:0/4013695663 learned_addr learned my addr 172.21.15.64:0/4013695663 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.592+0000 7f2a6259c700 1 -- 172.21.15.64:0/4013695663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a64105aa0 con 0x7f2a64076b80 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.592+0000 7f2a6259c700 1 --2- 172.21.15.64:0/4013695663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a64105420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a540098e0 tx=0x7f2a54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324ce5e363de9528 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.592+0000 7f2a6159a700 1 -- 172.21.15.64:0/4013695663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a5400b860 con 0x7f2a64076b80 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a6159a700 1 -- 172.21.15.64:0/4013695663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a5400be90 con 0x7f2a64076b80 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a6159a700 1 -- 172.21.15.64:0/4013695663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a540103c0 con 0x7f2a64076b80 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a68821700 1 -- 172.21.15.64:0/4013695663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 msgr2=0x7f2a64105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a68821700 1 --2- 172.21.15.64:0/4013695663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a64105420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a540098e0 tx=0x7f2a54009250 comp rx=0 tx=0).stop 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a68821700 1 -- 172.21.15.64:0/4013695663 shutdown_connections 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a68821700 1 --2- 172.21.15.64:0/4013695663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a64105420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:48.084 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a68821700 1 -- 172.21.15.64:0/4013695663 >> 172.21.15.64:0/4013695663 conn(0x7f2a640fcdd0 msgr2=0x7f2a640ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a68821700 1 -- 172.21.15.64:0/4013695663 shutdown_connections 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.593+0000 7f2a68821700 1 -- 172.21.15.64:0/4013695663 wait complete. 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a68821700 1 Processor -- start 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a68821700 1 -- start start 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a68821700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a641118a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a68821700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a64111de0 con 0x7f2a64076b80 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a6259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a641118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a6259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a641118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53732/0 (socket says 172.21.15.64:53732) 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a6259c700 1 -- 172.21.15.64:0/1484395494 learned_addr learned my addr 172.21.15.64:0/1484395494 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a6259c700 1 -- 172.21.15.64:0/1484395494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540094f0 con 0x7f2a64076b80 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.594+0000 7f2a6259c700 1 --2- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a641118a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a540090b0 tx=0x7f2a54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.595+0000 7f2a52ffd700 1 -- 172.21.15.64:0/1484395494 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a540118f0 con 0x7f2a64076b80 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.595+0000 7f2a52ffd700 1 -- 172.21.15.64:0/1484395494 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a54010d10 con 0x7f2a64076b80 2024-02-16T22:47:48.085 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.595+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a64111fe0 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.595+0000 7f2a52ffd700 1 -- 172.21.15.64:0/1484395494 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a5401aea0 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.595+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a64112c80 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.595+0000 7f2a52ffd700 1 -- 172.21.15.64:0/1484395494 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2a54010840 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.595+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6405f5c0 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.596+0000 7f2a52ffd700 1 --2- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f2a4c036740 0x7f2a4c038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.596+0000 7f2a52ffd700 1 -- 172.21.15.64:0/1484395494 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2a5404a600 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.596+0000 7f2a61d9b700 1 --2- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f2a4c036740 0x7f2a4c038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.596+0000 7f2a61d9b700 1 --2- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f2a4c036740 0x7f2a4c038bf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009990 tx=0x7f2a58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.600+0000 7f2a52ffd700 1 -- 172.21.15.64:0/1484395494 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2a54022ae0 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:46.848+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f2a6404bed0 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.570+0000 7f2a52ffd700 1 -- 172.21.15.64:0/1484395494 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f2a5400fdc0 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.571+0000 7f2a52ffd700 1 -- 172.21.15.64:0/1484395494 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2a54022ae0 con 0x7f2a64076b80 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f2a4c036740 msgr2=0x7f2a4c038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:48.086 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 --2- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f2a4c036740 0x7f2a4c038bf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009990 tx=0x7f2a58006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 msgr2=0x7f2a641118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 --2- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a641118a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a540090b0 tx=0x7f2a54009250 comp rx=0 tx=0).stop 2024-02-16T22:47:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 shutdown_connections 2024-02-16T22:47:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 --2- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f2a4c036740 0x7f2a4c038bf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 --2- 172.21.15.64:0/1484395494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a64076b80 0x7f2a641118a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 >> 172.21.15.64:0/1484395494 conn(0x7f2a640fcdd0 msgr2=0x7f2a640ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.573+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 shutdown_connections 2024-02-16T22:47:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:47.574+0000 7f2a68821700 1 -- 172.21.15.64:0/1484395494 wait complete. 2024-02-16T22:47:48.730 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:48 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1484395494' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-16T22:47:48.730 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:48 smithi064 ceph-mon[29226]: mgrmap e9: smithi064.kubmlk(active, since 23s) 2024-02-16T22:47:49.314 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "smithi064.kubmlk", 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f376e70b700 1 Processor -- start 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f376e70b700 1 -- start start 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f376e70b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768104190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f376e70b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37681046d0 con 0x7f3768103d80 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f3767fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f3767fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768104190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53750/0 (socket says 172.21.15.64:53750) 2024-02-16T22:47:49.315 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f3767fff700 1 -- 172.21.15.64:0/3932999284 learned_addr learned my addr 172.21.15.64:0/3932999284 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f3767fff700 1 -- 172.21.15.64:0/3932999284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3768104810 con 0x7f3768103d80 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.524+0000 7f3767fff700 1 --2- 172.21.15.64:0/3932999284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768104190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f37580098e0 tx=0x7f3758009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e35a2710da32514 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f3766ffd700 1 -- 172.21.15.64:0/3932999284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f375800b860 con 0x7f3768103d80 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f3766ffd700 1 -- 172.21.15.64:0/3932999284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f375800be90 con 0x7f3768103d80 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f3766ffd700 1 -- 172.21.15.64:0/3932999284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37580103c0 con 0x7f3768103d80 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f376e70b700 1 -- 172.21.15.64:0/3932999284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 msgr2=0x7f3768104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f376e70b700 1 --2- 172.21.15.64:0/3932999284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768104190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f37580098e0 tx=0x7f3758009250 comp rx=0 tx=0).stop 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f376e70b700 1 -- 172.21.15.64:0/3932999284 shutdown_connections 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f376e70b700 1 --2- 172.21.15.64:0/3932999284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768104190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f376e70b700 1 -- 172.21.15.64:0/3932999284 >> 172.21.15.64:0/3932999284 conn(0x7f37680fc580 msgr2=0x7f37680fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f376e70b700 1 -- 172.21.15.64:0/3932999284 shutdown_connections 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.525+0000 7f376e70b700 1 -- 172.21.15.64:0/3932999284 wait complete. 2024-02-16T22:47:49.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.526+0000 7f376e70b700 1 Processor -- start 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.526+0000 7f376e70b700 1 -- start start 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.526+0000 7f376e70b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768111870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.526+0000 7f376e70b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3768111db0 con 0x7f3768103d80 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.526+0000 7f3767fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768111870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.526+0000 7f3767fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768111870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53762/0 (socket says 172.21.15.64:53762) 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.526+0000 7f3767fff700 1 -- 172.21.15.64:0/2388841759 learned_addr learned my addr 172.21.15.64:0/2388841759 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.527+0000 7f3767fff700 1 -- 172.21.15.64:0/2388841759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37580094f0 con 0x7f3768103d80 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.527+0000 7f3767fff700 1 --2- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768111870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3758006b00 tx=0x7f3758009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.527+0000 7f3764ff9700 1 -- 172.21.15.64:0/2388841759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37580109a0 con 0x7f3768103d80 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.527+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3768111fb0 con 0x7f3768103d80 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.527+0000 7f3764ff9700 1 -- 172.21.15.64:0/2388841759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f375801a430 con 0x7f3768103d80 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.527+0000 7f3764ff9700 1 -- 172.21.15.64:0/2388841759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37580245b0 con 0x7f3768103d80 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.527+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37681123c0 con 0x7f3768103d80 2024-02-16T22:47:49.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.528+0000 7f3764ff9700 1 -- 172.21.15.64:0/2388841759 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f3758024830 con 0x7f3768103d80 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.528+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376804bed0 con 0x7f3768103d80 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.528+0000 7f3764ff9700 1 --2- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f375003abf0 0x7f375003d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.528+0000 7f3764ff9700 1 -- 172.21.15.64:0/2388841759 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f375804a810 con 0x7f3768103d80 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.528+0000 7f37677fe700 1 -- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f375003abf0 msgr2=0x7f375003d0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3482665154 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.528+0000 7f37677fe700 1 --2- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f375003abf0 0x7f375003d0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.533+0000 7f3764ff9700 1 -- 172.21.15.64:0/2388841759 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f37580189a0 con 0x7f3768103d80 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.729+0000 7f37677fe700 1 -- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f375003abf0 msgr2=0x7f375003d0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3482665154 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.729+0000 7f37677fe700 1 --2- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f375003abf0 0x7f375003d0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.779+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3768064ad0 con 0x7f3768103d80 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.780+0000 7f3764ff9700 1 -- 172.21.15.64:0/2388841759 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f375801a5a0 con 0x7f3768103d80 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.781+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f375003abf0 msgr2=0x7f375003d0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 --2- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f375003abf0 0x7f375003d0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 msgr2=0x7f3768111870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:49.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 --2- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768111870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3758006b00 tx=0x7f3758009250 comp rx=0 tx=0).stop 2024-02-16T22:47:49.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 shutdown_connections 2024-02-16T22:47:49.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 --2- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f375003abf0 0x7f375003d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:49.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 --2- 172.21.15.64:0/2388841759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3768103d80 0x7f3768111870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:49.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 >> 172.21.15.64:0/2388841759 conn(0x7f37680fc580 msgr2=0x7f37680fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:49.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 shutdown_connections 2024-02-16T22:47:49.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:48.782+0000 7f376e70b700 1 -- 172.21.15.64:0/2388841759 wait complete. 2024-02-16T22:47:49.319 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-02-16T22:47:49.319 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 9... 2024-02-16T22:47:49.839 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:49 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2388841759' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-16T22:47:54.586 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: Active manager daemon smithi064.kubmlk restarted 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: Activating manager daemon smithi064.kubmlk 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: osdmap e3: 0 total, 0 up, 0 in 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: mgrmap e10: smithi064.kubmlk(active, starting, since 0.00555494s) 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: Manager daemon smithi064.kubmlk is now available 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:47:54.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:54 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:47:55.585 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:55 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:47:55.585 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:55 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:47:55.585 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:55 smithi064 ceph-mon[29226]: mgrmap e11: smithi064.kubmlk(active, since 1.00598s) 2024-02-16T22:47:55.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:47:55.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-16T22:47:55.975 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.705+0000 7f5261a05700 1 Processor -- start 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.705+0000 7f5261a05700 1 -- start start 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.705+0000 7f5261a05700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c0771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.705+0000 7f5261a05700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0790f0 con 0x7f525c078ce0 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.706+0000 7f525affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c0771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.706+0000 7f525affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c0771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53776/0 (socket says 172.21.15.64:53776) 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.706+0000 7f525affd700 1 -- 172.21.15.64:0/1032195252 learned_addr learned my addr 172.21.15.64:0/1032195252 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.706+0000 7f525affd700 1 -- 172.21.15.64:0/1032195252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c077730 con 0x7f525c078ce0 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.706+0000 7f525affd700 1 --2- 172.21.15.64:0/1032195252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c0771f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f524c0098e0 tx=0x7f524c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3852f9563d941394 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.706+0000 7f5259ffb700 1 -- 172.21.15.64:0/1032195252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f524c00b860 con 0x7f525c078ce0 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.706+0000 7f5259ffb700 1 -- 172.21.15.64:0/1032195252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f524c00be90 con 0x7f525c078ce0 2024-02-16T22:47:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.706+0000 7f5259ffb700 1 -- 172.21.15.64:0/1032195252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f524c0103c0 con 0x7f525c078ce0 2024-02-16T22:47:55.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.707+0000 7f5261a05700 1 -- 172.21.15.64:0/1032195252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 msgr2=0x7f525c0771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:55.977 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.707+0000 7f5261a05700 1 --2- 172.21.15.64:0/1032195252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c0771f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f524c0098e0 tx=0x7f524c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:55.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.707+0000 7f5261a05700 1 -- 172.21.15.64:0/1032195252 shutdown_connections 2024-02-16T22:47:55.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.707+0000 7f5261a05700 1 --2- 172.21.15.64:0/1032195252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c0771f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:55.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.707+0000 7f5261a05700 1 -- 172.21.15.64:0/1032195252 >> 172.21.15.64:0/1032195252 conn(0x7f525c0fded0 msgr2=0x7f525c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:55.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.707+0000 7f5261a05700 1 -- 172.21.15.64:0/1032195252 shutdown_connections 2024-02-16T22:47:55.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.707+0000 7f5261a05700 1 -- 172.21.15.64:0/1032195252 wait complete. 2024-02-16T22:47:55.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.708+0000 7f5261a05700 1 Processor -- start 2024-02-16T22:47:55.979 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.708+0000 7f5261a05700 1 -- start start 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.708+0000 7f5261a05700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c1137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.708+0000 7f5261a05700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c113d20 con 0x7f525c078ce0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.708+0000 7f525affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c1137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.708+0000 7f525affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c1137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53786/0 (socket says 172.21.15.64:53786) 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.708+0000 7f525affd700 1 -- 172.21.15.64:0/3005597274 learned_addr learned my addr 172.21.15.64:0/3005597274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.709+0000 7f525affd700 1 -- 172.21.15.64:0/3005597274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0094f0 con 0x7f525c078ce0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.709+0000 7f525affd700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c1137e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f524c0094c0 tx=0x7f524c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.709+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f524c010890 con 0x7f525c078ce0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.709+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f524c010ec0 con 0x7f525c078ce0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.709+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c113f20 con 0x7f525c078ce0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.709+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f524c0243f0 con 0x7f525c078ce0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.709+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c114bc0 con 0x7f525c078ce0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.710+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f524c02d3f0 con 0x7f525c078ce0 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.710+0000 7f52437fe700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 0x7f5244038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:55.980 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.710+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 --> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52440393a0 con 0x7f52440367e0 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.710+0000 7f525a7fc700 1 -- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 msgr2=0x7f5244038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3482665154 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.710+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f524c04a4a0 con 0x7f525c078ce0 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.710+0000 7f525a7fc700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 0x7f5244038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.911+0000 7f525a7fc700 1 -- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 msgr2=0x7f5244038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3482665154 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:49.911+0000 7f525a7fc700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 0x7f5244038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:50.312+0000 7f525a7fc700 1 -- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 msgr2=0x7f5244038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3482665154 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:50.312+0000 7f525a7fc700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 0x7f5244038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:51.113+0000 7f525a7fc700 1 -- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 msgr2=0x7f5244038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3482665154 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:51.113+0000 7f525a7fc700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 0x7f5244038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:52.714+0000 7f525a7fc700 1 -- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 msgr2=0x7f5244038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3482665154 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:52.714+0000 7f525a7fc700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 0x7f5244038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:54.265+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f524c02d740 con 0x7f525c078ce0 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:54.265+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 msgr2=0x7f5244038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:54.265+0000 7f52437fe700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 0x7f5244038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:55.981 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.265+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f524c0109f0 con 0x7f525c078ce0 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.265+0000 7f52437fe700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f524403a070 0x7f524403c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.265+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 --> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52440393a0 con 0x7f524403a070 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.265+0000 7f525a7fc700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f524403a070 0x7f524403c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.265+0000 7f525a7fc700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f524403a070 0x7f524403c450 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f52500039d0 tx=0x7f52500092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.266+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mgr.14162 v2:172.21.15.64:6800/600121841 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f52440393a0 con 0x7f524403a070 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.272+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 --> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f525c1143b0 con 0x7f524403a070 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.272+0000 7f52437fe700 1 -- 172.21.15.64:0/3005597274 <== mgr.14162 v2:172.21.15.64:6800/600121841 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f525c1143b0 con 0x7f524403a070 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f524403a070 msgr2=0x7f524403c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f524403a070 0x7f524403c450 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f52500039d0 tx=0x7f52500092f0 comp rx=0 tx=0).stop 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 msgr2=0x7f525c1137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c1137e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f524c0094c0 tx=0x7f524c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 shutdown_connections 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f524403a070 0x7f524403c450 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525c078ce0 0x7f525c1137e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:55.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 --2- 172.21.15.64:0/3005597274 >> [v2:172.21.15.64:6800/3482665154,v1:172.21.15.64:6801/3482665154] conn(0x7f52440367e0 0x7f5244038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:55.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 >> 172.21.15.64:0/3005597274 conn(0x7f525c0fded0 msgr2=0x7f525c0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:55.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 shutdown_connections 2024-02-16T22:47:55.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:55.273+0000 7f5261a05700 1 -- 172.21.15.64:0/3005597274 wait complete. 2024-02-16T22:47:55.983 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 9 is available 2024-02-16T22:47:55.983 INFO:teuthology.orchestra.run.smithi064.stdout:Generating a dashboard self-signed certificate... 2024-02-16T22:47:56.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:56 smithi064 ceph-mon[29226]: [16/Feb/2024:22:47:55] ENGINE Bus STARTING 2024-02-16T22:47:56.588 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:56 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:57.405 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-16T22:47:57.405 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.375+0000 7f345ebe9700 1 Processor -- start 2024-02-16T22:47:57.405 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.375+0000 7f345ebe9700 1 -- start start 2024-02-16T22:47:57.405 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.375+0000 7f345ebe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458102b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:57.405 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.375+0000 7f345ebe9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581030d0 con 0x7f3458102780 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.375+0000 7f345c985700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.375+0000 7f345c985700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33098/0 (socket says 172.21.15.64:33098) 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.375+0000 7f345c985700 1 -- 172.21.15.64:0/3251417737 learned_addr learned my addr 172.21.15.64:0/3251417737 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.376+0000 7f345c985700 1 -- 172.21.15.64:0/3251417737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458103210 con 0x7f3458102780 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.376+0000 7f345c985700 1 --2- 172.21.15.64:0/3251417737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458102b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f344c0098e0 tx=0x7f344c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d954bbb5301c49 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.376+0000 7f34577fe700 1 -- 172.21.15.64:0/3251417737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f344c00b860 con 0x7f3458102780 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.376+0000 7f34577fe700 1 -- 172.21.15.64:0/3251417737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f344c00be90 con 0x7f3458102780 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.376+0000 7f34577fe700 1 -- 172.21.15.64:0/3251417737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f344c0103c0 con 0x7f3458102780 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.376+0000 7f345ebe9700 1 -- 172.21.15.64:0/3251417737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 msgr2=0x7f3458102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.376+0000 7f345ebe9700 1 --2- 172.21.15.64:0/3251417737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458102b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f344c0098e0 tx=0x7f344c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 -- 172.21.15.64:0/3251417737 shutdown_connections 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 --2- 172.21.15.64:0/3251417737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458102b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 -- 172.21.15.64:0/3251417737 >> 172.21.15.64:0/3251417737 conn(0x7f34580fdf00 msgr2=0x7f3458100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 -- 172.21.15.64:0/3251417737 shutdown_connections 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 -- 172.21.15.64:0/3251417737 wait complete. 2024-02-16T22:47:57.406 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 Processor -- start 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 -- start start 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.377+0000 7f345ebe9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3458111ad0 con 0x7f3458102780 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f345c985700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f345c985700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33112/0 (socket says 172.21.15.64:33112) 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f345c985700 1 -- 172.21.15.64:0/478671699 learned_addr learned my addr 172.21.15.64:0/478671699 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f345c985700 1 -- 172.21.15.64:0/478671699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c0094f0 con 0x7f3458102780 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f345c985700 1 --2- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458111590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f344c0090b0 tx=0x7f344c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f34557fa700 1 -- 172.21.15.64:0/478671699 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f344c010840 con 0x7f3458102780 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f34557fa700 1 -- 172.21.15.64:0/478671699 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f344c010e70 con 0x7f3458102780 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3458111cd0 con 0x7f3458102780 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f34557fa700 1 -- 172.21.15.64:0/478671699 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f344c024e40 con 0x7f3458102780 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.378+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3458112970 con 0x7f3458102780 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.379+0000 7f34557fa700 1 -- 172.21.15.64:0/478671699 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f344c01a4c0 con 0x7f3458102780 2024-02-16T22:47:57.407 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.379+0000 7f34557fa700 1 --2- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f34480366b0 0x7f3448038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.379+0000 7f34557fa700 1 -- 172.21.15.64:0/478671699 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f344c04a6e0 con 0x7f3458102780 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.379+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345805f5c0 con 0x7f3458102780 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.379+0000 7f3457fff700 1 --2- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f34480366b0 0x7f3448038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.380+0000 7f3457fff700 1 --2- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f34480366b0 0x7f3448038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3444009990 tx=0x7f3444006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.384+0000 7f34557fa700 1 -- 172.21.15.64:0/478671699 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f344c018940 con 0x7f3458102780 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.586+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 --> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f34581001b0 con 0x7f34480366b0 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.667+0000 7f34557fa700 1 -- 172.21.15.64:0/478671699 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f344c024430 con 0x7f3458102780 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.707+0000 7f34557fa700 1 -- 172.21.15.64:0/478671699 <== mgr.14162 v2:172.21.15.64:6800/600121841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f34581001b0 con 0x7f34480366b0 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f34480366b0 msgr2=0x7f3448038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 --2- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f34480366b0 0x7f3448038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3444009990 tx=0x7f3444006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 msgr2=0x7f3458111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 --2- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458111590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f344c0090b0 tx=0x7f344c009250 comp rx=0 tx=0).stop 2024-02-16T22:47:57.408 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 shutdown_connections 2024-02-16T22:47:57.409 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 --2- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f34480366b0 0x7f3448038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:57.409 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 --2- 172.21.15.64:0/478671699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3458102780 0x7f3458111590 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:57.409 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 >> 172.21.15.64:0/478671699 conn(0x7f34580fdf00 msgr2=0x7f34580ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:57.409 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 shutdown_connections 2024-02-16T22:47:57.409 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:56.709+0000 7f345ebe9700 1 -- 172.21.15.64:0/478671699 wait complete. 2024-02-16T22:47:57.409 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial admin user... 2024-02-16T22:47:57.751 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:57 smithi064 ceph-mon[29226]: [16/Feb/2024:22:47:55] ENGINE Serving on https://172.21.15.64:7150 2024-02-16T22:47:57.751 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:57 smithi064 ceph-mon[29226]: [16/Feb/2024:22:47:55] ENGINE Bus STARTED 2024-02-16T22:47:57.751 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:57 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:57.751 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:57 smithi064 ceph-mon[29226]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:57.751 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:57 smithi064 ceph-mon[29226]: mgrmap e12: smithi064.kubmlk(active, since 2s) 2024-02-16T22:47:57.751 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:57 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:57.751 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:57 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:58.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8a0mq9R40slZepHZuJclgOMGyMu8KW08F9b7vd9sr.xexl4LgQXfu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708123678, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-16T22:47:58.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.774+0000 7f7d66fc5700 1 Processor -- start 2024-02-16T22:47:58.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.774+0000 7f7d66fc5700 1 -- start start 2024-02-16T22:47:58.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.774+0000 7f7d66fc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60105360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.774+0000 7f7d66fc5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d601058a0 con 0x7f7d60076c40 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.774+0000 7f7d64d61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60105360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.774+0000 7f7d64d61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60105360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33122/0 (socket says 172.21.15.64:33122) 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.774+0000 7f7d64d61700 1 -- 172.21.15.64:0/3044491677 learned_addr learned my addr 172.21.15.64:0/3044491677 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.775+0000 7f7d64d61700 1 -- 172.21.15.64:0/3044491677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d601059e0 con 0x7f7d60076c40 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.775+0000 7f7d64d61700 1 --2- 172.21.15.64:0/3044491677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60105360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d540098e0 tx=0x7f7d54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394237eeefc4361e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.775+0000 7f7d5f7fe700 1 -- 172.21.15.64:0/3044491677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d5400b860 con 0x7f7d60076c40 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.775+0000 7f7d5f7fe700 1 -- 172.21.15.64:0/3044491677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d5400be90 con 0x7f7d60076c40 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.775+0000 7f7d5f7fe700 1 -- 172.21.15.64:0/3044491677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d540103c0 con 0x7f7d60076c40 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 -- 172.21.15.64:0/3044491677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 msgr2=0x7f7d60105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 --2- 172.21.15.64:0/3044491677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60105360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d540098e0 tx=0x7f7d54009250 comp rx=0 tx=0).stop 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 -- 172.21.15.64:0/3044491677 shutdown_connections 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 --2- 172.21.15.64:0/3044491677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60105360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 -- 172.21.15.64:0/3044491677 >> 172.21.15.64:0/3044491677 conn(0x7f7d600fcdd0 msgr2=0x7f7d600ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:58.939 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 -- 172.21.15.64:0/3044491677 shutdown_connections 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 -- 172.21.15.64:0/3044491677 wait complete. 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 Processor -- start 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 -- start start 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60111870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.776+0000 7f7d66fc5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60111db0 con 0x7f7d60076c40 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d64d61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60111870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d64d61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60111870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33128/0 (socket says 172.21.15.64:33128) 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d64d61700 1 -- 172.21.15.64:0/2690509392 learned_addr learned my addr 172.21.15.64:0/2690509392 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d64d61700 1 -- 172.21.15.64:0/2690509392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540094f0 con 0x7f7d60076c40 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d64d61700 1 --2- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60111870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d54003080 tx=0x7f7d54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d5d7fa700 1 -- 172.21.15.64:0/2690509392 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d54011920 con 0x7f7d60076c40 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d5d7fa700 1 -- 172.21.15.64:0/2690509392 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d54010d60 con 0x7f7d60076c40 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d60111fb0 con 0x7f7d60076c40 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d5d7fa700 1 -- 172.21.15.64:0/2690509392 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d540243f0 con 0x7f7d60076c40 2024-02-16T22:47:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.777+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d60112c50 con 0x7f7d60076c40 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.778+0000 7f7d5d7fa700 1 -- 172.21.15.64:0/2690509392 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f7d5402d3f0 con 0x7f7d60076c40 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.778+0000 7f7d5d7fa700 1 --2- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f7d500367e0 0x7f7d50038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.778+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6005f5c0 con 0x7f7d60076c40 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.778+0000 7f7d5d7fa700 1 -- 172.21.15.64:0/2690509392 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7d5404a450 con 0x7f7d60076c40 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.778+0000 7f7d5ffff700 1 --2- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f7d500367e0 0x7f7d50038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.779+0000 7f7d5ffff700 1 --2- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f7d500367e0 0x7f7d50038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009990 tx=0x7f7d4c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.783+0000 7f7d5d7fa700 1 -- 172.21.15.64:0/2690509392 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7d5402b8e0 con 0x7f7d60076c40 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:57.992+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 --> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7d6005f3e0 con 0x7f7d500367e0 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.239+0000 7f7d5d7fa700 1 -- 172.21.15.64:0/2690509392 <== mgr.14162 v2:172.21.15.64:6800/600121841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f7d6005f3e0 con 0x7f7d500367e0 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f7d500367e0 msgr2=0x7f7d50038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 --2- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f7d500367e0 0x7f7d50038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009990 tx=0x7f7d4c006e40 comp rx=0 tx=0).stop 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 msgr2=0x7f7d60111870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 --2- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60111870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d54003080 tx=0x7f7d54009250 comp rx=0 tx=0).stop 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 shutdown_connections 2024-02-16T22:47:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 --2- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f7d500367e0 0x7f7d50038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:58.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 --2- 172.21.15.64:0/2690509392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d60076c40 0x7f7d60111870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:47:58.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 >> 172.21.15.64:0/2690509392 conn(0x7f7d600fcdd0 msgr2=0x7f7d600ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:47:58.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 shutdown_connections 2024-02-16T22:47:58.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:58.241+0000 7f7d66fc5700 1 -- 172.21.15.64:0/2690509392 wait complete. 2024-02-16T22:47:58.942 INFO:teuthology.orchestra.run.smithi064.stdout:Fetching dashboard port number... 2024-02-16T22:47:59.057 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:58 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:47:59.057 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:58 smithi064 ceph-mon[29226]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:47:59.057 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:47:58 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:00.338 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:00 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/8933590' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-16T22:48:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 8443 2024-02-16T22:48:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.747+0000 7fb23aca2700 1 Processor -- start 2024-02-16T22:48:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.747+0000 7fb23aca2700 1 -- start start 2024-02-16T22:48:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.747+0000 7fb23aca2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.747+0000 7fb23aca2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2341030c0 con 0x7fb234102770 2024-02-16T22:48:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.747+0000 7fb238a3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.747+0000 7fb238a3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33136/0 (socket says 172.21.15.64:33136) 2024-02-16T22:48:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.747+0000 7fb238a3e700 1 -- 172.21.15.64:0/735566470 learned_addr learned my addr 172.21.15.64:0/735566470 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.748+0000 7fb238a3e700 1 -- 172.21.15.64:0/735566470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb234103200 con 0x7fb234102770 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.748+0000 7fb238a3e700 1 --2- 172.21.15.64:0/735566470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234102b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb2280098e0 tx=0x7fb228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327cd115f7136972 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.748+0000 7fb2337fe700 1 -- 172.21.15.64:0/735566470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb22800b860 con 0x7fb234102770 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.748+0000 7fb2337fe700 1 -- 172.21.15.64:0/735566470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb22800be90 con 0x7fb234102770 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.748+0000 7fb2337fe700 1 -- 172.21.15.64:0/735566470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2280103c0 con 0x7fb234102770 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 -- 172.21.15.64:0/735566470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 msgr2=0x7fb234102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 --2- 172.21.15.64:0/735566470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234102b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb2280098e0 tx=0x7fb228009250 comp rx=0 tx=0).stop 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 -- 172.21.15.64:0/735566470 shutdown_connections 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 --2- 172.21.15.64:0/735566470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234102b80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 -- 172.21.15.64:0/735566470 >> 172.21.15.64:0/735566470 conn(0x7fb2340fded0 msgr2=0x7fb234100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 -- 172.21.15.64:0/735566470 shutdown_connections 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 -- 172.21.15.64:0/735566470 wait complete. 2024-02-16T22:48:00.802 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 Processor -- start 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.749+0000 7fb23aca2700 1 -- start start 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb23aca2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234111560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb23aca2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb234111aa0 con 0x7fb234102770 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb238a3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234111560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb238a3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234111560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33150/0 (socket says 172.21.15.64:33150) 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb238a3e700 1 -- 172.21.15.64:0/8933590 learned_addr learned my addr 172.21.15.64:0/8933590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb238a3e700 1 -- 172.21.15.64:0/8933590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280094f0 con 0x7fb234102770 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb238a3e700 1 --2- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234111560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb228006b00 tx=0x7fb228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb2317fa700 1 -- 172.21.15.64:0/8933590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2280109a0 con 0x7fb234102770 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb234111ca0 con 0x7fb234102770 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb2317fa700 1 -- 172.21.15.64:0/8933590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb22801a430 con 0x7fb234102770 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb2317fa700 1 -- 172.21.15.64:0/8933590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2280245b0 con 0x7fb234102770 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.750+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb234112940 con 0x7fb234102770 2024-02-16T22:48:00.803 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.751+0000 7fb2317fa700 1 -- 172.21.15.64:0/8933590 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fb228024830 con 0x7fb234102770 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.751+0000 7fb2317fa700 1 --2- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7fb2240367e0 0x7fb224038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.751+0000 7fb2317fa700 1 -- 172.21.15.64:0/8933590 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb22804a960 con 0x7fb234102770 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.751+0000 7fb233fff700 1 --2- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7fb2240367e0 0x7fb224038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.751+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb234112130 con 0x7fb234102770 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.752+0000 7fb233fff700 1 --2- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7fb2240367e0 0x7fb224038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb220009990 tx=0x7fb220006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.756+0000 7fb2317fa700 1 -- 172.21.15.64:0/8933590 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb228018710 con 0x7fb234102770 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.949+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb23405f5c0 con 0x7fb234102770 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.950+0000 7fb2317fa700 1 -- 172.21.15.64:0/8933590 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fb228010b00 con 0x7fb234102770 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7fb2240367e0 msgr2=0x7fb224038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 --2- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7fb2240367e0 0x7fb224038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb220009990 tx=0x7fb220006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 msgr2=0x7fb234111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:00.804 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 --2- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234111560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb228006b00 tx=0x7fb228009250 comp rx=0 tx=0).stop 2024-02-16T22:48:00.805 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 shutdown_connections 2024-02-16T22:48:00.805 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 --2- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7fb2240367e0 0x7fb224038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:00.805 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 --2- 172.21.15.64:0/8933590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb234102770 0x7fb234111560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:00.805 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 >> 172.21.15.64:0/8933590 conn(0x7fb2340fded0 msgr2=0x7fb2340fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:00.805 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 shutdown_connections 2024-02-16T22:48:00.805 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:47:59.952+0000 7fb23aca2700 1 -- 172.21.15.64:0/8933590 wait complete. 2024-02-16T22:48:00.808 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T22:48:00.808 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-02-16T22:48:00.816 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T22:48:00.816 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-02-16T22:48:00.816 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-02-16T22:48:00.816 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-16T22:48:00.818 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph Dashboard is now available at: 2024-02-16T22:48:00.818 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:00.818 INFO:teuthology.orchestra.run.smithi064.stdout: URL: https://smithi064.front.sepia.ceph.com:8443/ 2024-02-16T22:48:00.818 INFO:teuthology.orchestra.run.smithi064.stdout: User: admin 2024-02-16T22:48:00.818 INFO:teuthology.orchestra.run.smithi064.stdout: Password: 1z3fm6coux 2024-02-16T22:48:00.818 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:00.818 INFO:teuthology.orchestra.run.smithi064.stdout:Saving cluster configuration to /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config directory 2024-02-16T22:48:00.819 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling autotune for osd_memory_target 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f3587b60700 1 Processor -- start 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f3587b60700 1 -- start start 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f3587b60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f3580105380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f3587b60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35801058c0 con 0x7f3580076c20 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f35858fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f3580105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f35858fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f3580105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44390/0 (socket says 172.21.15.64:44390) 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f35858fc700 1 -- 172.21.15.64:0/407643462 learned_addr learned my addr 172.21.15.64:0/407643462 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f35858fc700 1 -- 172.21.15.64:0/407643462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3580105a00 con 0x7f3580076c20 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.248+0000 7f35858fc700 1 --2- 172.21.15.64:0/407643462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f3580105380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35740098e0 tx=0x7f3574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e1be6b17b38d17 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f35848fa700 1 -- 172.21.15.64:0/407643462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f357400b860 con 0x7f3580076c20 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f35848fa700 1 -- 172.21.15.64:0/407643462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f357400be90 con 0x7f3580076c20 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f35848fa700 1 -- 172.21.15.64:0/407643462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35740103c0 con 0x7f3580076c20 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f3587b60700 1 -- 172.21.15.64:0/407643462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 msgr2=0x7f3580105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f3587b60700 1 --2- 172.21.15.64:0/407643462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f3580105380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35740098e0 tx=0x7f3574009250 comp rx=0 tx=0).stop 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f3587b60700 1 -- 172.21.15.64:0/407643462 shutdown_connections 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f3587b60700 1 --2- 172.21.15.64:0/407643462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f3580105380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f3587b60700 1 -- 172.21.15.64:0/407643462 >> 172.21.15.64:0/407643462 conn(0x7f35800fcdd0 msgr2=0x7f35800ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f3587b60700 1 -- 172.21.15.64:0/407643462 shutdown_connections 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.249+0000 7f3587b60700 1 -- 172.21.15.64:0/407643462 wait complete. 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.250+0000 7f3587b60700 1 Processor -- start 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.250+0000 7f3587b60700 1 -- start start 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.250+0000 7f3587b60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f35801115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.250+0000 7f3587b60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3580111b20 con 0x7f3580076c20 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.250+0000 7f35858fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f35801115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.250+0000 7f35858fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f35801115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44406/0 (socket says 172.21.15.64:44406) 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.250+0000 7f35858fc700 1 -- 172.21.15.64:0/3483947765 learned_addr learned my addr 172.21.15.64:0/3483947765 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.250+0000 7f35858fc700 1 -- 172.21.15.64:0/3483947765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740094f0 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.251+0000 7f35858fc700 1 --2- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f35801115e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f35740094c0 tx=0x7f3574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.251+0000 7f35727fc700 1 -- 172.21.15.64:0/3483947765 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3574010890 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.251+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3580111d20 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.251+0000 7f35727fc700 1 -- 172.21.15.64:0/3483947765 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3574010ec0 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.251+0000 7f35727fc700 1 -- 172.21.15.64:0/3483947765 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35740243f0 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.251+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35801129c0 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.252+0000 7f35727fc700 1 -- 172.21.15.64:0/3483947765 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f35740109f0 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.252+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358005f5c0 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.252+0000 7f35727fc700 1 --2- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f356c036880 0x7f356c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.252+0000 7f35727fc700 1 -- 172.21.15.64:0/3483947765 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f357404a470 con 0x7f3580076c20 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.252+0000 7f35850fb700 1 --2- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f356c036880 0x7f356c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.252+0000 7f35850fb700 1 --2- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f356c036880 0x7f356c038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f357c009990 tx=0x7f357c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.257+0000 7f35727fc700 1 -- 172.21.15.64:0/3483947765 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f357402b8e0 con 0x7f3580076c20 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.447+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f358004bed0 con 0x7f3580076c20 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.447+0000 7f35727fc700 1 -- 172.21.15.64:0/3483947765 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f357402b2f0 con 0x7f3580076c20 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f356c036880 msgr2=0x7f356c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 --2- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f356c036880 0x7f356c038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f357c009990 tx=0x7f357c006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 msgr2=0x7f35801115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 --2- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f35801115e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f35740094c0 tx=0x7f3574009250 comp rx=0 tx=0).stop 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 shutdown_connections 2024-02-16T22:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 --2- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f356c036880 0x7f356c038d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 --2- 172.21.15.64:0/3483947765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3580076c20 0x7f35801115e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 >> 172.21.15.64:0/3483947765 conn(0x7f35800fcdd0 msgr2=0x7f35800fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 shutdown_connections 2024-02-16T22:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:01.449+0000 7f3587b60700 1 -- 172.21.15.64:0/3483947765 wait complete. 2024-02-16T22:48:02.268 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:02 smithi064 ceph-mon[29226]: mgrmap e13: smithi064.kubmlk(active, since 6s) 2024-02-16T22:48:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.370+0000 7f70d88ff700 1 Processor -- start 2024-02-16T22:48:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.370+0000 7f70d88ff700 1 -- start start 2024-02-16T22:48:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.370+0000 7f70d88ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d4102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.370+0000 7f70d88ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d41030c0 con 0x7f70d4102770 2024-02-16T22:48:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.370+0000 7f70d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d4102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.370+0000 7f70d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d4102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44412/0 (socket says 172.21.15.64:44412) 2024-02-16T22:48:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.370+0000 7f70d259c700 1 -- 172.21.15.64:0/173307090 learned_addr learned my addr 172.21.15.64:0/173307090 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.370+0000 7f70d259c700 1 -- 172.21.15.64:0/173307090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4103200 con 0x7f70d4102770 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d259c700 1 --2- 172.21.15.64:0/173307090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d4102b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098e0 tx=0x7f70c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6303e5eb1dc964bd server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d159a700 1 -- 172.21.15.64:0/173307090 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c400b860 con 0x7f70d4102770 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d159a700 1 -- 172.21.15.64:0/173307090 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70c400be90 con 0x7f70d4102770 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d159a700 1 -- 172.21.15.64:0/173307090 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c40103c0 con 0x7f70d4102770 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d88ff700 1 -- 172.21.15.64:0/173307090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 msgr2=0x7f70d4102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d88ff700 1 --2- 172.21.15.64:0/173307090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d4102b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098e0 tx=0x7f70c4009250 comp rx=0 tx=0).stop 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d88ff700 1 -- 172.21.15.64:0/173307090 shutdown_connections 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d88ff700 1 --2- 172.21.15.64:0/173307090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d4102b80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d88ff700 1 -- 172.21.15.64:0/173307090 >> 172.21.15.64:0/173307090 conn(0x7f70d40fded0 msgr2=0x7f70d4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d88ff700 1 -- 172.21.15.64:0/173307090 shutdown_connections 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.371+0000 7f70d88ff700 1 -- 172.21.15.64:0/173307090 wait complete. 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.372+0000 7f70d88ff700 1 Processor -- start 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.372+0000 7f70d88ff700 1 -- start start 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.372+0000 7f70d88ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d41115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.372+0000 7f70d88ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4111b00 con 0x7f70d4102770 2024-02-16T22:48:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.372+0000 7f70d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d41115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.372+0000 7f70d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d41115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44428/0 (socket says 172.21.15.64:44428) 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.372+0000 7f70d259c700 1 -- 172.21.15.64:0/3865700373 learned_addr learned my addr 172.21.15.64:0/3865700373 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.373+0000 7f70d259c700 1 -- 172.21.15.64:0/3865700373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c40094f0 con 0x7f70d4102770 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.373+0000 7f70d259c700 1 --2- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d41115c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f70c4011680 tx=0x7f70c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.373+0000 7f70c2ffd700 1 -- 172.21.15.64:0/3865700373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c4010890 con 0x7f70d4102770 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.373+0000 7f70c2ffd700 1 -- 172.21.15.64:0/3865700373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70c4010ec0 con 0x7f70d4102770 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.373+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d4111d00 con 0x7f70d4102770 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.373+0000 7f70c2ffd700 1 -- 172.21.15.64:0/3865700373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c40243f0 con 0x7f70d4102770 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.373+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d41129a0 con 0x7f70d4102770 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.374+0000 7f70c2ffd700 1 -- 172.21.15.64:0/3865700373 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f70c402d3f0 con 0x7f70d4102770 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.374+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d405f5c0 con 0x7f70d4102770 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.374+0000 7f70c2ffd700 1 --2- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70bc0368d0 0x7f70bc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.374+0000 7f70c2ffd700 1 -- 172.21.15.64:0/3865700373 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f70c404a490 con 0x7f70d4102770 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.374+0000 7f70d1d9b700 1 --2- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70bc0368d0 0x7f70bc038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.374+0000 7f70d1d9b700 1 --2- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70bc0368d0 0x7f70bc038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009990 tx=0x7f70c8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.379+0000 7f70c2ffd700 1 -- 172.21.15.64:0/3865700373 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f70c4006d70 con 0x7f70d4102770 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.677+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f70d404bed0 con 0x7f70d4102770 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.679+0000 7f70c2ffd700 1 -- 172.21.15.64:0/3865700373 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f70c4006d70 con 0x7f70d4102770 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70bc0368d0 msgr2=0x7f70bc038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 --2- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70bc0368d0 0x7f70bc038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009990 tx=0x7f70c8006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 msgr2=0x7f70d41115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 --2- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d41115c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f70c4011680 tx=0x7f70c4009250 comp rx=0 tx=0).stop 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 shutdown_connections 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 --2- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70bc0368d0 0x7f70bc038d80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 --2- 172.21.15.64:0/3865700373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d4102770 0x7f70d41115c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 >> 172.21.15.64:0/3865700373 conn(0x7f70d40fded0 msgr2=0x7f70d40ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 shutdown_connections 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-02-16T22:48:02.681+0000 7f70d88ff700 1 -- 172.21.15.64:0/3865700373 wait complete. 2024-02-16T22:48:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout:Or, if you are only running a single cluster on this host: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: ceph telemetry on 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout:For more information see: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-02-16T22:48:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:03.251 INFO:teuthology.orchestra.run.smithi064.stdout:Bootstrap complete. 2024-02-16T22:48:03.281 INFO:tasks.cephadm:Fetching config... 2024-02-16T22:48:03.281 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:48:03.281 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-16T22:48:03.339 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-16T22:48:03.340 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:48:03.340 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-16T22:48:03.390 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-16T22:48:03.390 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:48:03.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/keyring of=/dev/stdout 2024-02-16T22:48:03.457 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-16T22:48:03.457 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:48:03.457 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-16T22:48:03.512 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-16T22:48:03.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3CnEaDMje6txJ1TRToJpnjbXSpi7SgKfCr4jvuI1gF5sSZjBgq6DS5/cmKN+t9+N4N2Kgv4yrFgWXVcbLktk5nslubJBNjKNx1zB5328G73/zbwY3HKsfLQBkfmaaDKsLR86GCvNJQ0ekA6EJlRvuocMk1N8A+nbPNwt2PsEC0Ap+DcPX16ut2Hwg3Ioe5JT1uOCp6Qcl8vDZP0ZmgcFsGVjjTv1YSz1DrOKtHDUlatgE7shYsxeycmxUlzfaZC5XeL+Z0mtuu7UHP8sMDwc0gkmhT0swZjNxi1daTGT4hd0ixfb1ltqqW6G5NPElCSiB33RW8l2qJY73bKuZ6LVwZyml/DaIAvwtdXLzfCC0F2ZN/jAT4+37T0GrPZnB0DQN/W57NTF/O7APukYrSSifN0v4y+6pIqWXWCLX/mwQBd86mxYdIlLjT0jWVU7n4olDBGvlOBBBUZbl8dDrGEYPomDv5s5dkWImJulWR3CWUxzJChLk48mJ7bsVUOwskqU= ceph-2912dc76-cd1d-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-16T22:48:03.592 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3CnEaDMje6txJ1TRToJpnjbXSpi7SgKfCr4jvuI1gF5sSZjBgq6DS5/cmKN+t9+N4N2Kgv4yrFgWXVcbLktk5nslubJBNjKNx1zB5328G73/zbwY3HKsfLQBkfmaaDKsLR86GCvNJQ0ekA6EJlRvuocMk1N8A+nbPNwt2PsEC0Ap+DcPX16ut2Hwg3Ioe5JT1uOCp6Qcl8vDZP0ZmgcFsGVjjTv1YSz1DrOKtHDUlatgE7shYsxeycmxUlzfaZC5XeL+Z0mtuu7UHP8sMDwc0gkmhT0swZjNxi1daTGT4hd0ixfb1ltqqW6G5NPElCSiB33RW8l2qJY73bKuZ6LVwZyml/DaIAvwtdXLzfCC0F2ZN/jAT4+37T0GrPZnB0DQN/W57NTF/O7APukYrSSifN0v4y+6pIqWXWCLX/mwQBd86mxYdIlLjT0jWVU7n4olDBGvlOBBBUZbl8dDrGEYPomDv5s5dkWImJulWR3CWUxzJChLk48mJ7bsVUOwskqU= ceph-2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:48:03.606 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3CnEaDMje6txJ1TRToJpnjbXSpi7SgKfCr4jvuI1gF5sSZjBgq6DS5/cmKN+t9+N4N2Kgv4yrFgWXVcbLktk5nslubJBNjKNx1zB5328G73/zbwY3HKsfLQBkfmaaDKsLR86GCvNJQ0ekA6EJlRvuocMk1N8A+nbPNwt2PsEC0Ap+DcPX16ut2Hwg3Ioe5JT1uOCp6Qcl8vDZP0ZmgcFsGVjjTv1YSz1DrOKtHDUlatgE7shYsxeycmxUlzfaZC5XeL+Z0mtuu7UHP8sMDwc0gkmhT0swZjNxi1daTGT4hd0ixfb1ltqqW6G5NPElCSiB33RW8l2qJY73bKuZ6LVwZyml/DaIAvwtdXLzfCC0F2ZN/jAT4+37T0GrPZnB0DQN/W57NTF/O7APukYrSSifN0v4y+6pIqWXWCLX/mwQBd86mxYdIlLjT0jWVU7n4olDBGvlOBBBUZbl8dDrGEYPomDv5s5dkWImJulWR3CWUxzJChLk48mJ7bsVUOwskqU= ceph-2912dc76-cd1d-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-16T22:48:03.651 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3CnEaDMje6txJ1TRToJpnjbXSpi7SgKfCr4jvuI1gF5sSZjBgq6DS5/cmKN+t9+N4N2Kgv4yrFgWXVcbLktk5nslubJBNjKNx1zB5328G73/zbwY3HKsfLQBkfmaaDKsLR86GCvNJQ0ekA6EJlRvuocMk1N8A+nbPNwt2PsEC0Ap+DcPX16ut2Hwg3Ioe5JT1uOCp6Qcl8vDZP0ZmgcFsGVjjTv1YSz1DrOKtHDUlatgE7shYsxeycmxUlzfaZC5XeL+Z0mtuu7UHP8sMDwc0gkmhT0swZjNxi1daTGT4hd0ixfb1ltqqW6G5NPElCSiB33RW8l2qJY73bKuZ6LVwZyml/DaIAvwtdXLzfCC0F2ZN/jAT4+37T0GrPZnB0DQN/W57NTF/O7APukYrSSifN0v4y+6pIqWXWCLX/mwQBd86mxYdIlLjT0jWVU7n4olDBGvlOBBBUZbl8dDrGEYPomDv5s5dkWImJulWR3CWUxzJChLk48mJ7bsVUOwskqU= ceph-2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:48:03.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-16T22:48:03.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:03 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3865700373' entity='client.admin' 2024-02-16T22:48:03.922 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.048+0000 7ff08700c700 1 -- 172.21.15.64:0/1456440462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 msgr2=0x7ff0800fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.048+0000 7ff08700c700 1 --2- 172.21.15.64:0/1456440462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 0x7ff0800fcfd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff0740048c0 tx=0x7ff074009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.048+0000 7ff08700c700 1 -- 172.21.15.64:0/1456440462 shutdown_connections 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.048+0000 7ff08700c700 1 --2- 172.21.15.64:0/1456440462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 0x7ff0800fcfd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.048+0000 7ff08700c700 1 -- 172.21.15.64:0/1456440462 >> 172.21.15.64:0/1456440462 conn(0x7ff0800f82a0 msgr2=0x7ff0800fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.048+0000 7ff08700c700 1 -- 172.21.15.64:0/1456440462 shutdown_connections 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.048+0000 7ff08700c700 1 -- 172.21.15.64:0/1456440462 wait complete. 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.049+0000 7ff08700c700 1 Processor -- start 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.049+0000 7ff08700c700 1 -- start start 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.049+0000 7ff08700c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 0x7ff08010cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:06.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.049+0000 7ff08700c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08010d400 con 0x7ff0800fcbc0 2024-02-16T22:48:06.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.049+0000 7ff084da8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 0x7ff08010cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:06.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.049+0000 7ff084da8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 0x7ff08010cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44458/0 (socket says 172.21.15.64:44458) 2024-02-16T22:48:06.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.049+0000 7ff084da8700 1 -- 172.21.15.64:0/3847834841 learned_addr learned my addr 172.21.15.64:0/3847834841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:06.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.050+0000 7ff084da8700 1 -- 172.21.15.64:0/3847834841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074005040 con 0x7ff0800fcbc0 2024-02-16T22:48:06.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.050+0000 7ff084da8700 1 --2- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 0x7ff08010cec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff074009680 tx=0x7ff074009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:06.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.050+0000 7ff07d7fa700 1 -- 172.21.15.64:0/3847834841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff07400bed0 con 0x7ff0800fcbc0 2024-02-16T22:48:06.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.050+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08010d600 con 0x7ff0800fcbc0 2024-02-16T22:48:06.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.050+0000 7ff07d7fa700 1 -- 172.21.15.64:0/3847834841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff07400b5c0 con 0x7ff0800fcbc0 2024-02-16T22:48:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.050+0000 7ff07d7fa700 1 -- 172.21.15.64:0/3847834841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff07401c5b0 con 0x7ff0800fcbc0 2024-02-16T22:48:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.050+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08010e2a0 con 0x7ff0800fcbc0 2024-02-16T22:48:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.051+0000 7ff07d7fa700 1 -- 172.21.15.64:0/3847834841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ff07401c820 con 0x7ff0800fcbc0 2024-02-16T22:48:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.051+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08005f5c0 con 0x7ff0800fcbc0 2024-02-16T22:48:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.051+0000 7ff07d7fa700 1 --2- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7ff070036820 0x7ff070038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.051+0000 7ff07d7fa700 1 -- 172.21.15.64:0/3847834841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff074049a70 con 0x7ff0800fcbc0 2024-02-16T22:48:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.051+0000 7ff07ffff700 1 --2- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7ff070036820 0x7ff070038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.051+0000 7ff07ffff700 1 --2- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7ff070036820 0x7ff070038cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009990 tx=0x7ff06c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:06.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.056+0000 7ff07d7fa700 1 -- 172.21.15.64:0/3847834841 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff074011db0 con 0x7ff0800fcbc0 2024-02-16T22:48:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.249+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff08010ee80 con 0x7ff0800fcbc0 2024-02-16T22:48:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.252+0000 7ff07d7fa700 1 -- 172.21.15.64:0/3847834841 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff074023990 con 0x7ff0800fcbc0 2024-02-16T22:48:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7ff070036820 msgr2=0x7ff070038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 --2- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7ff070036820 0x7ff070038cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009990 tx=0x7ff06c006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 msgr2=0x7ff08010cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 --2- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 0x7ff08010cec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff074009680 tx=0x7ff074009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 shutdown_connections 2024-02-16T22:48:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 --2- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7ff070036820 0x7ff070038cd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 --2- 172.21.15.64:0/3847834841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800fcbc0 0x7ff08010cec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 >> 172.21.15.64:0/3847834841 conn(0x7ff0800f82a0 msgr2=0x7ff0800f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 shutdown_connections 2024-02-16T22:48:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:06.254+0000 7ff08700c700 1 -- 172.21.15.64:0/3847834841 wait complete. 2024-02-16T22:48:06.266 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:06 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:06.266 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:06 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:06.266 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:06 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:48:06.267 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:06 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:06.267 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:06 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:48:06.267 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:06 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-16T22:48:06.267 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:06 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:48:06.267 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:06 smithi064 ceph-mon[29226]: Deploying daemon crash.smithi064 on smithi064 2024-02-16T22:48:06.862 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-16T22:48:06.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-16T22:48:07.121 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:48:07.533 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:07 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3847834841' entity='client.admin' 2024-02-16T22:48:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.667+0000 7f2407f03700 1 -- 172.21.15.64:0/3181040023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 msgr2=0x7f24000fbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.667+0000 7f2407f03700 1 --2- 172.21.15.64:0/3181040023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 0x7f24000fbed0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0048c0 tx=0x7f23fc009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.667+0000 7f2407f03700 1 -- 172.21.15.64:0/3181040023 shutdown_connections 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.667+0000 7f2407f03700 1 --2- 172.21.15.64:0/3181040023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 0x7f24000fbed0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.667+0000 7f2407f03700 1 -- 172.21.15.64:0/3181040023 >> 172.21.15.64:0/3181040023 conn(0x7f24000f71a0 msgr2=0x7f24000f95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.667+0000 7f2407f03700 1 -- 172.21.15.64:0/3181040023 shutdown_connections 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.667+0000 7f2407f03700 1 -- 172.21.15.64:0/3181040023 wait complete. 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.668+0000 7f2407f03700 1 Processor -- start 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.668+0000 7f2407f03700 1 -- start start 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.668+0000 7f2407f03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 0x7f240010cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.668+0000 7f2407f03700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240010d420 con 0x7f24000fbac0 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.668+0000 7f2405c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 0x7f240010cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.668+0000 7f2405c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 0x7f240010cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44482/0 (socket says 172.21.15.64:44482) 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.668+0000 7f2405c9f700 1 -- 172.21.15.64:0/3064345451 learned_addr learned my addr 172.21.15.64:0/3064345451 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.669+0000 7f2405c9f700 1 -- 172.21.15.64:0/3064345451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc005040 con 0x7f24000fbac0 2024-02-16T22:48:09.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.669+0000 7f2405c9f700 1 --2- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 0x7f240010cee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009680 tx=0x7f23fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:09.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.669+0000 7f23f67fc700 1 -- 172.21.15.64:0/3064345451 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23fc00bed0 con 0x7f24000fbac0 2024-02-16T22:48:09.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.669+0000 7f23f67fc700 1 -- 172.21.15.64:0/3064345451 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23fc00b5c0 con 0x7f24000fbac0 2024-02-16T22:48:09.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.669+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240010d620 con 0x7f24000fbac0 2024-02-16T22:48:09.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.669+0000 7f23f67fc700 1 -- 172.21.15.64:0/3064345451 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23fc01c5b0 con 0x7f24000fbac0 2024-02-16T22:48:09.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.669+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240010e2c0 con 0x7f24000fbac0 2024-02-16T22:48:09.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.670+0000 7f23f67fc700 1 -- 172.21.15.64:0/3064345451 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f23fc01c820 con 0x7f24000fbac0 2024-02-16T22:48:09.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.670+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240005f5c0 con 0x7f24000fbac0 2024-02-16T22:48:09.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.670+0000 7f23f67fc700 1 --2- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f23ec036820 0x7f23ec038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:09.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.670+0000 7f23f67fc700 1 -- 172.21.15.64:0/3064345451 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f23fc049ae0 con 0x7f24000fbac0 2024-02-16T22:48:09.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.670+0000 7f240549e700 1 --2- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f23ec036820 0x7f23ec038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:09.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.670+0000 7f240549e700 1 --2- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f23ec036820 0x7f23ec038cd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009990 tx=0x7f23f0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:09.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.675+0000 7f23f67fc700 1 -- 172.21.15.64:0/3064345451 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f23fc011dd0 con 0x7f24000fbac0 2024-02-16T22:48:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.870+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 --> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f24000f8b20 con 0x7f23ec036820 2024-02-16T22:48:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.874+0000 7f23f67fc700 1 -- 172.21.15.64:0/3064345451 <== mgr.14162 v2:172.21.15.64:6800/600121841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f24000f8b20 con 0x7f23ec036820 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.875+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f23ec036820 msgr2=0x7f23ec038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.875+0000 7f2407f03700 1 --2- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f23ec036820 0x7f23ec038cd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009990 tx=0x7f23f0006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.875+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 msgr2=0x7f240010cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.875+0000 7f2407f03700 1 --2- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 0x7f240010cee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009680 tx=0x7f23fc009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.876+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 shutdown_connections 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.876+0000 7f2407f03700 1 --2- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f23ec036820 0x7f23ec038cd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.876+0000 7f2407f03700 1 --2- 172.21.15.64:0/3064345451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24000fbac0 0x7f240010cee0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.876+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 >> 172.21.15.64:0/3064345451 conn(0x7f24000f71a0 msgr2=0x7f24000f7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.876+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 shutdown_connections 2024-02-16T22:48:09.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:09.876+0000 7f2407f03700 1 -- 172.21.15.64:0/3064345451 wait complete. 2024-02-16T22:48:10.768 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-02-16T22:48:10.768 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:48:10.768 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-02-16T22:48:10.787 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:48:10.787 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:48:10.844 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-02-16T22:48:10.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch host add smithi192 2024-02-16T22:48:11.077 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:10 smithi064 ceph-mon[29226]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:48:11.077 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:10 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:11.354 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:48:12.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:12 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:12.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:12 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:12.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:12 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:12.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:12 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:12.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:12 smithi064 ceph-mon[29226]: Deploying daemon node-exporter.smithi064 on smithi064 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.779+0000 7f4d2465a700 1 -- 172.21.15.64:0/4134989315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 msgr2=0x7f4d1c0fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.779+0000 7f4d2465a700 1 --2- 172.21.15.64:0/4134989315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 0x7f4d1c0fe630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4d180048c0 tx=0x7f4d18009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.779+0000 7f4d2465a700 1 -- 172.21.15.64:0/4134989315 shutdown_connections 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.779+0000 7f4d2465a700 1 --2- 172.21.15.64:0/4134989315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 0x7f4d1c0fe630 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.779+0000 7f4d2465a700 1 -- 172.21.15.64:0/4134989315 >> 172.21.15.64:0/4134989315 conn(0x7f4d1c0f6920 msgr2=0x7f4d1c0f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.779+0000 7f4d2465a700 1 -- 172.21.15.64:0/4134989315 shutdown_connections 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.779+0000 7f4d2465a700 1 -- 172.21.15.64:0/4134989315 wait complete. 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.780+0000 7f4d2465a700 1 Processor -- start 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.780+0000 7f4d2465a700 1 -- start start 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.780+0000 7f4d2465a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 0x7f4d1c10cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.780+0000 7f4d2465a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c10d440 con 0x7f4d1c0fe220 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.780+0000 7f4d223f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 0x7f4d1c10cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.780+0000 7f4d223f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 0x7f4d1c10cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39558/0 (socket says 172.21.15.64:39558) 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.780+0000 7f4d223f6700 1 -- 172.21.15.64:0/3411819300 learned_addr learned my addr 172.21.15.64:0/3411819300 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.781+0000 7f4d223f6700 1 -- 172.21.15.64:0/3411819300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18005040 con 0x7f4d1c0fe220 2024-02-16T22:48:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.781+0000 7f4d223f6700 1 --2- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 0x7f4d1c10cf00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009680 tx=0x7f4d18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.781+0000 7f4d12ffd700 1 -- 172.21.15.64:0/3411819300 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d1800bed0 con 0x7f4d1c0fe220 2024-02-16T22:48:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.781+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c10d640 con 0x7f4d1c0fe220 2024-02-16T22:48:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.781+0000 7f4d12ffd700 1 -- 172.21.15.64:0/3411819300 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d1800b5c0 con 0x7f4d1c0fe220 2024-02-16T22:48:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.781+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c10e2e0 con 0x7f4d1c0fe220 2024-02-16T22:48:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.781+0000 7f4d12ffd700 1 -- 172.21.15.64:0/3411819300 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d18025560 con 0x7f4d1c0fe220 2024-02-16T22:48:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.782+0000 7f4d12ffd700 1 -- 172.21.15.64:0/3411819300 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f4d1801c3f0 con 0x7f4d1c0fe220 2024-02-16T22:48:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.782+0000 7f4d12ffd700 1 --2- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4d08036860 0x7f4d08038d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.782+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d000052e0 con 0x7f4d1c0fe220 2024-02-16T22:48:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.782+0000 7f4d12ffd700 1 -- 172.21.15.64:0/3411819300 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4d1804a890 con 0x7f4d1c0fe220 2024-02-16T22:48:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.782+0000 7f4d21bf5700 1 --2- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4d08036860 0x7f4d08038d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.783+0000 7f4d21bf5700 1 --2- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4d08036860 0x7f4d08038d10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009990 tx=0x7f4d0c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:12.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.788+0000 7f4d12ffd700 1 -- 172.21.15.64:0/3411819300 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4d180238f0 con 0x7f4d1c0fe220 2024-02-16T22:48:12.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:12.993+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 --> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d00000bf0 con 0x7f4d08036860 2024-02-16T22:48:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:13 smithi064 ceph-mon[29226]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:48:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.130+0000 7f4d12ffd700 1 -- 172.21.15.64:0/3411819300 <== mgr.14162 v2:172.21.15.64:6800/600121841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4d00000bf0 con 0x7f4d08036860 2024-02-16T22:48:15.132 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4d08036860 msgr2=0x7f4d08038d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 --2- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4d08036860 0x7f4d08038d10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009990 tx=0x7f4d0c006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 msgr2=0x7f4d1c10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 --2- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 0x7f4d1c10cf00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009680 tx=0x7f4d18009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 shutdown_connections 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 --2- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4d08036860 0x7f4d08038d10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 --2- 172.21.15.64:0/3411819300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d1c0fe220 0x7f4d1c10cf00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 >> 172.21.15.64:0/3411819300 conn(0x7f4d1c0f6920 msgr2=0x7f4d1c0f76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.132+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 shutdown_connections 2024-02-16T22:48:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:15.133+0000 7f4d2465a700 1 -- 172.21.15.64:0/3411819300 wait complete. 2024-02-16T22:48:15.434 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:15 smithi064 ceph-mon[29226]: Deploying cephadm binary to smithi192 2024-02-16T22:48:15.434 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:15 smithi064 ceph-mon[29226]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:15.434 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:15 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:15.434 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:15 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:16.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-16T22:48:16.437 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:16 smithi064 ceph-mon[29226]: Added host smithi192 2024-02-16T22:48:16.437 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:16 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:16.437 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:16 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:16.437 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:16 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:16.437 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:16 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:48:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:17 smithi064 ceph-mon[29226]: Deploying daemon alertmanager.smithi064 on smithi064 2024-02-16T22:48:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:17 smithi064 ceph-mon[29226]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.964+0000 7f22f11df700 1 -- 172.21.15.64:0/3845231811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 msgr2=0x7f22ec0fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.964+0000 7f22f11df700 1 --2- 172.21.15.64:0/3845231811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 0x7f22ec0fbee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0048c0 tx=0x7f22dc009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.964+0000 7f22f11df700 1 -- 172.21.15.64:0/3845231811 shutdown_connections 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.964+0000 7f22f11df700 1 --2- 172.21.15.64:0/3845231811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 0x7f22ec0fbee0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.964+0000 7f22f11df700 1 -- 172.21.15.64:0/3845231811 >> 172.21.15.64:0/3845231811 conn(0x7f22ec0f7190 msgr2=0x7f22ec0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.965+0000 7f22f11df700 1 -- 172.21.15.64:0/3845231811 shutdown_connections 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.965+0000 7f22f11df700 1 -- 172.21.15.64:0/3845231811 wait complete. 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.965+0000 7f22f11df700 1 Processor -- start 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.965+0000 7f22f11df700 1 -- start start 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.965+0000 7f22f11df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 0x7f22ec10fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.965+0000 7f22f11df700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec065d60 con 0x7f22ec0fbad0 2024-02-16T22:48:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22ead9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 0x7f22ec10fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22ead9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 0x7f22ec10fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39586/0 (socket says 172.21.15.64:39586) 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22ead9d700 1 -- 172.21.15.64:0/2573609145 learned_addr learned my addr 172.21.15.64:0/2573609145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22ead9d700 1 -- 172.21.15.64:0/2573609145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc005040 con 0x7f22ec0fbad0 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22ead9d700 1 --2- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 0x7f22ec10fbd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009680 tx=0x7f22dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22d37fe700 1 -- 172.21.15.64:0/2573609145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22dc00bed0 con 0x7f22ec0fbad0 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22d37fe700 1 -- 172.21.15.64:0/2573609145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22dc00b5c0 con 0x7f22ec0fbad0 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec065f60 con 0x7f22ec0fbad0 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22d37fe700 1 -- 172.21.15.64:0/2573609145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22dc01c5b0 con 0x7f22ec0fbad0 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.966+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec110350 con 0x7f22ec0fbad0 2024-02-16T22:48:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.967+0000 7f22d37fe700 1 -- 172.21.15.64:0/2573609145 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f22dc013420 con 0x7f22ec0fbad0 2024-02-16T22:48:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.967+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec1058a0 con 0x7f22ec0fbad0 2024-02-16T22:48:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.967+0000 7f22d37fe700 1 --2- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f22d4036820 0x7f22d4038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.967+0000 7f22d37fe700 1 -- 172.21.15.64:0/2573609145 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f22dc049b50 con 0x7f22ec0fbad0 2024-02-16T22:48:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.967+0000 7f22ea59c700 1 --2- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f22d4036820 0x7f22d4038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.968+0000 7f22ea59c700 1 --2- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f22d4036820 0x7f22d4038cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009990 tx=0x7f22e0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:18.972+0000 7f22d37fe700 1 -- 172.21.15.64:0/2573609145 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f22dc011550 con 0x7f22ec0fbad0 2024-02-16T22:48:19.044 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:18 smithi064 ceph-mon[29226]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:19.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.179+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 --> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22ec02b540 con 0x7f22d4036820 2024-02-16T22:48:19.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.181+0000 7f22d37fe700 1 -- 172.21.15.64:0/2573609145 <== mgr.14162 v2:172.21.15.64:6800/600121841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f22ec02b540 con 0x7f22d4036820 2024-02-16T22:48:19.182 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:48:19.182 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-02-16T22:48:19.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f22d4036820 msgr2=0x7f22d4038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:19.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 --2- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f22d4036820 0x7f22d4038cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009990 tx=0x7f22e0006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:19.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 msgr2=0x7f22ec10fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:19.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 --2- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 0x7f22ec10fbd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009680 tx=0x7f22dc009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:19.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 shutdown_connections 2024-02-16T22:48:19.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 --2- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f22d4036820 0x7f22d4038cd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:19.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 --2- 172.21.15.64:0/2573609145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22ec0fbad0 0x7f22ec10fbd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:19.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 >> 172.21.15.64:0/2573609145 conn(0x7f22ec0f7190 msgr2=0x7f22ec0f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:19.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 shutdown_connections 2024-02-16T22:48:19.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:19.183+0000 7f22f11df700 1 -- 172.21.15.64:0/2573609145 wait complete. 2024-02-16T22:48:20.390 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-16T22:48:20.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-16T22:48:20.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:20 smithi064 ceph-mon[29226]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:48:20.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:20 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:20.654 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:48:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:21 smithi064 ceph-mon[29226]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:23.352 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:23 smithi064 ceph-mon[29226]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.635+0000 7f48ea2ba700 1 -- 172.21.15.64:0/3620611454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 msgr2=0x7f48e40fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.635+0000 7f48ea2ba700 1 --2- 172.21.15.64:0/3620611454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 0x7f48e40fe610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48d40048c0 tx=0x7f48d4009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.635+0000 7f48ea2ba700 1 -- 172.21.15.64:0/3620611454 shutdown_connections 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.635+0000 7f48ea2ba700 1 --2- 172.21.15.64:0/3620611454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 0x7f48e40fe610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.635+0000 7f48ea2ba700 1 -- 172.21.15.64:0/3620611454 >> 172.21.15.64:0/3620611454 conn(0x7f48e40f69c0 msgr2=0x7f48e40f8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.636+0000 7f48ea2ba700 1 -- 172.21.15.64:0/3620611454 shutdown_connections 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.636+0000 7f48ea2ba700 1 -- 172.21.15.64:0/3620611454 wait complete. 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.636+0000 7f48ea2ba700 1 Processor -- start 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.636+0000 7f48ea2ba700 1 -- start start 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.636+0000 7f48ea2ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 0x7f48e410cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.636+0000 7f48ea2ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e410d480 con 0x7f48e40fe200 2024-02-16T22:48:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 0x7f48e410cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 0x7f48e410cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34540/0 (socket says 172.21.15.64:34540) 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48e3fff700 1 -- 172.21.15.64:0/4195024419 learned_addr learned my addr 172.21.15.64:0/4195024419 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48e3fff700 1 -- 172.21.15.64:0/4195024419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4005040 con 0x7f48e40fe200 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48e3fff700 1 --2- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 0x7f48e410cf40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009330 tx=0x7f48d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48e0ff9700 1 -- 172.21.15.64:0/4195024419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48d400be50 con 0x7f48e40fe200 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e410d680 con 0x7f48e40fe200 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48e0ff9700 1 -- 172.21.15.64:0/4195024419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48d40096b0 con 0x7f48e40fe200 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48e0ff9700 1 -- 172.21.15.64:0/4195024419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48d401c5b0 con 0x7f48e40fe200 2024-02-16T22:48:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.637+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e410e320 con 0x7f48e40fe200 2024-02-16T22:48:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.638+0000 7f48e0ff9700 1 -- 172.21.15.64:0/4195024419 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f48d401c870 con 0x7f48e40fe200 2024-02-16T22:48:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.638+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e405f5c0 con 0x7f48e40fe200 2024-02-16T22:48:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.638+0000 7f48e0ff9700 1 --2- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f48cc036870 0x7f48cc038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.638+0000 7f48e0ff9700 1 -- 172.21.15.64:0/4195024419 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f48d4049bc0 con 0x7f48e40fe200 2024-02-16T22:48:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.638+0000 7f48e37fe700 1 --2- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f48cc036870 0x7f48cc038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.639+0000 7f48e37fe700 1 --2- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f48cc036870 0x7f48cc038d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009990 tx=0x7f48d8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:24.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.643+0000 7f48e0ff9700 1 -- 172.21.15.64:0/4195024419 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f48d4011dd0 con 0x7f48e40fe200 2024-02-16T22:48:24.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:24.840+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f48e410ee70 con 0x7f48e40fe200 2024-02-16T22:48:25.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.325+0000 7f48e0ff9700 1 -- 172.21.15.64:0/4195024419 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f48d4011370 con 0x7f48e40fe200 2024-02-16T22:48:25.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f48cc036870 msgr2=0x7f48cc038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:25.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 --2- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f48cc036870 0x7f48cc038d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009990 tx=0x7f48d8006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:25.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 msgr2=0x7f48e410cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:25.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 --2- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 0x7f48e410cf40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009330 tx=0x7f48d4009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:25.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 shutdown_connections 2024-02-16T22:48:25.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 --2- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f48cc036870 0x7f48cc038d20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:25.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 --2- 172.21.15.64:0/4195024419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e40fe200 0x7f48e410cf40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:25.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 >> 172.21.15.64:0/4195024419 conn(0x7f48e40f69c0 msgr2=0x7f48e40f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:25.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 shutdown_connections 2024-02-16T22:48:25.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:48:25.327+0000 7f48ea2ba700 1 -- 172.21.15.64:0/4195024419 wait complete. 2024-02-16T22:48:25.329 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-02-16T22:48:25.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:25 smithi064 ceph-mon[29226]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:25.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:25 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/4195024419' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-16T22:48:25.947 INFO:tasks.cephadm:Adding mon.smithi064 on smithi064 2024-02-16T22:48:25.947 INFO:tasks.cephadm:Adding mon.smithi192 on smithi192 2024-02-16T22:48:25.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch apply mon '2;smithi064:172.21.15.64=smithi064;smithi192:172.21.15.192=smithi192' 2024-02-16T22:48:26.194 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:26.240 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:26.492 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:26 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/4195024419' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-16T22:48:26.492 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:26 smithi064 ceph-mon[29226]: osdmap e4: 0 total, 0 up, 0 in 2024-02-16T22:48:27.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:27.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:27.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:27.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:27.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:27.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:27.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:27.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-16T22:48:27.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:27 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:27.775 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 -- 172.21.15.192:0/670926907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 msgr2=0x7f37b00fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:27.776 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 --2- 172.21.15.192:0/670926907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 0x7f37b00fcfe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f37a00048c0 tx=0x7f37a0009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:27.776 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 -- 172.21.15.192:0/670926907 shutdown_connections 2024-02-16T22:48:27.776 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 --2- 172.21.15.192:0/670926907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 0x7f37b00fcfe0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:27.776 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 -- 172.21.15.192:0/670926907 >> 172.21.15.192:0/670926907 conn(0x7f37b00f8290 msgr2=0x7f37b00fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:27.776 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 -- 172.21.15.192:0/670926907 shutdown_connections 2024-02-16T22:48:27.776 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 -- 172.21.15.192:0/670926907 wait complete. 2024-02-16T22:48:27.777 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 Processor -- start 2024-02-16T22:48:27.777 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.774+0000 7f37b6099700 1 -- start start 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.775+0000 7f37b6099700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 0x7f37b0071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.775+0000 7f37b6099700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00702e0 con 0x7f37b00fcbd0 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.775+0000 7f37af7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 0x7f37b0071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.775+0000 7f37af7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 0x7f37b0071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:48530/0 (socket says 172.21.15.192:48530) 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.775+0000 7f37af7fe700 1 -- 172.21.15.192:0/2479436493 learned_addr learned my addr 172.21.15.192:0/2479436493 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.775+0000 7f37af7fe700 1 -- 172.21.15.192:0/2479436493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a0005040 con 0x7f37b00fcbd0 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.775+0000 7f37af7fe700 1 --2- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 0x7f37b0071e60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009680 tx=0x7f37a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.776+0000 7f37acff9700 1 -- 172.21.15.192:0/2479436493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37a000bed0 con 0x7f37b00fcbd0 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.776+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b00704e0 con 0x7f37b00fcbd0 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.776+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b00709d0 con 0x7f37b00fcbd0 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.776+0000 7f37acff9700 1 -- 172.21.15.192:0/2479436493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37a000b5c0 con 0x7f37b00fcbd0 2024-02-16T22:48:27.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.776+0000 7f37acff9700 1 -- 172.21.15.192:0/2479436493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37a001c5b0 con 0x7f37b00fcbd0 2024-02-16T22:48:27.779 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.777+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b01069a0 con 0x7f37b00fcbd0 2024-02-16T22:48:27.779 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.777+0000 7f37acff9700 1 -- 172.21.15.192:0/2479436493 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f37a0013a80 con 0x7f37b00fcbd0 2024-02-16T22:48:27.779 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.777+0000 7f37acff9700 1 --2- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f379c036820 0x7f379c038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:27.779 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.777+0000 7f37acff9700 1 -- 172.21.15.192:0/2479436493 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f37a0049be0 con 0x7f37b00fcbd0 2024-02-16T22:48:27.780 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.777+0000 7f37a7fff700 1 --2- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f379c036820 0x7f379c038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:27.780 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.778+0000 7f37a7fff700 1 --2- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f379c036820 0x7f379c038cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3798009990 tx=0x7f3798006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:27.783 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.782+0000 7f37acff9700 1 -- 172.21.15.192:0/2479436493 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f37a0011550 con 0x7f37b00fcbd0 2024-02-16T22:48:27.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.973+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 --> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi064:172.21.15.64=smithi064;smithi192:172.21.15.192=smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7f37b002b310 con 0x7f379c036820 2024-02-16T22:48:27.980 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.979+0000 7f37acff9700 1 -- 172.21.15.192:0/2479436493 <== mgr.14162 v2:172.21.15.64:6800/600121841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f37b002b310 con 0x7f379c036820 2024-02-16T22:48:27.981 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-02-16T22:48:27.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f379c036820 msgr2=0x7f379c038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 --2- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f379c036820 0x7f379c038cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3798009990 tx=0x7f3798006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 msgr2=0x7f37b0071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 --2- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 0x7f37b0071e60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009680 tx=0x7f37a0009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 shutdown_connections 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 --2- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f379c036820 0x7f379c038cd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 --2- 172.21.15.192:0/2479436493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b00fcbd0 0x7f37b0071e60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 >> 172.21.15.192:0/2479436493 conn(0x7f37b00f8290 msgr2=0x7f37b00f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 shutdown_connections 2024-02-16T22:48:27.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:27.981+0000 7f37b6099700 1 -- 172.21.15.192:0/2479436493 wait complete. 2024-02-16T22:48:28.569 DEBUG:teuthology.orchestra.run.smithi192:mon.smithi192> sudo journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi192.service 2024-02-16T22:48:28.571 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:28.572 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:28.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:28 smithi064 ceph-mon[29226]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-16T22:48:28.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:28 smithi064 ceph-mon[29226]: Deploying daemon grafana.smithi064 on smithi064 2024-02-16T22:48:28.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:28 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:28.847 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:28.894 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:29.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:29 smithi064 ceph-mon[29226]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi064:172.21.15.64=smithi064;smithi192:172.21.15.192=smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:48:29.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:29 smithi064 ceph-mon[29226]: Saving service mon spec with placement smithi064:172.21.15.64=smithi064;smithi192:172.21.15.192=smithi192;count:2 2024-02-16T22:48:29.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:29 smithi064 ceph-mon[29226]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:29.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:29 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 -- 172.21.15.192:0/3339496797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 msgr2=0x7f606c0fd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 --2- 172.21.15.192:0/3339496797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 0x7f606c0fd4a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f605c0048c0 tx=0x7f605c009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 -- 172.21.15.192:0/3339496797 shutdown_connections 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 --2- 172.21.15.192:0/3339496797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 0x7f606c0fd4a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 -- 172.21.15.192:0/3339496797 >> 172.21.15.192:0/3339496797 conn(0x7f606c0f6920 msgr2=0x7f606c0f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 -- 172.21.15.192:0/3339496797 shutdown_connections 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 -- 172.21.15.192:0/3339496797 wait complete. 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 Processor -- start 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.406+0000 7f60712c7700 1 -- start start 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f60712c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 0x7f606c10cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:30.409 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f60712c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c10d430 con 0x7f606c0fd090 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f606affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 0x7f606c10cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f606affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 0x7f606c10cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:55508/0 (socket says 172.21.15.192:55508) 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f606affd700 1 -- 172.21.15.192:0/2098583771 learned_addr learned my addr 172.21.15.192:0/2098583771 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f606affd700 1 -- 172.21.15.192:0/2098583771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c005040 con 0x7f606c0fd090 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f606affd700 1 --2- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 0x7f606c10cef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f605c009b80 tx=0x7f605c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f60537fe700 1 -- 172.21.15.192:0/2098583771 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f605c00b5c0 con 0x7f606c0fd090 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f60537fe700 1 -- 172.21.15.192:0/2098583771 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f605c00be70 con 0x7f606c0fd090 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f60537fe700 1 -- 172.21.15.192:0/2098583771 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f605c01c660 con 0x7f606c0fd090 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606c10d630 con 0x7f606c0fd090 2024-02-16T22:48:30.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.407+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c10e2d0 con 0x7f606c0fd090 2024-02-16T22:48:30.411 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.408+0000 7f60537fe700 1 -- 172.21.15.192:0/2098583771 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f605c013420 con 0x7f606c0fd090 2024-02-16T22:48:30.411 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.408+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606c05f5c0 con 0x7f606c0fd090 2024-02-16T22:48:30.411 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.408+0000 7f60537fe700 1 --2- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f6054036870 0x7f6054038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:30.411 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.408+0000 7f60537fe700 1 -- 172.21.15.192:0/2098583771 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f605c049a80 con 0x7f606c0fd090 2024-02-16T22:48:30.411 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.409+0000 7f606a7fc700 1 --2- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f6054036870 0x7f6054038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:30.411 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.409+0000 7f606a7fc700 1 --2- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f6054036870 0x7f6054038d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6060009990 tx=0x7f6060006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:30.415 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.413+0000 7f60537fe700 1 -- 172.21.15.192:0/2098583771 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f605c011550 con 0x7f606c0fd090 2024-02-16T22:48:30.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.666+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f606c10e650 con 0x7f606c0fd090 2024-02-16T22:48:30.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.667+0000 7f60537fe700 1 -- 172.21.15.192:0/2098583771 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f605c011370 con 0x7f606c0fd090 2024-02-16T22:48:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:30.669 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:30.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.668+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f6054036870 msgr2=0x7f6054038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:30.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.668+0000 7f60712c7700 1 --2- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f6054036870 0x7f6054038d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6060009990 tx=0x7f6060006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:30.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.668+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 msgr2=0x7f606c10cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:30.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.669+0000 7f60712c7700 1 --2- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 0x7f606c10cef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f605c009b80 tx=0x7f605c009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:30.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.669+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 shutdown_connections 2024-02-16T22:48:30.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.669+0000 7f60712c7700 1 --2- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f6054036870 0x7f6054038d20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:30.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.669+0000 7f60712c7700 1 --2- 172.21.15.192:0/2098583771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606c0fd090 0x7f606c10cef0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:30.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.669+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 >> 172.21.15.192:0/2098583771 conn(0x7f606c0f6920 msgr2=0x7f606c0f7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:30.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.669+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 shutdown_connections 2024-02-16T22:48:30.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:30.669+0000 7f60712c7700 1 -- 172.21.15.192:0/2098583771 wait complete. 2024-02-16T22:48:30.672 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:48:31.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:31 smithi064 ceph-mon[29226]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:31.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:31 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/2098583771' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:48:32.251 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:32.251 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:32.547 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:33.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:33 smithi064 ceph-mon[29226]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:34.034 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2484363325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 msgr2=0x7f2bb80fbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:34.034 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 --2- 172.21.15.192:0/2484363325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 0x7f2bb80fbed0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40048c0 tx=0x7f2bb4009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:34.034 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2484363325 shutdown_connections 2024-02-16T22:48:34.034 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 --2- 172.21.15.192:0/2484363325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 0x7f2bb80fbed0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:34.034 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2484363325 >> 172.21.15.192:0/2484363325 conn(0x7f2bb80f71a0 msgr2=0x7f2bb80f95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:34.034 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2484363325 shutdown_connections 2024-02-16T22:48:34.034 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2484363325 wait complete. 2024-02-16T22:48:34.035 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 Processor -- start 2024-02-16T22:48:34.035 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 -- start start 2024-02-16T22:48:34.035 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.032+0000 7f2bbfba3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 0x7f2bb810fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:34.035 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bbfba3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8066810 con 0x7f2bb80fbac0 2024-02-16T22:48:34.035 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bbd93f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 0x7f2bb810fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:34.035 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bbd93f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 0x7f2bb810fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:55536/0 (socket says 172.21.15.192:55536) 2024-02-16T22:48:34.035 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bbd93f700 1 -- 172.21.15.192:0/2940620405 learned_addr learned my addr 172.21.15.192:0/2940620405 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:34.035 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bbd93f700 1 -- 172.21.15.192:0/2940620405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb4005040 con 0x7f2bb80fbac0 2024-02-16T22:48:34.036 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bbd93f700 1 --2- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 0x7f2bb810fc00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009680 tx=0x7f2bb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bae7fc700 1 -- 172.21.15.192:0/2940620405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bb400bed0 con 0x7f2bb80fbac0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bae7fc700 1 -- 172.21.15.192:0/2940620405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bb400b5c0 con 0x7f2bb80fbac0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb8066a10 con 0x7f2bb80fbac0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bae7fc700 1 -- 172.21.15.192:0/2940620405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bb401c5b0 con 0x7f2bb80fbac0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.033+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb8110380 con 0x7f2bb80fbac0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.034+0000 7f2bae7fc700 1 -- 172.21.15.192:0/2940620405 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f2bb4013a80 con 0x7f2bb80fbac0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.034+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb8105890 con 0x7f2bb80fbac0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.034+0000 7f2bae7fc700 1 --2- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f2ba4036870 0x7f2ba4038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.034+0000 7f2bae7fc700 1 -- 172.21.15.192:0/2940620405 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2bb4049a90 con 0x7f2bb80fbac0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.035+0000 7f2bbd13e700 1 --2- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f2ba4036870 0x7f2ba4038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:34.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.035+0000 7f2bbd13e700 1 --2- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f2ba4036870 0x7f2ba4038d20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009990 tx=0x7f2ba8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:34.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.040+0000 7f2bae7fc700 1 -- 172.21.15.192:0/2940620405 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2bb4011940 con 0x7f2bb80fbac0 2024-02-16T22:48:34.299 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.297+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2bb804bed0 con 0x7f2bb80fbac0 2024-02-16T22:48:34.299 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.297+0000 7f2bae7fc700 1 -- 172.21.15.192:0/2940620405 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2bb4011370 con 0x7f2bb80fbac0 2024-02-16T22:48:34.299 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:34.299 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f2ba4036870 msgr2=0x7f2ba4038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 --2- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f2ba4036870 0x7f2ba4038d20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009990 tx=0x7f2ba8006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 msgr2=0x7f2bb810fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 --2- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 0x7f2bb810fc00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009680 tx=0x7f2bb4009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 shutdown_connections 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 --2- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f2ba4036870 0x7f2ba4038d20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 --2- 172.21.15.192:0/2940620405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bb80fbac0 0x7f2bb810fc00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 >> 172.21.15.192:0/2940620405 conn(0x7f2bb80f71a0 msgr2=0x7f2bb80f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:34.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.299+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 shutdown_connections 2024-02-16T22:48:34.302 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:34.300+0000 7f2bbfba3700 1 -- 172.21.15.192:0/2940620405 wait complete. 2024-02-16T22:48:34.302 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:48:34.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:34 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/2940620405' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:48:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:35 smithi064 ceph-mon[29226]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:35.915 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:35.916 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:36.159 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:36.207 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:37.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:37 smithi064 ceph-mon[29226]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 -- 172.21.15.192:0/295839331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 msgr2=0x7f58f40fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 --2- 172.21.15.192:0/295839331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 0x7f58f40fb650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f58e80048c0 tx=0x7f58e8009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 -- 172.21.15.192:0/295839331 shutdown_connections 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 --2- 172.21.15.192:0/295839331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 0x7f58f40fb650 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 -- 172.21.15.192:0/295839331 >> 172.21.15.192:0/295839331 conn(0x7f58f40f6920 msgr2=0x7f58f40f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 -- 172.21.15.192:0/295839331 shutdown_connections 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 -- 172.21.15.192:0/295839331 wait complete. 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 Processor -- start 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.658+0000 7f58faab4700 1 -- start start 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58faab4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 0x7f58f410cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58faab4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f410d400 con 0x7f58f40fb240 2024-02-16T22:48:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58f8850700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 0x7f58f410cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58f8850700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 0x7f58f410cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:55544/0 (socket says 172.21.15.192:55544) 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58f8850700 1 -- 172.21.15.192:0/992491492 learned_addr learned my addr 172.21.15.192:0/992491492 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58f8850700 1 -- 172.21.15.192:0/992491492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e8005040 con 0x7f58f40fb240 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58f8850700 1 --2- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 0x7f58f410cec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009b80 tx=0x7f58e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58f17fa700 1 -- 172.21.15.192:0/992491492 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58e800b5c0 con 0x7f58f40fb240 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58f17fa700 1 -- 172.21.15.192:0/992491492 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58e800be70 con 0x7f58f40fb240 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.659+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f410d600 con 0x7f58f40fb240 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.660+0000 7f58f17fa700 1 -- 172.21.15.192:0/992491492 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58e801c660 con 0x7f58f40fb240 2024-02-16T22:48:37.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.660+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f410e2a0 con 0x7f58f40fb240 2024-02-16T22:48:37.663 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.660+0000 7f58f17fa700 1 -- 172.21.15.192:0/992491492 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f58e801c8d0 con 0x7f58f40fb240 2024-02-16T22:48:37.663 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.661+0000 7f58f17fa700 1 --2- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f58e4036870 0x7f58e4038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:37.663 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.661+0000 7f58f17fa700 1 -- 172.21.15.192:0/992491492 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f58e8049c60 con 0x7f58f40fb240 2024-02-16T22:48:37.663 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.661+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f405f5c0 con 0x7f58f40fb240 2024-02-16T22:48:37.663 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.661+0000 7f58f3fff700 1 --2- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f58e4036870 0x7f58e4038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:37.663 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.661+0000 7f58f3fff700 1 --2- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f58e4036870 0x7f58e4038d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009990 tx=0x7f58e0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:37.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.666+0000 7f58f17fa700 1 -- 172.21.15.192:0/992491492 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f58e80119c0 con 0x7f58f40fb240 2024-02-16T22:48:37.925 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.924+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f58f404bed0 con 0x7f58f40fb240 2024-02-16T22:48:37.926 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.924+0000 7f58f17fa700 1 -- 172.21.15.192:0/992491492 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f58e8011370 con 0x7f58f40fb240 2024-02-16T22:48:37.926 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:37.926 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.926+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f58e4036870 msgr2=0x7f58e4038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.926+0000 7f58faab4700 1 --2- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f58e4036870 0x7f58e4038d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009990 tx=0x7f58e0006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.926+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 msgr2=0x7f58f410cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.926+0000 7f58faab4700 1 --2- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 0x7f58f410cec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009b80 tx=0x7f58e8009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.927+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 shutdown_connections 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.927+0000 7f58faab4700 1 --2- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f58e4036870 0x7f58e4038d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.927+0000 7f58faab4700 1 --2- 172.21.15.192:0/992491492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f40fb240 0x7f58f410cec0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.927+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 >> 172.21.15.192:0/992491492 conn(0x7f58f40f6920 msgr2=0x7f58f40f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:37.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.927+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 shutdown_connections 2024-02-16T22:48:37.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:37.927+0000 7f58faab4700 1 -- 172.21.15.192:0/992491492 wait complete. 2024-02-16T22:48:37.929 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:48:39.067 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:38 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/992491492' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:48:39.067 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:38 smithi064 ceph-mon[29226]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:39.571 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:39.572 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:39.819 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:39.865 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:40.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:40 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:40.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:40 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:40.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:40 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:40.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:40 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:40.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:40 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:40.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:40 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:40.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:40 smithi064 ceph-mon[29226]: Deploying daemon prometheus.smithi064 on smithi064 2024-02-16T22:48:41.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.223+0000 7f70e5cd8700 1 -- 172.21.15.192:0/3506909562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 msgr2=0x7f70e00fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.223+0000 7f70e5cd8700 1 --2- 172.21.15.192:0/3506909562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 0x7f70e00fb290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70d00048c0 tx=0x7f70d0009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.223+0000 7f70e5cd8700 1 -- 172.21.15.192:0/3506909562 shutdown_connections 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.223+0000 7f70e5cd8700 1 --2- 172.21.15.192:0/3506909562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 0x7f70e00fb290 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.223+0000 7f70e5cd8700 1 -- 172.21.15.192:0/3506909562 >> 172.21.15.192:0/3506909562 conn(0x7f70e00f6920 msgr2=0x7f70e00f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.223+0000 7f70e5cd8700 1 -- 172.21.15.192:0/3506909562 shutdown_connections 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.223+0000 7f70e5cd8700 1 -- 172.21.15.192:0/3506909562 wait complete. 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.224+0000 7f70e5cd8700 1 Processor -- start 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.224+0000 7f70e5cd8700 1 -- start start 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.224+0000 7f70e5cd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 0x7f70e010cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.224+0000 7f70e5cd8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e010d420 con 0x7f70e00fae80 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.224+0000 7f70df7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 0x7f70e010cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.224+0000 7f70df7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 0x7f70e010cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:56992/0 (socket says 172.21.15.192:56992) 2024-02-16T22:48:41.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.224+0000 7f70df7fe700 1 -- 172.21.15.192:0/412734145 learned_addr learned my addr 172.21.15.192:0/412734145 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.225+0000 7f70df7fe700 1 -- 172.21.15.192:0/412734145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0005040 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.225+0000 7f70df7fe700 1 --2- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 0x7f70e010cee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009040 tx=0x7f70d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.225+0000 7f70dcff9700 1 -- 172.21.15.192:0/412734145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70d000bed0 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.225+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e010d620 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.225+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e010e2c0 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.225+0000 7f70dcff9700 1 -- 172.21.15.192:0/412734145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70d000b5c0 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.225+0000 7f70dcff9700 1 -- 172.21.15.192:0/412734145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70d001c5b0 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.226+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e005f5c0 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.226+0000 7f70dcff9700 1 -- 172.21.15.192:0/412734145 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f70d0004d30 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.226+0000 7f70dcff9700 1 --2- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70cc036870 0x7f70cc038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.226+0000 7f70dcff9700 1 -- 172.21.15.192:0/412734145 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f70d0049b00 con 0x7f70e00fae80 2024-02-16T22:48:41.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.226+0000 7f70d6ffd700 1 --2- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70cc036870 0x7f70cc038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:41.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.227+0000 7f70d6ffd700 1 --2- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70cc036870 0x7f70cc038d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009990 tx=0x7f70c8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:41.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.235+0000 7f70dcff9700 1 -- 172.21.15.192:0/412734145 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f70d0011550 con 0x7f70e00fae80 2024-02-16T22:48:41.494 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.493+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f70e010ee30 con 0x7f70e00fae80 2024-02-16T22:48:41.495 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.493+0000 7f70dcff9700 1 -- 172.21.15.192:0/412734145 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f70d0011370 con 0x7f70e00fae80 2024-02-16T22:48:41.495 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:41.495 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70cc036870 msgr2=0x7f70cc038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 --2- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70cc036870 0x7f70cc038d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009990 tx=0x7f70c8006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 msgr2=0x7f70e010cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 --2- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 0x7f70e010cee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009040 tx=0x7f70d0009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 shutdown_connections 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 --2- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f70cc036870 0x7f70cc038d20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 --2- 172.21.15.192:0/412734145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70e00fae80 0x7f70e010cee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 >> 172.21.15.192:0/412734145 conn(0x7f70e00f6920 msgr2=0x7f70e00f76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 shutdown_connections 2024-02-16T22:48:41.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:41.495+0000 7f70e5cd8700 1 -- 172.21.15.192:0/412734145 wait complete. 2024-02-16T22:48:41.498 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:48:41.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:41 smithi064 ceph-mon[29226]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:42.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:42 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/412734145' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:48:43.022 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:43.022 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:43.267 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:43.314 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:43.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:43 smithi064 ceph-mon[29226]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:44.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 -- 172.21.15.192:0/1763215066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 msgr2=0x7f0e680fd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:44.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 --2- 172.21.15.192:0/1763215066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 0x7f0e680fd910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e580048c0 tx=0x7f0e58009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:44.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 -- 172.21.15.192:0/1763215066 shutdown_connections 2024-02-16T22:48:44.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 --2- 172.21.15.192:0/1763215066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 0x7f0e680fd910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:44.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 -- 172.21.15.192:0/1763215066 >> 172.21.15.192:0/1763215066 conn(0x7f0e680f71a0 msgr2=0x7f0e680f95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:44.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 -- 172.21.15.192:0/1763215066 shutdown_connections 2024-02-16T22:48:44.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 -- 172.21.15.192:0/1763215066 wait complete. 2024-02-16T22:48:44.768 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 Processor -- start 2024-02-16T22:48:44.768 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.764+0000 7f0e6d842700 1 -- start start 2024-02-16T22:48:44.768 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e6d842700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 0x7f0e681134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:44.768 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e6d842700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e681139e0 con 0x7f0e680fb4f0 2024-02-16T22:48:44.768 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e66ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 0x7f0e681134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e66ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 0x7f0e681134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:57004/0 (socket says 172.21.15.192:57004) 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e66ffd700 1 -- 172.21.15.192:0/3672169005 learned_addr learned my addr 172.21.15.192:0/3672169005 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e66ffd700 1 -- 172.21.15.192:0/3672169005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58005040 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e66ffd700 1 --2- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 0x7f0e681134a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009680 tx=0x7f0e58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e4f7fe700 1 -- 172.21.15.192:0/3672169005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e5800b5c0 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e4f7fe700 1 -- 172.21.15.192:0/3672169005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e58004e60 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e4f7fe700 1 -- 172.21.15.192:0/3672169005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e5801c590 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.765+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e68113be0 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.766+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e68114880 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.766+0000 7f0e4f7fe700 1 -- 172.21.15.192:0/3672169005 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f0e5801c7c0 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.766+0000 7f0e4f7fe700 1 --2- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0e500368c0 0x7f0e50038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.766+0000 7f0e4f7fe700 1 -- 172.21.15.192:0/3672169005 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0e58049b20 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.767+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6805f5c0 con 0x7f0e680fb4f0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.767+0000 7f0e667fc700 1 --2- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0e500368c0 0x7f0e50038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:44.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.767+0000 7f0e667fc700 1 --2- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0e500368c0 0x7f0e50038d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009990 tx=0x7f0e5c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:44.773 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:44.772+0000 7f0e4f7fe700 1 -- 172.21.15.192:0/3672169005 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0e58023300 con 0x7f0e680fb4f0 2024-02-16T22:48:45.054 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.053+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0e68115330 con 0x7f0e680fb4f0 2024-02-16T22:48:45.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.054+0000 7f0e4f7fe700 1 -- 172.21.15.192:0/3672169005 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0e58034090 con 0x7f0e680fb4f0 2024-02-16T22:48:45.055 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:45.055 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:45.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0e500368c0 msgr2=0x7f0e50038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:45.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 --2- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0e500368c0 0x7f0e50038d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009990 tx=0x7f0e5c006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:45.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 msgr2=0x7f0e681134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:45.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 --2- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 0x7f0e681134a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009680 tx=0x7f0e58009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:45.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 shutdown_connections 2024-02-16T22:48:45.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 --2- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0e500368c0 0x7f0e50038d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:45.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 --2- 172.21.15.192:0/3672169005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e680fb4f0 0x7f0e681134a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:45.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 >> 172.21.15.192:0/3672169005 conn(0x7f0e680f71a0 msgr2=0x7f0e680f7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:45.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 shutdown_connections 2024-02-16T22:48:45.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:45.056+0000 7f0e6d842700 1 -- 172.21.15.192:0/3672169005 wait complete. 2024-02-16T22:48:45.058 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:48:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:45 smithi064 ceph-mon[29226]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:45 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:45 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/3672169005' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:48:46.610 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:46.610 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:46.855 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:46.901 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:47.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:47 smithi064 ceph-mon[29226]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:48.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.309+0000 7f4ae8251700 1 -- 172.21.15.192:0/2286529375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 msgr2=0x7f4ae00fbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:48.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.309+0000 7f4ae8251700 1 --2- 172.21.15.192:0/2286529375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 0x7f4ae00fbed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0048c0 tx=0x7f4adc009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:48.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.309+0000 7f4ae8251700 1 -- 172.21.15.192:0/2286529375 shutdown_connections 2024-02-16T22:48:48.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.309+0000 7f4ae8251700 1 --2- 172.21.15.192:0/2286529375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 0x7f4ae00fbed0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.309+0000 7f4ae8251700 1 -- 172.21.15.192:0/2286529375 >> 172.21.15.192:0/2286529375 conn(0x7f4ae00f71a0 msgr2=0x7f4ae00f95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.309+0000 7f4ae8251700 1 -- 172.21.15.192:0/2286529375 shutdown_connections 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.309+0000 7f4ae8251700 1 -- 172.21.15.192:0/2286529375 wait complete. 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.310+0000 7f4ae8251700 1 Processor -- start 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.310+0000 7f4ae8251700 1 -- start start 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.310+0000 7f4ae8251700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 0x7f4ae010fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.310+0000 7f4ae8251700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0066810 con 0x7f4ae00fbac0 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.310+0000 7f4ae5fed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 0x7f4ae010fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.310+0000 7f4ae5fed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 0x7f4ae010fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:57024/0 (socket says 172.21.15.192:57024) 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.310+0000 7f4ae5fed700 1 -- 172.21.15.192:0/1800594144 learned_addr learned my addr 172.21.15.192:0/1800594144 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.311+0000 7f4ae5fed700 1 -- 172.21.15.192:0/1800594144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc005040 con 0x7f4ae00fbac0 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.311+0000 7f4ae5fed700 1 --2- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 0x7f4ae010fc00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009680 tx=0x7f4adc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.311+0000 7f4ad67fc700 1 -- 172.21.15.192:0/1800594144 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4adc00bed0 con 0x7f4ae00fbac0 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.311+0000 7f4ad67fc700 1 -- 172.21.15.192:0/1800594144 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4adc00b5c0 con 0x7f4ae00fbac0 2024-02-16T22:48:48.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.311+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae0066a10 con 0x7f4ae00fbac0 2024-02-16T22:48:48.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.311+0000 7f4ad67fc700 1 -- 172.21.15.192:0/1800594144 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4adc01c5b0 con 0x7f4ae00fbac0 2024-02-16T22:48:48.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.311+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae0110380 con 0x7f4ae00fbac0 2024-02-16T22:48:48.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.312+0000 7f4ad67fc700 1 -- 172.21.15.192:0/1800594144 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f4adc01c820 con 0x7f4ae00fbac0 2024-02-16T22:48:48.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.312+0000 7f4ad67fc700 1 --2- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4acc036820 0x7f4acc038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:48.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.312+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae0105890 con 0x7f4ae00fbac0 2024-02-16T22:48:48.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.312+0000 7f4ad67fc700 1 -- 172.21.15.192:0/1800594144 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4adc049c20 con 0x7f4ae00fbac0 2024-02-16T22:48:48.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.312+0000 7f4ae57ec700 1 --2- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4acc036820 0x7f4acc038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:48.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.313+0000 7f4ae57ec700 1 --2- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4acc036820 0x7f4acc038cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009990 tx=0x7f4ad0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:48.318 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.317+0000 7f4ad67fc700 1 -- 172.21.15.192:0/1800594144 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4adc011550 con 0x7f4ae00fbac0 2024-02-16T22:48:48.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.573+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ae004bed0 con 0x7f4ae00fbac0 2024-02-16T22:48:48.575 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.574+0000 7f4ad67fc700 1 -- 172.21.15.192:0/1800594144 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4adc011370 con 0x7f4ae00fbac0 2024-02-16T22:48:48.575 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:48.575 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:48.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.575+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4acc036820 msgr2=0x7f4acc038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:48.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.575+0000 7f4ae8251700 1 --2- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4acc036820 0x7f4acc038cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009990 tx=0x7f4ad0006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:48.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.575+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 msgr2=0x7f4ae010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:48.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.575+0000 7f4ae8251700 1 --2- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 0x7f4ae010fc00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009680 tx=0x7f4adc009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:48.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.576+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 shutdown_connections 2024-02-16T22:48:48.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.576+0000 7f4ae8251700 1 --2- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f4acc036820 0x7f4acc038cd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:48.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.576+0000 7f4ae8251700 1 --2- 172.21.15.192:0/1800594144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ae00fbac0 0x7f4ae010fc00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:48.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.576+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 >> 172.21.15.192:0/1800594144 conn(0x7f4ae00f71a0 msgr2=0x7f4ae00f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:48.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.576+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 shutdown_connections 2024-02-16T22:48:48.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:48.576+0000 7f4ae8251700 1 -- 172.21.15.192:0/1800594144 wait complete. 2024-02-16T22:48:48.578 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:48:49.026 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:48 smithi064 ceph-mon[29226]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:49.026 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:48 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/1800594144' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:48:50.157 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:50.157 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:50.404 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:50.451 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:51.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:51 smithi064 ceph-mon[29226]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:48:51.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:51 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:51.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:51 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:51.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:51 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:51.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:51 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 -- 172.21.15.192:0/758620716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 msgr2=0x7f01600fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 --2- 172.21.15.192:0/758620716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 0x7f01600fe610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f01500048c0 tx=0x7f0150009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 -- 172.21.15.192:0/758620716 shutdown_connections 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 --2- 172.21.15.192:0/758620716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 0x7f01600fe610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 -- 172.21.15.192:0/758620716 >> 172.21.15.192:0/758620716 conn(0x7f01600f69c0 msgr2=0x7f01600f8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 -- 172.21.15.192:0/758620716 shutdown_connections 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 -- 172.21.15.192:0/758620716 wait complete. 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 Processor -- start 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 -- start start 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 0x7f01600fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.929+0000 7f0164f1a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01600fbec0 con 0x7f01600fe200 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f015e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 0x7f01600fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:51.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f015e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 0x7f01600fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:46992/0 (socket says 172.21.15.192:46992) 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f015e59c700 1 -- 172.21.15.192:0/3681082497 learned_addr learned my addr 172.21.15.192:0/3681082497 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f015e59c700 1 -- 172.21.15.192:0/3681082497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150005040 con 0x7f01600fe200 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f015e59c700 1 --2- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 0x7f01600fda40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0150009300 tx=0x7f0150009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f014effd700 1 -- 172.21.15.192:0/3681082497 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0150004e60 con 0x7f01600fe200 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01600fc0c0 con 0x7f01600fe200 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f014effd700 1 -- 172.21.15.192:0/3681082497 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0150009680 con 0x7f01600fe200 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f014effd700 1 -- 172.21.15.192:0/3681082497 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f015001c5b0 con 0x7f01600fe200 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.930+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01600fc5b0 con 0x7f01600fe200 2024-02-16T22:48:51.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.931+0000 7f014effd700 1 -- 172.21.15.192:0/3681082497 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f015001c870 con 0x7f01600fe200 2024-02-16T22:48:51.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.931+0000 7f014effd700 1 --2- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f01480368c0 0x7f0148038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:51.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.931+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0160106910 con 0x7f01600fe200 2024-02-16T22:48:51.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.931+0000 7f015dd9b700 1 -- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f01480368c0 msgr2=0x7f0148038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/600121841 2024-02-16T22:48:51.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.931+0000 7f014effd700 1 -- 172.21.15.192:0/3681082497 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0150049bf0 con 0x7f01600fe200 2024-02-16T22:48:51.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.931+0000 7f015dd9b700 1 --2- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f01480368c0 0x7f0148038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:48:51.941 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:51.940+0000 7f014effd700 1 -- 172.21.15.192:0/3681082497 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0150011bf0 con 0x7f01600fe200 2024-02-16T22:48:52.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.132+0000 7f015dd9b700 1 -- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f01480368c0 msgr2=0x7f0148038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/600121841 2024-02-16T22:48:52.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.132+0000 7f015dd9b700 1 --2- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f01480368c0 0x7f0148038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:48:52.196 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.195+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f016004bed0 con 0x7f01600fe200 2024-02-16T22:48:52.196 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.195+0000 7f014effd700 1 -- 172.21.15.192:0/3681082497 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f01500117e0 con 0x7f01600fe200 2024-02-16T22:48:52.196 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:52.197 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f01480368c0 msgr2=0x7f0148038d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 --2- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f01480368c0 0x7f0148038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 msgr2=0x7f01600fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 --2- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 0x7f01600fda40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0150009300 tx=0x7f0150009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 shutdown_connections 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 --2- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f01480368c0 0x7f0148038d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 --2- 172.21.15.192:0/3681082497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01600fe200 0x7f01600fda40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 >> 172.21.15.192:0/3681082497 conn(0x7f01600f69c0 msgr2=0x7f01600f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 shutdown_connections 2024-02-16T22:48:52.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:52.197+0000 7f0164f1a700 1 -- 172.21.15.192:0/3681082497 wait complete. 2024-02-16T22:48:52.200 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:48:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:52 smithi064 ceph-mon[29226]: from='mgr.14162 172.21.15.64:0/2094027297' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-16T22:48:52.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:52 smithi064 ceph-mon[29226]: mgrmap e14: smithi064.kubmlk(active, since 57s) 2024-02-16T22:48:52.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:52 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/3681082497' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:48:53.727 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:53.727 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:53.973 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:54.019 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:55.553 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.550+0000 7f0d66db3700 1 -- 172.21.15.192:0/3309849130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 msgr2=0x7f0d600fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.550+0000 7f0d66db3700 1 --2- 172.21.15.192:0/3309849130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 0x7f0d600fbee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d540048c0 tx=0x7f0d54009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.550+0000 7f0d66db3700 1 -- 172.21.15.192:0/3309849130 shutdown_connections 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.550+0000 7f0d66db3700 1 --2- 172.21.15.192:0/3309849130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 0x7f0d600fbee0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.550+0000 7f0d66db3700 1 -- 172.21.15.192:0/3309849130 >> 172.21.15.192:0/3309849130 conn(0x7f0d600f7190 msgr2=0x7f0d600f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.550+0000 7f0d66db3700 1 -- 172.21.15.192:0/3309849130 shutdown_connections 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.550+0000 7f0d66db3700 1 -- 172.21.15.192:0/3309849130 wait complete. 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.551+0000 7f0d66db3700 1 Processor -- start 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.551+0000 7f0d66db3700 1 -- start start 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.551+0000 7f0d66db3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 0x7f0d6010fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.551+0000 7f0d66db3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d60065dc0 con 0x7f0d600fbad0 2024-02-16T22:48:55.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.551+0000 7f0d64b4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 0x7f0d6010fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.551+0000 7f0d64b4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 0x7f0d6010fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:47010/0 (socket says 172.21.15.192:47010) 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.551+0000 7f0d64b4f700 1 -- 172.21.15.192:0/2102261667 learned_addr learned my addr 172.21.15.192:0/2102261667 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.551+0000 7f0d64b4f700 1 -- 172.21.15.192:0/2102261667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54005040 con 0x7f0d600fbad0 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.552+0000 7f0d64b4f700 1 --2- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 0x7f0d6010fb70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009680 tx=0x7f0d54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.552+0000 7f0d5d7fa700 1 -- 172.21.15.192:0/2102261667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d5400bed0 con 0x7f0d600fbad0 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.552+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d60065fc0 con 0x7f0d600fbad0 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.552+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d601102f0 con 0x7f0d600fbad0 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.552+0000 7f0d5d7fa700 1 -- 172.21.15.192:0/2102261667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d5400b5c0 con 0x7f0d600fbad0 2024-02-16T22:48:55.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.552+0000 7f0d5d7fa700 1 -- 172.21.15.192:0/2102261667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d5401c5b0 con 0x7f0d600fbad0 2024-02-16T22:48:55.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.553+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d440052e0 con 0x7f0d600fbad0 2024-02-16T22:48:55.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.553+0000 7f0d5d7fa700 1 -- 172.21.15.192:0/2102261667 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f0d54013420 con 0x7f0d600fbad0 2024-02-16T22:48:55.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.553+0000 7f0d5d7fa700 1 --2- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0d500368c0 0x7f0d50038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:55.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.553+0000 7f0d5d7fa700 1 -- 172.21.15.192:0/2102261667 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0d54049b60 con 0x7f0d600fbad0 2024-02-16T22:48:55.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.553+0000 7f0d5ffff700 1 -- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0d500368c0 msgr2=0x7f0d50038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/600121841 2024-02-16T22:48:55.557 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.553+0000 7f0d5ffff700 1 --2- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0d500368c0 0x7f0d50038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:48:55.559 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.558+0000 7f0d5d7fa700 1 -- 172.21.15.192:0/2102261667 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0d54025400 con 0x7f0d600fbad0 2024-02-16T22:48:55.755 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.754+0000 7f0d5ffff700 1 -- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0d500368c0 msgr2=0x7f0d50038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/600121841 2024-02-16T22:48:55.755 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.754+0000 7f0d5ffff700 1 --2- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0d500368c0 0x7f0d50038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:48:55.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.816+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0d44005590 con 0x7f0d600fbad0 2024-02-16T22:48:55.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.816+0000 7f0d5d7fa700 1 -- 172.21.15.192:0/2102261667 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0d54004d30 con 0x7f0d600fbad0 2024-02-16T22:48:55.818 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:55.818 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:55.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0d500368c0 msgr2=0x7f0d50038d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 --2- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0d500368c0 0x7f0d50038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 msgr2=0x7f0d6010fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 --2- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 0x7f0d6010fb70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009680 tx=0x7f0d54009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 shutdown_connections 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 --2- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:6800/600121841,v1:172.21.15.64:6801/600121841] conn(0x7f0d500368c0 0x7f0d50038d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 --2- 172.21.15.192:0/2102261667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d600fbad0 0x7f0d6010fb70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 >> 172.21.15.192:0/2102261667 conn(0x7f0d600f7190 msgr2=0x7f0d600f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.818+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 shutdown_connections 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:55.819+0000 7f0d66db3700 1 -- 172.21.15.192:0/2102261667 wait complete. 2024-02-16T22:48:55.821 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:48:56.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:55 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/2102261667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:48:57.381 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:48:57.382 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:48:57.627 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:57.673 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:48:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: Active manager daemon smithi064.kubmlk restarted 2024-02-16T22:48:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: Activating manager daemon smithi064.kubmlk 2024-02-16T22:48:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T22:48:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: mgrmap e15: smithi064.kubmlk(active, starting, since 0.0029569s) 2024-02-16T22:48:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: Manager daemon smithi064.kubmlk is now available 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:48:59.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:48:59.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.214+0000 7f2dd26f3700 1 -- 172.21.15.192:0/4079094334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 msgr2=0x7f2dcc0fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:59.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.214+0000 7f2dd26f3700 1 --2- 172.21.15.192:0/4079094334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 0x7f2dcc0fcfd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0048c0 tx=0x7f2dbc009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:59.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.214+0000 7f2dd26f3700 1 -- 172.21.15.192:0/4079094334 shutdown_connections 2024-02-16T22:48:59.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.214+0000 7f2dd26f3700 1 --2- 172.21.15.192:0/4079094334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 0x7f2dcc0fcfd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:59.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.214+0000 7f2dd26f3700 1 -- 172.21.15.192:0/4079094334 >> 172.21.15.192:0/4079094334 conn(0x7f2dcc0f82a0 msgr2=0x7f2dcc0fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:59.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.214+0000 7f2dd26f3700 1 -- 172.21.15.192:0/4079094334 shutdown_connections 2024-02-16T22:48:59.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.214+0000 7f2dd26f3700 1 -- 172.21.15.192:0/4079094334 wait complete. 2024-02-16T22:48:59.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.215+0000 7f2dd26f3700 1 Processor -- start 2024-02-16T22:48:59.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.215+0000 7f2dd26f3700 1 -- start start 2024-02-16T22:48:59.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.215+0000 7f2dd26f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 0x7f2dcc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:59.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.215+0000 7f2dd26f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0702e0 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.215+0000 7f2dcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 0x7f2dcc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:59.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.215+0000 7f2dcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 0x7f2dcc071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:55234/0 (socket says 172.21.15.192:55234) 2024-02-16T22:48:59.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.215+0000 7f2dcbfff700 1 -- 172.21.15.192:0/465504072 learned_addr learned my addr 172.21.15.192:0/465504072 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.216+0000 7f2dcbfff700 1 -- 172.21.15.192:0/465504072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc005040 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.216+0000 7f2dcbfff700 1 --2- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 0x7f2dcc071e60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009b80 tx=0x7f2dbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.216+0000 7f2dc8ff9700 1 -- 172.21.15.192:0/465504072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dbc00b5c0 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.216+0000 7f2dc8ff9700 1 -- 172.21.15.192:0/465504072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2dbc00be70 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.216+0000 7f2dc8ff9700 1 -- 172.21.15.192:0/465504072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dbc01c660 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.216+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc0704e0 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.216+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc0709d0 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.217+0000 7f2dc8ff9700 1 -- 172.21.15.192:0/465504072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7f2dbc01c8d0 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.217+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc106990 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.217+0000 7f2dc8ff9700 1 --2- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2db4036700 0x7f2db4038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.217+0000 7f2dc8ff9700 1 -- 172.21.15.192:0/465504072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f2dbc049a70 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.217+0000 7f2dcb7fe700 1 --2- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2db4036700 0x7f2db4038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:48:59.220 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.218+0000 7f2dcb7fe700 1 --2- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2db4036700 0x7f2db4038bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009990 tx=0x7f2dc0006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:48:59.223 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.222+0000 7f2dc8ff9700 1 -- 172.21.15.192:0/465504072 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2dbc013420 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.513+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2dcc04bed0 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.514+0000 7f2dc8ff9700 1 -- 172.21.15.192:0/465504072 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2dbc013420 con 0x7f2dcc0fcbc0 2024-02-16T22:48:59.515 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:48:59.515 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:48:59.517 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2db4036700 msgr2=0x7f2db4038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:59.517 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 --2- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2db4036700 0x7f2db4038bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009990 tx=0x7f2dc0006e40 comp rx=0 tx=0).stop 2024-02-16T22:48:59.517 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 msgr2=0x7f2dcc071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:48:59.517 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 --2- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 0x7f2dcc071e60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009b80 tx=0x7f2dbc009e30 comp rx=0 tx=0).stop 2024-02-16T22:48:59.518 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 shutdown_connections 2024-02-16T22:48:59.518 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 --2- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2db4036700 0x7f2db4038bb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:59.518 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 --2- 172.21.15.192:0/465504072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc0fcbc0 0x7f2dcc071e60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:48:59.518 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 >> 172.21.15.192:0/465504072 conn(0x7f2dcc0f82a0 msgr2=0x7f2dcc0f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:48:59.518 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 shutdown_connections 2024-02-16T22:48:59.518 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:48:59.516+0000 7f2dd26f3700 1 -- 172.21.15.192:0/465504072 wait complete. 2024-02-16T22:48:59.518 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:49:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:59 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:59 smithi064 ceph-mon[29226]: [16/Feb/2024:22:48:58] ENGINE Bus STARTING 2024-02-16T22:49:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:59 smithi064 ceph-mon[29226]: [16/Feb/2024:22:48:59] ENGINE Serving on https://172.21.15.64:7150 2024-02-16T22:49:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:59 smithi064 ceph-mon[29226]: [16/Feb/2024:22:48:59] ENGINE Bus STARTED 2024-02-16T22:49:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:59 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:59 smithi064 ceph-mon[29226]: mgrmap e16: smithi064.kubmlk(active, since 1.00475s) 2024-02-16T22:49:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:59 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:48:59 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/465504072' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:49:01.099 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:49:01.100 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:49:01.444 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:49:01.551 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:01.552 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:01 smithi064 ceph-mon[29226]: mgrmap e17: smithi064.kubmlk(active, since 2s) 2024-02-16T22:49:01.826 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:49:03.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 -- 172.21.15.192:0/2979719110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 msgr2=0x7fe17c0fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:03.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 --2- 172.21.15.192:0/2979719110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 0x7fe17c0fbee0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe1780048c0 tx=0x7fe178009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:03.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 -- 172.21.15.192:0/2979719110 shutdown_connections 2024-02-16T22:49:03.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 --2- 172.21.15.192:0/2979719110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 0x7fe17c0fbee0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 -- 172.21.15.192:0/2979719110 >> 172.21.15.192:0/2979719110 conn(0x7fe17c0f7190 msgr2=0x7fe17c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 -- 172.21.15.192:0/2979719110 shutdown_connections 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 -- 172.21.15.192:0/2979719110 wait complete. 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 Processor -- start 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.449+0000 7fe183b84700 1 -- start start 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.450+0000 7fe183b84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 0x7fe17c10cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.450+0000 7fe183b84700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c10d400 con 0x7fe17c0fbad0 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.450+0000 7fe181920700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 0x7fe17c10cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.450+0000 7fe181920700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 0x7fe17c10cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:55254/0 (socket says 172.21.15.192:55254) 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.450+0000 7fe181920700 1 -- 172.21.15.192:0/4070408565 learned_addr learned my addr 172.21.15.192:0/4070408565 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:49:03.453 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.450+0000 7fe181920700 1 -- 172.21.15.192:0/4070408565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe178005040 con 0x7fe17c0fbad0 2024-02-16T22:49:03.454 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.450+0000 7fe181920700 1 --2- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 0x7fe17c10cec0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe178009b80 tx=0x7fe178009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:03.454 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.450+0000 7fe1727fc700 1 -- 172.21.15.192:0/4070408565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe17800b5c0 con 0x7fe17c0fbad0 2024-02-16T22:49:03.454 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.451+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c10d600 con 0x7fe17c0fbad0 2024-02-16T22:49:03.454 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.451+0000 7fe1727fc700 1 -- 172.21.15.192:0/4070408565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe17800be70 con 0x7fe17c0fbad0 2024-02-16T22:49:03.454 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.451+0000 7fe1727fc700 1 -- 172.21.15.192:0/4070408565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe17801c660 con 0x7fe17c0fbad0 2024-02-16T22:49:03.454 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.451+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c10e2a0 con 0x7fe17c0fbad0 2024-02-16T22:49:03.455 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.451+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c05f5c0 con 0x7fe17c0fbad0 2024-02-16T22:49:03.455 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.451+0000 7fe1727fc700 1 -- 172.21.15.192:0/4070408565 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fe178004d30 con 0x7fe17c0fbad0 2024-02-16T22:49:03.455 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.452+0000 7fe1727fc700 1 --2- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe168036960 0x7fe168038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:03.455 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.452+0000 7fe1727fc700 1 -- 172.21.15.192:0/4070408565 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe178049cb0 con 0x7fe17c0fbad0 2024-02-16T22:49:03.455 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.452+0000 7fe18111f700 1 --2- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe168036960 0x7fe168038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:03.455 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.452+0000 7fe18111f700 1 --2- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe168036960 0x7fe168038e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009990 tx=0x7fe16c006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:03.458 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.457+0000 7fe1727fc700 1 -- 172.21.15.192:0/4070408565 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe178025400 con 0x7fe17c0fbad0 2024-02-16T22:49:03.723 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.722+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe17c10edf0 con 0x7fe17c0fbad0 2024-02-16T22:49:03.724 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.722+0000 7fe1727fc700 1 -- 172.21.15.192:0/4070408565 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe178011370 con 0x7fe17c0fbad0 2024-02-16T22:49:03.724 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:49:03.724 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:49:03.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe168036960 msgr2=0x7fe168038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:03.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 --2- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe168036960 0x7fe168038e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009990 tx=0x7fe16c006e40 comp rx=0 tx=0).stop 2024-02-16T22:49:03.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 msgr2=0x7fe17c10cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:03.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 --2- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 0x7fe17c10cec0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe178009b80 tx=0x7fe178009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:03.727 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 shutdown_connections 2024-02-16T22:49:03.727 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 --2- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe168036960 0x7fe168038e10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:03.727 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 --2- 172.21.15.192:0/4070408565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0fbad0 0x7fe17c10cec0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:03.727 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 >> 172.21.15.192:0/4070408565 conn(0x7fe17c0f7190 msgr2=0x7fe17c065960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:03.727 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 shutdown_connections 2024-02-16T22:49:03.727 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:03.725+0000 7fe183b84700 1 -- 172.21.15.192:0/4070408565 wait complete. 2024-02-16T22:49:03.727 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:49:04.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:03 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/4070408565' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:49:05.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:05.156 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:05.157 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:05.157 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:05.157 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:49:05.341 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:49:05.341 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:49:05.605 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:49:05.653 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:49:07.107 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.104+0000 7f1787f55700 1 -- 172.21.15.192:0/2354507406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 msgr2=0x7f17800fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.104+0000 7f1787f55700 1 --2- 172.21.15.192:0/2354507406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 0x7f17800fcfd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f17700048c0 tx=0x7f1770009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.104+0000 7f1787f55700 1 -- 172.21.15.192:0/2354507406 shutdown_connections 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.104+0000 7f1787f55700 1 --2- 172.21.15.192:0/2354507406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 0x7f17800fcfd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.104+0000 7f1787f55700 1 -- 172.21.15.192:0/2354507406 >> 172.21.15.192:0/2354507406 conn(0x7f17800f82a0 msgr2=0x7f17800fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.104+0000 7f1787f55700 1 -- 172.21.15.192:0/2354507406 shutdown_connections 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.104+0000 7f1787f55700 1 -- 172.21.15.192:0/2354507406 wait complete. 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.105+0000 7f1787f55700 1 Processor -- start 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.105+0000 7f1787f55700 1 -- start start 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.105+0000 7f1787f55700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 0x7f178010ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.105+0000 7f1787f55700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178010d3d0 con 0x7f17800fcbc0 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.105+0000 7f1785cf1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 0x7f178010ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:07.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.105+0000 7f1785cf1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 0x7f178010ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:55280/0 (socket says 172.21.15.192:55280) 2024-02-16T22:49:07.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.105+0000 7f1785cf1700 1 -- 172.21.15.192:0/1261839879 learned_addr learned my addr 172.21.15.192:0/1261839879 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:49:07.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.106+0000 7f1785cf1700 1 -- 172.21.15.192:0/1261839879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1770005040 con 0x7f17800fcbc0 2024-02-16T22:49:07.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.106+0000 7f1785cf1700 1 --2- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 0x7f178010ce90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1770009b80 tx=0x7f1770009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:07.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.106+0000 7f17767fc700 1 -- 172.21.15.192:0/1261839879 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f177000b5c0 con 0x7f17800fcbc0 2024-02-16T22:49:07.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.106+0000 7f17767fc700 1 -- 172.21.15.192:0/1261839879 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f177000be70 con 0x7f17800fcbc0 2024-02-16T22:49:07.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.106+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178010d5d0 con 0x7f17800fcbc0 2024-02-16T22:49:07.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.106+0000 7f17767fc700 1 -- 172.21.15.192:0/1261839879 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f177001c660 con 0x7f17800fcbc0 2024-02-16T22:49:07.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.106+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178010e270 con 0x7f17800fcbc0 2024-02-16T22:49:07.110 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.107+0000 7f17767fc700 1 -- 172.21.15.192:0/1261839879 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f1770013420 con 0x7f17800fcbc0 2024-02-16T22:49:07.110 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.107+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178005f5c0 con 0x7f17800fcbc0 2024-02-16T22:49:07.110 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.107+0000 7f17767fc700 1 --2- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f176c036960 0x7f176c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:07.110 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.107+0000 7f17767fc700 1 -- 172.21.15.192:0/1261839879 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1770049ce0 con 0x7f17800fcbc0 2024-02-16T22:49:07.110 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.107+0000 7f17854f0700 1 --2- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f176c036960 0x7f176c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:07.110 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.108+0000 7f17854f0700 1 --2- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f176c036960 0x7f176c038e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f177c009990 tx=0x7f177c006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:07.113 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.112+0000 7f17767fc700 1 -- 172.21.15.192:0/1261839879 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1770013b10 con 0x7f17800fcbc0 2024-02-16T22:49:07.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.366+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f178010edc0 con 0x7f17800fcbc0 2024-02-16T22:49:07.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.367+0000 7f17767fc700 1 -- 172.21.15.192:0/1261839879 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1770011370 con 0x7f17800fcbc0 2024-02-16T22:49:07.368 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:49:07.369 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f176c036960 msgr2=0x7f176c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 --2- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f176c036960 0x7f176c038e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f177c009990 tx=0x7f177c006e40 comp rx=0 tx=0).stop 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 msgr2=0x7f178010ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 --2- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 0x7f178010ce90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1770009b80 tx=0x7f1770009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 shutdown_connections 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 --2- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f176c036960 0x7f176c038e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 --2- 172.21.15.192:0/1261839879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17800fcbc0 0x7f178010ce90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 >> 172.21.15.192:0/1261839879 conn(0x7f17800f82a0 msgr2=0x7f17800f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 shutdown_connections 2024-02-16T22:49:07.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:07.369+0000 7f1787f55700 1 -- 172.21.15.192:0/1261839879 wait complete. 2024-02-16T22:49:07.372 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:49:07.699 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:07 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/1261839879' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:49:08.903 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:49:08.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:49:09.150 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:09.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:09.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:09.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:09.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:09.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:49:09.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:09.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:10.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:10.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:10.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:10.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:49:10.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-16T22:49:10.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.179+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2304165702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 msgr2=0x7f68b80fe020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.179+0000 7f68bdc6e700 1 --2- 172.21.15.192:0/2304165702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 0x7f68b80fe020 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68a80048c0 tx=0x7f68a8009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.179+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2304165702 shutdown_connections 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.179+0000 7f68bdc6e700 1 --2- 172.21.15.192:0/2304165702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 0x7f68b80fe020 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.179+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2304165702 >> 172.21.15.192:0/2304165702 conn(0x7f68b8072850 msgr2=0x7f68b8072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.179+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2304165702 shutdown_connections 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.179+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2304165702 wait complete. 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.179+0000 7f68bdc6e700 1 Processor -- start 2024-02-16T22:49:11.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.180+0000 7f68bdc6e700 1 -- start start 2024-02-16T22:49:11.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.180+0000 7f68bdc6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 0x7f68b8112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:11.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.180+0000 7f68bdc6e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8113500 con 0x7f68b80fdc50 2024-02-16T22:49:11.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.180+0000 7f68b77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 0x7f68b8112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:11.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.180+0000 7f68b77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 0x7f68b8112fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:58096/0 (socket says 172.21.15.192:58096) 2024-02-16T22:49:11.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.180+0000 7f68b77fe700 1 -- 172.21.15.192:0/2826893969 learned_addr learned my addr 172.21.15.192:0/2826893969 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:49:11.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.180+0000 7f68b77fe700 1 -- 172.21.15.192:0/2826893969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8005040 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.180+0000 7f68b77fe700 1 --2- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 0x7f68b8112fc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009300 tx=0x7f68a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.181+0000 7f689ffff700 1 -- 172.21.15.192:0/2826893969 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68a8004e60 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.181+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b8113700 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.181+0000 7f689ffff700 1 -- 172.21.15.192:0/2826893969 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68a8013420 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.181+0000 7f689ffff700 1 -- 172.21.15.192:0/2826893969 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68a801c5b0 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.181+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b8113b70 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.181+0000 7f689ffff700 1 -- 172.21.15.192:0/2826893969 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f68a8013a90 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.181+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b805f5c0 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.182+0000 7f689ffff700 1 --2- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f68a0036960 0x7f68a0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.182+0000 7f689ffff700 1 -- 172.21.15.192:0/2826893969 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f68a8049bf0 con 0x7f68b80fdc50 2024-02-16T22:49:11.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.182+0000 7f68b6ffd700 1 --2- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f68a0036960 0x7f68a0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:11.185 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.182+0000 7f68b6ffd700 1 --2- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f68a0036960 0x7f68a0038e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009990 tx=0x7f68ac006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:11.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.187+0000 7f689ffff700 1 -- 172.21.15.192:0/2826893969 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f68a8011960 con 0x7f68b80fdc50 2024-02-16T22:49:11.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:10 smithi064 ceph-mon[29226]: Deploying daemon crash.smithi192 on smithi192 2024-02-16T22:49:11.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.443+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f68b8106f30 con 0x7f68b80fdc50 2024-02-16T22:49:11.446 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.444+0000 7f689ffff700 1 -- 172.21.15.192:0/2826893969 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f68a8023560 con 0x7f68b80fdc50 2024-02-16T22:49:11.446 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:49:11.446 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f68a0036960 msgr2=0x7f68a0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 --2- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f68a0036960 0x7f68a0038e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009990 tx=0x7f68ac006e40 comp rx=0 tx=0).stop 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 msgr2=0x7f68b8112fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 --2- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 0x7f68b8112fc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009300 tx=0x7f68a8009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 shutdown_connections 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 --2- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f68a0036960 0x7f68a0038e10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 --2- 172.21.15.192:0/2826893969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b80fdc50 0x7f68b8112fc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 >> 172.21.15.192:0/2826893969 conn(0x7f68b8072850 msgr2=0x7f68b8101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 shutdown_connections 2024-02-16T22:49:11.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:11.446+0000 7f68bdc6e700 1 -- 172.21.15.192:0/2826893969 wait complete. 2024-02-16T22:49:11.449 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:49:12.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:11 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/2826893969' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:49:13.209 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:49:13.210 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:49:13.592 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:14.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:14.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:14.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:14.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:14.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:13 smithi064 ceph-mon[29226]: Deploying daemon node-exporter.smithi192 on smithi192 2024-02-16T22:49:14.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:14.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:49:15.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.137+0000 7f582d0fa700 1 -- 172.21.15.192:0/3935900675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 msgr2=0x7f5828070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.137+0000 7f582d0fa700 1 --2- 172.21.15.192:0/3935900675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 0x7f5828070680 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f58180048c0 tx=0x7f5818009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.137+0000 7f582d0fa700 1 -- 172.21.15.192:0/3935900675 shutdown_connections 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.137+0000 7f582d0fa700 1 --2- 172.21.15.192:0/3935900675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 0x7f5828070680 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.137+0000 7f582d0fa700 1 -- 172.21.15.192:0/3935900675 >> 172.21.15.192:0/3935900675 conn(0x7f58280f8760 msgr2=0x7f58280fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.137+0000 7f582d0fa700 1 -- 172.21.15.192:0/3935900675 shutdown_connections 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.137+0000 7f582d0fa700 1 -- 172.21.15.192:0/3935900675 wait complete. 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.138+0000 7f582d0fa700 1 Processor -- start 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.138+0000 7f582d0fa700 1 -- start start 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.138+0000 7f582d0fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 0x7f5828115260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.138+0000 7f582d0fa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281157a0 con 0x7f58280702b0 2024-02-16T22:49:15.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.138+0000 7f5826d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 0x7f5828115260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.138+0000 7f5826d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 0x7f5828115260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:58128/0 (socket says 172.21.15.192:58128) 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.139+0000 7f5826d9d700 1 -- 172.21.15.192:0/2897739739 learned_addr learned my addr 172.21.15.192:0/2897739739 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.139+0000 7f5826d9d700 1 -- 172.21.15.192:0/2897739739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818005040 con 0x7f58280702b0 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.139+0000 7f5826d9d700 1 --2- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 0x7f5828115260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5818009330 tx=0x7f5818009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.139+0000 7f581ffff700 1 -- 172.21.15.192:0/2897739739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f581800be50 con 0x7f58280702b0 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.139+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58281159a0 con 0x7f58280702b0 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.139+0000 7f581ffff700 1 -- 172.21.15.192:0/2897739739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5818009630 con 0x7f58280702b0 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.139+0000 7f581ffff700 1 -- 172.21.15.192:0/2897739739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f581801c3f0 con 0x7f58280702b0 2024-02-16T22:49:15.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.139+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5828115e10 con 0x7f58280702b0 2024-02-16T22:49:15.143 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.140+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582805f5c0 con 0x7f58280702b0 2024-02-16T22:49:15.143 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.140+0000 7f581ffff700 1 -- 172.21.15.192:0/2897739739 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f58180097a0 con 0x7f58280702b0 2024-02-16T22:49:15.143 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.140+0000 7f581ffff700 1 --2- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5814036960 0x7f5814038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:15.143 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.140+0000 7f581ffff700 1 -- 172.21.15.192:0/2897739739 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5818049bb0 con 0x7f58280702b0 2024-02-16T22:49:15.143 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.140+0000 7f581e59c700 1 --2- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5814036960 0x7f5814038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:15.143 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.141+0000 7f581e59c700 1 --2- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5814036960 0x7f5814038e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5810009990 tx=0x7f5810006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:15.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.146+0000 7f581ffff700 1 -- 172.21.15.192:0/2897739739 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5818011940 con 0x7f58280702b0 2024-02-16T22:49:15.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.435+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f58281108e0 con 0x7f58280702b0 2024-02-16T22:49:15.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.436+0000 7f581ffff700 1 -- 172.21.15.192:0/2897739739 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5818011370 con 0x7f58280702b0 2024-02-16T22:49:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:49:15.438 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:49:15.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5814036960 msgr2=0x7f5814038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:15.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 --2- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5814036960 0x7f5814038e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5810009990 tx=0x7f5810006e40 comp rx=0 tx=0).stop 2024-02-16T22:49:15.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 msgr2=0x7f5828115260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:15.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 --2- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 0x7f5828115260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5818009330 tx=0x7f5818009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:15.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 shutdown_connections 2024-02-16T22:49:15.441 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 --2- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5814036960 0x7f5814038e10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:15.441 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 --2- 172.21.15.192:0/2897739739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58280702b0 0x7f5828115260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:15.441 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 >> 172.21.15.192:0/2897739739 conn(0x7f58280f8760 msgr2=0x7f58281037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:15.441 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 shutdown_connections 2024-02-16T22:49:15.441 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:15.438+0000 7f582d0fa700 1 -- 172.21.15.192:0/2897739739 wait complete. 2024-02-16T22:49:15.442 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:49:15.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:15 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/2897739739' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:49:18.172 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:49:18.172 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:49:18.946 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: Deploying daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:19.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.878+0000 7f606a616700 1 -- 172.21.15.192:0/1458079131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 msgr2=0x7f6064101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.878+0000 7f606a616700 1 --2- 172.21.15.192:0/1458079131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 0x7f6064101f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f60540048c0 tx=0x7f6054009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.878+0000 7f606a616700 1 -- 172.21.15.192:0/1458079131 shutdown_connections 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.878+0000 7f606a616700 1 --2- 172.21.15.192:0/1458079131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 0x7f6064101f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.878+0000 7f606a616700 1 -- 172.21.15.192:0/1458079131 >> 172.21.15.192:0/1458079131 conn(0x7f60640720b0 msgr2=0x7f60640724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.878+0000 7f606a616700 1 -- 172.21.15.192:0/1458079131 shutdown_connections 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.878+0000 7f606a616700 1 -- 172.21.15.192:0/1458079131 wait complete. 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.879+0000 7f606a616700 1 Processor -- start 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.879+0000 7f606a616700 1 -- start start 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.879+0000 7f606a616700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 0x7f606410bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.879+0000 7f606a616700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606410c1b0 con 0x7f6064101b50 2024-02-16T22:49:20.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.879+0000 7f6063fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 0x7f606410bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.879+0000 7f6063fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 0x7f606410bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:33428/0 (socket says 172.21.15.192:33428) 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.879+0000 7f6063fff700 1 -- 172.21.15.192:0/2166930915 learned_addr learned my addr 172.21.15.192:0/2166930915 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.880+0000 7f6063fff700 1 -- 172.21.15.192:0/2166930915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6054005040 con 0x7f6064101b50 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.880+0000 7f6063fff700 1 --2- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 0x7f606410bc70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6054009300 tx=0x7f6054009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.880+0000 7f6060ff9700 1 -- 172.21.15.192:0/2166930915 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6054004e60 con 0x7f6064101b50 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.880+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606410c3b0 con 0x7f6064101b50 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.880+0000 7f6060ff9700 1 -- 172.21.15.192:0/2166930915 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6054013420 con 0x7f6064101b50 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.880+0000 7f6060ff9700 1 -- 172.21.15.192:0/2166930915 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f605401c590 con 0x7f6064101b50 2024-02-16T22:49:20.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.880+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606410c820 con 0x7f6064101b50 2024-02-16T22:49:20.884 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.881+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6064100e40 con 0x7f6064101b50 2024-02-16T22:49:20.884 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.881+0000 7f6060ff9700 1 -- 172.21.15.192:0/2166930915 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f6054013590 con 0x7f6064101b50 2024-02-16T22:49:20.884 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.881+0000 7f6060ff9700 1 --2- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f604c036910 0x7f604c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:20.884 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.881+0000 7f6060ff9700 1 -- 172.21.15.192:0/2166930915 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f6054049d30 con 0x7f6064101b50 2024-02-16T22:49:20.884 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.881+0000 7f60637fe700 1 --2- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f604c036910 0x7f604c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:20.884 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.882+0000 7f60637fe700 1 --2- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f604c036910 0x7f604c038dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6058009990 tx=0x7f6058006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:20.888 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:20.886+0000 7f6060ff9700 1 -- 172.21.15.192:0/2166930915 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6054011960 con 0x7f6064101b50 2024-02-16T22:49:21.156 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.154+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f606405f5c0 con 0x7f6064101b50 2024-02-16T22:49:21.156 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.154+0000 7f6060ff9700 1 -- 172.21.15.192:0/2166930915 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f606405f5c0 con 0x7f6064101b50 2024-02-16T22:49:21.156 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:49:21.156 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:49:21.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f604c036910 msgr2=0x7f604c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:21.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 --2- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f604c036910 0x7f604c038dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6058009990 tx=0x7f6058006e40 comp rx=0 tx=0).stop 2024-02-16T22:49:21.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 msgr2=0x7f606410bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:21.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 --2- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 0x7f606410bc70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6054009300 tx=0x7f6054009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:21.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 shutdown_connections 2024-02-16T22:49:21.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 --2- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f604c036910 0x7f604c038dc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:21.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 --2- 172.21.15.192:0/2166930915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6064101b50 0x7f606410bc70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:21.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 >> 172.21.15.192:0/2166930915 conn(0x7f60640720b0 msgr2=0x7f60640f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:21.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 shutdown_connections 2024-02-16T22:49:21.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:21.158+0000 7f606a616700 1 -- 172.21.15.192:0/2166930915 wait complete. 2024-02-16T22:49:21.160 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:49:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:21 smithi064 ceph-mon[29226]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:49:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:22.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:22 smithi064 ceph-mon[29226]: Deploying daemon mon.smithi192 on smithi192 2024-02-16T22:49:22.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:22 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/2166930915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:49:22.736 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:49:22.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:49:23.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:23 smithi064 ceph-mon[29226]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:23.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:23 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:24.032 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:49:25.421 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:25 smithi064 ceph-mon[29226]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.500+0000 7fc8fb89b700 1 -- 172.21.15.192:0/3882269134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 msgr2=0x7fc8f41030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.500+0000 7fc8fb89b700 1 --2- 172.21.15.192:0/3882269134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 0x7fc8f41030a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00048c0 tx=0x7fc8f0009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.500+0000 7fc8fb89b700 1 -- 172.21.15.192:0/3882269134 shutdown_connections 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.500+0000 7fc8fb89b700 1 --2- 172.21.15.192:0/3882269134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 0x7fc8f41030a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.500+0000 7fc8fb89b700 1 -- 172.21.15.192:0/3882269134 >> 172.21.15.192:0/3882269134 conn(0x7fc8f40f8860 msgr2=0x7fc8f40fac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.500+0000 7fc8fb89b700 1 -- 172.21.15.192:0/3882269134 shutdown_connections 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.500+0000 7fc8fb89b700 1 -- 172.21.15.192:0/3882269134 wait complete. 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.501+0000 7fc8fb89b700 1 Processor -- start 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.501+0000 7fc8fb89b700 1 -- start start 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.501+0000 7fc8fb89b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 0x7fc8f4070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:25.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.501+0000 7fc8fb89b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40707d0 con 0x7fc8f4102cd0 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.501+0000 7fc8f9637700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 0x7fc8f4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.501+0000 7fc8f9637700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 0x7fc8f4070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.192:33460/0 (socket says 172.21.15.192:33460) 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.501+0000 7fc8f9637700 1 -- 172.21.15.192:0/570002551 learned_addr learned my addr 172.21.15.192:0/570002551 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.502+0000 7fc8f9637700 1 -- 172.21.15.192:0/570002551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f0005040 con 0x7fc8f4102cd0 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.502+0000 7fc8f9637700 1 --2- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 0x7fc8f4070290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009300 tx=0x7fc8f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.502+0000 7fc8e9ffb700 1 -- 172.21.15.192:0/570002551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8f0004e60 con 0x7fc8f4102cd0 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.502+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f40709d0 con 0x7fc8f4102cd0 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.502+0000 7fc8e9ffb700 1 -- 172.21.15.192:0/570002551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f0013420 con 0x7fc8f4102cd0 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.502+0000 7fc8e9ffb700 1 -- 172.21.15.192:0/570002551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8f001c5b0 con 0x7fc8f4102cd0 2024-02-16T22:49:25.505 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.502+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f4070ec0 con 0x7fc8f4102cd0 2024-02-16T22:49:25.506 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.503+0000 7fc8e9ffb700 1 -- 172.21.15.192:0/570002551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fc8f0009600 con 0x7fc8f4102cd0 2024-02-16T22:49:25.506 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.503+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f4071100 con 0x7fc8f4102cd0 2024-02-16T22:49:25.506 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.503+0000 7fc8e9ffb700 1 --2- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8e0036970 0x7fc8e0038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:25.506 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.503+0000 7fc8e9ffb700 1 -- 172.21.15.192:0/570002551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc8f0049c40 con 0x7fc8f4102cd0 2024-02-16T22:49:25.506 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.503+0000 7fc8f8e36700 1 --2- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8e0036970 0x7fc8e0038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:25.506 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.504+0000 7fc8f8e36700 1 --2- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8e0036970 0x7fc8e0038e20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009990 tx=0x7fc8e4006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:25.509 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.508+0000 7fc8e9ffb700 1 -- 172.21.15.192:0/570002551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc8f4071100 con 0x7fc8f4102cd0 2024-02-16T22:49:25.810 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.808+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc8f4071100 con 0x7fc8f4102cd0 2024-02-16T22:49:25.810 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.809+0000 7fc8e9ffb700 1 -- 172.21.15.192:0/570002551 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc8f4071100 con 0x7fc8f4102cd0 2024-02-16T22:49:25.811 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:49:25.811 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:46:53.076262Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8e0036970 msgr2=0x7fc8e0038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 --2- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8e0036970 0x7fc8e0038e20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009990 tx=0x7fc8e4006e40 comp rx=0 tx=0).stop 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 msgr2=0x7fc8f4070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 --2- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 0x7fc8f4070290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009300 tx=0x7fc8f0009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 shutdown_connections 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 --2- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8e0036970 0x7fc8e0038e20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 --2- 172.21.15.192:0/570002551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f4102cd0 0x7fc8f4070290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 >> 172.21.15.192:0/570002551 conn(0x7fc8f40f8860 msgr2=0x7fc8f40f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 shutdown_connections 2024-02-16T22:49:25.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:25.811+0000 7fc8fb89b700 1 -- 172.21.15.192:0/570002551 wait complete. 2024-02-16T22:49:25.814 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-02-16T22:49:26.516 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:26 smithi192 systemd[1]: Started Ceph mon.smithi192 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T22:49:27.541 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:49:27.542 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:49:27.798 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:49:31.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:49:31.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: mon.smithi064 calling monitor election 2024-02-16T22:49:31.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:31.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/2785096458' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: mon.smithi192 calling monitor election 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: mon.smithi064 is new leader, mons smithi064,smithi192 in quorum (ranks 0,1) 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: monmap e2: 2 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: fsmap 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: mgrmap e17: smithi064.kubmlk(active, since 32s) 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: Standby manager daemon smithi192.pauvih started 2024-02-16T22:49:31.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/2785096458' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:49:31.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: overall HEALTH_OK 2024-02-16T22:49:31.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/2785096458' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:49:31.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/2785096458' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:49:31.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:49:31.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: mon.smithi064 calling monitor election 2024-02-16T22:49:31.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/2785096458' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: mon.smithi192 calling monitor election 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: mon.smithi064 is new leader, mons smithi064,smithi192 in quorum (ranks 0,1) 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: monmap e2: 2 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:49:31.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: fsmap 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: mgrmap e17: smithi064.kubmlk(active, since 32s) 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: Standby manager daemon smithi192.pauvih started 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/2785096458' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: overall HEALTH_OK 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/2785096458' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/2785096458' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:31.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:32.290 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.287+0000 7fd008a2f700 1 -- 172.21.15.192:0/208606549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 msgr2=0x7fcfec005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:32.290 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.287+0000 7fd008a2f700 1 --2- 172.21.15.192:0/208606549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 0x7fcfec005a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd0040fcd10 tx=0x7fcff800b040 comp rx=0 tx=0).stop 2024-02-16T22:49:32.290 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.287+0000 7fd008a2f700 1 -- 172.21.15.192:0/208606549 shutdown_connections 2024-02-16T22:49:32.290 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.287+0000 7fd008a2f700 1 --2- 172.21.15.192:0/208606549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 0x7fcfec005a00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:32.290 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.287+0000 7fd008a2f700 1 -- 172.21.15.192:0/208606549 >> 172.21.15.192:0/208606549 conn(0x7fd0040f8980 msgr2=0x7fd0040fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:32.290 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.287+0000 7fd008a2f700 1 -- 172.21.15.192:0/208606549 shutdown_connections 2024-02-16T22:49:32.290 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.287+0000 7fd008a2f700 1 -- 172.21.15.192:0/208606549 wait complete. 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd008a2f700 1 Processor -- start 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd008a2f700 1 -- start start 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd008a2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 0x7fd0041131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd008a2f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 0x7fd00410d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd008a2f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004113d40 con 0x7fcfec003580 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd008a2f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004113eb0 con 0x7fd004113730 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd001d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 0x7fd00410d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd001d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 0x7fd00410d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:41138/0 (socket says 172.21.15.192:41138) 2024-02-16T22:49:32.292 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd001d9b700 1 -- 172.21.15.192:0/1961798611 learned_addr learned my addr 172.21.15.192:0/1961798611 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd00259c700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 0x7fd0041131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd001d9b700 1 -- 172.21.15.192:0/1961798611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 msgr2=0x7fd00410d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd001d9b700 1 -- 172.21.15.192:0/1961798611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 msgr2=0x7fd00410d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd001d9b700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 0x7fd00410d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.288+0000 7fd001d9b700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 0x7fd00410d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fd00259c700 1 -- 172.21.15.192:0/1961798611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 msgr2=0x7fd00410d0d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fd00259c700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 0x7fd00410d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fd00259c700 1 -- 172.21.15.192:0/1961798611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff8005040 con 0x7fcfec003580 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fd00259c700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 0x7fd0041131f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcff800b010 tx=0x7fcff800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fcff2ffd700 1 -- 172.21.15.192:0/1961798611 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff801b070 con 0x7fcfec003580 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fcff2ffd700 1 -- 172.21.15.192:0/1961798611 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcff8020910 con 0x7fcfec003580 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00410d6d0 con 0x7fcfec003580 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fcff2ffd700 1 -- 172.21.15.192:0/1961798611 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff801f800 con 0x7fcfec003580 2024-02-16T22:49:32.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.289+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00410dbe0 con 0x7fcfec003580 2024-02-16T22:49:32.294 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:32 smithi192 ceph-mon[36254]: mgrmap e18: smithi064.kubmlk(active, since 32s), standbys: smithi192.pauvih 2024-02-16T22:49:32.294 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi192.pauvih", "id": "smithi192.pauvih"}]: dispatch 2024-02-16T22:49:32.294 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:49:32.294 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:32.295 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.290+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00405f5c0 con 0x7fcfec003580 2024-02-16T22:49:32.295 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.291+0000 7fcff2ffd700 1 -- 172.21.15.192:0/1961798611 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fcff8020440 con 0x7fcfec003580 2024-02-16T22:49:32.295 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.291+0000 7fcff2ffd700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcfe4069360 0x7fcfe406b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:32.295 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.291+0000 7fcff2ffd700 1 -- 172.21.15.192:0/1961798611 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fcff8088110 con 0x7fcfec003580 2024-02-16T22:49:32.295 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.291+0000 7fd001d9b700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcfe4069360 0x7fcfe406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:32.295 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.292+0000 7fd001d9b700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcfe4069360 0x7fcfe406b810 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcff4006fd0 tx=0x7fcff4008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:32.298 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.297+0000 7fcff2ffd700 1 -- 172.21.15.192:0/1961798611 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fcff8055440 con 0x7fcfec003580 2024-02-16T22:49:32.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:32 smithi064 ceph-mon[29226]: mgrmap e18: smithi064.kubmlk(active, since 32s), standbys: smithi192.pauvih 2024-02-16T22:49:32.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi192.pauvih", "id": "smithi192.pauvih"}]: dispatch 2024-02-16T22:49:32.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:49:32.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:49:32.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.560+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd00410e8b0 con 0x7fcfec003580 2024-02-16T22:49:32.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.561+0000 7fcff2ffd700 1 -- 172.21.15.192:0/1961798611 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7fcff8024090 con 0x7fcfec003580 2024-02-16T22:49:32.562 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-02-16T22:49:32.562 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","modified":"2024-02-16T22:49:25.937661Z","created":"2024-02-16T22:46:53.076262Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi192","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcfe4069360 msgr2=0x7fcfe406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcfe4069360 0x7fcfe406b810 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcff4006fd0 tx=0x7fcff4008040 comp rx=0 tx=0).stop 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 msgr2=0x7fd0041131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 0x7fd0041131f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcff800b010 tx=0x7fcff800b040 comp rx=0 tx=0).stop 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 shutdown_connections 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcfe4069360 0x7fcfe406b810 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec003580 0x7fd0041131f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 --2- 172.21.15.192:0/1961798611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd004113730 0x7fd00410d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 >> 172.21.15.192:0/1961798611 conn(0x7fd0040f8980 msgr2=0x7fd0040fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 shutdown_connections 2024-02-16T22:49:32.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:49:32.563+0000 7fd008a2f700 1 -- 172.21.15.192:0/1961798611 wait complete. 2024-02-16T22:49:32.566 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-02-16T22:49:33.253 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:33 smithi192 ceph-mon[36254]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:33.253 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:33 smithi192 ceph-mon[36254]: from='client.? 172.21.15.192:0/1961798611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:49:33.253 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:33.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:33.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:33.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:49:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:33 smithi064 ceph-mon[29226]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:33 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/1961798611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:49:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:49:33.627 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-16T22:49:33.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-16T22:49:34.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:49:34.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:34.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:49:34.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:49:34.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:34.819 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:49:35.306 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:35 smithi192 ceph-mon[36254]: Reconfiguring mon.smithi064 (unknown last config time)... 2024-02-16T22:49:35.306 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:35 smithi192 ceph-mon[36254]: Reconfiguring daemon mon.smithi064 on smithi064 2024-02-16T22:49:35.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:35 smithi192 ceph-mon[36254]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:35.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:35 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:35.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:35 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:35.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:35 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi064.kubmlk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:49:35.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:35 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:49:35.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:35 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:35.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:35 smithi064 ceph-mon[29226]: Reconfiguring mon.smithi064 (unknown last config time)... 2024-02-16T22:49:35.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:35 smithi064 ceph-mon[29226]: Reconfiguring daemon mon.smithi064 on smithi064 2024-02-16T22:49:35.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:35 smithi064 ceph-mon[29226]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:35.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:35 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:35.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:35 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:35.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:35 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi064.kubmlk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:49:35.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:35 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:49:35.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:35 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:36.251 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:36 smithi064 ceph-mon[29226]: Reconfiguring mgr.smithi064.kubmlk (unknown last config time)... 2024-02-16T22:49:36.252 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:36 smithi064 ceph-mon[29226]: Reconfiguring daemon mgr.smithi064.kubmlk on smithi064 2024-02-16T22:49:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:36 smithi192 ceph-mon[36254]: Reconfiguring mgr.smithi064.kubmlk (unknown last config time)... 2024-02-16T22:49:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:36 smithi192 ceph-mon[36254]: Reconfiguring daemon mgr.smithi064.kubmlk on smithi064 2024-02-16T22:49:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.179+0000 7fd9b5ded700 1 -- 172.21.15.64:0/10776281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 msgr2=0x7fd9b00fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.179+0000 7fd9b5ded700 1 --2- 172.21.15.64:0/10776281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 0x7fd9b00fe390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0004940 tx=0x7fd9a0009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.179+0000 7fd9b5ded700 1 -- 172.21.15.64:0/10776281 shutdown_connections 2024-02-16T22:49:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.179+0000 7fd9b5ded700 1 --2- 172.21.15.64:0/10776281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b00fe960 0x7fd9b0106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.179+0000 7fd9b5ded700 1 --2- 172.21.15.64:0/10776281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 0x7fd9b00fe390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.179+0000 7fd9b5ded700 1 -- 172.21.15.64:0/10776281 >> 172.21.15.64:0/10776281 conn(0x7fd9b00f7a10 msgr2=0x7fd9b00f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.179+0000 7fd9b5ded700 1 -- 172.21.15.64:0/10776281 shutdown_connections 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.179+0000 7fd9b5ded700 1 -- 172.21.15.64:0/10776281 wait complete. 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9b5ded700 1 Processor -- start 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9b5ded700 1 -- start start 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9b5ded700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 0x7fd9b0113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9b5ded700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b00fe960 0x7fd9b0113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9b5ded700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b01145a0 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9b5ded700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b010d8d0 con 0x7fd9b00fe960 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9af7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 0x7fd9b0113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9af7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 0x7fd9b0113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60278/0 (socket says 172.21.15.64:60278) 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9af7fe700 1 -- 172.21.15.64:0/2169502336 learned_addr learned my addr 172.21.15.64:0/2169502336 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9af7fe700 1 -- 172.21.15.64:0/2169502336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b00fe960 msgr2=0x7fd9b0113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9af7fe700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b00fe960 0x7fd9b0113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:37.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.180+0000 7fd9af7fe700 1 -- 172.21.15.64:0/2169502336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0005040 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.181+0000 7fd9af7fe700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 0x7fd9b0113a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0004890 tx=0x7fd9a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:37.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.181+0000 7fd9a7fff700 1 -- 172.21.15.64:0/2169502336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a000e040 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.181+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b010dbb0 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.181+0000 7fd9a7fff700 1 -- 172.21.15.64:0/2169502336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9a0004290 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.181+0000 7fd9a7fff700 1 -- 172.21.15.64:0/2169502336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a001c630 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.181+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b010e0f0 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.182+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b01045b0 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.182+0000 7fd9a7fff700 1 -- 172.21.15.64:0/2169502336 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fd9a000bbf0 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.182+0000 7fd9a7fff700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fd99c069210 0x7fd99c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:37.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.183+0000 7fd9a7fff700 1 -- 172.21.15.64:0/2169502336 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd9a00874d0 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.183+0000 7fd9aeffd700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fd99c069210 0x7fd99c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:37.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.183+0000 7fd9aeffd700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fd99c069210 0x7fd99c06b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd9b010f240 tx=0x7fd998007d40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:37.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.187+0000 7fd9a7fff700 1 -- 172.21.15.64:0/2169502336 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd9a0058370 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.379+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd9b0109f80 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.379+0000 7fd9a7fff700 1 -- 172.21.15.64:0/2169502336 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7fd9a0086c80 con 0x7fd9b00fdfc0 2024-02-16T22:49:37.381 INFO:teuthology.orchestra.run.smithi064.stdout:# minimal ceph.conf for 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:49:37.381 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-02-16T22:49:37.381 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:49:37.381 INFO:teuthology.orchestra.run.smithi064.stdout: mon_host = [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-02-16T22:49:37.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fd99c069210 msgr2=0x7fd99c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fd99c069210 0x7fd99c06b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd9b010f240 tx=0x7fd998007d40 comp rx=0 tx=0).stop 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 msgr2=0x7fd9b0113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 0x7fd9b0113a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0004890 tx=0x7fd9a0009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 shutdown_connections 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fd99c069210 0x7fd99c06b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9b00fdfc0 0x7fd9b0113a10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 --2- 172.21.15.64:0/2169502336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b00fe960 0x7fd9b0113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 >> 172.21.15.64:0/2169502336 conn(0x7fd9b00f7a10 msgr2=0x7fd9b0101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 shutdown_connections 2024-02-16T22:49:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:37.381+0000 7fd9b5ded700 1 -- 172.21.15.64:0/2169502336 wait complete. 2024-02-16T22:49:37.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:37 smithi064 ceph-mon[29226]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:37 smithi192 ceph-mon[36254]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:38.167 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:38 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2169502336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:38.167 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:38.167 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:38.168 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:49:38.168 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:38.169 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-16T22:49:38.169 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:49:38.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-02-16T22:49:38.199 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:49:38.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:49:38.266 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:49:38.267 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-02-16T22:49:38.303 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:49:38.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:49:38.374 INFO:tasks.cephadm:Deploying OSDs... 2024-02-16T22:49:38.374 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:49:38.374 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-02-16T22:49:38.389 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-16T22:49:38.389 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-02-16 22:48:49.660940667 +0000 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-16 22:43:18.633704158 +0000 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-16 22:43:18.633704158 +0000 2024-02-16T22:49:38.444 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-02-16 22:43:18.633704158 +0000 2024-02-16T22:49:38.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-16T22:49:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-02-16T22:49:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-02-16T22:49:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000194099 s, 2.6 MB/s 2024-02-16T22:49:38.513 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-16T22:49:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:38 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2169502336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:49:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:38.570 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-02-16T22:49:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-16T22:49:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:49:38.627 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-16T22:49:38.627 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:49:38.627 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:49:38.627 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-02-16 22:48:49.661940647 +0000 2024-02-16T22:49:38.627 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-16 22:43:18.968697684 +0000 2024-02-16T22:49:38.627 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-16 22:43:18.968697684 +0000 2024-02-16T22:49:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-02-16 22:43:18.968697684 +0000 2024-02-16T22:49:38.628 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-16T22:49:38.693 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-02-16T22:49:38.693 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-02-16T22:49:38.693 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000186526 s, 2.7 MB/s 2024-02-16T22:49:38.694 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-16T22:49:38.750 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-02-16T22:49:38.804 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-16T22:49:38.804 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:49:38.804 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-16T22:49:38.804 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:49:38.804 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:49:38.805 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-02-16 22:48:49.661940647 +0000 2024-02-16T22:49:38.805 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-16 22:43:19.306691152 +0000 2024-02-16T22:49:38.805 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-16 22:43:19.306691152 +0000 2024-02-16T22:49:38.805 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-02-16 22:43:19.306691152 +0000 2024-02-16T22:49:38.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-16T22:49:38.870 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-02-16T22:49:38.870 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-02-16T22:49:38.870 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000189607 s, 2.7 MB/s 2024-02-16T22:49:38.871 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-16T22:49:38.926 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-02-16 22:48:49.661940647 +0000 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-16 22:43:19.633684832 +0000 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-16 22:43:19.633684832 +0000 2024-02-16T22:49:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-02-16 22:43:19.633684832 +0000 2024-02-16T22:49:38.982 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-16T22:49:39.046 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-02-16T22:49:39.046 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-02-16T22:49:39.046 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000203174 s, 2.5 MB/s 2024-02-16T22:49:39.047 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-16T22:49:39.102 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:49:39.102 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-02-16T22:49:39.120 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-16T22:49:39.121 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-02-16T22:49:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-16T22:49:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:49:39.178 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-16T22:49:39.179 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:49:39.179 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:49:39.179 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-02-16 22:49:24.226160418 +0000 2024-02-16T22:49:39.179 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-02-16 22:43:16.577865031 +0000 2024-02-16T22:49:39.179 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-02-16 22:43:16.577865031 +0000 2024-02-16T22:49:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-02-16 22:43:16.577865031 +0000 2024-02-16T22:49:39.179 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-16T22:49:39.247 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-02-16T22:49:39.247 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-02-16T22:49:39.248 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000232723 s, 2.2 MB/s 2024-02-16T22:49:39.249 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-16T22:49:39.306 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-02-16T22:49:39.342 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:39 smithi064 ceph-mon[29226]: Reconfiguring crash.smithi064 (monmap changed)... 2024-02-16T22:49:39.342 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:39 smithi064 ceph-mon[29226]: Reconfiguring daemon crash.smithi064 on smithi064 2024-02-16T22:49:39.342 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:39 smithi064 ceph-mon[29226]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:39.363 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-16T22:49:39.363 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:49:39.363 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-16T22:49:39.363 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:49:39.363 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:49:39.363 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-02-16 22:49:24.227160398 +0000 2024-02-16T22:49:39.363 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-02-16 22:43:16.921857864 +0000 2024-02-16T22:49:39.364 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-02-16 22:43:16.921857864 +0000 2024-02-16T22:49:39.364 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-02-16 22:43:16.921857864 +0000 2024-02-16T22:49:39.364 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-16T22:49:39.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:39 smithi192 ceph-mon[36254]: Reconfiguring crash.smithi064 (monmap changed)... 2024-02-16T22:49:39.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:39 smithi192 ceph-mon[36254]: Reconfiguring daemon crash.smithi064 on smithi064 2024-02-16T22:49:39.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:39 smithi192 ceph-mon[36254]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:39.431 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-02-16T22:49:39.431 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-02-16T22:49:39.431 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000217166 s, 2.4 MB/s 2024-02-16T22:49:39.432 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-16T22:49:39.488 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-02-16 22:49:24.227160398 +0000 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-02-16 22:43:17.259850822 +0000 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-02-16 22:43:17.259850822 +0000 2024-02-16T22:49:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-02-16 22:43:17.259850822 +0000 2024-02-16T22:49:39.545 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-16T22:49:39.611 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-02-16T22:49:39.612 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-02-16T22:49:39.612 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000220465 s, 2.3 MB/s 2024-02-16T22:49:39.613 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-16T22:49:39.669 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-02-16T22:49:39.724 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-16T22:49:39.724 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:49:39.724 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-16T22:49:39.725 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:49:39.725 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:49:39.725 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-02-16 22:49:24.228160378 +0000 2024-02-16T22:49:39.725 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-02-16 22:43:17.597843780 +0000 2024-02-16T22:49:39.725 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-02-16 22:43:17.597843780 +0000 2024-02-16T22:49:39.725 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-02-16 22:43:17.597843780 +0000 2024-02-16T22:49:39.725 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-16T22:49:39.790 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-02-16T22:49:39.790 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-02-16T22:49:39.791 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000192092 s, 2.7 MB/s 2024-02-16T22:49:39.792 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-16T22:49:39.847 INFO:tasks.cephadm:Deploying osd.0 on smithi064 with /dev/vg_nvme/lv_4... 2024-02-16T22:49:39.848 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-16T22:49:40.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:40 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:40.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:40 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:40.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:40 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:40.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:40 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:41.426 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:41 smithi064 ceph-mon[29226]: Reconfiguring alertmanager.smithi064 (dependencies changed)... 2024-02-16T22:49:41.426 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:41 smithi064 ceph-mon[29226]: Reconfiguring daemon alertmanager.smithi064 on smithi064 2024-02-16T22:49:41.427 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:41 smithi064 ceph-mon[29226]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:41 smithi192 ceph-mon[36254]: Reconfiguring alertmanager.smithi064 (dependencies changed)... 2024-02-16T22:49:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:41 smithi192 ceph-mon[36254]: Reconfiguring daemon alertmanager.smithi064 on smithi064 2024-02-16T22:49:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:41 smithi192 ceph-mon[36254]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:41.802 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:49:43.615 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:43 smithi064 ceph-mon[29226]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:43.615 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:43 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:49:43.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:43 smithi192 ceph-mon[36254]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:43.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:43 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:49:44.091 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-16T22:49:44.092 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-16T22:49:44.092 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-02-16T22:49:44.092 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-02-16T22:49:44.092 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124504 s, 842 MB/s 2024-02-16T22:49:44.092 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-02-16T22:49:45.047 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-02-16T22:49:45.334 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:49:45.610 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:45 smithi064 ceph-mon[29226]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:45.724 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:45 smithi192 ceph-mon[36254]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:46.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:46 smithi064 ceph-mon[29226]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:46 smithi192 ceph-mon[36254]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:47.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.421+0000 7fb68103a700 1 -- 172.21.15.64:0/1593935807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fbe40 msgr2=0x7fb67c0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:47.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.421+0000 7fb68103a700 1 --2- 172.21.15.64:0/1593935807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fbe40 0x7fb67c0fc250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb66c004940 tx=0x7fb66c009e30 comp rx=0 tx=0).stop 2024-02-16T22:49:47.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.421+0000 7fb68103a700 1 -- 172.21.15.64:0/1593935807 shutdown_connections 2024-02-16T22:49:47.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.421+0000 7fb68103a700 1 --2- 172.21.15.64:0/1593935807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c0fcfb0 0x7fb67c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:47.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.421+0000 7fb68103a700 1 --2- 172.21.15.64:0/1593935807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fbe40 0x7fb67c0fc250 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:47.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.421+0000 7fb68103a700 1 -- 172.21.15.64:0/1593935807 >> 172.21.15.64:0/1593935807 conn(0x7fb67c0f7520 msgr2=0x7fb67c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:49:47.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.421+0000 7fb68103a700 1 -- 172.21.15.64:0/1593935807 shutdown_connections 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.421+0000 7fb68103a700 1 -- 172.21.15.64:0/1593935807 wait complete. 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb68103a700 1 Processor -- start 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb68103a700 1 -- start start 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb68103a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c0fbe40 0x7fb67c0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb68103a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fcfb0 0x7fb67c0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb68103a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0fea10 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb68103a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0feb80 con 0x7fb67c0fbe40 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fcfb0 0x7fb67c0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fcfb0 0x7fb67c0fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43488/0 (socket says 172.21.15.64:43488) 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67a59c700 1 -- 172.21.15.64:0/3591356018 learned_addr learned my addr 172.21.15.64:0/3591356018 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:49:47.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67ad9d700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c0fbe40 0x7fb67c0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67a59c700 1 -- 172.21.15.64:0/3591356018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c0fbe40 msgr2=0x7fb67c0fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67a59c700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c0fbe40 0x7fb67c0fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67a59c700 1 -- 172.21.15.64:0/3591356018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c005040 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67ad9d700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c0fbe40 0x7fb67c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb67a59c700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fcfb0 0x7fb67c0fe380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb66400b790 tx=0x7fb664009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb6737fe700 1 -- 172.21.15.64:0/3591356018 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb66400f770 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb6737fe700 1 -- 172.21.15.64:0/3591356018 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb66400fda0 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c0fee60 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.422+0000 7fb6737fe700 1 -- 172.21.15.64:0/3591356018 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb66400d310 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.423+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c0ff1e0 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.423+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c04bed0 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.424+0000 7fb6737fe700 1 -- 172.21.15.64:0/3591356018 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fb664010420 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.424+0000 7fb6737fe700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb668069210 0x7fb66806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:49:47.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.424+0000 7fb6737fe700 1 -- 172.21.15.64:0/3591356018 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb6640860a0 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.424+0000 7fb67ad9d700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb668069210 0x7fb66806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:49:47.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.425+0000 7fb67ad9d700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb668069210 0x7fb66806b6c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009fd0 tx=0x7fb66c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:49:47.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.429+0000 7fb6737fe700 1 -- 172.21.15.64:0/3591356018 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb664056f40 con 0x7fb67c0fcfb0 2024-02-16T22:49:47.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:49:47.632+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb67c066660 con 0x7fb668069210 2024-02-16T22:49:48.238 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:47 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:48.239 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:47 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:48.239 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:47 smithi064 ceph-mon[29226]: Reconfiguring grafana.smithi064 (dependencies changed)... 2024-02-16T22:49:48.239 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:47 smithi064 ceph-mon[29226]: Reconfiguring daemon grafana.smithi064 on smithi064 2024-02-16T22:49:48.239 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:47 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:49:48.239 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:47 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:49:48.239 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:47 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:47 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:47 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:47 smithi192 ceph-mon[36254]: Reconfiguring grafana.smithi064 (dependencies changed)... 2024-02-16T22:49:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:47 smithi192 ceph-mon[36254]: Reconfiguring daemon grafana.smithi064 on smithi064 2024-02-16T22:49:48.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:47 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:49:48.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:47 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:49:48.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:47 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:49:49.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:48 smithi064 ceph-mon[29226]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:49:49.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:48 smithi064 ceph-mon[29226]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:49.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:48 smithi192 ceph-mon[36254]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:49:49.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:48 smithi192 ceph-mon[36254]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:51.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:51 smithi064 ceph-mon[29226]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:51 smithi192 ceph-mon[36254]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:53.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:53 smithi064 ceph-mon[29226]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:53.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:53.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:53.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:53 smithi064 ceph-mon[29226]: Reconfiguring prometheus.smithi064 (dependencies changed)... 2024-02-16T22:49:53.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:53 smithi064 ceph-mon[29226]: Reconfiguring daemon prometheus.smithi064 on smithi064 2024-02-16T22:49:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:53 smithi192 ceph-mon[36254]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:49:53.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:53 smithi192 ceph-mon[36254]: Reconfiguring prometheus.smithi064 (dependencies changed)... 2024-02-16T22:49:53.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:53 smithi192 ceph-mon[36254]: Reconfiguring daemon prometheus.smithi064 on smithi064 2024-02-16T22:49:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:54 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/820268212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a680d8c-0559-46c6-9bfc-95fca252a657"}]: dispatch 2024-02-16T22:49:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:54 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/820268212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a680d8c-0559-46c6-9bfc-95fca252a657"}]': finished 2024-02-16T22:49:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:54 smithi064 ceph-mon[29226]: osdmap e6: 1 total, 0 up, 1 in 2024-02-16T22:49:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:54 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:49:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:54 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/66489298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:49:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:54 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/820268212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a680d8c-0559-46c6-9bfc-95fca252a657"}]: dispatch 2024-02-16T22:49:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:54 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/820268212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a680d8c-0559-46c6-9bfc-95fca252a657"}]': finished 2024-02-16T22:49:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:54 smithi192 ceph-mon[36254]: osdmap e6: 1 total, 0 up, 1 in 2024-02-16T22:49:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:54 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:49:54.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:54 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/66489298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:49:55.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:55 smithi064 ceph-mon[29226]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:55 smithi192 ceph-mon[36254]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:57.539 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:57 smithi064 ceph-mon[29226]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:57.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:57 smithi192 ceph-mon[36254]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:59.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:58 smithi192 ceph-mon[36254]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:59.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:49:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:49:59.090 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:58 smithi064 ceph-mon[29226]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:49:59.090 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:49:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:50:00.331 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:00 smithi064 ceph-mon[29226]: overall HEALTH_OK 2024-02-16T22:50:00.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:00 smithi192 ceph-mon[36254]: overall HEALTH_OK 2024-02-16T22:50:01.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:01 smithi064 ceph-mon[29226]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:01.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:01 smithi192 ceph-mon[36254]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:03.332 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:03 smithi064 ceph-mon[29226]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:03.332 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:03 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:03.332 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:03 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:03.332 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:03 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:50:03.332 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:03 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:03.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:03 smithi192 ceph-mon[36254]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:03.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:03 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:03.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:03 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:03.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:03 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:50:03.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:03 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:04.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:04 smithi064 ceph-mon[29226]: Reconfiguring crash.smithi192 (monmap changed)... 2024-02-16T22:50:04.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:04 smithi064 ceph-mon[29226]: Reconfiguring daemon crash.smithi192 on smithi192 2024-02-16T22:50:04.501 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:04 smithi192 ceph-mon[36254]: Reconfiguring crash.smithi192 (monmap changed)... 2024-02-16T22:50:04.501 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:04 smithi192 ceph-mon[36254]: Reconfiguring daemon crash.smithi192 on smithi192 2024-02-16T22:50:05.445 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: Reconfiguring mgr.smithi192.pauvih (monmap changed)... 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: Deploying daemon osd.0 on smithi064 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:05.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:05 smithi192 ceph-mon[36254]: Reconfiguring daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: Reconfiguring mgr.smithi192.pauvih (monmap changed)... 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: Deploying daemon osd.0 on smithi064 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:05.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:05 smithi064 ceph-mon[29226]: Reconfiguring daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:50:06.690 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:06 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:06.691 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:06 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:06.691 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:06 smithi064 ceph-mon[29226]: Reconfiguring mon.smithi192 (monmap changed)... 2024-02-16T22:50:06.691 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:06 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:50:06.691 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:06 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:50:06.691 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:06 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:06.691 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:06 smithi064 ceph-mon[29226]: Reconfiguring daemon mon.smithi192 on smithi192 2024-02-16T22:50:06.691 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:06 smithi064 ceph-mon[29226]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:06.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:06 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:06.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:06 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:06.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:06 smithi192 ceph-mon[36254]: Reconfiguring mon.smithi192 (monmap changed)... 2024-02-16T22:50:06.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:06 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:50:06.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:06 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:50:06.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:06 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:06.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:06 smithi192 ceph-mon[36254]: Reconfiguring daemon mon.smithi192 on smithi192 2024-02-16T22:50:06.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:06 smithi192 ceph-mon[36254]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:08.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:50:08.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:50:08.062 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T22:50:08.062 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T22:50:08.062 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.062 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:07 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:50:08.081 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.081 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.081 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:50:08.081 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi064.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T22:50:08.082 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi064.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T22:50:08.083 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:08.083 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:07 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:50:09.022 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:08 smithi064 ceph-mon[29226]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:09.023 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:09.023 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:08 smithi192 ceph-mon[36254]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:08 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:09.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:08 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:10.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:10 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:10.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:10 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:10.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:10 smithi064 ceph-mon[29226]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:11.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:10 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:11.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:10 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:11.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:10 smithi192 ceph-mon[36254]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:13.478 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:13 smithi064 ceph-mon[29226]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:13.478 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:50:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:13 smithi192 ceph-mon[36254]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:50:14.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.358+0000 7fb6737fe700 1 -- 172.21.15.64:0/3591356018 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb67c066660 con 0x7fb668069210 2024-02-16T22:50:14.360 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 0 on host 'smithi064' 2024-02-16T22:50:14.360 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:14 smithi064 ceph-mon[29226]: from='osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T22:50:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb668069210 msgr2=0x7fb66806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:50:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb668069210 0x7fb66806b6c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009fd0 tx=0x7fb66c009e30 comp rx=0 tx=0).stop 2024-02-16T22:50:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fcfb0 msgr2=0x7fb67c0fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:50:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fcfb0 0x7fb67c0fe380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb66400b790 tx=0x7fb664009250 comp rx=0 tx=0).stop 2024-02-16T22:50:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 shutdown_connections 2024-02-16T22:50:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c0fbe40 0x7fb67c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb668069210 0x7fb66806b6c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 --2- 172.21.15.64:0/3591356018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb67c0fcfb0 0x7fb67c0fe380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.360+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 >> 172.21.15.64:0/3591356018 conn(0x7fb67c0f7520 msgr2=0x7fb67c065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:50:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.361+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 shutdown_connections 2024-02-16T22:50:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:14.361+0000 7fb68103a700 1 -- 172.21.15.64:0/3591356018 wait complete. 2024-02-16T22:50:14.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:14 smithi192 ceph-mon[36254]: from='osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T22:50:14.968 DEBUG:teuthology.orchestra.run.smithi064:osd.0> sudo journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.0.service 2024-02-16T22:50:14.970 INFO:tasks.cephadm:Deploying osd.1 on smithi064 with /dev/vg_nvme/lv_3... 2024-02-16T22:50:14.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-16T22:50:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:50:15.368 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[47946]: 2024-02-16T22:50:15.150+0000 7fed1257c700 -1 osd.0 0 waiting for initial osdmap 2024-02-16T22:50:15.369 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[47946]: 2024-02-16T22:50:15.154+0000 7fed0c375700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: from='osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: osdmap e7: 1 total, 0 up, 1 in 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: from='osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:15.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:15.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:15.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: from='osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T22:50:15.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: osdmap e7: 1 total, 0 up, 1 in 2024-02-16T22:50:15.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:15.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: from='osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T22:50:15.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:15.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:15.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:15.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:16.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:16 smithi064 ceph-mon[29226]: from='osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-16T22:50:16.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:16 smithi064 ceph-mon[29226]: osdmap e8: 1 total, 0 up, 1 in 2024-02-16T22:50:16.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:16 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:16.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:16 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:16.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:16 smithi192 ceph-mon[36254]: from='osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-16T22:50:16.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:16 smithi192 ceph-mon[36254]: osdmap e8: 1 total, 0 up, 1 in 2024-02-16T22:50:16.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:16 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:16.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:16 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:17.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:17 smithi192 ceph-mon[36254]: purged_snaps scrub starts 2024-02-16T22:50:17.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:17 smithi192 ceph-mon[36254]: purged_snaps scrub ok 2024-02-16T22:50:17.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:17 smithi192 ceph-mon[36254]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:17.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:17 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:17.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:17 smithi192 ceph-mon[36254]: osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] boot 2024-02-16T22:50:17.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:17 smithi192 ceph-mon[36254]: osdmap e9: 1 total, 1 up, 1 in 2024-02-16T22:50:17.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:17 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:17 smithi064 ceph-mon[29226]: purged_snaps scrub starts 2024-02-16T22:50:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:17 smithi064 ceph-mon[29226]: purged_snaps scrub ok 2024-02-16T22:50:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:17 smithi064 ceph-mon[29226]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:50:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:17 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:17 smithi064 ceph-mon[29226]: osd.0 [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] boot 2024-02-16T22:50:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:17 smithi064 ceph-mon[29226]: osdmap e9: 1 total, 1 up, 1 in 2024-02-16T22:50:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:17 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:50:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-16T22:50:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-16T22:50:17.725 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-02-16T22:50:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-02-16T22:50:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142291 s, 737 MB/s 2024-02-16T22:50:17.725 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 2024-02-16T22:50:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-02-16T22:50:18.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:18 smithi192 ceph-mon[36254]: osdmap e10: 1 total, 1 up, 1 in 2024-02-16T22:50:18.297 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:18 smithi064 ceph-mon[29226]: osdmap e10: 1 total, 1 up, 1 in 2024-02-16T22:50:18.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-02-16T22:50:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:50:19.347 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:19 smithi064 ceph-mon[29226]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:19.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:19 smithi192 ceph-mon[36254]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:20.816 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: Detected new or changed devices on smithi064 2024-02-16T22:50:20.816 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:20.816 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:20.817 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:50:20.817 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: Adjusting osd_memory_target on smithi064 to 12864M 2024-02-16T22:50:20.817 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:20.817 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:20.817 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:50:20.817 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:20.817 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:50:20.817 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:20 smithi064 ceph-mon[29226]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: Detected new or changed devices on smithi064 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: Adjusting osd_memory_target on smithi064 to 12864M 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:50:20.982 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:20 smithi192 ceph-mon[36254]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.006+0000 7fa53761b700 1 -- 172.21.15.64:0/1613953579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 msgr2=0x7fa5300fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.006+0000 7fa53761b700 1 --2- 172.21.15.64:0/1613953579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 0x7fa5300fcce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa520004940 tx=0x7fa520009e30 comp rx=0 tx=0).stop 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 -- 172.21.15.64:0/1613953579 shutdown_connections 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 --2- 172.21.15.64:0/1613953579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 0x7fa5300fcce0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 --2- 172.21.15.64:0/1613953579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fb5c0 0x7fa5300fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 -- 172.21.15.64:0/1613953579 >> 172.21.15.64:0/1613953579 conn(0x7fa5300f6d20 msgr2=0x7fa5300f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 -- 172.21.15.64:0/1613953579 shutdown_connections 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 -- 172.21.15.64:0/1613953579 wait complete. 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 Processor -- start 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 -- start start 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fb5c0 0x7fa53010dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:50:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 0x7fa53010e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53010efe0 con 0x7fa5300fc870 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa53761b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53010f150 con 0x7fa5300fb5c0 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa534bb6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 0x7fa53010e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa534bb6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 0x7fa53010e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35472/0 (socket says 172.21.15.64:35472) 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.007+0000 7fa534bb6700 1 -- 172.21.15.64:0/3343441718 learned_addr learned my addr 172.21.15.64:0/3343441718 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa5353b7700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fb5c0 0x7fa53010dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa534bb6700 1 -- 172.21.15.64:0/3343441718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fb5c0 msgr2=0x7fa53010dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa534bb6700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fb5c0 0x7fa53010dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa534bb6700 1 -- 172.21.15.64:0/3343441718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520005040 con 0x7fa5300fc870 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa534bb6700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 0x7fa53010e280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5200093b0 tx=0x7fa520009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa525ffb700 1 -- 172.21.15.64:0/3343441718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa52000e040 con 0x7fa5300fc870 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa525ffb700 1 -- 172.21.15.64:0/3343441718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa52000bcc0 con 0x7fa5300fc870 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa525ffb700 1 -- 172.21.15.64:0/3343441718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa52001c630 con 0x7fa5300fc870 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53010f3d0 con 0x7fa5300fc870 2024-02-16T22:50:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.008+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53010f910 con 0x7fa5300fc870 2024-02-16T22:50:21.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.009+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53004bed0 con 0x7fa5300fc870 2024-02-16T22:50:21.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.010+0000 7fa525ffb700 1 -- 172.21.15.64:0/3343441718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fa5200139f0 con 0x7fa5300fc870 2024-02-16T22:50:21.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.010+0000 7fa525ffb700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa51c069210 0x7fa51c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:50:21.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.010+0000 7fa525ffb700 1 -- 172.21.15.64:0/3343441718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fa520087610 con 0x7fa5300fc870 2024-02-16T22:50:21.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.010+0000 7fa5353b7700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa51c069210 0x7fa51c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:50:21.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.010+0000 7fa5353b7700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa51c069210 0x7fa51c06b6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006fd0 tx=0x7fa52c008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:50:21.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.015+0000 7fa525ffb700 1 -- 172.21.15.64:0/3343441718 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa5200582f0 con 0x7fa5300fc870 2024-02-16T22:50:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:21.223+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa53010fb80 con 0x7fa51c069210 2024-02-16T22:50:21.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:21 smithi064 ceph-mon[29226]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:50:21.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:50:21.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:50:21.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:21.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:21.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:21 smithi192 ceph-mon[36254]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:50:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:21 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:50:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:21 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:50:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:21 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:21 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:21 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:22.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:22 smithi064 ceph-mon[29226]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:22 smithi192 ceph-mon[36254]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:25.404 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:25 smithi064 ceph-mon[29226]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:25.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:25 smithi192 ceph-mon[36254]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:26.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/4201465411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a8757d-9f77-4f46-83d3-ff904c9e50b2"}]: dispatch 2024-02-16T22:50:26.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/4201465411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21a8757d-9f77-4f46-83d3-ff904c9e50b2"}]': finished 2024-02-16T22:50:26.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: osdmap e11: 2 total, 1 up, 2 in 2024-02-16T22:50:26.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:26.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/807194263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:50:26.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:26.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:26.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:26.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:50:26.537 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:26 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/4201465411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a8757d-9f77-4f46-83d3-ff904c9e50b2"}]: dispatch 2024-02-16T22:50:26.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/4201465411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21a8757d-9f77-4f46-83d3-ff904c9e50b2"}]': finished 2024-02-16T22:50:26.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: osdmap e11: 2 total, 1 up, 2 in 2024-02-16T22:50:26.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:26.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/807194263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:50:26.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:26.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:26.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:26.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:50:26.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:26 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:27.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:27 smithi192 ceph-mon[36254]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:27.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:27 smithi064 ceph-mon[29226]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:28 smithi192 ceph-mon[36254]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:29.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:50:29.093 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:28 smithi064 ceph-mon[29226]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:29.093 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:50:31.202 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:31 smithi064 ceph-mon[29226]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:31.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:31 smithi192 ceph-mon[36254]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:33 smithi064 ceph-mon[29226]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:50:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:33.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:33 smithi064 ceph-mon[29226]: Deploying daemon osd.1 on smithi064 2024-02-16T22:50:33.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:33 smithi192 ceph-mon[36254]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:33.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:50:33.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:33.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:33 smithi192 ceph-mon[36254]: Deploying daemon osd.1 on smithi064 2024-02-16T22:50:35.220 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:35 smithi064 ceph-mon[29226]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:35.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:35 smithi192 ceph-mon[36254]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:37.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:37 smithi064 ceph-mon[29226]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:37.528 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:37 smithi192 ceph-mon[36254]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:50:38.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:38.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:50:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:39.684 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:39 smithi064 ceph-mon[29226]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:39.684 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:39 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:39.684 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:39 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:39.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:39 smithi192 ceph-mon[36254]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:39.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:39 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:39.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:39 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:40.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.840+0000 7fa525ffb700 1 -- 172.21.15.64:0/3343441718 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa53010fb80 con 0x7fa51c069210 2024-02-16T22:50:40.842 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 1 on host 'smithi064' 2024-02-16T22:50:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa51c069210 msgr2=0x7fa51c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:50:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa51c069210 0x7fa51c06b6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006fd0 tx=0x7fa52c008040 comp rx=0 tx=0).stop 2024-02-16T22:50:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 msgr2=0x7fa53010e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:50:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 0x7fa53010e280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5200093b0 tx=0x7fa520009e30 comp rx=0 tx=0).stop 2024-02-16T22:50:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 shutdown_connections 2024-02-16T22:50:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fb5c0 0x7fa53010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa51c069210 0x7fa51c06b6c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 --2- 172.21.15.64:0/3343441718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fc870 0x7fa53010e280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 >> 172.21.15.64:0/3343441718 conn(0x7fa5300f6d20 msgr2=0x7fa5300f8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:50:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 shutdown_connections 2024-02-16T22:50:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:40.842+0000 7fa53761b700 1 -- 172.21.15.64:0/3343441718 wait complete. 2024-02-16T22:50:41.659 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:41 smithi064 ceph-mon[29226]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:41.659 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:41.659 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:41.659 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:41 smithi064 ceph-mon[29226]: from='osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T22:50:41.661 DEBUG:teuthology.orchestra.run.smithi064:osd.1> sudo journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.1.service 2024-02-16T22:50:41.663 INFO:tasks.cephadm:Deploying osd.2 on smithi064 with /dev/vg_nvme/lv_2... 2024-02-16T22:50:41.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-16T22:50:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:41 smithi192 ceph-mon[36254]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:41 smithi192 ceph-mon[36254]: from='osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T22:50:42.354 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:50:43.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:42 smithi064 ceph-mon[29226]: from='osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T22:50:43.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:42 smithi064 ceph-mon[29226]: osdmap e12: 2 total, 1 up, 2 in 2024-02-16T22:50:43.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:42 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:43.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:42 smithi064 ceph-mon[29226]: from='osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T22:50:43.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:42 smithi064 ceph-mon[29226]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:43.120 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:50:42 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[55345]: 2024-02-16T22:50:42.834+0000 7f2ac3a49700 -1 osd.1 0 waiting for initial osdmap 2024-02-16T22:50:43.120 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:50:42 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[55345]: 2024-02-16T22:50:42.838+0000 7f2ac0047700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:50:43.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:42 smithi192 ceph-mon[36254]: from='osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T22:50:43.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:42 smithi192 ceph-mon[36254]: osdmap e12: 2 total, 1 up, 2 in 2024-02-16T22:50:43.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:42 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:43.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:42 smithi192 ceph-mon[36254]: from='osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T22:50:43.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:42 smithi192 ceph-mon[36254]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:43.977 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:43 smithi064 ceph-mon[29226]: from='osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-16T22:50:43.977 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:43 smithi064 ceph-mon[29226]: osdmap e13: 2 total, 1 up, 2 in 2024-02-16T22:50:43.977 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:43 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:43.978 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:43 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:43.978 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:43 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:50:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:43 smithi192 ceph-mon[36254]: from='osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-16T22:50:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:43 smithi192 ceph-mon[36254]: osdmap e13: 2 total, 1 up, 2 in 2024-02-16T22:50:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:43 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:43 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:43 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:50:44.871 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-16T22:50:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-16T22:50:44.872 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-02-16T22:50:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-02-16T22:50:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124744 s, 841 MB/s 2024-02-16T22:50:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-02-16T22:50:44.973 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:44 smithi064 ceph-mon[29226]: purged_snaps scrub starts 2024-02-16T22:50:44.973 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:44 smithi064 ceph-mon[29226]: purged_snaps scrub ok 2024-02-16T22:50:44.973 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:44 smithi064 ceph-mon[29226]: osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] boot 2024-02-16T22:50:44.973 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:44 smithi064 ceph-mon[29226]: osdmap e14: 2 total, 2 up, 2 in 2024-02-16T22:50:44.974 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:44 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:44.974 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:44 smithi064 ceph-mon[29226]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:44 smithi192 ceph-mon[36254]: purged_snaps scrub starts 2024-02-16T22:50:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:44 smithi192 ceph-mon[36254]: purged_snaps scrub ok 2024-02-16T22:50:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:44 smithi192 ceph-mon[36254]: osd.1 [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] boot 2024-02-16T22:50:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:44 smithi192 ceph-mon[36254]: osdmap e14: 2 total, 2 up, 2 in 2024-02-16T22:50:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:44 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:50:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:44 smithi192 ceph-mon[36254]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:50:45.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-02-16T22:50:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:45 smithi064 ceph-mon[29226]: osdmap e15: 2 total, 2 up, 2 in 2024-02-16T22:50:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:45 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:46.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:45 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:46.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:45 smithi192 ceph-mon[36254]: osdmap e15: 2 total, 2 up, 2 in 2024-02-16T22:50:46.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:45 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:46.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:45 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:50:47.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:46 smithi064 ceph-mon[29226]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:46 smithi192 ceph-mon[36254]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:49.000 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:49.001 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:49.001 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:49.001 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:50:49.001 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:50:49.001 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:49.001 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:49.001 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:50:49.001 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:49.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:49.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:49.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:49.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:50:49.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:50:49.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:49.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:49.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:50:49.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 -- 172.21.15.64:0/2160657094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 msgr2=0x7f10f80ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 --2- 172.21.15.64:0/2160657094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 0x7f10f80ffaf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009ce0 tx=0x7f10e8004d10 comp rx=0 tx=0).stop 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 -- 172.21.15.64:0/2160657094 shutdown_connections 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 --2- 172.21.15.64:0/2160657094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f81000c0 0x7f10f8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 --2- 172.21.15.64:0/2160657094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 0x7f10f80ffaf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 -- 172.21.15.64:0/2160657094 >> 172.21.15.64:0/2160657094 conn(0x7f10f80f74e0 msgr2=0x7f10f80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 -- 172.21.15.64:0/2160657094 shutdown_connections 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 -- 172.21.15.64:0/2160657094 wait complete. 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 Processor -- start 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 -- start start 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 0x7f10f810daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:50:49.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f81000c0 0x7f10f810e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f810e6c0 con 0x7f10f80663e0 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.455+0000 7f10fe262700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f810ef00 con 0x7f10f81000c0 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 0x7f10f810daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 0x7f10f810daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37974/0 (socket says 172.21.15.64:37974) 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10f77fe700 1 -- 172.21.15.64:0/854780594 learned_addr learned my addr 172.21.15.64:0/854780594 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10f6ffd700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f81000c0 0x7f10f810e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10f77fe700 1 -- 172.21.15.64:0/854780594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f81000c0 msgr2=0x7f10f810e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10f77fe700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f81000c0 0x7f10f810e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10f77fe700 1 -- 172.21.15.64:0/854780594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8009940 con 0x7f10f80663e0 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10f77fe700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 0x7f10f810daf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f10e8015040 tx=0x7f10e8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10effff700 1 -- 172.21.15.64:0/854780594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e800e040 con 0x7f10f80663e0 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f810f180 con 0x7f10f80663e0 2024-02-16T22:50:49.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10effff700 1 -- 172.21.15.64:0/854780594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10e80128e0 con 0x7f10f80663e0 2024-02-16T22:50:49.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10effff700 1 -- 172.21.15.64:0/854780594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e801c940 con 0x7f10f80663e0 2024-02-16T22:50:49.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.456+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f810f6c0 con 0x7f10f80663e0 2024-02-16T22:50:49.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.457+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f8107af0 con 0x7f10f80663e0 2024-02-16T22:50:49.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.458+0000 7f10effff700 1 -- 172.21.15.64:0/854780594 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f10e8012410 con 0x7f10f80663e0 2024-02-16T22:50:49.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.458+0000 7f10effff700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f10e4069210 0x7f10e406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:50:49.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.458+0000 7f10effff700 1 -- 172.21.15.64:0/854780594 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f10e8088230 con 0x7f10f80663e0 2024-02-16T22:50:49.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.458+0000 7f10f6ffd700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f10e4069210 0x7f10e406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:50:49.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.459+0000 7f10f6ffd700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f10e4069210 0x7f10e406b6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f10e0006fd0 tx=0x7f10e0008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:50:49.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.463+0000 7f10effff700 1 -- 172.21.15.64:0/854780594 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f10e8058c40 con 0x7f10f80663e0 2024-02-16T22:50:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:50:49.670+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f10f802b340 con 0x7f10e4069210 2024-02-16T22:50:50.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:49 smithi064 ceph-mon[29226]: Detected new or changed devices on smithi064 2024-02-16T22:50:50.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:49 smithi064 ceph-mon[29226]: Adjusting osd_memory_target on smithi064 to 6432M 2024-02-16T22:50:50.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:49 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:50:50.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:49 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:50:50.020 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:49 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:50:50.020 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:49 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:50.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:49 smithi192 ceph-mon[36254]: Detected new or changed devices on smithi064 2024-02-16T22:50:50.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:49 smithi192 ceph-mon[36254]: Adjusting osd_memory_target on smithi064 to 6432M 2024-02-16T22:50:50.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:49 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:50:50.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:49 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:50:50.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:49 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:50:50.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:49 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:51.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:50 smithi064 ceph-mon[29226]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:50:51.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:50 smithi064 ceph-mon[29226]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:51.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:50 smithi192 ceph-mon[36254]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:50:51.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:50 smithi192 ceph-mon[36254]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:53 smithi192 ceph-mon[36254]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:53.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:53 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1553434494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fcb2e12-54a0-46db-af11-d047cf25d50b"}]: dispatch 2024-02-16T22:50:53.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:53 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1553434494' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fcb2e12-54a0-46db-af11-d047cf25d50b"}]': finished 2024-02-16T22:50:53.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:53 smithi192 ceph-mon[36254]: osdmap e16: 3 total, 2 up, 3 in 2024-02-16T22:50:53.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:50:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:53 smithi064 ceph-mon[29226]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:53 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1553434494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fcb2e12-54a0-46db-af11-d047cf25d50b"}]: dispatch 2024-02-16T22:50:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:53 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1553434494' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fcb2e12-54a0-46db-af11-d047cf25d50b"}]': finished 2024-02-16T22:50:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:53 smithi064 ceph-mon[29226]: osdmap e16: 3 total, 2 up, 3 in 2024-02-16T22:50:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:50:54.474 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:54 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1503578397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:50:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:54 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1503578397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:50:55.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:55 smithi064 ceph-mon[29226]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:55.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:55.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:55.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:55.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:50:55.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:55.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:55 smithi192 ceph-mon[36254]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:55.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:55.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:55.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:50:55.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:50:55.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:50:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:57 smithi192 ceph-mon[36254]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:57.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:57 smithi064 ceph-mon[29226]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:58.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:58 smithi064 ceph-mon[29226]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:58.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:50:59.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:58 smithi192 ceph-mon[36254]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:50:59.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:59 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:51:00.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:50:59 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:00.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:59 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:51:00.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:50:59 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:01.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:00 smithi192 ceph-mon[36254]: Deploying daemon osd.2 on smithi064 2024-02-16T22:51:01.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:00 smithi192 ceph-mon[36254]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:01.329 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:00 smithi064 ceph-mon[29226]: Deploying daemon osd.2 on smithi064 2024-02-16T22:51:01.329 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:00 smithi064 ceph-mon[29226]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:03.310 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:03 smithi064 ceph-mon[29226]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:03.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:03 smithi192 ceph-mon[36254]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:05.402 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:05 smithi064 ceph-mon[29226]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:05.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:05.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:05.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:05 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:05.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:05 smithi192 ceph-mon[36254]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:05.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:05.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:05.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:05 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:07.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:07 smithi064 ceph-mon[29226]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:07.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:07 smithi192 ceph-mon[36254]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:07.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.782+0000 7f10effff700 1 -- 172.21.15.64:0/854780594 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f10f802b340 con 0x7f10e4069210 2024-02-16T22:51:07.784 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 2 on host 'smithi064' 2024-02-16T22:51:07.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f10e4069210 msgr2=0x7f10e406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:07.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f10e4069210 0x7f10e406b6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f10e0006fd0 tx=0x7f10e0008040 comp rx=0 tx=0).stop 2024-02-16T22:51:07.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 msgr2=0x7f10f810daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:07.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 0x7f10f810daf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f10e8015040 tx=0x7f10e8004d10 comp rx=0 tx=0).stop 2024-02-16T22:51:07.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 shutdown_connections 2024-02-16T22:51:07.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f10e4069210 0x7f10e406b6c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:07.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f80663e0 0x7f10f810daf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:07.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 --2- 172.21.15.64:0/854780594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f81000c0 0x7f10f810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:07.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 >> 172.21.15.64:0/854780594 conn(0x7f10f80f74e0 msgr2=0x7f10f80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:51:07.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 shutdown_connections 2024-02-16T22:51:07.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:51:07.785+0000 7f10fe262700 1 -- 172.21.15.64:0/854780594 wait complete. 2024-02-16T22:51:08.740 DEBUG:teuthology.orchestra.run.smithi064:osd.2> sudo journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.2.service 2024-02-16T22:51:08.742 INFO:tasks.cephadm:Deploying osd.3 on smithi192 with /dev/vg_nvme/lv_4... 2024-02-16T22:51:08.742 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-16T22:51:09.003 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:51:09.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:08 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:09.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:08 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:09.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:08 smithi192 ceph-mon[36254]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:09.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:08 smithi192 ceph-mon[36254]: from='osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T22:51:09.070 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:09.070 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:08 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:09.070 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:08 smithi064 ceph-mon[29226]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:09.070 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:08 smithi064 ceph-mon[29226]: from='osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T22:51:10.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:09 smithi064 ceph-mon[29226]: from='osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T22:51:10.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:09 smithi064 ceph-mon[29226]: osdmap e17: 3 total, 2 up, 3 in 2024-02-16T22:51:10.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:09 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:10.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:09 smithi064 ceph-mon[29226]: from='osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T22:51:10.119 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:09 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[62667]: 2024-02-16T22:51:09.798+0000 7f94d9fb6700 -1 osd.2 0 waiting for initial osdmap 2024-02-16T22:51:10.119 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:09 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[62667]: 2024-02-16T22:51:09.802+0000 7f94d3daf700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:51:10.135 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:09 smithi192 ceph-mon[36254]: from='osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T22:51:10.135 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:09 smithi192 ceph-mon[36254]: osdmap e17: 3 total, 2 up, 3 in 2024-02-16T22:51:10.136 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:09 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:10.136 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:09 smithi192 ceph-mon[36254]: from='osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T22:51:10.939 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-16T22:51:10.939 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-16T22:51:10.939 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-02-16T22:51:10.939 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-02-16T22:51:10.939 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137185 s, 764 MB/s 2024-02-16T22:51:10.939 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-02-16T22:51:11.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:10 smithi064 ceph-mon[29226]: from='osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-16T22:51:11.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:10 smithi064 ceph-mon[29226]: osdmap e18: 3 total, 2 up, 3 in 2024-02-16T22:51:11.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:10 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:11.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:10 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:11.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:10 smithi064 ceph-mon[29226]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:11.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:10 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:10 smithi192 ceph-mon[36254]: from='osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-02-16T22:51:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:10 smithi192 ceph-mon[36254]: osdmap e18: 3 total, 2 up, 3 in 2024-02-16T22:51:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:10 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:11.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:10 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:11.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:10 smithi192 ceph-mon[36254]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T22:51:11.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:10 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:11.565 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-02-16T22:51:11.813 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:51:11.909 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:11 smithi192 ceph-mon[36254]: purged_snaps scrub starts 2024-02-16T22:51:11.909 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:11 smithi192 ceph-mon[36254]: purged_snaps scrub ok 2024-02-16T22:51:11.909 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:11 smithi192 ceph-mon[36254]: osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] boot 2024-02-16T22:51:11.909 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:11 smithi192 ceph-mon[36254]: osdmap e19: 3 total, 3 up, 3 in 2024-02-16T22:51:11.910 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:11 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:11.910 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:11 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:11.910 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:11 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:12.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:11 smithi064 ceph-mon[29226]: purged_snaps scrub starts 2024-02-16T22:51:12.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:11 smithi064 ceph-mon[29226]: purged_snaps scrub ok 2024-02-16T22:51:12.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:11 smithi064 ceph-mon[29226]: osd.2 [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] boot 2024-02-16T22:51:12.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:11 smithi064 ceph-mon[29226]: osdmap e19: 3 total, 3 up, 3 in 2024-02-16T22:51:12.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:11 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:51:12.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:11 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:12.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:11 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:12.985 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:12 smithi192 ceph-mon[36254]: osdmap e20: 3 total, 3 up, 3 in 2024-02-16T22:51:12.986 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:12 smithi192 ceph-mon[36254]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:12.986 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:12 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-16T22:51:13.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:12 smithi064 ceph-mon[29226]: osdmap e20: 3 total, 3 up, 3 in 2024-02-16T22:51:13.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:12 smithi064 ceph-mon[29226]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:13.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:12 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-16T22:51:13.186 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1566918233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fcfb0 msgr2=0x7f8d780fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:13.186 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 --2- 172.21.15.192:0/1566918233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fcfb0 0x7f8d780fd420 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8d68004940 tx=0x7f8d68009e30 comp rx=0 tx=0).stop 2024-02-16T22:51:13.186 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1566918233 shutdown_connections 2024-02-16T22:51:13.186 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 --2- 172.21.15.192:0/1566918233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fcfb0 0x7f8d780fd420 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 --2- 172.21.15.192:0/1566918233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d780fbe40 0x7f8d780fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1566918233 >> 172.21.15.192:0/1566918233 conn(0x7f8d780f7520 msgr2=0x7f8d780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1566918233 shutdown_connections 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1566918233 wait complete. 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 Processor -- start 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 -- start start 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fbe40 0x7f8d780fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d780fcfb0 0x7f8d780fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780fe980 con 0x7f8d780fcfb0 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7fd42700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780feaf0 con 0x7f8d780fbe40 2024-02-16T22:51:13.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7dade700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fbe40 0x7f8d780fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7dade700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fbe40 0x7f8d780fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:51850/0 (socket says 172.21.15.192:51850) 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.183+0000 7f8d7dade700 1 -- 172.21.15.192:0/1644124685 learned_addr learned my addr 172.21.15.192:0/1644124685 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d7d2dd700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d780fcfb0 0x7f8d780fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d7dade700 1 -- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d780fcfb0 msgr2=0x7f8d780fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d7dade700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d780fcfb0 0x7f8d780fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d7dade700 1 -- 172.21.15.192:0/1644124685 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68005040 con 0x7f8d780fbe40 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d7dade700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fbe40 0x7f8d780fff00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c830 tx=0x7f8d74004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d6e7fc700 1 -- 172.21.15.192:0/1644124685 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d740106f0 con 0x7f8d780fbe40 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d6e7fc700 1 -- 172.21.15.192:0/1644124685 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d74010d20 con 0x7f8d780fbe40 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d6e7fc700 1 -- 172.21.15.192:0/1644124685 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d740113a0 con 0x7f8d780fbe40 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d780fedd0 con 0x7f8d780fbe40 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.184+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d78115970 con 0x7f8d780fbe40 2024-02-16T22:51:13.188 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.185+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7804bed0 con 0x7f8d780fbe40 2024-02-16T22:51:13.189 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.185+0000 7f8d6e7fc700 1 -- 172.21.15.192:0/1644124685 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f8d74012400 con 0x7f8d780fbe40 2024-02-16T22:51:13.189 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.186+0000 7f8d6e7fc700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8d64069210 0x7f8d6406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:51:13.189 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.186+0000 7f8d6e7fc700 1 -- 172.21.15.192:0/1644124685 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f8d74086690 con 0x7f8d780fbe40 2024-02-16T22:51:13.190 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.186+0000 7f8d7d2dd700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8d64069210 0x7f8d6406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:51:13.190 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.186+0000 7f8d7d2dd700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8d64069210 0x7f8d6406b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d68004f90 tx=0x7f8d68009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:51:13.191 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.190+0000 7f8d6e7fc700 1 -- 172.21.15.192:0/1644124685 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8d74056e90 con 0x7f8d780fbe40 2024-02-16T22:51:13.382 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:13.380+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d78066660 con 0x7f8d64069210 2024-02-16T22:51:14.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-16T22:51:14.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:13 smithi192 ceph-mon[36254]: osdmap e21: 3 total, 3 up, 3 in 2024-02-16T22:51:14.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-16T22:51:14.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:14.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:13 smithi192 ceph-mon[36254]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:51:14.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:51:14.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:51:14.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-16T22:51:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:13 smithi064 ceph-mon[29226]: osdmap e21: 3 total, 3 up, 3 in 2024-02-16T22:51:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-16T22:51:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:13 smithi064 ceph-mon[29226]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:51:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:51:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:51:14.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:14.908 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:14 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-16T22:51:14.908 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:14 smithi192 ceph-mon[36254]: osdmap e22: 3 total, 3 up, 3 in 2024-02-16T22:51:14.908 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:14 smithi192 ceph-mon[36254]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:15.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:14 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-16T22:51:15.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:14 smithi064 ceph-mon[29226]: osdmap e22: 3 total, 3 up, 3 in 2024-02-16T22:51:15.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:14 smithi064 ceph-mon[29226]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:15.119 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:51:14 smithi064 sudo[66978]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:51:15.119 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:51:14 smithi064 sudo[66978]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:15.119 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:51:14 smithi064 sudo[66978]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:15.119 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:51:14 smithi064 sudo[66978]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:15.525 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67068]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:51:15.525 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67068]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:15.525 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67068]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:15.525 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67068]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:15.786 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67160]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:51:15.786 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:15.786 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67160]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:15.786 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67160]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:15.841 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:15.840+0000 7f8d6e7fc700 1 -- 172.21.15.192:0/1644124685 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8d740537d0 con 0x7f8d780fbe40 2024-02-16T22:51:16.052 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67182]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:51:16.052 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67182]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:16.052 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67182]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:16.052 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 22:51:15 smithi064 sudo[67182]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: osdmap e23: 3 total, 3 up, 3 in 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: Detected new or changed devices on smithi064 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: Adjusting osd_memory_target on smithi064 to 4288M 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.053 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/1648028457' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a58c77-96fa-4835-9ea4-d16e6d995620"}]: dispatch 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a58c77-96fa-4835-9ea4-d16e6d995620"}]: dispatch 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7a58c77-96fa-4835-9ea4-d16e6d995620"}]': finished 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: osdmap e24: 4 total, 3 up, 4 in 2024-02-16T22:51:16.054 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:15 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:16.097 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: osdmap e23: 3 total, 3 up, 3 in 2024-02-16T22:51:16.097 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: Detected new or changed devices on smithi064 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: Adjusting osd_memory_target on smithi064 to 4288M 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='client.? 172.21.15.192:0/1648028457' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a58c77-96fa-4835-9ea4-d16e6d995620"}]: dispatch 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a58c77-96fa-4835-9ea4-d16e6d995620"}]: dispatch 2024-02-16T22:51:16.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7a58c77-96fa-4835-9ea4-d16e6d995620"}]': finished 2024-02-16T22:51:16.099 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: osdmap e24: 4 total, 3 up, 4 in 2024-02-16T22:51:16.099 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:15 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:16.326 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67185]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:51:16.326 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67185]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:16.326 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67185]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:16.326 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67185]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:16.584 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67268]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:51:16.585 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67268]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:16.585 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67268]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:16.585 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67268]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:16.849 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67293]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:51:16.850 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67293]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:16.850 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67293]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:16.850 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67293]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:17.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67298]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:51:17.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67298]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:17.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67298]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:17.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:16 smithi064 sudo[67298]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:17.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 sudo[39806]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:51:17.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 sudo[39806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:17.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 sudo[39806]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:17.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 sudo[39806]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:17.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:17 smithi064 ceph-mon[29226]: mgrmap e19: smithi064.kubmlk(active, since 2m), standbys: smithi192.pauvih 2024-02-16T22:51:17.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:17 smithi064 ceph-mon[29226]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T22:51:17.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 sudo[39995]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:51:17.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 sudo[39995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:51:17.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 sudo[39995]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:51:17.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 sudo[39995]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:51:17.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 ceph-mon[36254]: mgrmap e19: smithi064.kubmlk(active, since 2m), standbys: smithi192.pauvih 2024-02-16T22:51:17.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:17 smithi192 ceph-mon[36254]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T22:51:18.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: pgmap v81: 1 pgs: 1 active+clean; 449 KiB data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:18.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/897131833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:51:18.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T22:51:18.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:51:18.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:51:18.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T22:51:18.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:51:18.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:51:18.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T22:51:18.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:18 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:18.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: pgmap v81: 1 pgs: 1 active+clean; 449 KiB data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:18.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='client.? 172.21.15.192:0/897131833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:51:18.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T22:51:18.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:51:18.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:51:18.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T22:51:18.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:51:18.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:51:18.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T22:51:18.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:18 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:19.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:19 smithi064 ceph-mon[29226]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:19.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:19 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:19.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:19 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:19.641 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:19 smithi192 ceph-mon[36254]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:19.641 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:19 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:19.641 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:19 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:20.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:20.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:51:20.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:20.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:20.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:51:20.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:21.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:21 smithi064 ceph-mon[29226]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 6.2 KiB/s rd, 300 KiB/s wr, 20 op/s 2024-02-16T22:51:21.643 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:21 smithi192 ceph-mon[36254]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 6.2 KiB/s rd, 300 KiB/s wr, 20 op/s 2024-02-16T22:51:23.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:23 smithi192 ceph-mon[36254]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.2 KiB/s rd, 205 KiB/s wr, 14 op/s 2024-02-16T22:51:23.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:23 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:51:23.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:23 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:23 smithi064 ceph-mon[29226]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.2 KiB/s rd, 205 KiB/s wr, 14 op/s 2024-02-16T22:51:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:23 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:51:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:23 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:24.639 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:24 smithi192 ceph-mon[36254]: Deploying daemon osd.3 on smithi192 2024-02-16T22:51:24.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:24 smithi064 ceph-mon[29226]: Deploying daemon osd.3 on smithi192 2024-02-16T22:51:25.401 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:25 smithi192 ceph-mon[36254]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.2 KiB/s rd, 156 KiB/s wr, 10 op/s 2024-02-16T22:51:25.770 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:25 smithi064 ceph-mon[29226]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.2 KiB/s rd, 156 KiB/s wr, 10 op/s 2024-02-16T22:51:27.403 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:27 smithi192 ceph-mon[36254]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.6 KiB/s rd, 126 KiB/s wr, 8 op/s 2024-02-16T22:51:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:27 smithi064 ceph-mon[29226]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.6 KiB/s rd, 126 KiB/s wr, 8 op/s 2024-02-16T22:51:28.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:28.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:28.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:28.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:28 smithi192 ceph-mon[36254]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-16T22:51:28.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:28.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:28.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:28 smithi064 ceph-mon[29226]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-16T22:51:28.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:31.039 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.038+0000 7f8d6e7fc700 1 -- 172.21.15.192:0/1644124685 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8d78066660 con 0x7f8d64069210 2024-02-16T22:51:31.040 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 3 on host 'smithi192' 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8d64069210 msgr2=0x7f8d6406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8d64069210 0x7f8d6406b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d68004f90 tx=0x7f8d68009e30 comp rx=0 tx=0).stop 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fbe40 msgr2=0x7f8d780fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fbe40 0x7f8d780fff00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c830 tx=0x7f8d74004470 comp rx=0 tx=0).stop 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 shutdown_connections 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d780fbe40 0x7f8d780fff00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8d64069210 0x7f8d6406b6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 --2- 172.21.15.192:0/1644124685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d780fcfb0 0x7f8d780fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.040+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 >> 172.21.15.192:0/1644124685 conn(0x7f8d780f7520 msgr2=0x7f8d78065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.041+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 shutdown_connections 2024-02-16T22:51:31.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:31.041+0000 7f8d7fd42700 1 -- 172.21.15.192:0/1644124685 wait complete. 2024-02-16T22:51:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:31 smithi192 ceph-mon[36254]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-16T22:51:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:31 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:31.615 DEBUG:teuthology.orchestra.run.smithi192:osd.3> sudo journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.3.service 2024-02-16T22:51:31.617 INFO:tasks.cephadm:Deploying osd.4 on smithi192 with /dev/vg_nvme/lv_3... 2024-02-16T22:51:31.617 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-16T22:51:31.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:31 smithi064 ceph-mon[29226]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-16T22:51:31.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:31.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:31 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:31.938 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:51:32.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:32 smithi192 ceph-mon[36254]: from='osd.3 [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:51:32.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:32 smithi192 ceph-mon[36254]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:51:32.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:32.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:32.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:32 smithi064 ceph-mon[29226]: from='osd.3 [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:51:32.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:32 smithi064 ceph-mon[29226]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:51:32.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:32.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:33.168 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:33 smithi192 ceph-mon[36254]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:33.168 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:33 smithi192 ceph-mon[36254]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T22:51:33.169 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:33 smithi192 ceph-mon[36254]: osdmap e25: 4 total, 3 up, 4 in 2024-02-16T22:51:33.169 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:33.169 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:33 smithi192 ceph-mon[36254]: from='osd.3 [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:51:33.169 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:33 smithi192 ceph-mon[36254]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:51:33.496 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 22:51:33 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[41814]: 2024-02-16T22:51:33.166+0000 7f16cff68700 -1 osd.3 0 waiting for initial osdmap 2024-02-16T22:51:33.496 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 22:51:33 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[41814]: 2024-02-16T22:51:33.173+0000 7f16cad63700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:51:33.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:33 smithi064 ceph-mon[29226]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:33.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:33 smithi064 ceph-mon[29226]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T22:51:33.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:33 smithi064 ceph-mon[29226]: osdmap e25: 4 total, 3 up, 4 in 2024-02-16T22:51:33.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:33.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:33 smithi064 ceph-mon[29226]: from='osd.3 [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:51:33.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:33 smithi064 ceph-mon[29226]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:51:33.726 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-16T22:51:33.726 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-16T22:51:33.726 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-02-16T22:51:33.727 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-02-16T22:51:33.727 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125438 s, 836 MB/s 2024-02-16T22:51:33.727 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-02-16T22:51:34.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:34 smithi192 ceph-mon[36254]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-02-16T22:51:34.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:34 smithi192 ceph-mon[36254]: osdmap e26: 4 total, 3 up, 4 in 2024-02-16T22:51:34.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:34.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:34 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:34.614 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-02-16T22:51:34.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:34 smithi064 ceph-mon[29226]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-02-16T22:51:34.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:34 smithi064 ceph-mon[29226]: osdmap e26: 4 total, 3 up, 4 in 2024-02-16T22:51:34.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:34.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:34 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:35.082 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:51:35.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:35 smithi192 ceph-mon[36254]: purged_snaps scrub starts 2024-02-16T22:51:35.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:35 smithi192 ceph-mon[36254]: purged_snaps scrub ok 2024-02-16T22:51:35.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:35 smithi192 ceph-mon[36254]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:35.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:35 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:35.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:35 smithi192 ceph-mon[36254]: osd.3 [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] boot 2024-02-16T22:51:35.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:35 smithi192 ceph-mon[36254]: osdmap e27: 4 total, 4 up, 4 in 2024-02-16T22:51:35.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:35 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:35 smithi064 ceph-mon[29226]: purged_snaps scrub starts 2024-02-16T22:51:35.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:35 smithi064 ceph-mon[29226]: purged_snaps scrub ok 2024-02-16T22:51:35.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:35 smithi064 ceph-mon[29226]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T22:51:35.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:35 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:35.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:35 smithi064 ceph-mon[29226]: osd.3 [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] boot 2024-02-16T22:51:35.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:35 smithi064 ceph-mon[29226]: osdmap e27: 4 total, 4 up, 4 in 2024-02-16T22:51:35.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:35 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:51:36.480 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:36 smithi064 ceph-mon[29226]: osdmap e28: 4 total, 4 up, 4 in 2024-02-16T22:51:36.480 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:36 smithi064 ceph-mon[29226]: osdmap e29: 4 total, 4 up, 4 in 2024-02-16T22:51:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:36 smithi192 ceph-mon[36254]: osdmap e28: 4 total, 4 up, 4 in 2024-02-16T22:51:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:36 smithi192 ceph-mon[36254]: osdmap e29: 4 total, 4 up, 4 in 2024-02-16T22:51:37.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 -- 172.21.15.192:0/2877710179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e80fda80 msgr2=0x7f08e80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 --2- 172.21.15.192:0/2877710179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e80fda80 0x7f08e80ffe60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f08e4004940 tx=0x7f08e4009e30 comp rx=0 tx=0).stop 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 -- 172.21.15.192:0/2877710179 shutdown_connections 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 --2- 172.21.15.192:0/2877710179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08e81003a0 0x7f08e8102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 --2- 172.21.15.192:0/2877710179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e80fda80 0x7f08e80ffe60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 -- 172.21.15.192:0/2877710179 >> 172.21.15.192:0/2877710179 conn(0x7f08e80f7520 msgr2=0x7f08e80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 -- 172.21.15.192:0/2877710179 shutdown_connections 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 -- 172.21.15.192:0/2877710179 wait complete. 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 Processor -- start 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 -- start start 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.213+0000 7f08f055f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08e80fda80 0x7f08e810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08f055f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e81003a0 0x7f08e810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08f055f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e810e4e0 con 0x7f08e80fda80 2024-02-16T22:51:37.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08f055f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e810ed20 con 0x7f08e81003a0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08edafa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e81003a0 0x7f08e810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08edafa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e81003a0 0x7f08e810de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52536/0 (socket says 172.21.15.192:52536) 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08edafa700 1 -- 172.21.15.192:0/1498910283 learned_addr learned my addr 172.21.15.192:0/1498910283 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08ee2fb700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08e80fda80 0x7f08e810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08edafa700 1 -- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08e80fda80 msgr2=0x7f08e810d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08edafa700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08e80fda80 0x7f08e810d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08edafa700 1 -- 172.21.15.192:0/1498910283 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4005040 con 0x7f08e81003a0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08edafa700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e81003a0 0x7f08e810de50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b790 tx=0x7f08d8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08deffd700 1 -- 172.21.15.192:0/1498910283 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08d800f6f0 con 0x7f08e81003a0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e810f000 con 0x7f08e81003a0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08deffd700 1 -- 172.21.15.192:0/1498910283 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08d800fd20 con 0x7f08e81003a0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08deffd700 1 -- 172.21.15.192:0/1498910283 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08d80103a0 con 0x7f08e81003a0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.214+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e810f540 con 0x7f08e81003a0 2024-02-16T22:51:37.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.215+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e8107ad0 con 0x7f08e81003a0 2024-02-16T22:51:37.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.216+0000 7f08deffd700 1 -- 172.21.15.192:0/1498910283 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f08d8011400 con 0x7f08e81003a0 2024-02-16T22:51:37.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.216+0000 7f08deffd700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f08d40692a0 0x7f08d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:51:37.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.216+0000 7f08deffd700 1 -- 172.21.15.192:0/1498910283 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f08d80867f0 con 0x7f08e81003a0 2024-02-16T22:51:37.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.216+0000 7f08ee2fb700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f08d40692a0 0x7f08d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:51:37.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.217+0000 7f08ee2fb700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f08d40692a0 0x7f08d406b750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08e400b3b0 tx=0x7f08e4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:51:37.222 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.221+0000 7f08deffd700 1 -- 172.21.15.192:0/1498910283 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f08d8056cf0 con 0x7f08e81003a0 2024-02-16T22:51:37.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:37 smithi064 ceph-mon[29226]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:37.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:37.416+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f08e810f880 con 0x7f08d40692a0 2024-02-16T22:51:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:37 smithi192 ceph-mon[36254]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: osdmap e30: 4 total, 4 up, 4 in 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:38.530 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:51:38.531 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.531 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:38.531 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:38.531 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:51:38.531 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.531 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:38 smithi192 ceph-mon[36254]: osdmap e31: 4 total, 4 up, 4 in 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: osdmap e30: 4 total, 4 up, 4 in 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:38.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:51:38.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:38.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:38.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:51:38.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:38.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:38 smithi064 ceph-mon[29226]: osdmap e31: 4 total, 4 up, 4 in 2024-02-16T22:51:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:39 smithi192 ceph-mon[36254]: Detected new or changed devices on smithi192 2024-02-16T22:51:39.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:39 smithi192 ceph-mon[36254]: Adjusting osd_memory_target on smithi192 to 15936M 2024-02-16T22:51:39.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:39 smithi192 ceph-mon[36254]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:39.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:39 smithi064 ceph-mon[29226]: Detected new or changed devices on smithi192 2024-02-16T22:51:39.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:39 smithi064 ceph-mon[29226]: Adjusting osd_memory_target on smithi192 to 15936M 2024-02-16T22:51:39.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:39 smithi064 ceph-mon[29226]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:40.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:40 smithi192 ceph-mon[36254]: from='client.? 172.21.15.192:0/911208803' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e094c5-2494-457a-a4a5-7b2742be9279"}]: dispatch 2024-02-16T22:51:40.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:40 smithi192 ceph-mon[36254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e094c5-2494-457a-a4a5-7b2742be9279"}]: dispatch 2024-02-16T22:51:40.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:40 smithi192 ceph-mon[36254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04e094c5-2494-457a-a4a5-7b2742be9279"}]': finished 2024-02-16T22:51:40.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:40 smithi192 ceph-mon[36254]: osdmap e32: 5 total, 4 up, 5 in 2024-02-16T22:51:40.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:40 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:40.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:40 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/911208803' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e094c5-2494-457a-a4a5-7b2742be9279"}]: dispatch 2024-02-16T22:51:40.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:40 smithi064 ceph-mon[29226]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e094c5-2494-457a-a4a5-7b2742be9279"}]: dispatch 2024-02-16T22:51:40.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:40 smithi064 ceph-mon[29226]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04e094c5-2494-457a-a4a5-7b2742be9279"}]': finished 2024-02-16T22:51:40.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:40 smithi064 ceph-mon[29226]: osdmap e32: 5 total, 4 up, 5 in 2024-02-16T22:51:40.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:40 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:41.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:41 smithi192 ceph-mon[36254]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-16T22:51:41.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:41 smithi192 ceph-mon[36254]: from='client.? 172.21.15.192:0/3040885246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:51:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:41 smithi064 ceph-mon[29226]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-16T22:51:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:41 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/3040885246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:51:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:43 smithi192 ceph-mon[36254]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-16T22:51:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:43 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:43.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:43 smithi064 ceph-mon[29226]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-16T22:51:43.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:43 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:45.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:45 smithi192 ceph-mon[36254]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-16T22:51:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:45 smithi064 ceph-mon[29226]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-16T22:51:47.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:47 smithi192 ceph-mon[36254]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-16T22:51:47.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:47 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:51:47.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:47 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:47 smithi064 ceph-mon[29226]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-16T22:51:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:47 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:51:47.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:47 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:51:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:48 smithi192 ceph-mon[36254]: Deploying daemon osd.4 on smithi192 2024-02-16T22:51:48.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:48 smithi064 ceph-mon[29226]: Deploying daemon osd.4 on smithi192 2024-02-16T22:51:49.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:49 smithi192 ceph-mon[36254]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-16T22:51:49.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:49 smithi064 ceph-mon[29226]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-16T22:51:51.495 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:51 smithi192 ceph-mon[36254]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-02-16T22:51:51.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:51 smithi064 ceph-mon[29226]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-02-16T22:51:52.759 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:52 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:52.760 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:52 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:52.760 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:52 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:52.760 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:52 smithi192 ceph-mon[36254]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:52.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:52 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:52.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:52 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:52.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:52 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:51:52.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:52 smithi064 ceph-mon[29226]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:54.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.827+0000 7f08deffd700 1 -- 172.21.15.192:0/1498910283 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f08e810f880 con 0x7f08d40692a0 2024-02-16T22:51:54.828 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 4 on host 'smithi192' 2024-02-16T22:51:54.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f08d40692a0 msgr2=0x7f08d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:54.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f08d40692a0 0x7f08d406b750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08e400b3b0 tx=0x7f08e4009e30 comp rx=0 tx=0).stop 2024-02-16T22:51:54.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e81003a0 msgr2=0x7f08e810de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:51:54.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e81003a0 0x7f08e810de50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b790 tx=0x7f08d8009250 comp rx=0 tx=0).stop 2024-02-16T22:51:54.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 shutdown_connections 2024-02-16T22:51:54.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f08d40692a0 0x7f08d406b750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:54.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08e80fda80 0x7f08e810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:54.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 --2- 172.21.15.192:0/1498910283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08e81003a0 0x7f08e810de50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:51:54.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 >> 172.21.15.192:0/1498910283 conn(0x7f08e80f7520 msgr2=0x7f08e80f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:51:54.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 shutdown_connections 2024-02-16T22:51:54.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:51:54.829+0000 7f08f055f700 1 -- 172.21.15.192:0/1498910283 wait complete. 2024-02-16T22:51:55.402 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:55 smithi064 ceph-mon[29226]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:55.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:55.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:55.451 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:55 smithi192 ceph-mon[36254]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:55.451 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:55.451 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:55.725 DEBUG:teuthology.orchestra.run.smithi192:osd.4> sudo journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.4.service 2024-02-16T22:51:55.727 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_2... 2024-02-16T22:51:55.727 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-16T22:51:56.025 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:51:56.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:56 smithi192 ceph-mon[36254]: from='osd.4 [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:51:56.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:56 smithi192 ceph-mon[36254]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:51:56.430 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:56 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:56.430 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:56 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:56.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:56 smithi064 ceph-mon[29226]: from='osd.4 [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:51:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:56 smithi064 ceph-mon[29226]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:51:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:56 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:56 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:51:57.453 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:57 smithi192 ceph-mon[36254]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:57.453 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:57 smithi192 ceph-mon[36254]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T22:51:57.453 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:57 smithi192 ceph-mon[36254]: osdmap e33: 5 total, 4 up, 5 in 2024-02-16T22:51:57.454 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:57 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:57.454 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:57 smithi192 ceph-mon[36254]: from='osd.4 [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:51:57.454 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:57 smithi192 ceph-mon[36254]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:51:57.454 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 22:51:57 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[48675]: 2024-02-16T22:51:57.165+0000 7f5ae45b5700 -1 osd.4 0 waiting for initial osdmap 2024-02-16T22:51:57.454 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 22:51:57 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[48675]: 2024-02-16T22:51:57.171+0000 7f5ade3ae700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:51:57.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:57 smithi064 ceph-mon[29226]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:57.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:57 smithi064 ceph-mon[29226]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T22:51:57.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:57 smithi064 ceph-mon[29226]: osdmap e33: 5 total, 4 up, 5 in 2024-02-16T22:51:57.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:57 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:57.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:57 smithi064 ceph-mon[29226]: from='osd.4 [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:51:57.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:57 smithi064 ceph-mon[29226]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:51:58.476 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:58 smithi192 ceph-mon[36254]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-02-16T22:51:58.476 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:58 smithi192 ceph-mon[36254]: osdmap e34: 5 total, 4 up, 5 in 2024-02-16T22:51:58.476 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:58.476 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:58.476 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:58.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:58 smithi064 ceph-mon[29226]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-02-16T22:51:58.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:58 smithi064 ceph-mon[29226]: osdmap e34: 5 total, 4 up, 5 in 2024-02-16T22:51:58.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:58.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:58.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:51:58.870 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-16T22:51:58.870 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-16T22:51:58.870 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-02-16T22:51:58.870 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-02-16T22:51:58.870 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012366 s, 848 MB/s 2024-02-16T22:51:58.870 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-02-16T22:51:59.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:59 smithi192 ceph-mon[36254]: purged_snaps scrub starts 2024-02-16T22:51:59.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:59 smithi192 ceph-mon[36254]: purged_snaps scrub ok 2024-02-16T22:51:59.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:59 smithi192 ceph-mon[36254]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:59.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:59 smithi192 ceph-mon[36254]: osd.4 [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] boot 2024-02-16T22:51:59.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:59 smithi192 ceph-mon[36254]: osdmap e35: 5 total, 5 up, 5 in 2024-02-16T22:51:59.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:51:59 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:59.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:59 smithi064 ceph-mon[29226]: purged_snaps scrub starts 2024-02-16T22:51:59.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:59 smithi064 ceph-mon[29226]: purged_snaps scrub ok 2024-02-16T22:51:59.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:59 smithi064 ceph-mon[29226]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T22:51:59.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:59 smithi064 ceph-mon[29226]: osd.4 [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] boot 2024-02-16T22:51:59.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:59 smithi064 ceph-mon[29226]: osdmap e35: 5 total, 5 up, 5 in 2024-02-16T22:51:59.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:51:59 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:51:59.623 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-02-16T22:52:00.260 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:52:00.456 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:00 smithi192 ceph-mon[36254]: osdmap e36: 5 total, 5 up, 5 in 2024-02-16T22:52:00.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:00 smithi064 ceph-mon[29226]: osdmap e36: 5 total, 5 up, 5 in 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: Detected new or changed devices on smithi192 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: Adjusting osd_memory_target on smithi192 to 7968M 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.457 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:52:01.458 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:01.458 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:52:01.458 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: Detected new or changed devices on smithi192 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: Adjusting osd_memory_target on smithi192 to 7968M 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:52:01.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:01.665 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.662+0000 7f306c9fa700 1 -- 172.21.15.192:0/529571793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fcef0 msgr2=0x7f30680fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:01.665 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.662+0000 7f306c9fa700 1 --2- 172.21.15.192:0/529571793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fcef0 0x7f30680fd320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3058004970 tx=0x7f3058009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:01.665 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.662+0000 7f306c9fa700 1 -- 172.21.15.192:0/529571793 shutdown_connections 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.662+0000 7f306c9fa700 1 --2- 172.21.15.192:0/529571793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30680fe0e0 0x7f30680fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.662+0000 7f306c9fa700 1 --2- 172.21.15.192:0/529571793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fcef0 0x7f30680fd320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.662+0000 7f306c9fa700 1 -- 172.21.15.192:0/529571793 >> 172.21.15.192:0/529571793 conn(0x7f30680f8620 msgr2=0x7f30680faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.662+0000 7f306c9fa700 1 -- 172.21.15.192:0/529571793 shutdown_connections 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.662+0000 7f306c9fa700 1 -- 172.21.15.192:0/529571793 wait complete. 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f306c9fa700 1 Processor -- start 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f306c9fa700 1 -- start start 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f306c9fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30680fcef0 0x7f3068071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f306c9fa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fe0e0 0x7f30680702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f306c9fa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30680708e0 con 0x7f30680fcef0 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f306c9fa700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068070a50 con 0x7f30680fe0e0 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f3065d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fe0e0 0x7f30680702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f3065d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fe0e0 0x7f30680702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:45704/0 (socket says 172.21.15.192:45704) 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f3065d9b700 1 -- 172.21.15.192:0/3643618182 learned_addr learned my addr 172.21.15.192:0/3643618182 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:52:01.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f306659c700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30680fcef0 0x7f3068071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f3065d9b700 1 -- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30680fcef0 msgr2=0x7f3068071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f3065d9b700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30680fcef0 0x7f3068071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.663+0000 7f3065d9b700 1 -- 172.21.15.192:0/3643618182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3058005040 con 0x7f30680fe0e0 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.664+0000 7f3065d9b700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fe0e0 0x7f30680702e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f305c002a80 tx=0x7f305c0046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.664+0000 7f3056ffd700 1 -- 172.21.15.192:0/3643618182 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f305c007cc0 con 0x7f30680fe0e0 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.664+0000 7f3056ffd700 1 -- 172.21.15.192:0/3643618182 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f305c00f040 con 0x7f30680fe0e0 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.664+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3068070d30 con 0x7f30680fe0e0 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.664+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3068115920 con 0x7f30680fe0e0 2024-02-16T22:52:01.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.664+0000 7f3056ffd700 1 -- 172.21.15.192:0/3643618182 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f305c0096f0 con 0x7f30680fe0e0 2024-02-16T22:52:01.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.665+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306804bed0 con 0x7f30680fe0e0 2024-02-16T22:52:01.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.665+0000 7f3056ffd700 1 -- 172.21.15.192:0/3643618182 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f305c00d690 con 0x7f30680fe0e0 2024-02-16T22:52:01.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.666+0000 7f3056ffd700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30500692a0 0x7f305006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:01.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.666+0000 7f3056ffd700 1 -- 172.21.15.192:0/3643618182 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f305c086460 con 0x7f30680fe0e0 2024-02-16T22:52:01.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.666+0000 7f306659c700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30500692a0 0x7f305006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:01.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.666+0000 7f306659c700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30500692a0 0x7f305006b750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3058004940 tx=0x7f3058009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:01.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.670+0000 7f3056ffd700 1 -- 172.21.15.192:0/3643618182 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f305c04cbd0 con 0x7f30680fe0e0 2024-02-16T22:52:01.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:01.859+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3068102ac0 con 0x7f30500692a0 2024-02-16T22:52:02.525 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:02 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:52:02.525 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:02 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:52:02.525 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:02 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:02.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:02 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:52:02.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:02 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:52:02.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:02 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:03.409 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:03 smithi192 ceph-mon[36254]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:52:03.409 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:03 smithi192 ceph-mon[36254]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:03.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:03 smithi064 ceph-mon[29226]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:52:03.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:03 smithi064 ceph-mon[29226]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:04 smithi192 ceph-mon[36254]: from='client.? 172.21.15.192:0/3790275134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0"}]: dispatch 2024-02-16T22:52:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:04 smithi192 ceph-mon[36254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0"}]: dispatch 2024-02-16T22:52:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:04 smithi192 ceph-mon[36254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0"}]': finished 2024-02-16T22:52:04.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:04 smithi192 ceph-mon[36254]: osdmap e37: 6 total, 5 up, 6 in 2024-02-16T22:52:04.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:04 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:04 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/3790275134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0"}]: dispatch 2024-02-16T22:52:04.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:04 smithi064 ceph-mon[29226]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0"}]: dispatch 2024-02-16T22:52:04.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:04 smithi064 ceph-mon[29226]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0"}]': finished 2024-02-16T22:52:04.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:04 smithi064 ceph-mon[29226]: osdmap e37: 6 total, 5 up, 6 in 2024-02-16T22:52:04.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:05.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:05 smithi192 ceph-mon[36254]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:05.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:05 smithi192 ceph-mon[36254]: from='client.? 172.21.15.192:0/2305855076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:52:05.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:05 smithi064 ceph-mon[29226]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:05.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:05 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/2305855076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:52:07.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:07 smithi192 ceph-mon[36254]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:07.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:07 smithi064 ceph-mon[29226]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:09.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:08 smithi192 ceph-mon[36254]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:09.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:08 smithi064 ceph-mon[29226]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:11 smithi192 ceph-mon[36254]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:11.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:11 smithi064 ceph-mon[29226]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:12 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:52:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:12 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:12.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:12 smithi192 ceph-mon[36254]: Deploying daemon osd.5 on smithi192 2024-02-16T22:52:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:12 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:52:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:12 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:12 smithi064 ceph-mon[29226]: Deploying daemon osd.5 on smithi192 2024-02-16T22:52:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:13 smithi192 ceph-mon[36254]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:52:13.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:13 smithi064 ceph-mon[29226]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:13.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:52:15.462 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:15 smithi192 ceph-mon[36254]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:15.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:15 smithi064 ceph-mon[29226]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:17.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:17 smithi192 ceph-mon[36254]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:17.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:17 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:17.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:17 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:17.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:17 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:52:17.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:17 smithi064 ceph-mon[29226]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:17.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:17 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:17.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:17 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:17 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:52:19.112 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:18 smithi192 ceph-mon[36254]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:19.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:18 smithi064 ceph-mon[29226]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:19.822 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.820+0000 7f3056ffd700 1 -- 172.21.15.192:0/3643618182 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3068102ac0 con 0x7f30500692a0 2024-02-16T22:52:19.822 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-02-16T22:52:19.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30500692a0 msgr2=0x7f305006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30500692a0 0x7f305006b750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3058004940 tx=0x7f3058009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fe0e0 msgr2=0x7f30680702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fe0e0 0x7f30680702e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f305c002a80 tx=0x7f305c0046b0 comp rx=0 tx=0).stop 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 shutdown_connections 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30500692a0 0x7f305006b750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30680fcef0 0x7f3068071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 --2- 172.21.15.192:0/3643618182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30680fe0e0 0x7f30680702e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 >> 172.21.15.192:0/3643618182 conn(0x7f30680f8620 msgr2=0x7f30681013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 shutdown_connections 2024-02-16T22:52:19.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:19.823+0000 7f306c9fa700 1 -- 172.21.15.192:0/3643618182 wait complete. 2024-02-16T22:52:20.434 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.5.service 2024-02-16T22:52:20.436 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-16T22:52:20.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-16T22:52:20.711 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:21.098 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:21.098 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:21.098 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:20 smithi064 ceph-mon[29226]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:21.098 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:20 smithi064 ceph-mon[29226]: from='osd.5 [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:52:21.098 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:20 smithi064 ceph-mon[29226]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:52:21.098 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:21.098 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:20 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:21.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:21.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:21.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:20 smithi192 ceph-mon[36254]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:21.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:20 smithi192 ceph-mon[36254]: from='osd.5 [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:52:21.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:20 smithi192 ceph-mon[36254]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:52:21.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:21.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:20 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:21 smithi192 ceph-mon[36254]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T22:52:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:21 smithi192 ceph-mon[36254]: osdmap e38: 6 total, 5 up, 6 in 2024-02-16T22:52:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:21 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:21 smithi192 ceph-mon[36254]: from='osd.5 [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:52:22.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:21 smithi192 ceph-mon[36254]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:52:22.036 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 22:52:21 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[55165]: 2024-02-16T22:52:21.828+0000 7fe98bf65700 -1 osd.5 0 waiting for initial osdmap 2024-02-16T22:52:22.036 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 22:52:21 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[55165]: 2024-02-16T22:52:21.838+0000 7fe986d60700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:52:22.085 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:21 smithi064 ceph-mon[29226]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T22:52:22.086 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:21 smithi064 ceph-mon[29226]: osdmap e38: 6 total, 5 up, 6 in 2024-02-16T22:52:22.086 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:21 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:22.086 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:21 smithi064 ceph-mon[29226]: from='osd.5 [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:52:22.086 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:21 smithi064 ceph-mon[29226]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 -- 172.21.15.64:0/2817232260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 msgr2=0x7f52740fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 --2- 172.21.15.64:0/2817232260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 0x7f52740fd340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5268004940 tx=0x7f5268009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 -- 172.21.15.64:0/2817232260 shutdown_connections 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 --2- 172.21.15.64:0/2817232260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52740fe1e0 0x7f52740fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 --2- 172.21.15.64:0/2817232260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 0x7f52740fd340 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 -- 172.21.15.64:0/2817232260 >> 172.21.15.64:0/2817232260 conn(0x7f52740f86b0 msgr2=0x7f52740faac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 -- 172.21.15.64:0/2817232260 shutdown_connections 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 -- 172.21.15.64:0/2817232260 wait complete. 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 Processor -- start 2024-02-16T22:52:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.413+0000 7f527ab5c700 1 -- start start 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f527ab5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 0x7f5274071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f527ab5c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52740fe1e0 0x7f52740702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f527ab5c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740708e0 con 0x7f52740fcf30 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f527ab5c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5274070a50 con 0x7f52740fe1e0 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52788f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 0x7f5274071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52788f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 0x7f5274071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51388/0 (socket says 172.21.15.64:51388) 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52788f8700 1 -- 172.21.15.64:0/4159819950 learned_addr learned my addr 172.21.15.64:0/4159819950 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f5273fff700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52740fe1e0 0x7f52740702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52788f8700 1 -- 172.21.15.64:0/4159819950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52740fe1e0 msgr2=0x7f52740702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52788f8700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52740fe1e0 0x7f52740702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52788f8700 1 -- 172.21.15.64:0/4159819950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268005040 con 0x7f52740fcf30 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52788f8700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 0x7f5274071e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f52680093b0 tx=0x7f5268009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52717fa700 1 -- 172.21.15.64:0/4159819950 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f526800e040 con 0x7f52740fcf30 2024-02-16T22:52:22.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52717fa700 1 -- 172.21.15.64:0/4159819950 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f526800bcc0 con 0x7f52740fcf30 2024-02-16T22:52:22.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f52717fa700 1 -- 172.21.15.64:0/4159819950 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f526801c630 con 0x7f52740fcf30 2024-02-16T22:52:22.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5274070cd0 con 0x7f52740fcf30 2024-02-16T22:52:22.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.414+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5274115990 con 0x7f52740fcf30 2024-02-16T22:52:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.415+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527404bed0 con 0x7f52740fcf30 2024-02-16T22:52:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.416+0000 7f52717fa700 1 -- 172.21.15.64:0/4159819950 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5268013980 con 0x7f52740fcf30 2024-02-16T22:52:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.416+0000 7f52717fa700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f52640692a0 0x7f526406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.416+0000 7f52717fa700 1 -- 172.21.15.64:0/4159819950 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f5268088000 con 0x7f52740fcf30 2024-02-16T22:52:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.416+0000 7f5273fff700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f52640692a0 0x7f526406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.417+0000 7f5273fff700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f52640692a0 0x7f526406b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5260004f40 tx=0x7f526000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.421+0000 7f52717fa700 1 -- 172.21.15.64:0/4159819950 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f52680582b0 con 0x7f52740fcf30 2024-02-16T22:52:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.614+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5274105a20 con 0x7f52740fcf30 2024-02-16T22:52:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.615+0000 7f52717fa700 1 -- 172.21.15.64:0/4159819950 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f5274105a20 con 0x7f52740fcf30 2024-02-16T22:52:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.616+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f52640692a0 msgr2=0x7f526406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.616+0000 7f527ab5c700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f52640692a0 0x7f526406b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5260004f40 tx=0x7f526000b040 comp rx=0 tx=0).stop 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.616+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 msgr2=0x7f5274071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.616+0000 7f527ab5c700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 0x7f5274071e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f52680093b0 tx=0x7f5268009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.617+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 shutdown_connections 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.617+0000 7f527ab5c700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f52640692a0 0x7f526406b750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.617+0000 7f527ab5c700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740fcf30 0x7f5274071e60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.617+0000 7f527ab5c700 1 --2- 172.21.15.64:0/4159819950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52740fe1e0 0x7f52740702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.617+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 >> 172.21.15.64:0/4159819950 conn(0x7f52740f86b0 msgr2=0x7f5274101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.617+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 shutdown_connections 2024-02-16T22:52:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:22.617+0000 7f527ab5c700 1 -- 172.21.15.64:0/4159819950 wait complete. 2024-02-16T22:52:22.967 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:22 smithi064 ceph-mon[29226]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-02-16T22:52:22.967 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:22 smithi064 ceph-mon[29226]: osdmap e39: 6 total, 5 up, 6 in 2024-02-16T22:52:22.967 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:22 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:22.967 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:22 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:22.967 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:22 smithi064 ceph-mon[29226]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:22.967 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:22 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/4159819950' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T22:52:23.272 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:22 smithi192 ceph-mon[36254]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-02-16T22:52:23.273 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:22 smithi192 ceph-mon[36254]: osdmap e39: 6 total, 5 up, 6 in 2024-02-16T22:52:23.273 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:22 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:23.273 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:22 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:23.273 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:22 smithi192 ceph-mon[36254]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T22:52:23.273 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:22 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/4159819950' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T22:52:23.311 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1708123918,"num_in_osds":6,"osd_in_since":1708123924,"num_remapped_pgs":0} 2024-02-16T22:52:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:23 smithi064 ceph-mon[29226]: purged_snaps scrub starts 2024-02-16T22:52:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:23 smithi064 ceph-mon[29226]: purged_snaps scrub ok 2024-02-16T22:52:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:23 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:23 smithi064 ceph-mon[29226]: osd.5 [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] boot 2024-02-16T22:52:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:23 smithi064 ceph-mon[29226]: osdmap e40: 6 total, 6 up, 6 in 2024-02-16T22:52:23.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:23 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:23 smithi192 ceph-mon[36254]: purged_snaps scrub starts 2024-02-16T22:52:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:23 smithi192 ceph-mon[36254]: purged_snaps scrub ok 2024-02-16T22:52:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:23 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:23 smithi192 ceph-mon[36254]: osd.5 [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] boot 2024-02-16T22:52:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:23 smithi192 ceph-mon[36254]: osdmap e40: 6 total, 6 up, 6 in 2024-02-16T22:52:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:23 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:52:24.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-16T22:52:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:24.858 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: osdmap e41: 6 total, 6 up, 6 in 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: Detected new or changed devices on smithi192 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: Adjusting osd_memory_target on smithi192 to 5312M 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:24.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:24.860 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:52:24.860 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:25.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: osdmap e41: 6 total, 6 up, 6 in 2024-02-16T22:52:25.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: Detected new or changed devices on smithi192 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: Adjusting osd_memory_target on smithi192 to 5312M 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:52:25.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:52:26.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 -- 172.21.15.64:0/3582470746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fbe50 msgr2=0x7f5fac0fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:26.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 --2- 172.21.15.64:0/3582470746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fbe50 0x7f5fac0fc260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c004940 tx=0x7f5f9c009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:26.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 -- 172.21.15.64:0/3582470746 shutdown_connections 2024-02-16T22:52:26.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 --2- 172.21.15.64:0/3582470746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fac0fcfc0 0x7f5fac0fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 --2- 172.21.15.64:0/3582470746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fbe50 0x7f5fac0fc260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 -- 172.21.15.64:0/3582470746 >> 172.21.15.64:0/3582470746 conn(0x7f5fac0f7510 msgr2=0x7f5fac0f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 -- 172.21.15.64:0/3582470746 shutdown_connections 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 -- 172.21.15.64:0/3582470746 wait complete. 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.222+0000 7f5fb2681700 1 Processor -- start 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fb2681700 1 -- start start 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fb2681700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fac0fbe50 0x7f5fac110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fb2681700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fcfc0 0x7f5fac112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fb2681700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac066cf0 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fb2681700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac066e30 con 0x7f5fac0fbe50 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fcfc0 0x7f5fac112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fcfc0 0x7f5fac112bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51410/0 (socket says 172.21.15.64:51410) 2024-02-16T22:52:26.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fab7fe700 1 -- 172.21.15.64:0/2187845777 learned_addr learned my addr 172.21.15.64:0/2187845777 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fabfff700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fac0fbe50 0x7f5fac110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fab7fe700 1 -- 172.21.15.64:0/2187845777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fac0fbe50 msgr2=0x7f5fac110670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fab7fe700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fac0fbe50 0x7f5fac110670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fab7fe700 1 -- 172.21.15.64:0/2187845777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c005040 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fab7fe700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fcfc0 0x7f5fac112bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0002a50 tx=0x7f5fa00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.223+0000 7f5fa8ff9700 1 -- 172.21.15.64:0/2187845777 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fa0007cc0 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.224+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac1131c0 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.224+0000 7f5fa8ff9700 1 -- 172.21.15.64:0/2187845777 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fa000f040 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.224+0000 7f5fa8ff9700 1 -- 172.21.15.64:0/2187845777 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fa000da30 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.224+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac113700 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.225+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac04bed0 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.225+0000 7f5fa8ff9700 1 -- 172.21.15.64:0/2187845777 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5fa000dbf0 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.225+0000 7f5fa8ff9700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5f940692a0 0x7f5f9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:26.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.225+0000 7f5fa8ff9700 1 -- 172.21.15.64:0/2187845777 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5fa0086c00 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.225+0000 7f5fabfff700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5f940692a0 0x7f5f9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:26.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.226+0000 7f5fabfff700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5f940692a0 0x7f5f9406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c004890 tx=0x7f5f9c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.230+0000 7f5fa8ff9700 1 -- 172.21.15.64:0/2187845777 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5fa008c020 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.423+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5fac104940 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.424+0000 7f5fa8ff9700 1 -- 172.21.15.64:0/2187845777 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f5fa0053260 con 0x7f5fac0fcfc0 2024-02-16T22:52:26.425 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:52:26.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.425+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5f940692a0 msgr2=0x7f5f9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:26.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.425+0000 7f5fb2681700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5f940692a0 0x7f5f9406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c004890 tx=0x7f5f9c009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:26.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.425+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fcfc0 msgr2=0x7f5fac112bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:26.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.425+0000 7f5fb2681700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fcfc0 0x7f5fac112bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0002a50 tx=0x7f5fa00046b0 comp rx=0 tx=0).stop 2024-02-16T22:52:26.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.426+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 shutdown_connections 2024-02-16T22:52:26.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.426+0000 7f5fb2681700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fac0fbe50 0x7f5fac110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:26.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.426+0000 7f5fb2681700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5f940692a0 0x7f5f9406b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:26.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.426+0000 7f5fb2681700 1 --2- 172.21.15.64:0/2187845777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac0fcfc0 0x7f5fac112bc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:26.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.426+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 >> 172.21.15.64:0/2187845777 conn(0x7f5fac0f7510 msgr2=0x7f5fac100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:26.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.426+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 shutdown_connections 2024-02-16T22:52:26.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:26.426+0000 7f5fb2681700 1 -- 172.21.15.64:0/2187845777 wait complete. 2024-02-16T22:52:27.108 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1708123942,"num_in_osds":6,"osd_in_since":1708123924,"num_remapped_pgs":0} 2024-02-16T22:52:27.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T22:52:27.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:27 smithi064 ceph-mon[29226]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:27.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:27 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2187845777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T22:52:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:27.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:27 smithi192 ceph-mon[36254]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:27.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:27 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2187845777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T22:52:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:28 smithi192 ceph-mon[36254]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.048+0000 7f054677d700 1 -- 172.21.15.64:0/3606883469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 msgr2=0x7f05400fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.048+0000 7f054677d700 1 --2- 172.21.15.64:0/3606883469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 0x7f05400fcf60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0534004940 tx=0x7f0534009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.048+0000 7f054677d700 1 -- 172.21.15.64:0/3606883469 shutdown_connections 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.048+0000 7f054677d700 1 --2- 172.21.15.64:0/3606883469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 0x7f05400fcf60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.048+0000 7f054677d700 1 --2- 172.21.15.64:0/3606883469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0540102f50 0x7f0540103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.048+0000 7f054677d700 1 -- 172.21.15.64:0/3606883469 >> 172.21.15.64:0/3606883469 conn(0x7f05400f8ae0 msgr2=0x7f05400faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.048+0000 7f054677d700 1 -- 172.21.15.64:0/3606883469 shutdown_connections 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.048+0000 7f054677d700 1 -- 172.21.15.64:0/3606883469 wait complete. 2024-02-16T22:52:29.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f054677d700 1 Processor -- start 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f054677d700 1 -- start start 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f054677d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0540102f50 0x7f0540113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f054677d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 0x7f0540113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f054677d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0540114620 con 0x7f05401038f0 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f054677d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054010d950 con 0x7f0540102f50 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f053f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 0x7f0540113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f053f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 0x7f0540113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51418/0 (socket says 172.21.15.64:51418) 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.049+0000 7f053f7fe700 1 -- 172.21.15.64:0/815194054 learned_addr learned my addr 172.21.15.64:0/815194054 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f053ffff700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0540102f50 0x7f0540113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f053f7fe700 1 -- 172.21.15.64:0/815194054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0540102f50 msgr2=0x7f0540113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:29.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f053f7fe700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0540102f50 0x7f0540113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:29.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f053f7fe700 1 -- 172.21.15.64:0/815194054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534005040 con 0x7f05401038f0 2024-02-16T22:52:29.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f053f7fe700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 0x7f0540113fd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f05340090d0 tx=0x7f0534009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f053cff9700 1 -- 172.21.15.64:0/815194054 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f053400e040 con 0x7f05401038f0 2024-02-16T22:52:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f053cff9700 1 -- 172.21.15.64:0/815194054 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0534009430 con 0x7f05401038f0 2024-02-16T22:52:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054010dbd0 con 0x7f05401038f0 2024-02-16T22:52:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f053cff9700 1 -- 172.21.15.64:0/815194054 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f053401c3f0 con 0x7f05401038f0 2024-02-16T22:52:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.050+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054010e110 con 0x7f05401038f0 2024-02-16T22:52:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.051+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0540105420 con 0x7f05401038f0 2024-02-16T22:52:29.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.052+0000 7f053cff9700 1 -- 172.21.15.64:0/815194054 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f053400bbf0 con 0x7f05401038f0 2024-02-16T22:52:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.052+0000 7f053cff9700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05280692a0 0x7f052806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.052+0000 7f053ffff700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05280692a0 0x7f052806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.052+0000 7f053cff9700 1 -- 172.21.15.64:0/815194054 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0534087f00 con 0x7f05401038f0 2024-02-16T22:52:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.053+0000 7f053ffff700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05280692a0 0x7f052806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0530004eb0 tx=0x7f053000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:29.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.057+0000 7f053cff9700 1 -- 172.21.15.64:0/815194054 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0534058000 con 0x7f05401038f0 2024-02-16T22:52:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:28 smithi064 ceph-mon[29226]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:52:29.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.248+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f054010a030 con 0x7f05401038f0 2024-02-16T22:52:29.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.249+0000 7f053cff9700 1 -- 172.21.15.64:0/815194054 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11071 (secure 0 0 0) 0x7f0534057b90 con 0x7f05401038f0 2024-02-16T22:52:29.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:52:29.250 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":41,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","created":"2024-02-16T22:46:56.779041+0000","modified":"2024-02-16T22:52:23.833005+0000","last_up_change":"2024-02-16T22:52:22.826789+0000","last_in_change":"2024-02-16T22:52:04.211979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:51:12.402108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7a680d8c-0559-46c6-9bfc-95fca252a657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6803","nonce":3967835180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6805","nonce":3967835180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6809","nonce":3967835180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6807","nonce":3967835180}]},"public_addr":"172.21.15.64:6803/3967835180","cluster_addr":"172.21.15.64:6805/3967835180","heartbeat_back_addr":"172.21.15.64:6809/3967835180","heartbeat_front_addr":"172.21.15.64:6807/3967835180","state":["exists","up"]},{"osd":1,"uuid":"21a8757d-9f77-4f46-83d3-ff904c9e50b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6811","nonce":2618370222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6813","nonce":2618370222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6817","nonce":2618370222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6815","nonce":2618370222}]},"public_addr":"172.21.15.64:6811/2618370222","cluster_addr":"172.21.15.64:6813/2618370222","heartbeat_back_addr":"172.21.15.64:6817/2618370222","heartbeat_front_addr":"172.21.15.64:6815/2618370222","state":["exists","up"]},{"osd":2,"uuid":"3fcb2e12-54a0-46db-af11-d047cf25d50b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6819","nonce":2434652781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6821","nonce":2434652781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6825","nonce":2434652781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6823","nonce":2434652781}]},"public_addr":"172.21.15.64:6819/2434652781","cluster_addr":"172.21.15.64:6821/2434652781","heartbeat_back_addr":"172.21.15.64:6825/2434652781","heartbeat_front_addr":"172.21.15.64:6823/2434652781","state":["exists","up"]},{"osd":3,"uuid":"a7a58c77-96fa-4835-9ea4-d16e6d995620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6801","nonce":1577479253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6803","nonce":1577479253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6807","nonce":1577479253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6805","nonce":1577479253}]},"public_addr":"172.21.15.192:6801/1577479253","cluster_addr":"172.21.15.192:6803/1577479253","heartbeat_back_addr":"172.21.15.192:6807/1577479253","heartbeat_front_addr":"172.21.15.192:6805/1577479253","state":["exists","up"]},{"osd":4,"uuid":"04e094c5-2494-457a-a4a5-7b2742be9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6809","nonce":523969674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6811","nonce":523969674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6815","nonce":523969674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6813","nonce":523969674}]},"public_addr":"172.21.15.192:6809/523969674","cluster_addr":"172.21.15.192:6811/523969674","heartbeat_back_addr":"172.21.15.192:6815/523969674","heartbeat_front_addr":"172.21.15.192:6813/523969674","state":["exists","up"]},{"osd":5,"uuid":"bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6817","nonce":610438408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6819","nonce":610438408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6823","nonce":610438408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6821","nonce":610438408}]},"public_addr":"172.21.15.192:6817/610438408","cluster_addr":"172.21.15.192:6819/610438408","heartbeat_back_addr":"172.21.15.192:6823/610438408","heartbeat_front_addr":"172.21.15.192:6821/610438408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:14.481189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:42.243983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:09.715768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:32.358421+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:56.431975+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:52:21.204162+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1250745016":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/4290875509":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/4194870513":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1222682641":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/3827352044":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6801/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6800/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1538951056":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6800/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/3005970232":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1580822017":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6800/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1804571669":"2024-02-17T22:47:54.261948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:52:29.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05280692a0 msgr2=0x7f052806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:29.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05280692a0 0x7f052806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0530004eb0 tx=0x7f053000b040 comp rx=0 tx=0).stop 2024-02-16T22:52:29.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 msgr2=0x7f0540113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:29.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 0x7f0540113fd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f05340090d0 tx=0x7f0534009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 shutdown_connections 2024-02-16T22:52:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0540102f50 0x7f0540113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05280692a0 0x7f052806b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 --2- 172.21.15.64:0/815194054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05401038f0 0x7f0540113fd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 >> 172.21.15.64:0/815194054 conn(0x7f05400f8ae0 msgr2=0x7f05400fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 shutdown_connections 2024-02-16T22:52:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:29.251+0000 7f054677d700 1 -- 172.21.15.64:0/815194054 wait complete. 2024-02-16T22:52:29.968 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:51:12.402108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-02-16T22:52:29.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-16T22:52:30.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:29 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/815194054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:52:30.227 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:29 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/815194054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:52:31.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:30 smithi064 ceph-mon[29226]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:30 smithi192 ceph-mon[36254]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.747+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2438704880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54102f50 msgr2=0x7f9c54103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.747+0000 7f9c59e5c700 1 --2- 172.21.15.64:0/2438704880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54102f50 0x7f9c54103320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004940 tx=0x7f9c4c009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.747+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2438704880 shutdown_connections 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.747+0000 7f9c59e5c700 1 --2- 172.21.15.64:0/2438704880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c541038f0 0x7f9c540fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.747+0000 7f9c59e5c700 1 --2- 172.21.15.64:0/2438704880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54102f50 0x7f9c54103320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.747+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2438704880 >> 172.21.15.64:0/2438704880 conn(0x7f9c540f8ae0 msgr2=0x7f9c540faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.747+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2438704880 shutdown_connections 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.747+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2438704880 wait complete. 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c59e5c700 1 Processor -- start 2024-02-16T22:52:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c59e5c700 1 -- start start 2024-02-16T22:52:31.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c59e5c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c54102f50 0x7f9c54113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c59e5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c541038f0 0x7f9c54113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c59e5c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54114670 con 0x7f9c541038f0 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c59e5c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5410d910 con 0x7f9c54102f50 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c52ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c541038f0 0x7f9c54113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c52ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c541038f0 0x7f9c54113f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59222/0 (socket says 172.21.15.64:59222) 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c52ffd700 1 -- 172.21.15.64:0/2270680309 learned_addr learned my addr 172.21.15.64:0/2270680309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.748+0000 7f9c537fe700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c54102f50 0x7f9c54113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c52ffd700 1 -- 172.21.15.64:0/2270680309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c54102f50 msgr2=0x7f9c54113a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c52ffd700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c54102f50 0x7f9c54113a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c52ffd700 1 -- 172.21.15.64:0/2270680309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c005040 con 0x7f9c541038f0 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c52ffd700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c541038f0 0x7f9c54113f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b790 tx=0x7f9c3c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c4bfff700 1 -- 172.21.15.64:0/2270680309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c00f6f0 con 0x7f9c541038f0 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5410dbf0 con 0x7f9c541038f0 2024-02-16T22:52:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c4bfff700 1 -- 172.21.15.64:0/2270680309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c3c00fd20 con 0x7f9c541038f0 2024-02-16T22:52:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c4bfff700 1 -- 172.21.15.64:0/2270680309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c0103a0 con 0x7f9c541038f0 2024-02-16T22:52:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.749+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5410e130 con 0x7f9c541038f0 2024-02-16T22:52:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.750+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c540fc6e0 con 0x7f9c541038f0 2024-02-16T22:52:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.750+0000 7f9c4bfff700 1 -- 172.21.15.64:0/2270680309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9c3c011400 con 0x7f9c541038f0 2024-02-16T22:52:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.751+0000 7f9c4bfff700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f9c400692a0 0x7f9c4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:31.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.751+0000 7f9c4bfff700 1 -- 172.21.15.64:0/2270680309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9c3c086c20 con 0x7f9c541038f0 2024-02-16T22:52:31.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.751+0000 7f9c537fe700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f9c400692a0 0x7f9c4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:31.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.751+0000 7f9c537fe700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f9c400692a0 0x7f9c4006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004f90 tx=0x7f9c4c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:31.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.755+0000 7f9c4bfff700 1 -- 172.21.15.64:0/2270680309 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9c3c056d20 con 0x7f9c541038f0 2024-02-16T22:52:31.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.946+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c5410f030 con 0x7f9c541038f0 2024-02-16T22:52:31.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.947+0000 7f9c4bfff700 1 -- 172.21.15.64:0/2270680309 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c3c0568b0 con 0x7f9c541038f0 2024-02-16T22:52:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-02-16T22:52:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f9c400692a0 msgr2=0x7f9c4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f9c400692a0 0x7f9c4006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004f90 tx=0x7f9c4c009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c541038f0 msgr2=0x7f9c54113f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c541038f0 0x7f9c54113f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b790 tx=0x7f9c3c009220 comp rx=0 tx=0).stop 2024-02-16T22:52:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 shutdown_connections 2024-02-16T22:52:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c54102f50 0x7f9c54113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f9c400692a0 0x7f9c4006b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 --2- 172.21.15.64:0/2270680309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c541038f0 0x7f9c54113f90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 >> 172.21.15.64:0/2270680309 conn(0x7f9c540f8ae0 msgr2=0x7f9c540fa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 shutdown_connections 2024-02-16T22:52:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:31.949+0000 7f9c59e5c700 1 -- 172.21.15.64:0/2270680309 wait complete. 2024-02-16T22:52:32.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:32 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2270680309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:52:32.362 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:31 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2270680309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:52:32.555 INFO:tasks.cephadm:Setting up client nodes... 2024-02-16T22:52:32.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-16T22:52:32.814 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:33.242 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:33 smithi064 ceph-mon[29226]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:33.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:33 smithi192 ceph-mon[36254]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.434+0000 7f1b1e1df700 1 -- 172.21.15.64:0/3802120653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fbe50 msgr2=0x7f1b180fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.434+0000 7f1b1e1df700 1 --2- 172.21.15.64:0/3802120653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fbe50 0x7f1b180fc260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1b08004940 tx=0x7f1b08009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.434+0000 7f1b1e1df700 1 -- 172.21.15.64:0/3802120653 shutdown_connections 2024-02-16T22:52:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.434+0000 7f1b1e1df700 1 --2- 172.21.15.64:0/3802120653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b180fcfc0 0x7f1b180fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.434+0000 7f1b1e1df700 1 --2- 172.21.15.64:0/3802120653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fbe50 0x7f1b180fc260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.434+0000 7f1b1e1df700 1 -- 172.21.15.64:0/3802120653 >> 172.21.15.64:0/3802120653 conn(0x7f1b180f7510 msgr2=0x7f1b180f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.434+0000 7f1b1e1df700 1 -- 172.21.15.64:0/3802120653 shutdown_connections 2024-02-16T22:52:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.434+0000 7f1b1e1df700 1 -- 172.21.15.64:0/3802120653 wait complete. 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b1e1df700 1 Processor -- start 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b1e1df700 1 -- start start 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b1e1df700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b180fbe50 0x7f1b1810c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b1e1df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fcfc0 0x7f1b1810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b1e1df700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1810d430 con 0x7f1b180fcfc0 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b1e1df700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1810dc70 con 0x7f1b180fbe50 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b16ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fcfc0 0x7f1b1810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b16ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fcfc0 0x7f1b1810cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59228/0 (socket says 172.21.15.64:59228) 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b16ffd700 1 -- 172.21.15.64:0/1600343202 learned_addr learned my addr 172.21.15.64:0/1600343202 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b177fe700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b180fbe50 0x7f1b1810c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b16ffd700 1 -- 172.21.15.64:0/1600343202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b180fbe50 msgr2=0x7f1b1810c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b16ffd700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b180fbe50 0x7f1b1810c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.435+0000 7f1b16ffd700 1 -- 172.21.15.64:0/1600343202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b08005040 con 0x7f1b180fcfc0 2024-02-16T22:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.436+0000 7f1b16ffd700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fcfc0 0x7f1b1810cda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00daf0 tx=0x7f1b0c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.436+0000 7f1afffff700 1 -- 172.21.15.64:0/1600343202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c00ecc0 con 0x7f1b180fcfc0 2024-02-16T22:52:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.436+0000 7f1afffff700 1 -- 172.21.15.64:0/1600343202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b0c00ee20 con 0x7f1b180fcfc0 2024-02-16T22:52:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.436+0000 7f1afffff700 1 -- 172.21.15.64:0/1600343202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c016940 con 0x7f1b180fcfc0 2024-02-16T22:52:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.436+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1810df50 con 0x7f1b180fcfc0 2024-02-16T22:52:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.436+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1810e490 con 0x7f1b180fcfc0 2024-02-16T22:52:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.437+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1804bed0 con 0x7f1b180fcfc0 2024-02-16T22:52:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.437+0000 7f1afffff700 1 -- 172.21.15.64:0/1600343202 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1b0c014070 con 0x7f1b180fcfc0 2024-02-16T22:52:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.438+0000 7f1afffff700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f1b000692a0 0x7f1b0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.438+0000 7f1b177fe700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f1b000692a0 0x7f1b0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.438+0000 7f1afffff700 1 -- 172.21.15.64:0/1600343202 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1b0c088e50 con 0x7f1b180fcfc0 2024-02-16T22:52:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.438+0000 7f1b177fe700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f1b000692a0 0x7f1b0006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b08004f90 tx=0x7f1b08009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:34.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.442+0000 7f1afffff700 1 -- 172.21.15.64:0/1600343202 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1b0c058ff0 con 0x7f1b180fcfc0 2024-02-16T22:52:34.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.714+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f1b1810e9b0 con 0x7f1b180fcfc0 2024-02-16T22:52:34.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.717+0000 7f1afffff700 1 -- 172.21.15.64:0/1600343202 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f1b0c058b80 con 0x7f1b180fcfc0 2024-02-16T22:52:34.719 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-02-16T22:52:34.719 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAy589lxKCjKhAAeaj7PeUGOQ0od6dXUUSd8w== 2024-02-16T22:52:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f1b000692a0 msgr2=0x7f1b0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f1b000692a0 0x7f1b0006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b08004f90 tx=0x7f1b08009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fcfc0 msgr2=0x7f1b1810cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fcfc0 0x7f1b1810cda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00daf0 tx=0x7f1b0c00b4f0 comp rx=0 tx=0).stop 2024-02-16T22:52:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 shutdown_connections 2024-02-16T22:52:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b180fbe50 0x7f1b1810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f1b000692a0 0x7f1b0006b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 --2- 172.21.15.64:0/1600343202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b180fcfc0 0x7f1b1810cda0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:34.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.719+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 >> 172.21.15.64:0/1600343202 conn(0x7f1b180f7510 msgr2=0x7f1b18100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:34.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.720+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 shutdown_connections 2024-02-16T22:52:34.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:34.720+0000 7f1b1e1df700 1 -- 172.21.15.64:0/1600343202 wait complete. 2024-02-16T22:52:35.337 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:35 smithi064 ceph-mon[29226]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:35.338 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:35 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1600343202' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:52:35.338 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:35 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1600343202' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T22:52:35.361 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:52:35.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-16T22:52:35.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-16T22:52:35.404 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-16T22:52:35.435 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:35 smithi192 ceph-mon[36254]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:35.435 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:35 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1600343202' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:52:35.435 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:35 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1600343202' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T22:52:35.681 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi192/config 2024-02-16T22:52:37.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.175+0000 7f6e1a44d700 1 -- 172.21.15.192:0/664323248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fbe40 msgr2=0x7f6e140fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:37.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.175+0000 7f6e1a44d700 1 --2- 172.21.15.192:0/664323248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fbe40 0x7f6e140fc250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6e04004940 tx=0x7f6e04009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:37.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.175+0000 7f6e1a44d700 1 -- 172.21.15.192:0/664323248 shutdown_connections 2024-02-16T22:52:37.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.175+0000 7f6e1a44d700 1 --2- 172.21.15.192:0/664323248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140fcfb0 0x7f6e140fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:37.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.175+0000 7f6e1a44d700 1 --2- 172.21.15.192:0/664323248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fbe40 0x7f6e140fc250 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:37.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.175+0000 7f6e1a44d700 1 -- 172.21.15.192:0/664323248 >> 172.21.15.192:0/664323248 conn(0x7f6e140f74e0 msgr2=0x7f6e140f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:37.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.175+0000 7f6e1a44d700 1 -- 172.21.15.192:0/664323248 shutdown_connections 2024-02-16T22:52:37.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.175+0000 7f6e1a44d700 1 -- 172.21.15.192:0/664323248 wait complete. 2024-02-16T22:52:37.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:37 smithi192 ceph-mon[36254]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e1a44d700 1 Processor -- start 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e1a44d700 1 -- start start 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e1a44d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140fbe40 0x7f6e1410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e1a44d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fcfb0 0x7f6e1410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e1a44d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410e4a0 con 0x7f6e140fbe40 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e1a44d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410ed40 con 0x7f6e140fcfb0 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e137fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fcfb0 0x7f6e1410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e137fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fcfb0 0x7f6e1410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:35346/0 (socket says 172.21.15.192:35346) 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e137fe700 1 -- 172.21.15.192:0/720438500 learned_addr learned my addr 172.21.15.192:0/720438500 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.176+0000 7f6e13fff700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140fbe40 0x7f6e1410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:37.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e137fe700 1 -- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140fbe40 msgr2=0x7f6e1410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e137fe700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140fbe40 0x7f6e1410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e137fe700 1 -- 172.21.15.192:0/720438500 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04005040 con 0x7f6e140fcfb0 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e137fe700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fcfb0 0x7f6e1410dea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e08002a50 tx=0x7f6e080046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e10ff9700 1 -- 172.21.15.192:0/720438500 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e08007cc0 con 0x7f6e140fcfb0 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e10ff9700 1 -- 172.21.15.192:0/720438500 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e0800f040 con 0x7f6e140fcfb0 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1410f020 con 0x7f6e140fcfb0 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e10ff9700 1 -- 172.21.15.192:0/720438500 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0800da30 con 0x7f6e140fcfb0 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.177+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1410f560 con 0x7f6e140fcfb0 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.178+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1404bed0 con 0x7f6e140fcfb0 2024-02-16T22:52:37.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.179+0000 7f6e10ff9700 1 -- 172.21.15.192:0/720438500 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6e0800dbb0 con 0x7f6e140fcfb0 2024-02-16T22:52:37.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.179+0000 7f6e10ff9700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6dfc0692a0 0x7f6dfc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:37.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.179+0000 7f6e10ff9700 1 -- 172.21.15.192:0/720438500 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6e08086b00 con 0x7f6e140fcfb0 2024-02-16T22:52:37.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.179+0000 7f6e13fff700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6dfc0692a0 0x7f6dfc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:37.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.180+0000 7f6e13fff700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6dfc0692a0 0x7f6dfc06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400b3b0 tx=0x7f6e04009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:37.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.183+0000 7f6e10ff9700 1 -- 172.21.15.192:0/720438500 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6e08056c00 con 0x7f6e140fcfb0 2024-02-16T22:52:37.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.435+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f6e14104930 con 0x7f6e140fcfb0 2024-02-16T22:52:37.441 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.440+0000 7f6e10ff9700 1 -- 172.21.15.192:0/720438500 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f6e08056790 con 0x7f6e140fcfb0 2024-02-16T22:52:37.441 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-02-16T22:52:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQA1589lN64RGhAAlYYv1hLLDNR58I3aan0qRg== 2024-02-16T22:52:37.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6dfc0692a0 msgr2=0x7f6dfc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:37.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6dfc0692a0 0x7f6dfc06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400b3b0 tx=0x7f6e04009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:37.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fcfb0 msgr2=0x7f6e1410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:37.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fcfb0 0x7f6e1410dea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e08002a50 tx=0x7f6e080046b0 comp rx=0 tx=0).stop 2024-02-16T22:52:37.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 shutdown_connections 2024-02-16T22:52:37.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6dfc0692a0 0x7f6dfc06b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:37.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140fbe40 0x7f6e1410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:37.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 --2- 172.21.15.192:0/720438500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e140fcfb0 0x7f6e1410dea0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:37.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 >> 172.21.15.192:0/720438500 conn(0x7f6e140f74e0 msgr2=0x7f6e140656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:37.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 shutdown_connections 2024-02-16T22:52:37.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:52:37.442+0000 7f6e1a44d700 1 -- 172.21.15.192:0/720438500 wait complete. 2024-02-16T22:52:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:37 smithi064 ceph-mon[29226]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:38.050 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:52:38.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-16T22:52:38.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-16T22:52:38.135 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-16T22:52:38.135 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-16T22:52:38.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-16T22:52:38.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:38 smithi192 ceph-mon[36254]: from='client.? 172.21.15.192:0/720438500' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:52:38.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:38 smithi192 ceph-mon[36254]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:52:38.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:38 smithi192 ceph-mon[36254]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T22:52:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:38.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:38 smithi064 ceph-mon[29226]: from='client.? 172.21.15.192:0/720438500' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:52:38.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:38 smithi064 ceph-mon[29226]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:52:38.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:38 smithi064 ceph-mon[29226]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T22:52:39.494 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:39 smithi064 ceph-mon[29226]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:39 smithi192 ceph-mon[36254]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:39.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 -- 172.21.15.64:0/2544834304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fe960 msgr2=0x7fc2b0106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 --2- 172.21.15.64:0/2544834304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fe960 0x7fc2b0106fa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc298004940 tx=0x7fc298009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 -- 172.21.15.64:0/2544834304 shutdown_connections 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 --2- 172.21.15.64:0/2544834304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fe960 0x7fc2b0106fa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 --2- 172.21.15.64:0/2544834304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2b00fdfc0 0x7fc2b00fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 -- 172.21.15.64:0/2544834304 >> 172.21.15.64:0/2544834304 conn(0x7fc2b00f7a10 msgr2=0x7fc2b00f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 -- 172.21.15.64:0/2544834304 shutdown_connections 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 -- 172.21.15.64:0/2544834304 wait complete. 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.970+0000 7fc2b671d700 1 Processor -- start 2024-02-16T22:52:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2b671d700 1 -- start start 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2b671d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fdfc0 0x7fc2b0113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2b671d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2b00fe960 0x7fc2b0114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2b671d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b0114690 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2b671d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b010d9c0 con 0x7fc2b00fe960 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2affff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fdfc0 0x7fc2b0113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2affff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fdfc0 0x7fc2b0113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59236/0 (socket says 172.21.15.64:59236) 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2affff700 1 -- 172.21.15.64:0/3593301082 learned_addr learned my addr 172.21.15.64:0/3593301082 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2af7fe700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2b00fe960 0x7fc2b0114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2affff700 1 -- 172.21.15.64:0/3593301082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2b00fe960 msgr2=0x7fc2b0114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2affff700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2b00fe960 0x7fc2b0114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2affff700 1 -- 172.21.15.64:0/3593301082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc298005040 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2af7fe700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2b00fe960 0x7fc2b0114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:52:39.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2affff700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fdfc0 0x7fc2b0113b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0002a50 tx=0x7fc2a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:39.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.971+0000 7fc2acff9700 1 -- 172.21.15.64:0/3593301082 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a0007cc0 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.972+0000 7fc2acff9700 1 -- 172.21.15.64:0/3593301082 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2a000f040 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.972+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b010dca0 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.972+0000 7fc2acff9700 1 -- 172.21.15.64:0/3593301082 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a0009640 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.972+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b010e1e0 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.972+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b01045b0 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.973+0000 7fc2acff9700 1 -- 172.21.15.64:0/3593301082 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc2a00097a0 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.973+0000 7fc2acff9700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc29c0692a0 0x7fc29c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:39.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.973+0000 7fc2acff9700 1 -- 172.21.15.64:0/3593301082 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc2a0086c80 con 0x7fc2b00fdfc0 2024-02-16T22:52:39.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.973+0000 7fc2af7fe700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc29c0692a0 0x7fc29c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:39.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.974+0000 7fc2af7fe700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc29c0692a0 0x7fc29c06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2b010f330 tx=0x7fc298009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:39.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:39.978+0000 7fc2acff9700 1 -- 172.21.15.64:0/3593301082 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc2a0056d00 con 0x7fc2b00fdfc0 2024-02-16T22:52:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.226+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc2b004bed0 con 0x7fc2b00fdfc0 2024-02-16T22:52:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.229+0000 7fc2acff9700 1 -- 172.21.15.64:0/3593301082 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167167 (secure 0 0 0) 0x7fc2a0056890 con 0x7fc2b00fdfc0 2024-02-16T22:52:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:52:40.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc29c0692a0 msgr2=0x7fc29c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:40.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc29c0692a0 0x7fc29c06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2b010f330 tx=0x7fc298009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fdfc0 msgr2=0x7fc2b0113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fdfc0 0x7fc2b0113b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0002a50 tx=0x7fc2a00046b0 comp rx=0 tx=0).stop 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 shutdown_connections 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc29c0692a0 0x7fc29c06b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b00fdfc0 0x7fc2b0113b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 --2- 172.21.15.64:0/3593301082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2b00fe960 0x7fc2b0114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.231+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 >> 172.21.15.64:0/3593301082 conn(0x7fc2b00f7a10 msgr2=0x7fc2b0101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.232+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 shutdown_connections 2024-02-16T22:52:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:40.232+0000 7fc2b671d700 1 -- 172.21.15.64:0/3593301082 wait complete. 2024-02-16T22:52:40.901 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"active_gid":14208,"active_name":"smithi064.kubmlk","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":700343021},{"type":"v1","addr":"172.21.15.64:6801","nonce":700343021}]},"active_addr":"172.21.15.64:6801/700343021","active_change":"2024-02-16T22:48:58.067162+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14232,"name":"smithi192.pauvih","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.64:8443/","prometheus":"http://172.21.15.64:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":743141814}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":4258467456}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2780081097}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1046482636}]}]} 2024-02-16T22:52:40.905 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-16T22:52:40.905 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-16T22:52:40.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T22:52:41.163 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:41.422 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:41 smithi064 ceph-mon[29226]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:41.422 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:41 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3593301082' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-16T22:52:41.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:41 smithi192 ceph-mon[36254]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:41.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:41 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3593301082' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-16T22:52:42.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 -- 172.21.15.64:0/2420333260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc101e40 msgr2=0x7f03fc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:42.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 --2- 172.21.15.64:0/2420333260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc101e40 0x7f03fc102210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f03ec004940 tx=0x7f03ec009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:42.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 -- 172.21.15.64:0/2420333260 shutdown_connections 2024-02-16T22:52:42.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 --2- 172.21.15.64:0/2420333260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03fc1027e0 0x7f03fc0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:42.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 --2- 172.21.15.64:0/2420333260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc101e40 0x7f03fc102210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 -- 172.21.15.64:0/2420333260 >> 172.21.15.64:0/2420333260 conn(0x7f03fc071d90 msgr2=0x7f03fc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 -- 172.21.15.64:0/2420333260 shutdown_connections 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 -- 172.21.15.64:0/2420333260 wait complete. 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.836+0000 7f0402149700 1 Processor -- start 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f0402149700 1 -- start start 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f0402149700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03fc101e40 0x7f03fc102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f0402149700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc1027e0 0x7f03fc1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f0402149700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1037e0 con 0x7f03fc1027e0 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f0402149700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc103950 con 0x7f03fc101e40 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f03faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc1027e0 0x7f03fc1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f03faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc1027e0 0x7f03fc1031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49140/0 (socket says 172.21.15.64:49140) 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f03faffd700 1 -- 172.21.15.64:0/398577057 learned_addr learned my addr 172.21.15.64:0/398577057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:42.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f03fb7fe700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03fc101e40 0x7f03fc102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f03faffd700 1 -- 172.21.15.64:0/398577057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03fc101e40 msgr2=0x7f03fc102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f03faffd700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03fc101e40 0x7f03fc102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f03faffd700 1 -- 172.21.15.64:0/398577057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec005040 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.837+0000 7f03faffd700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc1027e0 0x7f03fc1031e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f03f000c800 tx=0x7f03f0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.838+0000 7f03e3fff700 1 -- 172.21.15.64:0/398577057 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03f00106f0 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.838+0000 7f03e3fff700 1 -- 172.21.15.64:0/398577057 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03f0010d20 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.838+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc119c90 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.838+0000 7f03e3fff700 1 -- 172.21.15.64:0/398577057 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03f00113a0 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.838+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc11a170 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.839+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc0738b0 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.839+0000 7f03e3fff700 1 -- 172.21.15.64:0/398577057 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f03f0010850 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.839+0000 7f03e3fff700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f03e40692a0 0x7f03e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.839+0000 7f03e3fff700 1 -- 172.21.15.64:0/398577057 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f03f0086e30 con 0x7f03fc1027e0 2024-02-16T22:52:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.839+0000 7f03fb7fe700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f03e40692a0 0x7f03e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:42.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.840+0000 7f03fb7fe700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f03e40692a0 0x7f03e406b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03ec004f90 tx=0x7f03ec009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:42.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:42.844+0000 7f03e3fff700 1 -- 172.21.15.64:0/398577057 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f03f00534a0 con 0x7f03fc1027e0 2024-02-16T22:52:43.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.033+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03fc108f40 con 0x7f03fc1027e0 2024-02-16T22:52:43.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.034+0000 7f03e3fff700 1 -- 172.21.15.64:0/398577057 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11071 (secure 0 0 0) 0x7f03f0056ac0 con 0x7f03fc1027e0 2024-02-16T22:52:43.036 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:52:43.036 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":41,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","created":"2024-02-16T22:46:56.779041+0000","modified":"2024-02-16T22:52:23.833005+0000","last_up_change":"2024-02-16T22:52:22.826789+0000","last_in_change":"2024-02-16T22:52:04.211979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:51:12.402108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7a680d8c-0559-46c6-9bfc-95fca252a657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6803","nonce":3967835180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6805","nonce":3967835180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6809","nonce":3967835180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6807","nonce":3967835180}]},"public_addr":"172.21.15.64:6803/3967835180","cluster_addr":"172.21.15.64:6805/3967835180","heartbeat_back_addr":"172.21.15.64:6809/3967835180","heartbeat_front_addr":"172.21.15.64:6807/3967835180","state":["exists","up"]},{"osd":1,"uuid":"21a8757d-9f77-4f46-83d3-ff904c9e50b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6811","nonce":2618370222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6813","nonce":2618370222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6817","nonce":2618370222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6815","nonce":2618370222}]},"public_addr":"172.21.15.64:6811/2618370222","cluster_addr":"172.21.15.64:6813/2618370222","heartbeat_back_addr":"172.21.15.64:6817/2618370222","heartbeat_front_addr":"172.21.15.64:6815/2618370222","state":["exists","up"]},{"osd":2,"uuid":"3fcb2e12-54a0-46db-af11-d047cf25d50b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6819","nonce":2434652781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6821","nonce":2434652781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6825","nonce":2434652781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6823","nonce":2434652781}]},"public_addr":"172.21.15.64:6819/2434652781","cluster_addr":"172.21.15.64:6821/2434652781","heartbeat_back_addr":"172.21.15.64:6825/2434652781","heartbeat_front_addr":"172.21.15.64:6823/2434652781","state":["exists","up"]},{"osd":3,"uuid":"a7a58c77-96fa-4835-9ea4-d16e6d995620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6801","nonce":1577479253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6803","nonce":1577479253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6807","nonce":1577479253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6805","nonce":1577479253}]},"public_addr":"172.21.15.192:6801/1577479253","cluster_addr":"172.21.15.192:6803/1577479253","heartbeat_back_addr":"172.21.15.192:6807/1577479253","heartbeat_front_addr":"172.21.15.192:6805/1577479253","state":["exists","up"]},{"osd":4,"uuid":"04e094c5-2494-457a-a4a5-7b2742be9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6809","nonce":523969674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6811","nonce":523969674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6815","nonce":523969674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6813","nonce":523969674}]},"public_addr":"172.21.15.192:6809/523969674","cluster_addr":"172.21.15.192:6811/523969674","heartbeat_back_addr":"172.21.15.192:6815/523969674","heartbeat_front_addr":"172.21.15.192:6813/523969674","state":["exists","up"]},{"osd":5,"uuid":"bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6817","nonce":610438408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6819","nonce":610438408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6823","nonce":610438408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6821","nonce":610438408}]},"public_addr":"172.21.15.192:6817/610438408","cluster_addr":"172.21.15.192:6819/610438408","heartbeat_back_addr":"172.21.15.192:6823/610438408","heartbeat_front_addr":"172.21.15.192:6821/610438408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:14.481189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:42.243983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:09.715768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:32.358421+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:56.431975+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:52:21.204162+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1250745016":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/4290875509":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/4194870513":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1222682641":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/3827352044":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6801/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6800/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1538951056":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6800/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/3005970232":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1580822017":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6800/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1804571669":"2024-02-17T22:47:54.261948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:52:43.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f03e40692a0 msgr2=0x7f03e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:43.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f03e40692a0 0x7f03e406b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03ec004f90 tx=0x7f03ec009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:43.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc1027e0 msgr2=0x7f03fc1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:43.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc1027e0 0x7f03fc1031e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f03f000c800 tx=0x7f03f0004470 comp rx=0 tx=0).stop 2024-02-16T22:52:43.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 shutdown_connections 2024-02-16T22:52:43.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03fc101e40 0x7f03fc102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:43.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f03e40692a0 0x7f03e406b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:43.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 --2- 172.21.15.64:0/398577057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03fc1027e0 0x7f03fc1031e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:43.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 >> 172.21.15.64:0/398577057 conn(0x7f03fc071d90 msgr2=0x7f03fc065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:43.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 shutdown_connections 2024-02-16T22:52:43.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:43.037+0000 7f0402149700 1 -- 172.21.15.64:0/398577057 wait complete. 2024-02-16T22:52:43.340 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:43 smithi064 ceph-mon[29226]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:43.340 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:43 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/398577057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:52:43.340 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:43 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:52:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:43 smithi192 ceph-mon[36254]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:43 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/398577057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:52:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:43 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:52:43.725 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-16T22:52:43.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T22:52:43.983 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:45.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:45 smithi064 ceph-mon[29226]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:45.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:45 smithi192 ceph-mon[36254]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 -- 172.21.15.64:0/1715087928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 msgr2=0x7f2f6c102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 --2- 172.21.15.64:0/1715087928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 0x7f2f6c102230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009ce0 tx=0x7f2f64004d10 comp rx=0 tx=0).stop 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 -- 172.21.15.64:0/1715087928 shutdown_connections 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 --2- 172.21.15.64:0/1715087928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f6c102800 0x7f2f6c0fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 --2- 172.21.15.64:0/1715087928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 0x7f2f6c102230 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 -- 172.21.15.64:0/1715087928 >> 172.21.15.64:0/1715087928 conn(0x7f2f6c0f7a10 msgr2=0x7f2f6c0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 -- 172.21.15.64:0/1715087928 shutdown_connections 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 -- 172.21.15.64:0/1715087928 wait complete. 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 Processor -- start 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.593+0000 7f2f71590700 1 -- start start 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f71590700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 0x7f2f6c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f71590700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f6c102800 0x7f2f6c103610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f71590700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c103ca0 con 0x7f2f6c101e60 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f71590700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c103e10 con 0x7f2f6c102800 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f6affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 0x7f2f6c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f6affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 0x7f2f6c1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49152/0 (socket says 172.21.15.64:49152) 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f6affd700 1 -- 172.21.15.64:0/1827030651 learned_addr learned my addr 172.21.15.64:0/1827030651 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f6a7fc700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f6c102800 0x7f2f6c103610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f6affd700 1 -- 172.21.15.64:0/1827030651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f6c102800 msgr2=0x7f2f6c103610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f6affd700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f6c102800 0x7f2f6c103610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f6affd700 1 -- 172.21.15.64:0/1827030651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64009940 con 0x7f2f6c101e60 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f6affd700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 0x7f2f6c1030d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400bf60 tx=0x7f2f64004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f637fe700 1 -- 172.21.15.64:0/1827030651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6400e040 con 0x7f2f6c101e60 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f637fe700 1 -- 172.21.15.64:0/1827030651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f64008ea0 con 0x7f2f6c101e60 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c119e70 con 0x7f2f6c101e60 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f637fe700 1 -- 172.21.15.64:0/1827030651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6401c7e0 con 0x7f2f6c101e60 2024-02-16T22:52:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.594+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c11a330 con 0x7f2f6c101e60 2024-02-16T22:52:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.595+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c0fb610 con 0x7f2f6c101e60 2024-02-16T22:52:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.596+0000 7f2f637fe700 1 -- 172.21.15.64:0/1827030651 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2f64012410 con 0x7f2f6c101e60 2024-02-16T22:52:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.596+0000 7f2f637fe700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2f580692a0 0x7f2f5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.596+0000 7f2f637fe700 1 -- 172.21.15.64:0/1827030651 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2f64088a80 con 0x7f2f6c101e60 2024-02-16T22:52:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.596+0000 7f2f6a7fc700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2f580692a0 0x7f2f5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.597+0000 7f2f6a7fc700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2f580692a0 0x7f2f5806b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c104ad0 tx=0x7f2f540091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:45.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.601+0000 7f2f637fe700 1 -- 172.21.15.64:0/1827030651 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2f64055070 con 0x7f2f6c101e60 2024-02-16T22:52:45.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.793+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f6c104810 con 0x7f2f6c101e60 2024-02-16T22:52:45.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.793+0000 7f2f637fe700 1 -- 172.21.15.64:0/1827030651 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11071 (secure 0 0 0) 0x7f2f64004e10 con 0x7f2f6c101e60 2024-02-16T22:52:45.795 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:52:45.795 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":41,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","created":"2024-02-16T22:46:56.779041+0000","modified":"2024-02-16T22:52:23.833005+0000","last_up_change":"2024-02-16T22:52:22.826789+0000","last_in_change":"2024-02-16T22:52:04.211979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:51:12.402108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7a680d8c-0559-46c6-9bfc-95fca252a657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6803","nonce":3967835180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6805","nonce":3967835180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6809","nonce":3967835180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6807","nonce":3967835180}]},"public_addr":"172.21.15.64:6803/3967835180","cluster_addr":"172.21.15.64:6805/3967835180","heartbeat_back_addr":"172.21.15.64:6809/3967835180","heartbeat_front_addr":"172.21.15.64:6807/3967835180","state":["exists","up"]},{"osd":1,"uuid":"21a8757d-9f77-4f46-83d3-ff904c9e50b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6811","nonce":2618370222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6813","nonce":2618370222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6817","nonce":2618370222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6815","nonce":2618370222}]},"public_addr":"172.21.15.64:6811/2618370222","cluster_addr":"172.21.15.64:6813/2618370222","heartbeat_back_addr":"172.21.15.64:6817/2618370222","heartbeat_front_addr":"172.21.15.64:6815/2618370222","state":["exists","up"]},{"osd":2,"uuid":"3fcb2e12-54a0-46db-af11-d047cf25d50b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6819","nonce":2434652781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6821","nonce":2434652781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6825","nonce":2434652781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6823","nonce":2434652781}]},"public_addr":"172.21.15.64:6819/2434652781","cluster_addr":"172.21.15.64:6821/2434652781","heartbeat_back_addr":"172.21.15.64:6825/2434652781","heartbeat_front_addr":"172.21.15.64:6823/2434652781","state":["exists","up"]},{"osd":3,"uuid":"a7a58c77-96fa-4835-9ea4-d16e6d995620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6801","nonce":1577479253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6803","nonce":1577479253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6807","nonce":1577479253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6805","nonce":1577479253}]},"public_addr":"172.21.15.192:6801/1577479253","cluster_addr":"172.21.15.192:6803/1577479253","heartbeat_back_addr":"172.21.15.192:6807/1577479253","heartbeat_front_addr":"172.21.15.192:6805/1577479253","state":["exists","up"]},{"osd":4,"uuid":"04e094c5-2494-457a-a4a5-7b2742be9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6809","nonce":523969674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6811","nonce":523969674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6815","nonce":523969674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6813","nonce":523969674}]},"public_addr":"172.21.15.192:6809/523969674","cluster_addr":"172.21.15.192:6811/523969674","heartbeat_back_addr":"172.21.15.192:6815/523969674","heartbeat_front_addr":"172.21.15.192:6813/523969674","state":["exists","up"]},{"osd":5,"uuid":"bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6817","nonce":610438408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6819","nonce":610438408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6823","nonce":610438408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6821","nonce":610438408}]},"public_addr":"172.21.15.192:6817/610438408","cluster_addr":"172.21.15.192:6819/610438408","heartbeat_back_addr":"172.21.15.192:6823/610438408","heartbeat_front_addr":"172.21.15.192:6821/610438408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:14.481189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:42.243983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:09.715768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:32.358421+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:56.431975+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:52:21.204162+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1250745016":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/4290875509":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/4194870513":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1222682641":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/3827352044":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6801/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6800/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1538951056":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6800/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/3005970232":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1580822017":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6800/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1804571669":"2024-02-17T22:47:54.261948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:52:45.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2f580692a0 msgr2=0x7f2f5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:45.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2f580692a0 0x7f2f5806b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c104ad0 tx=0x7f2f540091a0 comp rx=0 tx=0).stop 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 msgr2=0x7f2f6c1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 0x7f2f6c1030d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400bf60 tx=0x7f2f64004d10 comp rx=0 tx=0).stop 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 shutdown_connections 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f2f580692a0 0x7f2f5806b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c101e60 0x7f2f6c1030d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 --2- 172.21.15.64:0/1827030651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f6c102800 0x7f2f6c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 >> 172.21.15.64:0/1827030651 conn(0x7f2f6c0f7a10 msgr2=0x7f2f6c105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 shutdown_connections 2024-02-16T22:52:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:45.796+0000 7f2f71590700 1 -- 172.21.15.64:0/1827030651 wait complete. 2024-02-16T22:52:46.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:46 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1827030651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:52:46.491 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-16T22:52:46.491 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-16T22:52:46.491 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-16T22:52:46.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-16T22:52:46.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-16T22:52:46.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-16T22:52:46.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:46 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1827030651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:52:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:46.963 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:46.963 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:46.964 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:46.986 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:47.351 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:47 smithi064 ceph-mon[29226]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:47.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:47 smithi192 ceph-mon[36254]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:49.037 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:48 smithi064 ceph-mon[29226]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:49.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:48 smithi192 ceph-mon[36254]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:51 smithi064 ceph-mon[29226]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:51 smithi192 ceph-mon[36254]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.072+0000 7f78e5852700 1 -- 172.21.15.64:0/702149641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0102880 msgr2=0x7f78e00fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.072+0000 7f78e5852700 1 --2- 172.21.15.64:0/702149641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0102880 0x7f78e00fc680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f78c8004940 tx=0x7f78c8009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.072+0000 7f78e5852700 1 -- 172.21.15.64:0/702149641 shutdown_connections 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.072+0000 7f78e5852700 1 --2- 172.21.15.64:0/702149641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0102880 0x7f78e00fc680 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.072+0000 7f78e5852700 1 --2- 172.21.15.64:0/702149641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78e0101ee0 0x7f78e01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.072+0000 7f78e5852700 1 -- 172.21.15.64:0/702149641 >> 172.21.15.64:0/702149641 conn(0x7f78e0071560 msgr2=0x7f78e0071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.072+0000 7f78e5852700 1 -- 172.21.15.64:0/702149641 shutdown_connections 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.073+0000 7f78e5852700 1 -- 172.21.15.64:0/702149641 wait complete. 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.073+0000 7f78e5852700 1 Processor -- start 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.073+0000 7f78e5852700 1 -- start start 2024-02-16T22:52:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.073+0000 7f78e5852700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0101ee0 0x7f78e0112960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.073+0000 7f78e5852700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78e0102880 0x7f78e0112ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.073+0000 7f78e5852700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e0113580 con 0x7f78e0101ee0 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.073+0000 7f78e5852700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e010c820 con 0x7f78e0102880 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78deffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0101ee0 0x7f78e0112960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78deffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0101ee0 0x7f78e0112960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35914/0 (socket says 172.21.15.64:35914) 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78deffd700 1 -- 172.21.15.64:0/1919985086 learned_addr learned my addr 172.21.15.64:0/1919985086 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78de7fc700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78e0102880 0x7f78e0112ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78deffd700 1 -- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78e0102880 msgr2=0x7f78e0112ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78deffd700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78e0102880 0x7f78e0112ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78deffd700 1 -- 172.21.15.64:0/1919985086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8005040 con 0x7f78e0101ee0 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78deffd700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0101ee0 0x7f78e0112960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78d800bd20 tx=0x7f78d800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78d800b2e0 con 0x7f78e0101ee0 2024-02-16T22:52:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78d800b440 con 0x7f78e0101ee0 2024-02-16T22:52:52.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e010caa0 con 0x7f78e0101ee0 2024-02-16T22:52:52.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78d8017470 con 0x7f78e0101ee0 2024-02-16T22:52:52.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.074+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e010cfb0 con 0x7f78e0101ee0 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.075+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78e010d5b0 con 0x7f78e0101ee0 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.076+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f78d8014070 con 0x7f78e0101ee0 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.076+0000 7f78d77fe700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f78cc0692a0 0x7f78cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.076+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f78d8088b10 con 0x7f78e0101ee0 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.076+0000 7f78de7fc700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f78cc0692a0 0x7f78cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.076+0000 7f78d77fe700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] conn(0x7f78cc06ef70 0x7f78cc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.076+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 --> [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78cc071a30 con 0x7f78cc06ef70 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.076+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78d808d220 con 0x7f78e0101ee0 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.077+0000 7f78de7fc700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f78cc0692a0 0x7f78cc06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f78c8004890 tx=0x7f78c8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.077+0000 7f78df7fe700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] conn(0x7f78cc06ef70 0x7f78cc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.077+0000 7f78df7fe700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] conn(0x7f78cc06ef70 0x7f78cc071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:52.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.078+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 <== osd.5 v2:172.21.15.192:6816/610438408 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f78cc071a30 con 0x7f78cc06ef70 2024-02-16T22:52:52.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 --> [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78e010cc30 con 0x7f78cc06ef70 2024-02-16T22:52:52.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78d77fe700 1 -- 172.21.15.64:0/1919985086 <== osd.5 v2:172.21.15.192:6816/610438408 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f78e010cc30 con 0x7f78cc06ef70 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] conn(0x7f78cc06ef70 msgr2=0x7f78cc071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] conn(0x7f78cc06ef70 0x7f78cc071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f78cc0692a0 msgr2=0x7f78cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f78cc0692a0 0x7f78cc06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f78c8004890 tx=0x7f78c8009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0101ee0 msgr2=0x7f78e0112960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0101ee0 0x7f78e0112960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78d800bd20 tx=0x7f78d800ba50 comp rx=0 tx=0).stop 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 shutdown_connections 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:6816/610438408,v1:172.21.15.192:6817/610438408] conn(0x7f78cc06ef70 0x7f78cc071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:52.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f78cc0692a0 0x7f78cc06b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:52.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e0101ee0 0x7f78e0112960 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:52.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 --2- 172.21.15.64:0/1919985086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78e0102880 0x7f78e0112ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:52.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.093+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 >> 172.21.15.64:0/1919985086 conn(0x7f78e0071560 msgr2=0x7f78e00fa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:52.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.094+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 shutdown_connections 2024-02-16T22:52:52.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:52.094+0000 7f78e5852700 1 -- 172.21.15.64:0/1919985086 wait complete. 2024-02-16T22:52:52.685 INFO:teuthology.orchestra.run.smithi064.stdout:171798691849 2024-02-16T22:52:52.685 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-16T22:52:53.239 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:53 smithi064 ceph-mon[29226]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.028+0000 7f4624391700 1 -- 172.21.15.64:0/2528890968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c101e50 msgr2=0x7f461c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.028+0000 7f4624391700 1 --2- 172.21.15.64:0/2528890968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c101e50 0x7f461c102220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4618004940 tx=0x7f4618009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.028+0000 7f4624391700 1 -- 172.21.15.64:0/2528890968 shutdown_connections 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.028+0000 7f4624391700 1 --2- 172.21.15.64:0/2528890968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c1027f0 0x7f461c0fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.028+0000 7f4624391700 1 --2- 172.21.15.64:0/2528890968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c101e50 0x7f461c102220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.028+0000 7f4624391700 1 -- 172.21.15.64:0/2528890968 >> 172.21.15.64:0/2528890968 conn(0x7f461c0f79e0 msgr2=0x7f461c0f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.028+0000 7f4624391700 1 -- 172.21.15.64:0/2528890968 shutdown_connections 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.028+0000 7f4624391700 1 -- 172.21.15.64:0/2528890968 wait complete. 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f4624391700 1 Processor -- start 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f4624391700 1 -- start start 2024-02-16T22:52:53.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f4624391700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c101e50 0x7f461c113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f4624391700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c1027f0 0x7f461c113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f4624391700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c1145f0 con 0x7f461c1027f0 2024-02-16T22:52:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f4624391700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c10d920 con 0x7f461c101e50 2024-02-16T22:52:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f462192c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c1027f0 0x7f461c113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f462192c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c1027f0 0x7f461c113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35926/0 (socket says 172.21.15.64:35926) 2024-02-16T22:52:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f462192c700 1 -- 172.21.15.64:0/3397457237 learned_addr learned my addr 172.21.15.64:0/3397457237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f462212d700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c101e50 0x7f461c113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f462192c700 1 -- 172.21.15.64:0/3397457237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c101e50 msgr2=0x7f461c113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f462192c700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c101e50 0x7f461c113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f462192c700 1 -- 172.21.15.64:0/3397457237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618005040 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.029+0000 7f462192c700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c1027f0 0x7f461c113fa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f460c002a50 tx=0x7f460c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.030+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f460c007cc0 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.030+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c10dc00 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.030+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f460c00f040 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.030+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f460c00d9c0 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.030+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c10e140 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.030+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f461c10e6e0 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.031+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f460c009730 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.031+0000 7f4612ffd700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46080692a0 0x7f460806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.031+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f460c086bc0 con 0x7f461c1027f0 2024-02-16T22:52:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.031+0000 7f462212d700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46080692a0 0x7f460806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.032+0000 7f4612ffd700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] conn(0x7f460806ef70 0x7f4608071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.032+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 --> [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4608071a30 con 0x7f460806ef70 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.032+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f460c08b220 con 0x7f461c1027f0 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.032+0000 7f462292e700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] conn(0x7f460806ef70 0x7f4608071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.032+0000 7f462212d700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46080692a0 0x7f460806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4618004f90 tx=0x7f4618009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.032+0000 7f462292e700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] conn(0x7f460806ef70 0x7f4608071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.033+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 <== osd.2 v2:172.21.15.64:6818/2434652781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f4608071a30 con 0x7f460806ef70 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 --> [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f461c0669a0 con 0x7f460806ef70 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4612ffd700 1 -- 172.21.15.64:0/3397457237 <== osd.2 v2:172.21.15.64:6818/2434652781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f461c0669a0 con 0x7f460806ef70 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] conn(0x7f460806ef70 msgr2=0x7f4608071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] conn(0x7f460806ef70 0x7f4608071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46080692a0 msgr2=0x7f460806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46080692a0 0x7f460806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4618004f90 tx=0x7f4618009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c1027f0 msgr2=0x7f461c113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c1027f0 0x7f461c113fa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f460c002a50 tx=0x7f460c004600 comp rx=0 tx=0).stop 2024-02-16T22:52:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 shutdown_connections 2024-02-16T22:52:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6818/2434652781,v1:172.21.15.64:6819/2434652781] conn(0x7f460806ef70 0x7f4608071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c101e50 0x7f461c113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46080692a0 0x7f460806b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 --2- 172.21.15.64:0/3397457237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c1027f0 0x7f461c113fa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 >> 172.21.15.64:0/3397457237 conn(0x7f461c0f79e0 msgr2=0x7f461c065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 shutdown_connections 2024-02-16T22:52:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:53.048+0000 7f4624391700 1 -- 172.21.15.64:0/3397457237 wait complete. 2024-02-16T22:52:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:53 smithi192 ceph-mon[36254]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:53.984 INFO:teuthology.orchestra.run.smithi064.stdout:81604378648 2024-02-16T22:52:53.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-16T22:52:54.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.654+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2370872969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 msgr2=0x7f5eb00fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.654+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2370872969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 0x7f5eb00fcf60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0004940 tx=0x7f5ea0009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2370872969 shutdown_connections 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2370872969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 0x7f5eb00fcf60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2370872969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb0102f50 0x7f5eb0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2370872969 >> 172.21.15.64:0/2370872969 conn(0x7f5eb00f8ae0 msgr2=0x7f5eb00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2370872969 shutdown_connections 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2370872969 wait complete. 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 Processor -- start 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 -- start start 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb0102f50 0x7f5eb0113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 0x7f5eb0113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0114620 con 0x7f5eb01038f0 2024-02-16T22:52:54.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.655+0000 7f5eb74d8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb010d950 con 0x7f5eb0102f50 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb4a73700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 0x7f5eb0113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb4a73700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 0x7f5eb0113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35952/0 (socket says 172.21.15.64:35952) 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb4a73700 1 -- 172.21.15.64:0/2391286144 learned_addr learned my addr 172.21.15.64:0/2391286144 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb5274700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb0102f50 0x7f5eb0113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb4a73700 1 -- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb0102f50 msgr2=0x7f5eb0113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb4a73700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb0102f50 0x7f5eb0113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb4a73700 1 -- 172.21.15.64:0/2391286144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0005040 con 0x7f5eb01038f0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb4a73700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 0x7f5eb0113fd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00090d0 tx=0x7f5ea0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea000e040 con 0x7f5eb01038f0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ea000bbf0 con 0x7f5eb01038f0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb010dbd0 con 0x7f5eb01038f0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea001c630 con 0x7f5eb01038f0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.656+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb010e110 con 0x7f5eb01038f0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.657+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5eb010e650 con 0x7f5eb01038f0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5ea000bd60 con 0x7f5eb01038f0 2024-02-16T22:52:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5ea5ffb700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e9c0692a0 0x7f5e9c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5ea00881b0 con 0x7f5eb01038f0 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5eb5274700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e9c0692a0 0x7f5e9c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5ea5ffb700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] conn(0x7f5e9c06ef70 0x7f5e9c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 --> [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e9c071a30 con 0x7f5e9c06ef70 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea008a800 con 0x7f5eb01038f0 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5eb5274700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e9c0692a0 0x7f5e9c06b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009fd0 tx=0x7f5eac009e50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.658+0000 7f5eb5a75700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] conn(0x7f5e9c06ef70 0x7f5e9c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.659+0000 7f5eb5a75700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] conn(0x7f5e9c06ef70 0x7f5e9c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.660+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 <== osd.3 v2:172.21.15.192:6800/1577479253 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5e9c071a30 con 0x7f5e9c06ef70 2024-02-16T22:52:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.674+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 --> [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5eb005e690 con 0x7f5e9c06ef70 2024-02-16T22:52:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5ea5ffb700 1 -- 172.21.15.64:0/2391286144 <== osd.3 v2:172.21.15.192:6800/1577479253 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5eb005e690 con 0x7f5e9c06ef70 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] conn(0x7f5e9c06ef70 msgr2=0x7f5e9c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] conn(0x7f5e9c06ef70 0x7f5e9c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e9c0692a0 msgr2=0x7f5e9c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e9c0692a0 0x7f5e9c06b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009fd0 tx=0x7f5eac009e50 comp rx=0 tx=0).stop 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 msgr2=0x7f5eb0113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 0x7f5eb0113fd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00090d0 tx=0x7f5ea0009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 shutdown_connections 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:6800/1577479253,v1:172.21.15.192:6801/1577479253] conn(0x7f5e9c06ef70 0x7f5e9c071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb0102f50 0x7f5eb0113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e9c0692a0 0x7f5e9c06b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 --2- 172.21.15.64:0/2391286144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb01038f0 0x7f5eb0113fd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 >> 172.21.15.64:0/2391286144 conn(0x7f5eb00f8ae0 msgr2=0x7f5eb00fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 shutdown_connections 2024-02-16T22:52:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:54.675+0000 7f5eb74d8700 1 -- 172.21.15.64:0/2391286144 wait complete. 2024-02-16T22:52:55.257 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:55 smithi064 ceph-mon[29226]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:55.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:55 smithi192 ceph-mon[36254]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:55.596 INFO:teuthology.orchestra.run.smithi064.stdout:115964117011 2024-02-16T22:52:55.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.931+0000 7fb33b549700 1 -- 172.21.15.64:0/198027885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 msgr2=0x7fb3340fe300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.931+0000 7fb33b549700 1 --2- 172.21.15.64:0/198027885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 0x7fb3340fe300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb330004940 tx=0x7fb330009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.931+0000 7fb33b549700 1 -- 172.21.15.64:0/198027885 shutdown_connections 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.931+0000 7fb33b549700 1 --2- 172.21.15.64:0/198027885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3340fe8d0 0x7fb334106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.931+0000 7fb33b549700 1 --2- 172.21.15.64:0/198027885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 0x7fb3340fe300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.931+0000 7fb33b549700 1 -- 172.21.15.64:0/198027885 >> 172.21.15.64:0/198027885 conn(0x7fb3340f79d0 msgr2=0x7fb3340f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.931+0000 7fb33b549700 1 -- 172.21.15.64:0/198027885 shutdown_connections 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.931+0000 7fb33b549700 1 -- 172.21.15.64:0/198027885 wait complete. 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb33b549700 1 Processor -- start 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb33b549700 1 -- start start 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb33b549700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 0x7fb3340fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb33b549700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3340fe8d0 0x7fb3340fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb33b549700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334111880 con 0x7fb3340fdf30 2024-02-16T22:52:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb33b549700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3341119f0 con 0x7fb3340fe8d0 2024-02-16T22:52:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb3392e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 0x7fb3340fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb3392e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 0x7fb3340fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35972/0 (socket says 172.21.15.64:35972) 2024-02-16T22:52:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.932+0000 7fb3392e5700 1 -- 172.21.15.64:0/677303583 learned_addr learned my addr 172.21.15.64:0/677303583 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb338ae4700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3340fe8d0 0x7fb3340fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb3392e5700 1 -- 172.21.15.64:0/677303583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3340fe8d0 msgr2=0x7fb3340fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb3392e5700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3340fe8d0 0x7fb3340fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb3392e5700 1 -- 172.21.15.64:0/677303583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330005040 con 0x7fb3340fdf30 2024-02-16T22:52:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb3392e5700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 0x7fb3340fbdf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb330007f50 tx=0x7fb330009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb330021070 con 0x7fb3340fdf30 2024-02-16T22:52:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb330013d00 con 0x7fb3340fdf30 2024-02-16T22:52:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33000e070 con 0x7fb3340fdf30 2024-02-16T22:52:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3340fc960 con 0x7fb3340fdf30 2024-02-16T22:52:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.933+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33411e580 con 0x7fb3340fdf30 2024-02-16T22:52:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.934+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb33405e610 con 0x7fb3340fdf30 2024-02-16T22:52:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.935+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb33001ca60 con 0x7fb3340fdf30 2024-02-16T22:52:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.935+0000 7fb329ffb700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb320069230 0x7fb32006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.935+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb330088e40 con 0x7fb3340fdf30 2024-02-16T22:52:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.935+0000 7fb338ae4700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb320069230 0x7fb32006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.935+0000 7fb329ffb700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] conn(0x7fb32006ef00 0x7fb320071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.935+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 --> [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3200719c0 con 0x7fb32006ef00 2024-02-16T22:52:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.935+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb330089220 con 0x7fb3340fdf30 2024-02-16T22:52:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.936+0000 7fb339ae6700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] conn(0x7fb32006ef00 0x7fb320071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.936+0000 7fb338ae4700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb320069230 0x7fb32006b6e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3340fd7f0 tx=0x7fb32400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.936+0000 7fb339ae6700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] conn(0x7fb32006ef00 0x7fb320071310 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.936+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 <== osd.0 v2:172.21.15.64:6802/3967835180 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb3200719c0 con 0x7fb32006ef00 2024-02-16T22:52:56.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.951+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 --> [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb33405ea10 con 0x7fb32006ef00 2024-02-16T22:52:56.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb329ffb700 1 -- 172.21.15.64:0/677303583 <== osd.0 v2:172.21.15.64:6802/3967835180 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb33405ea10 con 0x7fb32006ef00 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] conn(0x7fb32006ef00 msgr2=0x7fb320071310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] conn(0x7fb32006ef00 0x7fb320071310 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb320069230 msgr2=0x7fb32006b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb320069230 0x7fb32006b6e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3340fd7f0 tx=0x7fb32400b040 comp rx=0 tx=0).stop 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 msgr2=0x7fb3340fbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 0x7fb3340fbdf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb330007f50 tx=0x7fb330009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 shutdown_connections 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6802/3967835180,v1:172.21.15.64:6803/3967835180] conn(0x7fb32006ef00 0x7fb320071310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb320069230 0x7fb32006b6e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3340fdf30 0x7fb3340fbdf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 --2- 172.21.15.64:0/677303583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3340fe8d0 0x7fb3340fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:56.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.952+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 >> 172.21.15.64:0/677303583 conn(0x7fb3340f79d0 msgr2=0x7fb334101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:56.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.953+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 shutdown_connections 2024-02-16T22:52:56.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:56.953+0000 7fb33b549700 1 -- 172.21.15.64:0/677303583 wait complete. 2024-02-16T22:52:57.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:57 smithi064 ceph-mon[29226]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:57 smithi192 ceph-mon[36254]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:57.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.785+0000 7fa461a82700 1 -- 172.21.15.64:0/2878513952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c1027e0 msgr2=0x7fa45c0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:57.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.785+0000 7fa461a82700 1 --2- 172.21.15.64:0/2878513952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c1027e0 0x7fa45c0fbea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa450004940 tx=0x7fa450009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:57.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.785+0000 7fa461a82700 1 -- 172.21.15.64:0/2878513952 shutdown_connections 2024-02-16T22:52:57.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.785+0000 7fa461a82700 1 --2- 172.21.15.64:0/2878513952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c1027e0 0x7fa45c0fbea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.785+0000 7fa461a82700 1 --2- 172.21.15.64:0/2878513952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa45c101e40 0x7fa45c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.785+0000 7fa461a82700 1 -- 172.21.15.64:0/2878513952 >> 172.21.15.64:0/2878513952 conn(0x7fa45c071d90 msgr2=0x7fa45c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:57.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.785+0000 7fa461a82700 1 -- 172.21.15.64:0/2878513952 shutdown_connections 2024-02-16T22:52:57.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.785+0000 7fa461a82700 1 -- 172.21.15.64:0/2878513952 wait complete. 2024-02-16T22:52:57.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa461a82700 1 Processor -- start 2024-02-16T22:52:57.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa461a82700 1 -- start start 2024-02-16T22:52:57.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa461a82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c101e40 0x7fa45c112970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa461a82700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa45c1027e0 0x7fa45c112eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa461a82700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c113500 con 0x7fa45c101e40 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa461a82700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c10c830 con 0x7fa45c1027e0 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa45b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c101e40 0x7fa45c112970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa45b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c101e40 0x7fa45c112970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35978/0 (socket says 172.21.15.64:35978) 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa45b7fe700 1 -- 172.21.15.64:0/260784240 learned_addr learned my addr 172.21.15.64:0/260784240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa45affd700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa45c1027e0 0x7fa45c112eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa45b7fe700 1 -- 172.21.15.64:0/260784240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa45c1027e0 msgr2=0x7fa45c112eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa45b7fe700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa45c1027e0 0x7fa45c112eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.786+0000 7fa45b7fe700 1 -- 172.21.15.64:0/260784240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa450005040 con 0x7fa45c101e40 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.787+0000 7fa45b7fe700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c101e40 0x7fa45c112970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa44c002a50 tx=0x7fa44c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.787+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44c007cc0 con 0x7fa45c101e40 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.787+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa44c00f040 con 0x7fa45c101e40 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.787+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c10cb10 con 0x7fa45c101e40 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.787+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44c009640 con 0x7fa45c101e40 2024-02-16T22:52:57.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.787+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c10d050 con 0x7fa45c101e40 2024-02-16T22:52:57.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.788+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa45c10d5f0 con 0x7fa45c101e40 2024-02-16T22:52:57.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.789+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa44c0097c0 con 0x7fa45c101e40 2024-02-16T22:52:57.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.789+0000 7fa443fff700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa4440692a0 0x7fa44406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.789+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa44c086c60 con 0x7fa45c101e40 2024-02-16T22:52:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.789+0000 7fa45affd700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa4440692a0 0x7fa44406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.789+0000 7fa443fff700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] conn(0x7fa44406ef70 0x7fa444071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.789+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 --> [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa444071a30 con 0x7fa44406ef70 2024-02-16T22:52:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.789+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c08c260 con 0x7fa45c101e40 2024-02-16T22:52:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.790+0000 7fa45bfff700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] conn(0x7fa44406ef70 0x7fa444071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.790+0000 7fa45affd700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa4440692a0 0x7fa44406b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b3b0 tx=0x7fa450009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.790+0000 7fa45bfff700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] conn(0x7fa44406ef70 0x7fa444071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:57.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.791+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 <== osd.1 v2:172.21.15.64:6810/2618370222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa444071a30 con 0x7fa44406ef70 2024-02-16T22:52:57.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.805+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 --> [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa45c10cca0 con 0x7fa44406ef70 2024-02-16T22:52:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa443fff700 1 -- 172.21.15.64:0/260784240 <== osd.1 v2:172.21.15.64:6810/2618370222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa45c10cca0 con 0x7fa44406ef70 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] conn(0x7fa44406ef70 msgr2=0x7fa444071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] conn(0x7fa44406ef70 0x7fa444071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa4440692a0 msgr2=0x7fa44406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa4440692a0 0x7fa44406b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b3b0 tx=0x7fa450009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c101e40 msgr2=0x7fa45c112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c101e40 0x7fa45c112970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa44c002a50 tx=0x7fa44c0046b0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 shutdown_connections 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6810/2618370222,v1:172.21.15.64:6811/2618370222] conn(0x7fa44406ef70 0x7fa444071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa4440692a0 0x7fa44406b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c101e40 0x7fa45c112970 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 --2- 172.21.15.64:0/260784240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa45c1027e0 0x7fa45c112eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 >> 172.21.15.64:0/260784240 conn(0x7fa45c071d90 msgr2=0x7fa45c0fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.806+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 shutdown_connections 2024-02-16T22:52:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.807+0000 7fa461a82700 1 -- 172.21.15.64:0/260784240 wait complete. 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 -- 172.21.15.64:0/1291445108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 msgr2=0x7ff02c0fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 --2- 172.21.15.64:0/1291445108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 0x7ff02c0fbe50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff014004940 tx=0x7ff014009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 -- 172.21.15.64:0/1291445108 shutdown_connections 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 --2- 172.21.15.64:0/1291445108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 0x7ff02c0fbe50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 --2- 172.21.15.64:0/1291445108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff02c101e40 0x7ff02c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 -- 172.21.15.64:0/1291445108 >> 172.21.15.64:0/1291445108 conn(0x7ff02c0f79d0 msgr2=0x7ff02c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 -- 172.21.15.64:0/1291445108 shutdown_connections 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 -- 172.21.15.64:0/1291445108 wait complete. 2024-02-16T22:52:57.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.977+0000 7ff03200a700 1 Processor -- start 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff03200a700 1 -- start start 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff03200a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff02c101e40 0x7ff02c113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff03200a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 0x7ff02c1140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff03200a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1146f0 con 0x7ff02c1027e0 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff03200a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c10da20 con 0x7ff02c101e40 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 0x7ff02c1140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 0x7ff02c1140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35994/0 (socket says 172.21.15.64:35994) 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02affd700 1 -- 172.21.15.64:0/845910024 learned_addr learned my addr 172.21.15.64:0/845910024 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02b7fe700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff02c101e40 0x7ff02c113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02affd700 1 -- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff02c101e40 msgr2=0x7ff02c113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02affd700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff02c101e40 0x7ff02c113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02affd700 1 -- 172.21.15.64:0/845910024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff014005040 con 0x7ff02c1027e0 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02b7fe700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff02c101e40 0x7ff02c113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.978+0000 7ff02affd700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 0x7ff02c1140a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff0140090d0 tx=0x7ff014009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.979+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff01400e040 con 0x7ff02c1027e0 2024-02-16T22:52:57.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.979+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff01400bbf0 con 0x7ff02c1027e0 2024-02-16T22:52:57.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.979+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c10dca0 con 0x7ff02c1027e0 2024-02-16T22:52:57.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.979+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff01401c630 con 0x7ff02c1027e0 2024-02-16T22:52:57.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.979+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c10e1e0 con 0x7ff02c1027e0 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.979+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff02c10e640 con 0x7ff02c1027e0 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.980+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff01400bd60 con 0x7ff02c1027e0 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.980+0000 7ff023fff700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0180692a0 0x7ff01806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.980+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff0140881b0 con 0x7ff02c1027e0 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.980+0000 7ff02b7fe700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0180692a0 0x7ff01806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.981+0000 7ff023fff700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] conn(0x7ff01806ef70 0x7ff018071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.981+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 --> [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff018071a30 con 0x7ff01806ef70 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.981+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff01408a7a0 con 0x7ff02c1027e0 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.981+0000 7ff02b7fe700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0180692a0 0x7ff01806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff024004550 tx=0x7ff024009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.981+0000 7ff02bfff700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] conn(0x7ff01806ef70 0x7ff018071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:52:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.981+0000 7ff02bfff700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] conn(0x7ff01806ef70 0x7ff018071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:52:57.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.982+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 <== osd.4 v2:172.21.15.192:6808/523969674 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff018071a30 con 0x7ff01806ef70 2024-02-16T22:52:57.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.997+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 --> [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff02c100ed0 con 0x7ff01806ef70 2024-02-16T22:52:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff023fff700 1 -- 172.21.15.64:0/845910024 <== osd.4 v2:172.21.15.192:6808/523969674 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff02c100ed0 con 0x7ff01806ef70 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] conn(0x7ff01806ef70 msgr2=0x7ff018071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] conn(0x7ff01806ef70 0x7ff018071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0180692a0 msgr2=0x7ff01806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0180692a0 0x7ff01806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff024004550 tx=0x7ff024009040 comp rx=0 tx=0).stop 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 msgr2=0x7ff02c1140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 0x7ff02c1140a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff0140090d0 tx=0x7ff014009e30 comp rx=0 tx=0).stop 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 shutdown_connections 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:6808/523969674,v1:172.21.15.192:6809/523969674] conn(0x7ff01806ef70 0x7ff018071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff02c101e40 0x7ff02c113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0180692a0 0x7ff01806b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 --2- 172.21.15.64:0/845910024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff02c1027e0 0x7ff02c1140a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 >> 172.21.15.64:0/845910024 conn(0x7ff02c0f79d0 msgr2=0x7ff02c066110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 shutdown_connections 2024-02-16T22:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:52:57.998+0000 7ff03200a700 1 -- 172.21.15.64:0/845910024 wait complete. 2024-02-16T22:52:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:52:58.751 INFO:teuthology.orchestra.run.smithi064.stdout:60129542173 2024-02-16T22:52:58.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-16T22:52:59.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:58 smithi064 ceph-mon[29226]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:52:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:52:59.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:58 smithi192 ceph-mon[36254]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:52:59.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:52:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:52:59.722 INFO:teuthology.orchestra.run.smithi064.stdout:150323855374 2024-02-16T22:52:59.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-16T22:53:00.601 INFO:teuthology.orchestra.run.smithi064.stdout:38654705699 2024-02-16T22:53:00.601 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-16T22:53:00.635 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:01.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:01 smithi192 ceph-mon[36254]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:01.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:01 smithi064 ceph-mon[29226]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.523+0000 7fbdafdbb700 1 -- 172.21.15.64:0/577194131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fdf60 msgr2=0x7fbda80fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.523+0000 7fbdafdbb700 1 --2- 172.21.15.64:0/577194131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fdf60 0x7fbda80fe330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbda4004940 tx=0x7fbda4009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.523+0000 7fbdafdbb700 1 -- 172.21.15.64:0/577194131 shutdown_connections 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.523+0000 7fbdafdbb700 1 --2- 172.21.15.64:0/577194131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda80fe900 0x7fbda8106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.523+0000 7fbdafdbb700 1 --2- 172.21.15.64:0/577194131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fdf60 0x7fbda80fe330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.523+0000 7fbdafdbb700 1 -- 172.21.15.64:0/577194131 >> 172.21.15.64:0/577194131 conn(0x7fbda80f7a10 msgr2=0x7fbda80f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdafdbb700 1 -- 172.21.15.64:0/577194131 shutdown_connections 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdafdbb700 1 -- 172.21.15.64:0/577194131 wait complete. 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdafdbb700 1 Processor -- start 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdafdbb700 1 -- start start 2024-02-16T22:53:02.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdafdbb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda80fdf60 0x7fbda8113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdafdbb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fe900 0x7fbda8113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdafdbb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8114630 con 0x7fbda80fe900 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdafdbb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda810d960 con 0x7fbda80fdf60 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.524+0000 7fbdad356700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fe900 0x7fbda8113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdad356700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fe900 0x7fbda8113fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53834/0 (socket says 172.21.15.64:53834) 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdad356700 1 -- 172.21.15.64:0/194605976 learned_addr learned my addr 172.21.15.64:0/194605976 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdadb57700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda80fdf60 0x7fbda8113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdad356700 1 -- 172.21.15.64:0/194605976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda80fdf60 msgr2=0x7fbda8113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdad356700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda80fdf60 0x7fbda8113aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdad356700 1 -- 172.21.15.64:0/194605976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4005040 con 0x7fbda80fe900 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdadb57700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda80fdf60 0x7fbda8113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdad356700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fe900 0x7fbda8113fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800c830 tx=0x7fbd98004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbd9e7fc700 1 -- 172.21.15.64:0/194605976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd9800a760 con 0x7fbda80fe900 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbd9e7fc700 1 -- 172.21.15.64:0/194605976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd9800ad90 con 0x7fbda80fe900 2024-02-16T22:53:02.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda810dc40 con 0x7fbda80fe900 2024-02-16T22:53:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbd9e7fc700 1 -- 172.21.15.64:0/194605976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd980113a0 con 0x7fbda80fe900 2024-02-16T22:53:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.525+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda810e180 con 0x7fbda80fe900 2024-02-16T22:53:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.526+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda8104550 con 0x7fbda80fe900 2024-02-16T22:53:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.527+0000 7fbd9e7fc700 1 -- 172.21.15.64:0/194605976 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbd98012400 con 0x7fbda80fe900 2024-02-16T22:53:02.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.527+0000 7fbd9e7fc700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd940692a0 0x7fbd9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.527+0000 7fbd9e7fc700 1 -- 172.21.15.64:0/194605976 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbd98086f10 con 0x7fbda80fe900 2024-02-16T22:53:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.527+0000 7fbdadb57700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd940692a0 0x7fbd9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:02.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.527+0000 7fbdadb57700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd940692a0 0x7fbd9406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbda4004560 tx=0x7fbda4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:02.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.532+0000 7fbd9e7fc700 1 -- 172.21.15.64:0/194605976 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbd98057010 con 0x7fbda80fe900 2024-02-16T22:53:02.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.728+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbda8109ff0 con 0x7fbda80fe900 2024-02-16T22:53:02.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.728+0000 7fbd9e7fc700 1 -- 172.21.15.64:0/194605976 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd98056ba0 con 0x7fbda80fe900 2024-02-16T22:53:02.730 INFO:teuthology.orchestra.run.smithi064.stdout:171798691851 2024-02-16T22:53:02.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd940692a0 msgr2=0x7fbd9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd940692a0 0x7fbd9406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbda4004560 tx=0x7fbda4009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fe900 msgr2=0x7fbda8113fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fe900 0x7fbda8113fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800c830 tx=0x7fbd98004470 comp rx=0 tx=0).stop 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 shutdown_connections 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda80fdf60 0x7fbda8113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd940692a0 0x7fbd9406b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 --2- 172.21.15.64:0/194605976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbda80fe900 0x7fbda8113fe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 >> 172.21.15.64:0/194605976 conn(0x7fbda80f7a10 msgr2=0x7fbda81016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 shutdown_connections 2024-02-16T22:53:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:02.730+0000 7fbdafdbb700 1 -- 172.21.15.64:0/194605976 wait complete. 2024-02-16T22:53:03.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:03 smithi064 ceph-mon[29226]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:03.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:03 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/194605976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-16T22:53:03.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:03 smithi192 ceph-mon[36254]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:03.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:03 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/194605976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-16T22:53:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.681+0000 7ffb1813a700 1 -- 172.21.15.64:0/1439862772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb101038f0 msgr2=0x7ffb100fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.681+0000 7ffb1813a700 1 --2- 172.21.15.64:0/1439862772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb101038f0 0x7ffb100fcf60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb00004940 tx=0x7ffb00009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.681+0000 7ffb1813a700 1 -- 172.21.15.64:0/1439862772 shutdown_connections 2024-02-16T22:53:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.681+0000 7ffb1813a700 1 --2- 172.21.15.64:0/1439862772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb101038f0 0x7ffb100fcf60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.681+0000 7ffb1813a700 1 --2- 172.21.15.64:0/1439862772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb10102f50 0x7ffb10103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.681+0000 7ffb1813a700 1 -- 172.21.15.64:0/1439862772 >> 172.21.15.64:0/1439862772 conn(0x7ffb100f8ae0 msgr2=0x7ffb100faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb1813a700 1 -- 172.21.15.64:0/1439862772 shutdown_connections 2024-02-16T22:53:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb1813a700 1 -- 172.21.15.64:0/1439862772 wait complete. 2024-02-16T22:53:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb1813a700 1 Processor -- start 2024-02-16T22:53:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb1813a700 1 -- start start 2024-02-16T22:53:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb1813a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb10102f50 0x7ffb10070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb1813a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb101038f0 0x7ffb100707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb1813a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb10070dd0 con 0x7ffb10102f50 2024-02-16T22:53:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb1813a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb10070f40 con 0x7ffb101038f0 2024-02-16T22:53:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb15ed6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb10102f50 0x7ffb10070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb15ed6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb10102f50 0x7ffb10070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53852/0 (socket says 172.21.15.64:53852) 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb15ed6700 1 -- 172.21.15.64:0/376280275 learned_addr learned my addr 172.21.15.64:0/376280275 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.682+0000 7ffb156d5700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb101038f0 0x7ffb100707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb15ed6700 1 -- 172.21.15.64:0/376280275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb101038f0 msgr2=0x7ffb100707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb15ed6700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb101038f0 0x7ffb100707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb15ed6700 1 -- 172.21.15.64:0/376280275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb00005040 con 0x7ffb10102f50 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb15ed6700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb10102f50 0x7ffb10070290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c002a50 tx=0x7ffb0c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb067fc700 1 -- 172.21.15.64:0/376280275 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb0c007cc0 con 0x7ffb10102f50 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb10119cd0 con 0x7ffb10102f50 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb067fc700 1 -- 172.21.15.64:0/376280275 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb0c00f040 con 0x7ffb10102f50 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb067fc700 1 -- 172.21.15.64:0/376280275 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb0c00da30 con 0x7ffb10102f50 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.683+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1011a180 con 0x7ffb10102f50 2024-02-16T22:53:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.684+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb10105420 con 0x7ffb10102f50 2024-02-16T22:53:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.685+0000 7ffb067fc700 1 -- 172.21.15.64:0/376280275 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ffb0c00dbb0 con 0x7ffb10102f50 2024-02-16T22:53:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.685+0000 7ffb067fc700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ffafc0692a0 0x7ffafc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.685+0000 7ffb067fc700 1 -- 172.21.15.64:0/376280275 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffb0c086c00 con 0x7ffb10102f50 2024-02-16T22:53:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.685+0000 7ffb156d5700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ffafc0692a0 0x7ffafc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.685+0000 7ffb156d5700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ffafc0692a0 0x7ffafc06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000b3b0 tx=0x7ffb00009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:03.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.689+0000 7ffb067fc700 1 -- 172.21.15.64:0/376280275 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ffb0c08c020 con 0x7ffb10102f50 2024-02-16T22:53:03.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.886+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffb1004bed0 con 0x7ffb10102f50 2024-02-16T22:53:03.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.886+0000 7ffb067fc700 1 -- 172.21.15.64:0/376280275 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb0c0531f0 con 0x7ffb10102f50 2024-02-16T22:53:03.888 INFO:teuthology.orchestra.run.smithi064.stdout:81604378650 2024-02-16T22:53:03.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ffafc0692a0 msgr2=0x7ffafc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:03.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ffafc0692a0 0x7ffafc06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000b3b0 tx=0x7ffb00009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb10102f50 msgr2=0x7ffb10070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb10102f50 0x7ffb10070290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c002a50 tx=0x7ffb0c0046b0 comp rx=0 tx=0).stop 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 shutdown_connections 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ffafc0692a0 0x7ffafc06b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb10102f50 0x7ffb10070290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 --2- 172.21.15.64:0/376280275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb101038f0 0x7ffb100707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 >> 172.21.15.64:0/376280275 conn(0x7ffb100f8ae0 msgr2=0x7ffb100fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 shutdown_connections 2024-02-16T22:53:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:03.888+0000 7ffb1813a700 1 -- 172.21.15.64:0/376280275 wait complete. 2024-02-16T22:53:04.529 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:04 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/376280275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-16T22:53:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:04 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/376280275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-16T22:53:05.175 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-02-16T22:53:05.175 DEBUG:teuthology.parallel:result is None 2024-02-16T22:53:05.403 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:05 smithi064 ceph-mon[29226]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:05.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:05 smithi192 ceph-mon[36254]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:05.732 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691851 for osd.5 2024-02-16T22:53:05.732 DEBUG:teuthology.parallel:result is None 2024-02-16T22:53:05.928 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:05.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.951+0000 7f994effd700 1 -- 172.21.15.64:0/3671741360 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994400bcb0 con 0x7f99501038e0 2024-02-16T22:53:05.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 -- 172.21.15.64:0/3671741360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 msgr2=0x7f99500fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:05.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 --2- 172.21.15.64:0/3671741360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 0x7f99500fcf50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9944004940 tx=0x7f9944009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:05.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 -- 172.21.15.64:0/3671741360 shutdown_connections 2024-02-16T22:53:05.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 --2- 172.21.15.64:0/3671741360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 0x7f99500fcf50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:05.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 --2- 172.21.15.64:0/3671741360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9950102f40 0x7f9950103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 -- 172.21.15.64:0/3671741360 >> 172.21.15.64:0/3671741360 conn(0x7f99500f8af0 msgr2=0x7f99500faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 -- 172.21.15.64:0/3671741360 shutdown_connections 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 -- 172.21.15.64:0/3671741360 wait complete. 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.952+0000 7f99569a7700 1 Processor -- start 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f99569a7700 1 -- start start 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f99569a7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9950102f40 0x7f9950113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f99569a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 0x7f9950113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f99569a7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950114610 con 0x7f99501038e0 2024-02-16T22:53:05.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f99569a7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995010d940 con 0x7f9950102f40 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f994f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 0x7f9950113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f994ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9950102f40 0x7f9950113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f994ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9950102f40 0x7f9950113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:46072/0 (socket says 172.21.15.64:46072) 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f994f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 0x7f9950113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53872/0 (socket says 172.21.15.64:53872) 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f994ffff700 1 -- 172.21.15.64:0/2159401453 learned_addr learned my addr 172.21.15.64:0/2159401453 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f994ffff700 1 -- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 msgr2=0x7f9950113fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f994ffff700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 0x7f9950113fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.953+0000 7f994ffff700 1 -- 172.21.15.64:0/2159401453 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9944005040 con 0x7f9950102f40 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.954+0000 7f994ffff700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9950102f40 0x7f9950113a80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9940002a80 tx=0x7f99400046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.954+0000 7f994cff9700 1 -- 172.21.15.64:0/2159401453 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9940007cc0 con 0x7f9950102f40 2024-02-16T22:53:05.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.954+0000 7f994cff9700 1 -- 172.21.15.64:0/2159401453 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f994000f040 con 0x7f9950102f40 2024-02-16T22:53:05.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.954+0000 7f994cff9700 1 -- 172.21.15.64:0/2159401453 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994000dac0 con 0x7f9950102f40 2024-02-16T22:53:05.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.954+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995010dbc0 con 0x7f9950102f40 2024-02-16T22:53:05.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.954+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995010dfc0 con 0x7f9950102f40 2024-02-16T22:53:05.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.955+0000 7f994cff9700 1 -- 172.21.15.64:0/2159401453 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f994000dc20 con 0x7f9950102f40 2024-02-16T22:53:05.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.955+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99500fc6f0 con 0x7f9950102f40 2024-02-16T22:53:05.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.956+0000 7f994cff9700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f993806d6b0 0x7f993806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:05.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.956+0000 7f994cff9700 1 -- 172.21.15.64:0/2159401453 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9940086b90 con 0x7f9950102f40 2024-02-16T22:53:05.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.956+0000 7f994f7fe700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f993806d6b0 0x7f993806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:05.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.957+0000 7f994f7fe700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f993806d6b0 0x7f993806fb60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f995010ec70 tx=0x7f9944009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:05.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:05.960+0000 7f994cff9700 1 -- 172.21.15.64:0/2159401453 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9940056b70 con 0x7f9950102f40 2024-02-16T22:53:05.971 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:06.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.155+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f995010f060 con 0x7f9950102f40 2024-02-16T22:53:06.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.155+0000 7f994cff9700 1 -- 172.21.15.64:0/2159401453 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9940056700 con 0x7f9950102f40 2024-02-16T22:53:06.156 INFO:teuthology.orchestra.run.smithi064.stdout:115964117013 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f993806d6b0 msgr2=0x7f993806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f993806d6b0 0x7f993806fb60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f995010ec70 tx=0x7f9944009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9950102f40 msgr2=0x7f9950113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9950102f40 0x7f9950113a80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9940002a80 tx=0x7f99400046b0 comp rx=0 tx=0).stop 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 shutdown_connections 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9950102f40 0x7f9950113a80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f993806d6b0 0x7f993806fb60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 --2- 172.21.15.64:0/2159401453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99501038e0 0x7f9950113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 >> 172.21.15.64:0/2159401453 conn(0x7f99500f8af0 msgr2=0x7f99500fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 shutdown_connections 2024-02-16T22:53:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:06.157+0000 7f99569a7700 1 -- 172.21.15.64:0/2159401453 wait complete. 2024-02-16T22:53:06.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:06 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2159401453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-16T22:53:06.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:06 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2159401453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-16T22:53:07.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:07 smithi064 ceph-mon[29226]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:07.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:07 smithi192 ceph-mon[36254]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:08.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117011 got 115964117013 for osd.3 2024-02-16T22:53:08.184 DEBUG:teuthology.parallel:result is None 2024-02-16T22:53:08.978 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:08 smithi064 ceph-mon[29226]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:09.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 -- 172.21.15.64:0/1615141521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 msgr2=0x7f6340106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:09.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 --2- 172.21.15.64:0/1615141521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 0x7f6340106f40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6334004940 tx=0x7f6334009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:09.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 -- 172.21.15.64:0/1615141521 shutdown_connections 2024-02-16T22:53:09.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 --2- 172.21.15.64:0/1615141521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 0x7f6340106f40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:09.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 --2- 172.21.15.64:0/1615141521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63400fdf60 0x7f63400fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:09.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 -- 172.21.15.64:0/1615141521 >> 172.21.15.64:0/1615141521 conn(0x7f63400f7a10 msgr2=0x7f63400f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:09.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 -- 172.21.15.64:0/1615141521 shutdown_connections 2024-02-16T22:53:09.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 -- 172.21.15.64:0/1615141521 wait complete. 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 Processor -- start 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 -- start start 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.191+0000 7f6346a43700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63400fdf60 0x7f6340113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f6346a43700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 0x7f6340114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f6346a43700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6340114730 con 0x7f63400fe900 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f6346a43700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634010d9d0 con 0x7f63400fdf60 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 0x7f6340114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 0x7f6340114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53886/0 (socket says 172.21.15.64:53886) 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633f7fe700 1 -- 172.21.15.64:0/2215561449 learned_addr learned my addr 172.21.15.64:0/2215561449 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633ffff700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63400fdf60 0x7f6340113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633f7fe700 1 -- 172.21.15.64:0/2215561449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63400fdf60 msgr2=0x7f6340113b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633f7fe700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63400fdf60 0x7f6340113b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633f7fe700 1 -- 172.21.15.64:0/2215561449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334005040 con 0x7f63400fe900 2024-02-16T22:53:09.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633f7fe700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 0x7f6340114050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f6334009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633cff9700 1 -- 172.21.15.64:0/2215561449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f633400e040 con 0x7f63400fe900 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633cff9700 1 -- 172.21.15.64:0/2215561449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6334009af0 con 0x7f63400fe900 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634010dc50 con 0x7f63400fe900 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f633cff9700 1 -- 172.21.15.64:0/2215561449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f633401c630 con 0x7f63400fe900 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.192+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634010e190 con 0x7f63400fe900 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.193+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6340104550 con 0x7f63400fe900 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.194+0000 7f633cff9700 1 -- 172.21.15.64:0/2215561449 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f633400bbf0 con 0x7f63400fe900 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.194+0000 7f633cff9700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f63280692a0 0x7f632806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:09.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.194+0000 7f633ffff700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f63280692a0 0x7f632806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:09.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.194+0000 7f633cff9700 1 -- 172.21.15.64:0/2215561449 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6334088240 con 0x7f63400fe900 2024-02-16T22:53:09.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.195+0000 7f633ffff700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f63280692a0 0x7f632806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6330006fd0 tx=0x7f6330008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:09.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.199+0000 7f633cff9700 1 -- 172.21.15.64:0/2215561449 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6334058340 con 0x7f63400fe900 2024-02-16T22:53:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:08 smithi192 ceph-mon[36254]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.395+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f63400fb610 con 0x7f63400fe900 2024-02-16T22:53:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.395+0000 7f633cff9700 1 -- 172.21.15.64:0/2215561449 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6334057ed0 con 0x7f63400fe900 2024-02-16T22:53:09.397 INFO:teuthology.orchestra.run.smithi064.stdout:150323855376 2024-02-16T22:53:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f63280692a0 msgr2=0x7f632806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f63280692a0 0x7f632806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6330006fd0 tx=0x7f6330008040 comp rx=0 tx=0).stop 2024-02-16T22:53:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 msgr2=0x7f6340114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 0x7f6340114050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f6334009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 shutdown_connections 2024-02-16T22:53:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63400fdf60 0x7f6340113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f63280692a0 0x7f632806b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 --2- 172.21.15.64:0/2215561449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63400fe900 0x7f6340114050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 >> 172.21.15.64:0/2215561449 conn(0x7f63400f7a10 msgr2=0x7f63401016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 shutdown_connections 2024-02-16T22:53:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:09.397+0000 7f6346a43700 1 -- 172.21.15.64:0/2215561449 wait complete. 2024-02-16T22:53:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:09 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2215561449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-16T22:53:10.258 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855374 got 150323855376 for osd.4 2024-02-16T22:53:10.259 DEBUG:teuthology.parallel:result is None 2024-02-16T22:53:10.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:09 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2215561449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-16T22:53:10.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 -- 172.21.15.64:0/3351054362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fdf60 msgr2=0x7f12dc0fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:10.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 --2- 172.21.15.64:0/3351054362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fdf60 0x7f12dc0fe330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f12cc004940 tx=0x7f12cc009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:10.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 -- 172.21.15.64:0/3351054362 shutdown_connections 2024-02-16T22:53:10.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 --2- 172.21.15.64:0/3351054362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12dc0fe900 0x7f12dc106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:10.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 --2- 172.21.15.64:0/3351054362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fdf60 0x7f12dc0fe330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:10.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 -- 172.21.15.64:0/3351054362 >> 172.21.15.64:0/3351054362 conn(0x7f12dc0f7a10 msgr2=0x7f12dc0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:10.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 -- 172.21.15.64:0/3351054362 shutdown_connections 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 -- 172.21.15.64:0/3351054362 wait complete. 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.575+0000 7f12e0f65700 1 Processor -- start 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12e0f65700 1 -- start start 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12e0f65700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12dc0fdf60 0x7f12dc0fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12e0f65700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fe900 0x7f12dc0fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12e0f65700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0fc8d0 con 0x7f12dc0fe900 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12e0f65700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0fca40 con 0x7f12dc0fdf60 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12d9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fe900 0x7f12dc0fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12d9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fe900 0x7f12dc0fc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46922/0 (socket says 172.21.15.64:46922) 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12d9d9b700 1 -- 172.21.15.64:0/4015064760 learned_addr learned my addr 172.21.15.64:0/4015064760 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12da59c700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12dc0fdf60 0x7f12dc0fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12d9d9b700 1 -- 172.21.15.64:0/4015064760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12dc0fdf60 msgr2=0x7f12dc0fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12d9d9b700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12dc0fdf60 0x7f12dc0fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12d9d9b700 1 -- 172.21.15.64:0/4015064760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc005040 con 0x7f12dc0fe900 2024-02-16T22:53:10.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.576+0000 7f12d9d9b700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fe900 0x7f12dc0fc2d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f12d0002a50 tx=0x7f12d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:10.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.577+0000 7f12caffd700 1 -- 172.21.15.64:0/4015064760 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d0007cc0 con 0x7f12dc0fe900 2024-02-16T22:53:10.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.577+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc119d90 con 0x7f12dc0fe900 2024-02-16T22:53:10.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.577+0000 7f12caffd700 1 -- 172.21.15.64:0/4015064760 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12d000f040 con 0x7f12dc0fe900 2024-02-16T22:53:10.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.577+0000 7f12caffd700 1 -- 172.21.15.64:0/4015064760 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d000da70 con 0x7f12dc0fe900 2024-02-16T22:53:10.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.577+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc11a270 con 0x7f12dc0fe900 2024-02-16T22:53:10.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.578+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc104550 con 0x7f12dc0fe900 2024-02-16T22:53:10.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.578+0000 7f12caffd700 1 -- 172.21.15.64:0/4015064760 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f12d000dc10 con 0x7f12dc0fe900 2024-02-16T22:53:10.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.578+0000 7f12caffd700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12c40692a0 0x7f12c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:10.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.579+0000 7f12caffd700 1 -- 172.21.15.64:0/4015064760 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f12d00864a0 con 0x7f12dc0fe900 2024-02-16T22:53:10.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.579+0000 7f12da59c700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12c40692a0 0x7f12c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:10.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.579+0000 7f12da59c700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12c40692a0 0x7f12c406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12cc004560 tx=0x7f12cc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:10.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.583+0000 7f12caffd700 1 -- 172.21.15.64:0/4015064760 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f12d004cbd0 con 0x7f12dc0fe900 2024-02-16T22:53:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.777+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f12dc0fb610 con 0x7f12dc0fe900 2024-02-16T22:53:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.778+0000 7f12caffd700 1 -- 172.21.15.64:0/4015064760 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f12d0052a90 con 0x7f12dc0fe900 2024-02-16T22:53:10.780 INFO:teuthology.orchestra.run.smithi064.stdout:60129542175 2024-02-16T22:53:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12c40692a0 msgr2=0x7f12c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12c40692a0 0x7f12c406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12cc004560 tx=0x7f12cc009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fe900 msgr2=0x7f12dc0fc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fe900 0x7f12dc0fc2d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f12d0002a50 tx=0x7f12d00046b0 comp rx=0 tx=0).stop 2024-02-16T22:53:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 shutdown_connections 2024-02-16T22:53:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12dc0fdf60 0x7f12dc0fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12c40692a0 0x7f12c406b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 --2- 172.21.15.64:0/4015064760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12dc0fe900 0x7f12dc0fc2d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 >> 172.21.15.64:0/4015064760 conn(0x7f12dc0f7a10 msgr2=0x7f12dc1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 shutdown_connections 2024-02-16T22:53:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:10.780+0000 7f12e0f65700 1 -- 172.21.15.64:0/4015064760 wait complete. 2024-02-16T22:53:10.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:10 smithi064 ceph-mon[29226]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:10.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:10 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/4015064760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-16T22:53:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:10 smithi192 ceph-mon[36254]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:10 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/4015064760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-16T22:53:11.662 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542175 for osd.1 2024-02-16T22:53:11.663 DEBUG:teuthology.parallel:result is None 2024-02-16T22:53:12.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.496+0000 7f21c44bd700 1 -- 172.21.15.64:0/1945990841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 msgr2=0x7f21bc0fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:12.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.496+0000 7f21c44bd700 1 --2- 172.21.15.64:0/1945990841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 0x7f21bc0fbe70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f21b8004940 tx=0x7f21b8009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:12.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.496+0000 7f21c44bd700 1 -- 172.21.15.64:0/1945990841 shutdown_connections 2024-02-16T22:53:12.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.496+0000 7f21c44bd700 1 --2- 172.21.15.64:0/1945990841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 0x7f21bc0fbe70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:12.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.496+0000 7f21c44bd700 1 --2- 172.21.15.64:0/1945990841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc101e60 0x7f21bc102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:12.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.496+0000 7f21c44bd700 1 -- 172.21.15.64:0/1945990841 >> 172.21.15.64:0/1945990841 conn(0x7f21bc0f7a10 msgr2=0x7f21bc0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:12.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.496+0000 7f21c44bd700 1 -- 172.21.15.64:0/1945990841 shutdown_connections 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.496+0000 7f21c44bd700 1 -- 172.21.15.64:0/1945990841 wait complete. 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c44bd700 1 Processor -- start 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c44bd700 1 -- start start 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c44bd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc101e60 0x7f21bc10eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c44bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 0x7f21bc10f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c44bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc10f670 con 0x7f21bc102800 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c44bd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc10f7b0 con 0x7f21bc101e60 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c1a58700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 0x7f21bc10f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c1a58700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 0x7f21bc10f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46928/0 (socket says 172.21.15.64:46928) 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c1a58700 1 -- 172.21.15.64:0/3964975208 learned_addr learned my addr 172.21.15.64:0/3964975208 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c2259700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc101e60 0x7f21bc10eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c1a58700 1 -- 172.21.15.64:0/3964975208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc101e60 msgr2=0x7f21bc10eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c1a58700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc101e60 0x7f21bc10eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c1a58700 1 -- 172.21.15.64:0/3964975208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b8005040 con 0x7f21bc102800 2024-02-16T22:53:12.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21c1a58700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 0x7f21bc10f0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f21b80090d0 tx=0x7f21b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:12.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.497+0000 7f21b2ffd700 1 -- 172.21.15.64:0/3964975208 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b800e040 con 0x7f21bc102800 2024-02-16T22:53:12.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.498+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc117ab0 con 0x7f21bc102800 2024-02-16T22:53:12.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.498+0000 7f21b2ffd700 1 -- 172.21.15.64:0/3964975208 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21b800bbf0 con 0x7f21bc102800 2024-02-16T22:53:12.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.498+0000 7f21b2ffd700 1 -- 172.21.15.64:0/3964975208 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b801c630 con 0x7f21bc102800 2024-02-16T22:53:12.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.498+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc117fc0 con 0x7f21bc102800 2024-02-16T22:53:12.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.498+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc104330 con 0x7f21bc102800 2024-02-16T22:53:12.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.499+0000 7f21b2ffd700 1 -- 172.21.15.64:0/3964975208 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f21b800bd60 con 0x7f21bc102800 2024-02-16T22:53:12.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.499+0000 7f21b2ffd700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f21a80692a0 0x7f21a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:12.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.499+0000 7f21b2ffd700 1 -- 172.21.15.64:0/3964975208 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f21b8088180 con 0x7f21bc102800 2024-02-16T22:53:12.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.499+0000 7f21c2259700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f21a80692a0 0x7f21a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:12.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.500+0000 7f21c2259700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f21a80692a0 0x7f21a806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009fd0 tx=0x7f21ac009e50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:12.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.504+0000 7f21b2ffd700 1 -- 172.21.15.64:0/3964975208 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f21b80547f0 con 0x7f21bc102800 2024-02-16T22:53:12.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.694+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f21bc0fb610 con 0x7f21bc102800 2024-02-16T22:53:12.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.695+0000 7f21b2ffd700 1 -- 172.21.15.64:0/3964975208 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f21b8013e60 con 0x7f21bc102800 2024-02-16T22:53:12.697 INFO:teuthology.orchestra.run.smithi064.stdout:38654705702 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.696+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f21a80692a0 msgr2=0x7f21a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.696+0000 7f21c44bd700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f21a80692a0 0x7f21a806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009fd0 tx=0x7f21ac009e50 comp rx=0 tx=0).stop 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.696+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 msgr2=0x7f21bc10f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.696+0000 7f21c44bd700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 0x7f21bc10f0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f21b80090d0 tx=0x7f21b8009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.697+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 shutdown_connections 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.697+0000 7f21c44bd700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc101e60 0x7f21bc10eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.697+0000 7f21c44bd700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f21a80692a0 0x7f21a806b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.697+0000 7f21c44bd700 1 --2- 172.21.15.64:0/3964975208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21bc102800 0x7f21bc10f0e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.697+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 >> 172.21.15.64:0/3964975208 conn(0x7f21bc0f7a10 msgr2=0x7f21bc0f9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.697+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 shutdown_connections 2024-02-16T22:53:12.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:12.697+0000 7f21c44bd700 1 -- 172.21.15.64:0/3964975208 wait complete. 2024-02-16T22:53:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:13 smithi064 ceph-mon[29226]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:13.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:13 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3964975208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-16T22:53:13.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:53:13.420 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705702 for osd.0 2024-02-16T22:53:13.420 DEBUG:teuthology.parallel:result is None 2024-02-16T22:53:13.420 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-16T22:53:13.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-16T22:53:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:13 smithi192 ceph-mon[36254]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:13 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3964975208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-16T22:53:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:53:13.682 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.251+0000 7f3189f7e700 1 -- 172.21.15.64:0/125221315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 msgr2=0x7f3184102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.251+0000 7f3189f7e700 1 --2- 172.21.15.64:0/125221315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 0x7f3184102230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f317c004940 tx=0x7f317c009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 -- 172.21.15.64:0/125221315 shutdown_connections 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 --2- 172.21.15.64:0/125221315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3184102800 0x7f31840fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 --2- 172.21.15.64:0/125221315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 0x7f3184102230 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 -- 172.21.15.64:0/125221315 >> 172.21.15.64:0/125221315 conn(0x7f31840f7a10 msgr2=0x7f31840f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 -- 172.21.15.64:0/125221315 shutdown_connections 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 -- 172.21.15.64:0/125221315 wait complete. 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 Processor -- start 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 -- start start 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 0x7f3184113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.252+0000 7f3189f7e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3184102800 0x7f3184113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f3189f7e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31841146d0 con 0x7f3184101e60 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f3189f7e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318410d970 con 0x7f3184102800 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f31837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 0x7f3184113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f31837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 0x7f3184113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46952/0 (socket says 172.21.15.64:46952) 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f31837fe700 1 -- 172.21.15.64:0/667127450 learned_addr learned my addr 172.21.15.64:0/667127450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f3182ffd700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3184102800 0x7f3184113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f31837fe700 1 -- 172.21.15.64:0/667127450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3184102800 msgr2=0x7f3184113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f31837fe700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3184102800 0x7f3184113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f31837fe700 1 -- 172.21.15.64:0/667127450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c005040 con 0x7f3184101e60 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f31837fe700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 0x7f3184113ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b3b0 tx=0x7f317c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f317bfff700 1 -- 172.21.15.64:0/667127450 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317c00e040 con 0x7f3184101e60 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318410dbf0 con 0x7f3184101e60 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f317bfff700 1 -- 172.21.15.64:0/667127450 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f317c00bbf0 con 0x7f3184101e60 2024-02-16T22:53:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f317bfff700 1 -- 172.21.15.64:0/667127450 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317c01c630 con 0x7f3184101e60 2024-02-16T22:53:15.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.253+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318410e130 con 0x7f3184101e60 2024-02-16T22:53:15.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.254+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31840fb610 con 0x7f3184101e60 2024-02-16T22:53:15.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.255+0000 7f317bfff700 1 -- 172.21.15.64:0/667127450 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f317c00bd60 con 0x7f3184101e60 2024-02-16T22:53:15.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.255+0000 7f317bfff700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f31700692a0 0x7f317006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.255+0000 7f317bfff700 1 -- 172.21.15.64:0/667127450 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f317c088250 con 0x7f3184101e60 2024-02-16T22:53:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.255+0000 7f3182ffd700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f31700692a0 0x7f317006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.256+0000 7f3182ffd700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f31700692a0 0x7f317006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f316c006fd0 tx=0x7f316c008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:15.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.260+0000 7f317bfff700 1 -- 172.21.15.64:0/667127450 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f317c058350 con 0x7f3184101e60 2024-02-16T22:53:15.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:15 smithi064 ceph-mon[29226]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:15.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.448+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f318410f100 con 0x7f31700692a0 2024-02-16T22:53:15.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.450+0000 7f317bfff700 1 -- 172.21.15.64:0/667127450 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28188 (secure 0 0 0) 0x7f318410f100 con 0x7f31700692a0 2024-02-16T22:53:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f31700692a0 msgr2=0x7f317006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f31700692a0 0x7f317006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f316c006fd0 tx=0x7f316c008040 comp rx=0 tx=0).stop 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 msgr2=0x7f3184113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 0x7f3184113ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b3b0 tx=0x7f317c009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 shutdown_connections 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f31700692a0 0x7f317006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3184101e60 0x7f3184113ab0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 --2- 172.21.15.64:0/667127450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3184102800 0x7f3184113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 >> 172.21.15.64:0/667127450 conn(0x7f31840f7a10 msgr2=0x7f3184105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 shutdown_connections 2024-02-16T22:53:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:15.452+0000 7f3189f7e700 1 -- 172.21.15.64:0/667127450 wait complete. 2024-02-16T22:53:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-02-16T22:53:15.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:15 smithi192 ceph-mon[36254]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:16.227 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-02-16T22:53:14.112976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001773"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":189,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-02-16T22:52:22.839278+0000","last_change":"2024-02-16T22:51:58.167133+0000","last_active":"2024-02-16T22:52:22.839278+0000","last_peered":"2024-02-16T22:52:22.839278+0000","last_clean":"2024-02-16T22:52:22.839278+0000","last_became_active":"2024-02-16T22:51:58.166761+0000","last_became_peered":"2024-02-16T22:51:58.166761+0000","last_unstale":"2024-02-16T22:52:22.839278+0000","last_undegraded":"2024-02-16T22:52:22.839278+0000","last_fullsized":"2024-02-16T22:52:22.839278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-16T22:51:12.814093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-16T22:51:12.814093+0000","last_clean_scrub_stamp":"2024-02-16T22:51:12.814093+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T10:38:28.764123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Feb 16 22:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri Feb 16 22:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Feb 16 22:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Feb 16 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.44}]},{"osd":2,"last update":"Fri Feb 16 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Feb 16 22:53:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Feb 16 22:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Feb 16 22:52:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Feb 16 22:53:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 16 22:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.434,"5min":0.505,"15min":0.505},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.28299999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.432,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Feb 16 22:52:13 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Feb 16 22:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Feb 16 22:52:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.73599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:52:49 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33200000000000002},"min":{"1min":0.27700000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.45300000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"min":{"1min":0.26300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.30499999999999999}]},{"osd":2,"last update":"Fri Feb 16 22:52:13 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Feb 16 22:52:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.504},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Feb 16 22:53:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-16T22:53:16.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-16T22:53:16.487 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:17.500 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:17 smithi064 ceph-mon[29226]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:53:17.500 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:17 smithi064 ceph-mon[29226]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:17.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:17 smithi192 ceph-mon[36254]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:53:17.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:17 smithi192 ceph-mon[36254]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:18.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 -- 172.21.15.64:0/371800277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 msgr2=0x7fad940fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 --2- 172.21.15.64:0/371800277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 0x7fad940fcf60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fad84004940 tx=0x7fad84009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 -- 172.21.15.64:0/371800277 shutdown_connections 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 --2- 172.21.15.64:0/371800277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 0x7fad940fcf60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 --2- 172.21.15.64:0/371800277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad94102f50 0x7fad94103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 -- 172.21.15.64:0/371800277 >> 172.21.15.64:0/371800277 conn(0x7fad940f8ae0 msgr2=0x7fad940faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 -- 172.21.15.64:0/371800277 shutdown_connections 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 -- 172.21.15.64:0/371800277 wait complete. 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.002+0000 7fad9ada9700 1 Processor -- start 2024-02-16T22:53:18.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad9ada9700 1 -- start start 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad9ada9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad94102f50 0x7fad94113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad9ada9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 0x7fad94114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad9ada9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad941146e0 con 0x7fad941038f0 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad9ada9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9410d980 con 0x7fad94102f50 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 0x7fad94114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 0x7fad94114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46972/0 (socket says 172.21.15.64:46972) 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad93fff700 1 -- 172.21.15.64:0/1156923277 learned_addr learned my addr 172.21.15.64:0/1156923277 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad98b45700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad94102f50 0x7fad94113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad93fff700 1 -- 172.21.15.64:0/1156923277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad94102f50 msgr2=0x7fad94113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad93fff700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad94102f50 0x7fad94113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad93fff700 1 -- 172.21.15.64:0/1156923277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad84005040 con 0x7fad941038f0 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad98b45700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad94102f50 0x7fad94113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:53:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.003+0000 7fad93fff700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 0x7fad94114000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad840090d0 tx=0x7fad84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.004+0000 7fad917fa700 1 -- 172.21.15.64:0/1156923277 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad8400e040 con 0x7fad941038f0 2024-02-16T22:53:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.004+0000 7fad917fa700 1 -- 172.21.15.64:0/1156923277 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad8400bbf0 con 0x7fad941038f0 2024-02-16T22:53:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.004+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9410dc00 con 0x7fad941038f0 2024-02-16T22:53:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.004+0000 7fad917fa700 1 -- 172.21.15.64:0/1156923277 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad8401c630 con 0x7fad941038f0 2024-02-16T22:53:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.004+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9410e140 con 0x7fad941038f0 2024-02-16T22:53:18.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.005+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad940fc6e0 con 0x7fad941038f0 2024-02-16T22:53:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.005+0000 7fad917fa700 1 -- 172.21.15.64:0/1156923277 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fad8400bd60 con 0x7fad941038f0 2024-02-16T22:53:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.005+0000 7fad917fa700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fad800692a0 0x7fad8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.006+0000 7fad917fa700 1 -- 172.21.15.64:0/1156923277 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fad840881b0 con 0x7fad941038f0 2024-02-16T22:53:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.006+0000 7fad98b45700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fad800692a0 0x7fad8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.006+0000 7fad98b45700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fad800692a0 0x7fad8006b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009a80 tx=0x7fad7c008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.010+0000 7fad917fa700 1 -- 172.21.15.64:0/1156923277 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fad840582b0 con 0x7fad941038f0 2024-02-16T22:53:18.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.196+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad9404bed0 con 0x7fad800692a0 2024-02-16T22:53:18.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.198+0000 7fad917fa700 1 -- 172.21.15.64:0/1156923277 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28130 (secure 0 0 0) 0x7fad9404bed0 con 0x7fad800692a0 2024-02-16T22:53:18.199 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fad800692a0 msgr2=0x7fad8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fad800692a0 0x7fad8006b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009a80 tx=0x7fad7c008040 comp rx=0 tx=0).stop 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 msgr2=0x7fad94114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 0x7fad94114000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad840090d0 tx=0x7fad84009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 shutdown_connections 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad94102f50 0x7fad94113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fad800692a0 0x7fad8006b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 --2- 172.21.15.64:0/1156923277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad941038f0 0x7fad94114000 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 >> 172.21.15.64:0/1156923277 conn(0x7fad940f8ae0 msgr2=0x7fad940fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:18.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 shutdown_connections 2024-02-16T22:53:18.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:18.200+0000 7fad9ada9700 1 -- 172.21.15.64:0/1156923277 wait complete. 2024-02-16T22:53:18.202 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-02-16T22:53:18.772 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-02-16T22:53:18.113578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001850"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":189,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-02-16T22:52:22.839278+0000","last_change":"2024-02-16T22:51:58.167133+0000","last_active":"2024-02-16T22:52:22.839278+0000","last_peered":"2024-02-16T22:52:22.839278+0000","last_clean":"2024-02-16T22:52:22.839278+0000","last_became_active":"2024-02-16T22:51:58.166761+0000","last_became_peered":"2024-02-16T22:51:58.166761+0000","last_unstale":"2024-02-16T22:52:22.839278+0000","last_undegraded":"2024-02-16T22:52:22.839278+0000","last_fullsized":"2024-02-16T22:52:22.839278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-16T22:51:12.814093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-16T22:51:12.814093+0000","last_clean_scrub_stamp":"2024-02-16T22:51:12.814093+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T10:38:28.764123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Feb 16 22:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri Feb 16 22:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Feb 16 22:52:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Feb 16 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.44}]},{"osd":2,"last update":"Fri Feb 16 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Feb 16 22:53:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35799999999999998,"15min":0.36399999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.184},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.23000000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.23000000000000001}]},{"osd":1,"last update":"Fri Feb 16 22:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38700000000000001,"15min":0.39400000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.505,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.40500000000000003,"15min":0.41399999999999998},"min":{"1min":0.246,"5min":0.245,"15min":0.245},"max":{"1min":0.51400000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.246}]},{"osd":3,"last update":"Fri Feb 16 22:52:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Feb 16 22:53:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 16 22:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.434,"5min":0.505,"15min":0.505},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.28299999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.432,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Feb 16 22:53:16 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.27300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.40100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.317},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.374,"15min":0.38},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.40300000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Fri Feb 16 22:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.63}]},{"osd":4,"last update":"Fri Feb 16 22:52:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:52:49 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33200000000000002},"min":{"1min":0.27700000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.45300000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"min":{"1min":0.26300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.46100000000000002}]},{"osd":2,"last update":"Fri Feb 16 22:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.504,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.504},{"interface":"front","average":{"1min":0.375,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Feb 16 22:52:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Feb 16 22:53:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-16T22:53:18.773 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-16T22:53:18.773 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-16T22:53:18.774 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-16T22:53:18.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-16T22:53:18.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:18 smithi064 ceph-mon[29226]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:18.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:18 smithi064 ceph-mon[29226]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:53:19.032 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:18 smithi192 ceph-mon[36254]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:18 smithi192 ceph-mon[36254]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:53:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 -- 172.21.15.64:0/3657424611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c8102f50 msgr2=0x7fc8c8103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 --2- 172.21.15.64:0/3657424611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c8102f50 0x7fc8c8103320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8004940 tx=0x7fc8b8009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 -- 172.21.15.64:0/3657424611 shutdown_connections 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 --2- 172.21.15.64:0/3657424611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8c81038f0 0x7fc8c80fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 --2- 172.21.15.64:0/3657424611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c8102f50 0x7fc8c8103320 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 -- 172.21.15.64:0/3657424611 >> 172.21.15.64:0/3657424611 conn(0x7fc8c80f8ae0 msgr2=0x7fc8c80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 -- 172.21.15.64:0/3657424611 shutdown_connections 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 -- 172.21.15.64:0/3657424611 wait complete. 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 Processor -- start 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.522+0000 7fc8ce168700 1 -- start start 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8ce168700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8c8102f50 0x7fc8c8113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8ce168700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c81038f0 0x7fc8c8113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8ce168700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c81146c0 con 0x7fc8c81038f0 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8ce168700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c810d960 con 0x7fc8c8102f50 2024-02-16T22:53:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c81038f0 0x7fc8c8113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c81038f0 0x7fc8c8113fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33548/0 (socket says 172.21.15.64:33548) 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8c6ffd700 1 -- 172.21.15.64:0/664564191 learned_addr learned my addr 172.21.15.64:0/664564191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8c77fe700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8c8102f50 0x7fc8c8113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8c6ffd700 1 -- 172.21.15.64:0/664564191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8c8102f50 msgr2=0x7fc8c8113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8c6ffd700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8c8102f50 0x7fc8c8113aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8c6ffd700 1 -- 172.21.15.64:0/664564191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8005040 con 0x7fc8c81038f0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8c6ffd700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c81038f0 0x7fc8c8113fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00b790 tx=0x7fc8bc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8affff700 1 -- 172.21.15.64:0/664564191 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8bc00f6f0 con 0x7fc8c81038f0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8affff700 1 -- 172.21.15.64:0/664564191 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8bc00fd20 con 0x7fc8c81038f0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c810dc40 con 0x7fc8c81038f0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8affff700 1 -- 172.21.15.64:0/664564191 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8bc0103a0 con 0x7fc8c81038f0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.523+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c810e180 con 0x7fc8c81038f0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.524+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c80fc6e0 con 0x7fc8c81038f0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.525+0000 7fc8affff700 1 -- 172.21.15.64:0/664564191 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc8bc011400 con 0x7fc8c81038f0 2024-02-16T22:53:20.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.525+0000 7fc8affff700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8b00692b0 0x7fc8b006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:20.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.525+0000 7fc8affff700 1 -- 172.21.15.64:0/664564191 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc8bc086c20 con 0x7fc8c81038f0 2024-02-16T22:53:20.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.525+0000 7fc8c77fe700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8b00692b0 0x7fc8b006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:20.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.526+0000 7fc8c77fe700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8b00692b0 0x7fc8b006b760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b3b0 tx=0x7fc8b8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:20.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.530+0000 7fc8affff700 1 -- 172.21.15.64:0/664564191 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc8bc056d20 con 0x7fc8c81038f0 2024-02-16T22:53:20.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.764+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc8c810f080 con 0x7fc8c81038f0 2024-02-16T22:53:20.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.764+0000 7fc8affff700 1 -- 172.21.15.64:0/664564191 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc8bc0568b0 con 0x7fc8c81038f0 2024-02-16T22:53:20.765 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:20.766 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-16T22:53:20.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8b00692b0 msgr2=0x7fc8b006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:20.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8b00692b0 0x7fc8b006b760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b3b0 tx=0x7fc8b8009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:20.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c81038f0 msgr2=0x7fc8c8113fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:20.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c81038f0 0x7fc8c8113fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00b790 tx=0x7fc8bc009220 comp rx=0 tx=0).stop 2024-02-16T22:53:20.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 shutdown_connections 2024-02-16T22:53:20.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8c8102f50 0x7fc8c8113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:20.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc8b00692b0 0x7fc8b006b760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:20.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 --2- 172.21.15.64:0/664564191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8c81038f0 0x7fc8c8113fe0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:20.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 >> 172.21.15.64:0/664564191 conn(0x7fc8c80f8ae0 msgr2=0x7fc8c80fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:20.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.766+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 shutdown_connections 2024-02-16T22:53:20.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:20.767+0000 7fc8ce168700 1 -- 172.21.15.64:0/664564191 wait complete. 2024-02-16T22:53:21.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:21 smithi064 ceph-mon[29226]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:21.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:21 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/664564191' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-16T22:53:21.455 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-16T22:53:21.455 INFO:tasks.cephadm:Setup complete, yielding 2024-02-16T22:53:21.455 INFO:teuthology.run_tasks:Running task print... 2024-02-16T22:53:21.465 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-02-16T22:53:21.465 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:53:21.507 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:53:21.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-02-16T22:53:21.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:21 smithi192 ceph-mon[36254]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:21.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:21 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/664564191' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-16T22:53:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:23.423 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:23 smithi064 ceph-mon[29226]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.449+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2651947599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 msgr2=0x7fcd540fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.449+0000 7fcd5b6ed700 1 --2- 172.21.15.64:0/2651947599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 0x7fcd540fd420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd44004940 tx=0x7fcd44009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.449+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2651947599 shutdown_connections 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.449+0000 7fcd5b6ed700 1 --2- 172.21.15.64:0/2651947599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 0x7fcd540fd420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.449+0000 7fcd5b6ed700 1 --2- 172.21.15.64:0/2651947599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd540fbe40 0x7fcd540fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.449+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2651947599 >> 172.21.15.64:0/2651947599 conn(0x7fcd540f7520 msgr2=0x7fcd540f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.449+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2651947599 shutdown_connections 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.449+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2651947599 wait complete. 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd5b6ed700 1 Processor -- start 2024-02-16T22:53:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd5b6ed700 1 -- start start 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd5b6ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd540fbe40 0x7fcd540fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd5b6ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 0x7fcd540fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd5b6ed700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540fea10 con 0x7fcd540fcfb0 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd5b6ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540feb80 con 0x7fcd540fbe40 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd58c88700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 0x7fcd540fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd58c88700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 0x7fcd540fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33562/0 (socket says 172.21.15.64:33562) 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.450+0000 7fcd58c88700 1 -- 172.21.15.64:0/2621273371 learned_addr learned my addr 172.21.15.64:0/2621273371 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd59489700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd540fbe40 0x7fcd540fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd58c88700 1 -- 172.21.15.64:0/2621273371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd540fbe40 msgr2=0x7fcd540fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd58c88700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd540fbe40 0x7fcd540fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd58c88700 1 -- 172.21.15.64:0/2621273371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd44005040 con 0x7fcd540fcfb0 2024-02-16T22:53:23.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd58c88700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 0x7fcd540fe380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd440093b0 tx=0x7fcd44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:23.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd49ffb700 1 -- 172.21.15.64:0/2621273371 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd4400e040 con 0x7fcd540fcfb0 2024-02-16T22:53:23.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd49ffb700 1 -- 172.21.15.64:0/2621273371 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd4400bcc0 con 0x7fcd540fcfb0 2024-02-16T22:53:23.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd540fee00 con 0x7fcd540fcfb0 2024-02-16T22:53:23.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd49ffb700 1 -- 172.21.15.64:0/2621273371 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd4401c630 con 0x7fcd540fcfb0 2024-02-16T22:53:23.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.451+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd54115950 con 0x7fcd540fcfb0 2024-02-16T22:53:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.452+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5404bed0 con 0x7fcd540fcfb0 2024-02-16T22:53:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.453+0000 7fcd49ffb700 1 -- 172.21.15.64:0/2621273371 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcd440139f0 con 0x7fcd540fcfb0 2024-02-16T22:53:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.453+0000 7fcd49ffb700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcd400692a0 0x7fcd4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.453+0000 7fcd49ffb700 1 -- 172.21.15.64:0/2621273371 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcd44088250 con 0x7fcd540fcfb0 2024-02-16T22:53:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.453+0000 7fcd59489700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcd400692a0 0x7fcd4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.453+0000 7fcd59489700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcd400692a0 0x7fcd4006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006fd0 tx=0x7fcd50008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:23.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.458+0000 7fcd49ffb700 1 -- 172.21.15.64:0/2621273371 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fcd44058350 con 0x7fcd540fcfb0 2024-02-16T22:53:23.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:23 smithi192 ceph-mon[36254]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.650+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fcd54115d50 con 0x7fcd540fcfb0 2024-02-16T22:53:23.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.658+0000 7fcd49ffb700 1 -- 172.21.15.64:0/2621273371 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fcd44057ee0 con 0x7fcd540fcfb0 2024-02-16T22:53:23.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcd400692a0 msgr2=0x7fcd4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:23.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcd400692a0 0x7fcd4006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006fd0 tx=0x7fcd50008040 comp rx=0 tx=0).stop 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 msgr2=0x7fcd540fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 0x7fcd540fe380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd440093b0 tx=0x7fcd44009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 shutdown_connections 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd540fbe40 0x7fcd540fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcd400692a0 0x7fcd4006b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 --2- 172.21.15.64:0/2621273371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd540fcfb0 0x7fcd540fe380 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 >> 172.21.15.64:0/2621273371 conn(0x7fcd540f7520 msgr2=0x7fcd54065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 shutdown_connections 2024-02-16T22:53:23.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:23.660+0000 7fcd5b6ed700 1 -- 172.21.15.64:0/2621273371 wait complete. 2024-02-16T22:53:24.561 INFO:teuthology.run_tasks:Running task print... 2024-02-16T22:53:24.570 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-02-16T22:53:24.570 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:53:24.614 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:53:24.614 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-16T22:53:24.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:24 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2621273371' entity='client.admin' 2024-02-16T22:53:24.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:53:24.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:24.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:53:24.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:24 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:24.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:24 smithi064 ceph-mon[29226]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:24.874 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:25.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:24 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2621273371' entity='client.admin' 2024-02-16T22:53:25.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:53:25.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:25.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:53:25.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:24 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:25.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:24 smithi192 ceph-mon[36254]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:26.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 -- 172.21.15.64:0/2015722799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 msgr2=0x7f6cfc0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 --2- 172.21.15.64:0/2015722799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 0x7f6cfc0fd420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4004940 tx=0x7f6ce4009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 -- 172.21.15.64:0/2015722799 shutdown_connections 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 --2- 172.21.15.64:0/2015722799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 0x7f6cfc0fd420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 --2- 172.21.15.64:0/2015722799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cfc0fbe40 0x7f6cfc0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 -- 172.21.15.64:0/2015722799 >> 172.21.15.64:0/2015722799 conn(0x7f6cfc0f74e0 msgr2=0x7f6cfc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 -- 172.21.15.64:0/2015722799 shutdown_connections 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 -- 172.21.15.64:0/2015722799 wait complete. 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.486+0000 7f6d02832700 1 Processor -- start 2024-02-16T22:53:26.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6d02832700 1 -- start start 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6d02832700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cfc0fbe40 0x7f6cfc10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6d02832700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 0x7f6cfc10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6d02832700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc10e4f0 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6d02832700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc10ed30 con 0x7f6cfc0fbe40 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 0x7f6cfc10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 0x7f6cfc10de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33582/0 (socket says 172.21.15.64:33582) 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfb7fe700 1 -- 172.21.15.64:0/3661528226 learned_addr learned my addr 172.21.15.64:0/3661528226 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfbfff700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cfc0fbe40 0x7f6cfc10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfb7fe700 1 -- 172.21.15.64:0/3661528226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cfc0fbe40 msgr2=0x7f6cfc10d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfb7fe700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cfc0fbe40 0x7f6cfc10d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfb7fe700 1 -- 172.21.15.64:0/3661528226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4005040 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfbfff700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cfc0fbe40 0x7f6cfc10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.487+0000 7f6cfb7fe700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 0x7f6cfc10de60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40093b0 tx=0x7f6ce4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.488+0000 7f6cf8ff9700 1 -- 172.21.15.64:0/3661528226 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce400e040 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.488+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cfc10efb0 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.488+0000 7f6cf8ff9700 1 -- 172.21.15.64:0/3661528226 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ce400bcc0 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.488+0000 7f6cf8ff9700 1 -- 172.21.15.64:0/3661528226 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce401c630 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.488+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc10f4f0 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.489+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cfc04bed0 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.489+0000 7f6cf8ff9700 1 -- 172.21.15.64:0/3661528226 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6ce40139f0 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.489+0000 7f6cf8ff9700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6ce80692a0 0x7f6ce806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:26.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.490+0000 7f6cf8ff9700 1 -- 172.21.15.64:0/3661528226 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6ce4088250 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.490+0000 7f6cfbfff700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6ce80692a0 0x7f6ce806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:26.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.490+0000 7f6cfbfff700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6ce80692a0 0x7f6ce806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cec006fd0 tx=0x7f6cec008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:26.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.494+0000 7f6cf8ff9700 1 -- 172.21.15.64:0/3661528226 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6ce4058350 con 0x7f6cfc0fcfb0 2024-02-16T22:53:26.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.702+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cfc10f140 con 0x7f6ce80692a0 2024-02-16T22:53:26.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.703+0000 7f6cf8ff9700 1 -- 172.21.15.64:0/3661528226 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6cfc10f140 con 0x7f6ce80692a0 2024-02-16T22:53:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:Backend: cephadm 2024-02-16T22:53:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:Available: Yes 2024-02-16T22:53:26.704 INFO:teuthology.orchestra.run.smithi064.stdout:Paused: No 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6ce80692a0 msgr2=0x7f6ce806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6ce80692a0 0x7f6ce806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cec006fd0 tx=0x7f6cec008040 comp rx=0 tx=0).stop 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 msgr2=0x7f6cfc10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 0x7f6cfc10de60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40093b0 tx=0x7f6ce4009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 shutdown_connections 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cfc0fbe40 0x7f6cfc10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6ce80692a0 0x7f6ce806b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 --2- 172.21.15.64:0/3661528226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cfc0fcfb0 0x7f6cfc10de60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 >> 172.21.15.64:0/3661528226 conn(0x7f6cfc0f74e0 msgr2=0x7f6cfc0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 shutdown_connections 2024-02-16T22:53:26.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:26.705+0000 7f6d02832700 1 -- 172.21.15.64:0/3661528226 wait complete. 2024-02-16T22:53:27.283 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:27 smithi064 ceph-mon[29226]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:27.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-16T22:53:27.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:27 smithi192 ceph-mon[36254]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:27.587 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:28.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:28 smithi192 ceph-mon[36254]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:28.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:53:28.577 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:28 smithi064 ceph-mon[29226]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:28.577 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:53:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.200+0000 7f89caac5700 1 -- 172.21.15.64:0/1981104467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4100060 msgr2=0x7f89c4102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.200+0000 7f89caac5700 1 --2- 172.21.15.64:0/1981104467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4100060 0x7f89c4102480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f89b0004940 tx=0x7f89b0009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.200+0000 7f89caac5700 1 -- 172.21.15.64:0/1981104467 shutdown_connections 2024-02-16T22:53:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.200+0000 7f89caac5700 1 --2- 172.21.15.64:0/1981104467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4100060 0x7f89c4102480 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.200+0000 7f89caac5700 1 --2- 172.21.15.64:0/1981104467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89c4066c50 0x7f89c40ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 -- 172.21.15.64:0/1981104467 >> 172.21.15.64:0/1981104467 conn(0x7f89c40f7590 msgr2=0x7f89c40f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 -- 172.21.15.64:0/1981104467 shutdown_connections 2024-02-16T22:53:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 -- 172.21.15.64:0/1981104467 wait complete. 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 Processor -- start 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 -- start start 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4066c50 0x7f89c410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89c4100060 0x7f89c410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c410e580 con 0x7f89c4066c50 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89caac5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c410edc0 con 0x7f89c4100060 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89c8861700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4066c50 0x7f89c410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89c8861700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4066c50 0x7f89c410d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33600/0 (socket says 172.21.15.64:33600) 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.201+0000 7f89c8861700 1 -- 172.21.15.64:0/3341318437 learned_addr learned my addr 172.21.15.64:0/3341318437 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89c8861700 1 -- 172.21.15.64:0/3341318437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89c4100060 msgr2=0x7f89c410def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89c8861700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89c4100060 0x7f89c410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89c8861700 1 -- 172.21.15.64:0/3341318437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b0005040 con 0x7f89c4066c50 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89c8861700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4066c50 0x7f89c410d9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f89b800b790 tx=0x7f89b8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89c17fa700 1 -- 172.21.15.64:0/3341318437 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89b800f6f0 con 0x7f89c4066c50 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89c17fa700 1 -- 172.21.15.64:0/3341318437 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89b800fd20 con 0x7f89c4066c50 2024-02-16T22:53:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c410f0a0 con 0x7f89c4066c50 2024-02-16T22:53:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89c17fa700 1 -- 172.21.15.64:0/3341318437 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89b80103a0 con 0x7f89c4066c50 2024-02-16T22:53:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.202+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c410f5e0 con 0x7f89c4066c50 2024-02-16T22:53:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.203+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c404bed0 con 0x7f89c4066c50 2024-02-16T22:53:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.203+0000 7f89c17fa700 1 -- 172.21.15.64:0/3341318437 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f89b8011400 con 0x7f89c4066c50 2024-02-16T22:53:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.204+0000 7f89c17fa700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f89b40692a0 0x7f89b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.204+0000 7f89c17fa700 1 -- 172.21.15.64:0/3341318437 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f89b8086c40 con 0x7f89c4066c50 2024-02-16T22:53:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.204+0000 7f89c3fff700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f89b40692a0 0x7f89b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.204+0000 7f89c3fff700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f89b40692a0 0x7f89b406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89b0004f90 tx=0x7f89b0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:29.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.211+0000 7f89c17fa700 1 -- 172.21.15.64:0/3341318437 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f89b8056d40 con 0x7f89c4066c50 2024-02-16T22:53:29.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.412+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f89c4100cc0 con 0x7f89b40692a0 2024-02-16T22:53:29.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.421+0000 7f89c17fa700 1 -- 172.21.15.64:0/3341318437 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f89c4100cc0 con 0x7f89b40692a0 2024-02-16T22:53:29.422 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:53:29.422 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (3m) 2m ago 5m 26.1M - 0.25.0 c8568f914cd2 829b066b8c26 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (5m) 2m ago 5m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (4m) 68s ago 4m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (3m) 2m ago 4m 76.8M - 9.4.7 954c08fa6188 529226ab33c5 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:9283 running (6m) 2m ago 6m 472M - 17.2.7-513-gb88fae66 870a0944ccb8 7189c74ed7fc 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (4m) 68s ago 4m 432M - 17.2.7-513-gb88fae66 870a0944ccb8 a017e081156a 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (6m) 2m ago 6m 41.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (4m) 68s ago 4m 38.1M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 c25d924fa381 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (4m) 68s ago 4m 16.4M - 1.5.0 0da6a335fe13 c32798cb2969 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (3m) 2m ago 3m 39.0M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (2m) 2m ago 2m 38.9M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (2m) 2m ago 2m 39.1M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (2m) 68s ago 2m 38.5M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (98s) 68s ago 97s 37.6M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (73s) 68s ago 73s 10.5M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:53:29.423 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (3m) 2m ago 4m 35.0M - 2.43.0 a07b618ecd1d dd20eff0542c 2024-02-16T22:53:29.424 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:29 smithi064 ceph-mon[29226]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:29.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f89b40692a0 msgr2=0x7f89b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:29.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f89b40692a0 0x7f89b406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89b0004f90 tx=0x7f89b0009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:29.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4066c50 msgr2=0x7f89c410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:29.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4066c50 0x7f89c410d9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f89b800b790 tx=0x7f89b8009220 comp rx=0 tx=0).stop 2024-02-16T22:53:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 shutdown_connections 2024-02-16T22:53:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f89b40692a0 0x7f89b406b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4066c50 0x7f89c410d9b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 --2- 172.21.15.64:0/3341318437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89c4100060 0x7f89c410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 >> 172.21.15.64:0/3341318437 conn(0x7f89c40f7590 msgr2=0x7f89c40f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 shutdown_connections 2024-02-16T22:53:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:29.423+0000 7f89caac5700 1 -- 172.21.15.64:0/3341318437 wait complete. 2024-02-16T22:53:29.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:29 smithi192 ceph-mon[36254]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:30.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-16T22:53:30.406 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:30.528 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:30 smithi064 ceph-mon[29226]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:30.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:30 smithi192 ceph-mon[36254]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:31.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:31 smithi064 ceph-mon[29226]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:31.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:31 smithi192 ceph-mon[36254]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.822+0000 7fa1ae41f700 1 -- 172.21.15.64:0/2247632787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 msgr2=0x7fa1a80fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.822+0000 7fa1ae41f700 1 --2- 172.21.15.64:0/2247632787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 0x7fa1a80fd420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa19c004940 tx=0x7fa19c009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.823+0000 7fa1ae41f700 1 -- 172.21.15.64:0/2247632787 shutdown_connections 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.823+0000 7fa1ae41f700 1 --2- 172.21.15.64:0/2247632787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 0x7fa1a80fd420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.823+0000 7fa1ae41f700 1 --2- 172.21.15.64:0/2247632787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a80fbe40 0x7fa1a80fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.823+0000 7fa1ae41f700 1 -- 172.21.15.64:0/2247632787 >> 172.21.15.64:0/2247632787 conn(0x7fa1a80f74e0 msgr2=0x7fa1a80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.823+0000 7fa1ae41f700 1 -- 172.21.15.64:0/2247632787 shutdown_connections 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.823+0000 7fa1ae41f700 1 -- 172.21.15.64:0/2247632787 wait complete. 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.823+0000 7fa1ae41f700 1 Processor -- start 2024-02-16T22:53:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.823+0000 7fa1ae41f700 1 -- start start 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1ae41f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a80fbe40 0x7fa1a810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1ae41f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 0x7fa1a810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1ae41f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a810e4a0 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1ae41f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a810ed40 con 0x7fa1a80fbe40 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 0x7fa1a810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 0x7fa1a810dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46292/0 (socket says 172.21.15.64:46292) 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a77fe700 1 -- 172.21.15.64:0/1328822049 learned_addr learned my addr 172.21.15.64:0/1328822049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a7fff700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a80fbe40 0x7fa1a810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a77fe700 1 -- 172.21.15.64:0/1328822049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a80fbe40 msgr2=0x7fa1a810d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a77fe700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a80fbe40 0x7fa1a810d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a77fe700 1 -- 172.21.15.64:0/1328822049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c005040 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a77fe700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 0x7fa1a810dea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0093b0 tx=0x7fa19c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a4ff9700 1 -- 172.21.15.64:0/1328822049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19c00e040 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a4ff9700 1 -- 172.21.15.64:0/1328822049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa19c00bcc0 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a810efc0 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1a4ff9700 1 -- 172.21.15.64:0/1328822049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19c01c630 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.824+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a810f500 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.825+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a804bed0 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.826+0000 7fa1a4ff9700 1 -- 172.21.15.64:0/1328822049 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa19c013980 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.826+0000 7fa1a4ff9700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1900692a0 0x7fa19006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.826+0000 7fa1a4ff9700 1 -- 172.21.15.64:0/1328822049 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa19c088110 con 0x7fa1a80fcfb0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.826+0000 7fa1a7fff700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1900692a0 0x7fa19006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.827+0000 7fa1a7fff700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1900692a0 0x7fa19006b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa1980041d0 tx=0x7fa1980043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:31.831+0000 7fa1a4ff9700 1 -- 172.21.15.64:0/1328822049 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa19c0582b0 con 0x7fa1a80fcfb0 2024-02-16T22:53:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.034+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1a810f9a0 con 0x7fa1900692a0 2024-02-16T22:53:32.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.038+0000 7fa1a4ff9700 1 -- 172.21.15.64:0/1328822049 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7fa1a810f9a0 con 0x7fa1900692a0 2024-02-16T22:53:32.039 INFO:teuthology.orchestra.run.smithi064.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-16T22:53:32.039 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-02-16T22:53:32.040 INFO:teuthology.orchestra.run.smithi064.stdout:crash 2/2 2m ago 5m * 2024-02-16T22:53:32.040 INFO:teuthology.orchestra.run.smithi064.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-02-16T22:53:32.040 INFO:teuthology.orchestra.run.smithi064.stdout:mgr 2/2 2m ago 5m count:2 2024-02-16T22:53:32.040 INFO:teuthology.orchestra.run.smithi064.stdout:mon 2/2 2m ago 5m smithi064:172.21.15.64=smithi064;smithi192:172.21.15.192=smithi192;count:2 2024-02-16T22:53:32.040 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-02-16T22:53:32.040 INFO:teuthology.orchestra.run.smithi064.stdout:osd 6 2m ago - 2024-02-16T22:53:32.040 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-02-16T22:53:32.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1900692a0 msgr2=0x7fa19006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1900692a0 0x7fa19006b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa1980041d0 tx=0x7fa1980043c0 comp rx=0 tx=0).stop 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 msgr2=0x7fa1a810dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 0x7fa1a810dea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0093b0 tx=0x7fa19c009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 shutdown_connections 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a80fbe40 0x7fa1a810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1900692a0 0x7fa19006b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 --2- 172.21.15.64:0/1328822049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1a80fcfb0 0x7fa1a810dea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 >> 172.21.15.64:0/1328822049 conn(0x7fa1a80f74e0 msgr2=0x7fa1a80656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 shutdown_connections 2024-02-16T22:53:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:32.040+0000 7fa1ae41f700 1 -- 172.21.15.64:0/1328822049 wait complete. 2024-02-16T22:53:32.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-16T22:53:32.856 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:33.263 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:33 smithi064 ceph-mon[29226]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:33.263 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:33 smithi064 ceph-mon[29226]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:33.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:33 smithi192 ceph-mon[36254]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:33.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:33 smithi192 ceph-mon[36254]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.269+0000 7f214652f700 1 -- 172.21.15.64:0/4056773543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140100060 msgr2=0x7f2140102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.269+0000 7f214652f700 1 --2- 172.21.15.64:0/4056773543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140100060 0x7f2140102440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2130004940 tx=0x7f2130009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.269+0000 7f214652f700 1 -- 172.21.15.64:0/4056773543 shutdown_connections 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.269+0000 7f214652f700 1 --2- 172.21.15.64:0/4056773543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140100060 0x7f2140102440 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.269+0000 7f214652f700 1 --2- 172.21.15.64:0/4056773543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2140066340 0x7f21400ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.269+0000 7f214652f700 1 -- 172.21.15.64:0/4056773543 >> 172.21.15.64:0/4056773543 conn(0x7f21400f7530 msgr2=0x7f21400f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.269+0000 7f214652f700 1 -- 172.21.15.64:0/4056773543 shutdown_connections 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.269+0000 7f214652f700 1 -- 172.21.15.64:0/4056773543 wait complete. 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f214652f700 1 Processor -- start 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f214652f700 1 -- start start 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f214652f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140066340 0x7f214010d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f214652f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2140100060 0x7f214010dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f214652f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214010e390 con 0x7f2140066340 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f214652f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214010ec30 con 0x7f2140100060 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f213ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140066340 0x7f214010d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f213ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140066340 0x7f214010d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46300/0 (socket says 172.21.15.64:46300) 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f213ffff700 1 -- 172.21.15.64:0/2165479985 learned_addr learned my addr 172.21.15.64:0/2165479985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f213f7fe700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2140100060 0x7f214010dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f213ffff700 1 -- 172.21.15.64:0/2165479985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2140100060 msgr2=0x7f214010dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f213ffff700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2140100060 0x7f214010dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.270+0000 7f213ffff700 1 -- 172.21.15.64:0/2165479985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130005040 con 0x7f2140066340 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.271+0000 7f213ffff700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140066340 0x7f214010d850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2128002a80 tx=0x7f2128004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.271+0000 7f213cff9700 1 -- 172.21.15.64:0/2165479985 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2128007cc0 con 0x7f2140066340 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.271+0000 7f213cff9700 1 -- 172.21.15.64:0/2165479985 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f212800f040 con 0x7f2140066340 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.271+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214010ef10 con 0x7f2140066340 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.271+0000 7f213cff9700 1 -- 172.21.15.64:0/2165479985 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f212800da30 con 0x7f2140066340 2024-02-16T22:53:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.271+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214010f450 con 0x7f2140066340 2024-02-16T22:53:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.272+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21401079f0 con 0x7f2140066340 2024-02-16T22:53:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.272+0000 7f213cff9700 1 -- 172.21.15.64:0/2165479985 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f212800db90 con 0x7f2140066340 2024-02-16T22:53:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.273+0000 7f213cff9700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f212c0692a0 0x7f212c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.273+0000 7f213cff9700 1 -- 172.21.15.64:0/2165479985 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2128086c30 con 0x7f2140066340 2024-02-16T22:53:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.273+0000 7f213f7fe700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f212c0692a0 0x7f212c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.273+0000 7f213f7fe700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f212c0692a0 0x7f212c06b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f213000b3b0 tx=0x7f2130009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.277+0000 7f213cff9700 1 -- 172.21.15.64:0/2165479985 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2128056cb0 con 0x7f2140066340 2024-02-16T22:53:34.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.476+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f214010f7e0 con 0x7f212c0692a0 2024-02-16T22:53:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.477+0000 7f213cff9700 1 -- 172.21.15.64:0/2165479985 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f214010f7e0 con 0x7f212c0692a0 2024-02-16T22:53:34.478 INFO:teuthology.orchestra.run.smithi064.stdout:HOST ADDR LABELS STATUS 2024-02-16T22:53:34.478 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 172.21.15.64 2024-02-16T22:53:34.478 INFO:teuthology.orchestra.run.smithi064.stdout:smithi192 172.21.15.192 2024-02-16T22:53:34.478 INFO:teuthology.orchestra.run.smithi064.stdout:2 hosts in cluster 2024-02-16T22:53:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f212c0692a0 msgr2=0x7f212c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f212c0692a0 0x7f212c06b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f213000b3b0 tx=0x7f2130009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140066340 msgr2=0x7f214010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140066340 0x7f214010d850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2128002a80 tx=0x7f2128004600 comp rx=0 tx=0).stop 2024-02-16T22:53:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 shutdown_connections 2024-02-16T22:53:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f212c0692a0 0x7f212c06b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2140066340 0x7f214010d850 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:34.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 --2- 172.21.15.64:0/2165479985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2140100060 0x7f214010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:34.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 >> 172.21.15.64:0/2165479985 conn(0x7f21400f7530 msgr2=0x7f21400f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:34.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 shutdown_connections 2024-02-16T22:53:34.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:34.479+0000 7f214652f700 1 -- 172.21.15.64:0/2165479985 wait complete. 2024-02-16T22:53:35.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:35 smithi064 ceph-mon[29226]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:35.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-16T22:53:35.508 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:35.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:35 smithi192 ceph-mon[36254]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:36 smithi192 ceph-mon[36254]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:36.547 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:36 smithi064 ceph-mon[29226]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:37.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 -- 172.21.15.64:0/3484029534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05280663c0 msgr2=0x7f05280ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:37.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 --2- 172.21.15.64:0/3484029534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05280663c0 0x7f05280ffb10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f051c004940 tx=0x7f051c009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:37.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 -- 172.21.15.64:0/3484029534 shutdown_connections 2024-02-16T22:53:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 --2- 172.21.15.64:0/3484029534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05281000e0 0x7f0528102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 --2- 172.21.15.64:0/3484029534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05280663c0 0x7f05280ffb10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 -- 172.21.15.64:0/3484029534 >> 172.21.15.64:0/3484029534 conn(0x7f05280f74e0 msgr2=0x7f05280f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 -- 172.21.15.64:0/3484029534 shutdown_connections 2024-02-16T22:53:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 -- 172.21.15.64:0/3484029534 wait complete. 2024-02-16T22:53:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 Processor -- start 2024-02-16T22:53:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.075+0000 7f052efac700 1 -- start start 2024-02-16T22:53:37.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f052efac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05280663c0 0x7f05280fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:37.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f052efac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05281000e0 0x7f05280fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:37.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f052efac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05280fc010 con 0x7f05281000e0 2024-02-16T22:53:37.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f052efac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05280fc180 con 0x7f05280663c0 2024-02-16T22:53:37.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f0527fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05281000e0 0x7f05280fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:37.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f0527fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05281000e0 0x7f05280fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46310/0 (socket says 172.21.15.64:46310) 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f0527fff700 1 -- 172.21.15.64:0/1138679357 learned_addr learned my addr 172.21.15.64:0/1138679357 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f052cd48700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05280663c0 0x7f05280fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f0527fff700 1 -- 172.21.15.64:0/1138679357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05280663c0 msgr2=0x7f05280fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f0527fff700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05280663c0 0x7f05280fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f0527fff700 1 -- 172.21.15.64:0/1138679357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c005040 con 0x7f05281000e0 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f0527fff700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05281000e0 0x7f05280fba10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0514002a50 tx=0x7f0514004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.076+0000 7f05257fa700 1 -- 172.21.15.64:0/1138679357 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0514007cc0 con 0x7f05281000e0 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.077+0000 7f05257fa700 1 -- 172.21.15.64:0/1138679357 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f051400f040 con 0x7f05281000e0 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.077+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05280fc460 con 0x7f05281000e0 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.077+0000 7f05257fa700 1 -- 172.21.15.64:0/1138679357 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f051400d9c0 con 0x7f05281000e0 2024-02-16T22:53:37.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.077+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0528115980 con 0x7f05281000e0 2024-02-16T22:53:37.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.078+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0528107a70 con 0x7f05281000e0 2024-02-16T22:53:37.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.078+0000 7f05257fa700 1 -- 172.21.15.64:0/1138679357 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0514009730 con 0x7f05281000e0 2024-02-16T22:53:37.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.078+0000 7f05257fa700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05180692a0 0x7f051806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:37.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.079+0000 7f05257fa700 1 -- 172.21.15.64:0/1138679357 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0514086bf0 con 0x7f05281000e0 2024-02-16T22:53:37.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.079+0000 7f052cd48700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05180692a0 0x7f051806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:37.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.079+0000 7f052cd48700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05180692a0 0x7f051806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f051c004890 tx=0x7f051c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:37.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.083+0000 7f05257fa700 1 -- 172.21.15.64:0/1138679357 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0514056c70 con 0x7f05281000e0 2024-02-16T22:53:37.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.284+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f052802b310 con 0x7f05180692a0 2024-02-16T22:53:37.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.285+0000 7f05257fa700 1 -- 172.21.15.64:0/1138679357 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f052802b310 con 0x7f05180692a0 2024-02-16T22:53:37.287 INFO:teuthology.orchestra.run.smithi064.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-16T22:53:37.287 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-16T22:53:37.287 INFO:teuthology.orchestra.run.smithi064.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 73s ago Has a FileSystem, LVM detected 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05180692a0 msgr2=0x7f051806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05180692a0 0x7f051806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f051c004890 tx=0x7f051c009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05281000e0 msgr2=0x7f05280fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05281000e0 0x7f05280fba10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0514002a50 tx=0x7f0514004600 comp rx=0 tx=0).stop 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 shutdown_connections 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05280663c0 0x7f05280fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f05180692a0 0x7f051806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 --2- 172.21.15.64:0/1138679357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05281000e0 0x7f05280fba10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.287+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 >> 172.21.15.64:0/1138679357 conn(0x7f05280f74e0 msgr2=0x7f05280f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.288+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 shutdown_connections 2024-02-16T22:53:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:37.288+0000 7f052efac700 1 -- 172.21.15.64:0/1138679357 wait complete. 2024-02-16T22:53:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:37 smithi192 ceph-mon[36254]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:37.561 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:37 smithi064 ceph-mon[29226]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:37.945 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-16T22:53:37.955 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:53:37.996 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:53:37.996 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-16T22:53:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:38.512 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:38 smithi064 ceph-mon[29226]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:38 smithi192 ceph-mon[36254]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:39.503 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:39 smithi064 ceph-mon[29226]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:39 smithi192 ceph-mon[36254]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.815+0000 7fe616539700 1 -- 172.21.15.64:0/1072839856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 msgr2=0x7fe6100fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.815+0000 7fe616539700 1 --2- 172.21.15.64:0/1072839856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 0x7fe6100fe550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe604004940 tx=0x7fe604009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.815+0000 7fe616539700 1 -- 172.21.15.64:0/1072839856 shutdown_connections 2024-02-16T22:53:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.815+0000 7fe616539700 1 --2- 172.21.15.64:0/1072839856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 0x7fe6100fe550 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.815+0000 7fe616539700 1 --2- 172.21.15.64:0/1072839856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6100fcef0 0x7fe6100fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.815+0000 7fe616539700 1 -- 172.21.15.64:0/1072839856 >> 172.21.15.64:0/1072839856 conn(0x7fe6100f8620 msgr2=0x7fe6100faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.815+0000 7fe616539700 1 -- 172.21.15.64:0/1072839856 shutdown_connections 2024-02-16T22:53:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.815+0000 7fe616539700 1 -- 172.21.15.64:0/1072839856 wait complete. 2024-02-16T22:53:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe616539700 1 Processor -- start 2024-02-16T22:53:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe616539700 1 -- start start 2024-02-16T22:53:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe616539700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6100fcef0 0x7fe610071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe616539700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 0x7fe6100702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe616539700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe610070970 con 0x7fe6100fe0e0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe616539700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe610070ae0 con 0x7fe6100fcef0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe60f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 0x7fe6100702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe60f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 0x7fe6100702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46326/0 (socket says 172.21.15.64:46326) 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.816+0000 7fe60f7fe700 1 -- 172.21.15.64:0/3318597564 learned_addr learned my addr 172.21.15.64:0/3318597564 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe60ffff700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6100fcef0 0x7fe610071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe60f7fe700 1 -- 172.21.15.64:0/3318597564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6100fcef0 msgr2=0x7fe610071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe60f7fe700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6100fcef0 0x7fe610071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe60f7fe700 1 -- 172.21.15.64:0/3318597564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604005040 con 0x7fe6100fe0e0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe60f7fe700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 0x7fe6100702e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe6040093b0 tx=0x7fe604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe60cff9700 1 -- 172.21.15.64:0/3318597564 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60400e040 con 0x7fe6100fe0e0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe610070d60 con 0x7fe6100fe0e0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe60cff9700 1 -- 172.21.15.64:0/3318597564 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe60400bcc0 con 0x7fe6100fe0e0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe60cff9700 1 -- 172.21.15.64:0/3318597564 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60401c630 con 0x7fe6100fe0e0 2024-02-16T22:53:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.817+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6101159c0 con 0x7fe6100fe0e0 2024-02-16T22:53:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.818+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61004bed0 con 0x7fe6100fe0e0 2024-02-16T22:53:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.818+0000 7fe60cff9700 1 -- 172.21.15.64:0/3318597564 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe6040139f0 con 0x7fe6100fe0e0 2024-02-16T22:53:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.819+0000 7fe60cff9700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe5f80692a0 0x7fe5f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.819+0000 7fe60cff9700 1 -- 172.21.15.64:0/3318597564 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe604088250 con 0x7fe6100fe0e0 2024-02-16T22:53:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.819+0000 7fe60ffff700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe5f80692a0 0x7fe5f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.819+0000 7fe60ffff700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe5f80692a0 0x7fe5f806b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe600006fd0 tx=0x7fe600008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:39.823+0000 7fe60cff9700 1 -- 172.21.15.64:0/3318597564 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe604058350 con 0x7fe6100fe0e0 2024-02-16T22:53:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:40.044+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe610102ac0 con 0x7fe5f80692a0 2024-02-16T22:53:40.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:40 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-16T22:53:41.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:40 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-16T22:53:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.382+0000 7fe60cff9700 1 -- 172.21.15.64:0/3318597564 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe610102ac0 con 0x7fe5f80692a0 2024-02-16T22:53:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe5f80692a0 msgr2=0x7fe5f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe5f80692a0 0x7fe5f806b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe600006fd0 tx=0x7fe600008040 comp rx=0 tx=0).stop 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 msgr2=0x7fe6100702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 0x7fe6100702e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe6040093b0 tx=0x7fe604009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 shutdown_connections 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6100fcef0 0x7fe610071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe5f80692a0 0x7fe5f806b750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 --2- 172.21.15.64:0/3318597564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6100fe0e0 0x7fe6100702e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.385+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 >> 172.21.15.64:0/3318597564 conn(0x7fe6100f8620 msgr2=0x7fe6101013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.386+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 shutdown_connections 2024-02-16T22:53:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:41.386+0000 7fe616539700 1 -- 172.21.15.64:0/3318597564 wait complete. 2024-02-16T22:53:41.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[29222]: 2024-02-16T22:53:41.360+0000 7f29b29b6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:53:41.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: osdmap e42: 6 total, 6 up, 6 in 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: osdmap e43: 6 total, 6 up, 6 in 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: osdmap e44: 6 total, 6 up, 6 in 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: fsmap cephfs:0 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: Saving service mds.cephfs spec with placement count:4 2024-02-16T22:53:41.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:41.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:41 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:53:41.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T22:53:42.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:53:42.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:42.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-16T22:53:42.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: osdmap e42: 6 total, 6 up, 6 in 2024-02-16T22:53:42.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: osdmap e43: 6 total, 6 up, 6 in 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: osdmap e44: 6 total, 6 up, 6 in 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: fsmap cephfs:0 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: Saving service mds.cephfs spec with placement count:4 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:42.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:41 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:53:42.264 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:42.705 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: pgmap v174: 65 pgs: 4 creating+peering, 6 active+clean, 55 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:42.705 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:42.705 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:53:42.705 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:42.705 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.tqesxx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:53:42.705 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.tqesxx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:53:42.706 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:42.706 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: Deploying daemon mds.cephfs.smithi064.tqesxx on smithi064 2024-02-16T22:53:42.706 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:42 smithi064 ceph-mon[29226]: osdmap e45: 6 total, 6 up, 6 in 2024-02-16T22:53:43.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: pgmap v174: 65 pgs: 4 creating+peering, 6 active+clean, 55 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:43.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:43.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:53:43.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:43.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.tqesxx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:53:43.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.tqesxx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:53:43.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:43.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: Deploying daemon mds.cephfs.smithi064.tqesxx on smithi064 2024-02-16T22:53:43.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:42 smithi192 ceph-mon[36254]: osdmap e45: 6 total, 6 up, 6 in 2024-02-16T22:53:44.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.022+0000 7f04cb772700 1 -- 172.21.15.64:0/1082101168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 msgr2=0x7f04c40fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:44.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.022+0000 7f04cb772700 1 --2- 172.21.15.64:0/1082101168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 0x7f04c40fd350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f04c0004940 tx=0x7f04c0009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:44.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.022+0000 7f04cb772700 1 -- 172.21.15.64:0/1082101168 shutdown_connections 2024-02-16T22:53:44.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.022+0000 7f04cb772700 1 --2- 172.21.15.64:0/1082101168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c40fe1f0 0x7f04c40fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:44.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.022+0000 7f04cb772700 1 --2- 172.21.15.64:0/1082101168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 0x7f04c40fd350 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:44.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.022+0000 7f04cb772700 1 -- 172.21.15.64:0/1082101168 >> 172.21.15.64:0/1082101168 conn(0x7f04c40f8680 msgr2=0x7f04c40faad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:44.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.022+0000 7f04cb772700 1 -- 172.21.15.64:0/1082101168 shutdown_connections 2024-02-16T22:53:44.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.022+0000 7f04cb772700 1 -- 172.21.15.64:0/1082101168 wait complete. 2024-02-16T22:53:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04cb772700 1 Processor -- start 2024-02-16T22:53:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04cb772700 1 -- start start 2024-02-16T22:53:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04cb772700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 0x7f04c410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04cb772700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c40fe1f0 0x7f04c410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04cb772700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c410e4d0 con 0x7f04c40fcf40 2024-02-16T22:53:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04cb772700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c410ed70 con 0x7f04c40fe1f0 2024-02-16T22:53:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04c950e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 0x7f04c410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04c950e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 0x7f04c410d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51118/0 (socket says 172.21.15.64:51118) 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04c950e700 1 -- 172.21.15.64:0/4018784988 learned_addr learned my addr 172.21.15.64:0/4018784988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.023+0000 7f04c8d0d700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c40fe1f0 0x7f04c410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04c950e700 1 -- 172.21.15.64:0/4018784988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c40fe1f0 msgr2=0x7f04c410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04c950e700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c40fe1f0 0x7f04c410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04c950e700 1 -- 172.21.15.64:0/4018784988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c0005040 con 0x7f04c40fcf40 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04c950e700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 0x7f04c410d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f04c000b3b0 tx=0x7f04c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04b9ffb700 1 -- 172.21.15.64:0/4018784988 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c000e040 con 0x7f04c40fcf40 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04b9ffb700 1 -- 172.21.15.64:0/4018784988 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04c0009b10 con 0x7f04c40fcf40 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c410eff0 con 0x7f04c40fcf40 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04b9ffb700 1 -- 172.21.15.64:0/4018784988 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c0013ea0 con 0x7f04c40fcf40 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.024+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c410f530 con 0x7f04c40fcf40 2024-02-16T22:53:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.025+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c404bed0 con 0x7f04c40fcf40 2024-02-16T22:53:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.026+0000 7f04b9ffb700 1 -- 172.21.15.64:0/4018784988 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f04c000bcc0 con 0x7f04c40fcf40 2024-02-16T22:53:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.026+0000 7f04b9ffb700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f04b00692a0 0x7f04b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.026+0000 7f04b9ffb700 1 -- 172.21.15.64:0/4018784988 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f04c00884a0 con 0x7f04c40fcf40 2024-02-16T22:53:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.026+0000 7f04c8d0d700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f04b00692a0 0x7f04b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.027+0000 7f04c8d0d700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f04b00692a0 0x7f04b006b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f04b4004f40 tx=0x7f04b400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:44.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.031+0000 7f04b9ffb700 1 -- 172.21.15.64:0/4018784988 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f04c0058300 con 0x7f04c40fcf40 2024-02-16T22:53:44.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:44 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:44.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:44 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:53:44.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:44 smithi064 ceph-mon[29226]: osdmap e46: 6 total, 6 up, 6 in 2024-02-16T22:53:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.277+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f04c4105a30 con 0x7f04c40fcf40 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.278+0000 7f04b9ffb700 1 -- 172.21.15.64:0/4018784988 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f04c0057e90 con 0x7f04c40fcf40 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:e2 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 2 2024-02-16T22:53:44.279 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:41.360957+0000 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:in 2024-02-16T22:53:44.280 INFO:teuthology.orchestra.run.smithi064.stdout:up {} 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data disabled 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 0 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:44.281 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.279+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f04b00692a0 msgr2=0x7f04b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.279+0000 7f04cb772700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f04b00692a0 0x7f04b006b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f04b4004f40 tx=0x7f04b400b040 comp rx=0 tx=0).stop 2024-02-16T22:53:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.279+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 msgr2=0x7f04c410d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.279+0000 7f04cb772700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 0x7f04c410d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f04c000b3b0 tx=0x7f04c0009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.280+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 shutdown_connections 2024-02-16T22:53:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.280+0000 7f04cb772700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f04b00692a0 0x7f04b006b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.280+0000 7f04cb772700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c40fcf40 0x7f04c410d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.280+0000 7f04cb772700 1 --2- 172.21.15.64:0/4018784988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c40fe1f0 0x7f04c410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.280+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 >> 172.21.15.64:0/4018784988 conn(0x7f04c40f8680 msgr2=0x7f04c4101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.280+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 shutdown_connections 2024-02-16T22:53:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:44.280+0000 7f04cb772700 1 -- 172.21.15.64:0/4018784988 wait complete. 2024-02-16T22:53:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2024-02-16T22:53:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:44 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:44 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:53:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:44 smithi192 ceph-mon[36254]: osdmap e46: 6 total, 6 up, 6 in 2024-02-16T22:53:44.918 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:53:44.928 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:53:44.928 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-02-16T22:53:45.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:45 smithi064 ceph-mon[29226]: pgmap v177: 65 pgs: 10 creating+peering, 28 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:45.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:45 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/4018784988' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:53:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:45.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:45 smithi192 ceph-mon[36254]: pgmap v177: 65 pgs: 10 creating+peering, 28 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:45.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:45 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/4018784988' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:53:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.053+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/3532362621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd0066c70 msgr2=0x7fcbd00ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.053+0000 7fcbd4a0f700 1 --2- 172.21.15.64:0/3532362621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd0066c70 0x7fcbd00ffb00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0004940 tx=0x7fcbc0009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.053+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/3532362621 shutdown_connections 2024-02-16T22:53:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.053+0000 7fcbd4a0f700 1 --2- 172.21.15.64:0/3532362621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbd01000d0 0x7fcbd0102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.053+0000 7fcbd4a0f700 1 --2- 172.21.15.64:0/3532362621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd0066c70 0x7fcbd00ffb00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.053+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/3532362621 >> 172.21.15.64:0/3532362621 conn(0x7fcbd00f7510 msgr2=0x7fcbd00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.053+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/3532362621 shutdown_connections 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.053+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/3532362621 wait complete. 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbd4a0f700 1 Processor -- start 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbd4a0f700 1 -- start start 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbd4a0f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbd0066c70 0x7fcbd00fd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbd4a0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd01000d0 0x7fcbd00fba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbd4a0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00fc000 con 0x7fcbd01000d0 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbd4a0f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00fc170 con 0x7fcbd0066c70 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbcdd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd01000d0 0x7fcbd00fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbcdd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd01000d0 0x7fcbd00fba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51134/0 (socket says 172.21.15.64:51134) 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbcdd9b700 1 -- 172.21.15.64:0/2278823306 learned_addr learned my addr 172.21.15.64:0/2278823306 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.054+0000 7fcbce59c700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbd0066c70 0x7fcbd00fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbcdd9b700 1 -- 172.21.15.64:0/2278823306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbd0066c70 msgr2=0x7fcbd00fd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbcdd9b700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbd0066c70 0x7fcbd00fd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbcdd9b700 1 -- 172.21.15.64:0/2278823306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc0005040 con 0x7fcbd01000d0 2024-02-16T22:53:47.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbcdd9b700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd01000d0 0x7fcbd00fba00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4002a50 tx=0x7fcbc4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:47.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbbeffd700 1 -- 172.21.15.64:0/2278823306 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbc4007cc0 con 0x7fcbd01000d0 2024-02-16T22:53:47.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd00fc450 con 0x7fcbd01000d0 2024-02-16T22:53:47.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbbeffd700 1 -- 172.21.15.64:0/2278823306 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbc400f040 con 0x7fcbd01000d0 2024-02-16T22:53:47.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbbeffd700 1 -- 172.21.15.64:0/2278823306 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbc400d9c0 con 0x7fcbd01000d0 2024-02-16T22:53:47.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.055+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd01159b0 con 0x7fcbd01000d0 2024-02-16T22:53:47.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.056+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd0107a80 con 0x7fcbd01000d0 2024-02-16T22:53:47.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.056+0000 7fcbbeffd700 1 -- 172.21.15.64:0/2278823306 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcbc4009730 con 0x7fcbd01000d0 2024-02-16T22:53:47.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.057+0000 7fcbbeffd700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcbb80692a0 0x7fcbb806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:47.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.057+0000 7fcbbeffd700 1 -- 172.21.15.64:0/2278823306 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fcbc4086f30 con 0x7fcbd01000d0 2024-02-16T22:53:47.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.057+0000 7fcbce59c700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcbb80692a0 0x7fcbb806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:47.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.058+0000 7fcbce59c700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcbb80692a0 0x7fcbb806b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0004890 tx=0x7fcbc0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:47.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.062+0000 7fcbbeffd700 1 -- 172.21.15.64:0/2278823306 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fcbc4056c70 con 0x7fcbd01000d0 2024-02-16T22:53:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.306+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fcbd0105a40 con 0x7fcbd01000d0 2024-02-16T22:53:47.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:47 smithi064 ceph-mon[29226]: pgmap v178: 65 pgs: 10 creating+peering, 55 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:47.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:47 smithi192 ceph-mon[36254]: pgmap v178: 65 pgs: 10 creating+peering, 55 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:47.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.821+0000 7fcbbeffd700 1 -- 172.21.15.64:0/2278823306 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v3) v1 ==== 105+0+0 (secure 0 0 0) 0x7fcbc4056800 con 0x7fcbd01000d0 2024-02-16T22:53:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.823+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcbb80692a0 msgr2=0x7fcbb806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.823+0000 7fcbd4a0f700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcbb80692a0 0x7fcbb806b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0004890 tx=0x7fcbc0009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.823+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd01000d0 msgr2=0x7fcbd00fba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.824+0000 7fcbd4a0f700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd01000d0 0x7fcbd00fba00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4002a50 tx=0x7fcbc4004600 comp rx=0 tx=0).stop 2024-02-16T22:53:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.824+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 shutdown_connections 2024-02-16T22:53:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.824+0000 7fcbd4a0f700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbd0066c70 0x7fcbd00fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.824+0000 7fcbd4a0f700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fcbb80692a0 0x7fcbb806b750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.824+0000 7fcbd4a0f700 1 --2- 172.21.15.64:0/2278823306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd01000d0 0x7fcbd00fba00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.824+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 >> 172.21.15.64:0/2278823306 conn(0x7fcbd00f7510 msgr2=0x7fcbd00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.824+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 shutdown_connections 2024-02-16T22:53:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:47.824+0000 7fcbd4a0f700 1 -- 172.21.15.64:0/2278823306 wait complete. 2024-02-16T22:53:48.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2278823306' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-16T22:53:48.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:48.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:48.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:48.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.tnppti", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:53:48.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.tnppti", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: daemon mds.cephfs.smithi064.tqesxx assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: Cluster is now healthy 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2278823306' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] up:boot 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:creating} 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T22:53:48.206 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:48 smithi192 ceph-mon[36254]: daemon mds.cephfs.smithi064.tqesxx is now active in filesystem cephfs as rank 0 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2278823306' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.tnppti", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.tnppti", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: daemon mds.cephfs.smithi064.tqesxx assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: Cluster is now healthy 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2278823306' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-16T22:53:48.599 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] up:boot 2024-02-16T22:53:48.600 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:creating} 2024-02-16T22:53:48.600 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T22:53:48.600 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:48 smithi064 ceph-mon[29226]: daemon mds.cephfs.smithi064.tqesxx is now active in filesystem cephfs as rank 0 2024-02-16T22:53:48.629 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:53:48.638 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:53:48.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-02-16T22:53:48.909 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:49.301 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:49 smithi064 ceph-mon[29226]: Deploying daemon mds.cephfs.smithi192.tnppti on smithi192 2024-02-16T22:53:49.301 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:49 smithi064 ceph-mon[29226]: pgmap v179: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:49.302 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:49 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] up:active 2024-02-16T22:53:49.302 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:49 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 2024-02-16T22:53:49.493 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:49 smithi192 ceph-mon[36254]: Deploying daemon mds.cephfs.smithi192.tnppti on smithi192 2024-02-16T22:53:49.493 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:49 smithi192 ceph-mon[36254]: pgmap v179: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:53:49.494 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:49 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] up:active 2024-02-16T22:53:49.494 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:49 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 2024-02-16T22:53:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.339+0000 7fa1c807f700 1 -- 172.21.15.64:0/1703160010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 msgr2=0x7fa1c00fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.339+0000 7fa1c807f700 1 --2- 172.21.15.64:0/1703160010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 0x7fa1c00fb630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc004940 tx=0x7fa1bc009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.339+0000 7fa1c807f700 1 -- 172.21.15.64:0/1703160010 shutdown_connections 2024-02-16T22:53:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.339+0000 7fa1c807f700 1 --2- 172.21.15.64:0/1703160010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c00fe2b0 0x7fa1c00fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.339+0000 7fa1c807f700 1 --2- 172.21.15.64:0/1703160010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 0x7fa1c00fb630 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.339+0000 7fa1c807f700 1 -- 172.21.15.64:0/1703160010 >> 172.21.15.64:0/1703160010 conn(0x7fa1c00f6ca0 msgr2=0x7fa1c00f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.339+0000 7fa1c807f700 1 -- 172.21.15.64:0/1703160010 shutdown_connections 2024-02-16T22:53:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.339+0000 7fa1c807f700 1 -- 172.21.15.64:0/1703160010 wait complete. 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c807f700 1 Processor -- start 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c807f700 1 -- start start 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c807f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 0x7fa1c010d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c807f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c00fe2b0 0x7fa1c010dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c807f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c010e4e0 con 0x7fa1c00fb220 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c807f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c010ed80 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c5e1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 0x7fa1c010d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c5e1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 0x7fa1c010d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51180/0 (socket says 172.21.15.64:51180) 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c5e1b700 1 -- 172.21.15.64:0/1753439219 learned_addr learned my addr 172.21.15.64:0/1753439219 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.340+0000 7fa1c561a700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c00fe2b0 0x7fa1c010dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1c561a700 1 -- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 msgr2=0x7fa1c010d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1c561a700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 0x7fa1c010d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1c561a700 1 -- 172.21.15.64:0/1753439219 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1bc005040 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1c5e1b700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 0x7fa1c010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:53:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1c561a700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c00fe2b0 0x7fa1c010dee0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009fd0 tx=0x7fa1b000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1b67fc700 1 -- 172.21.15.64:0/1753439219 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1b0004e90 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c010f000 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1b67fc700 1 -- 172.21.15.64:0/1753439219 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1b000f040 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1b67fc700 1 -- 172.21.15.64:0/1753439219 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1b000ca50 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.341+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c010f540 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.342+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a40052e0 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.343+0000 7fa1b67fc700 1 -- 172.21.15.64:0/1753439219 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa1b000cc00 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.343+0000 7fa1b67fc700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1ac0692a0 0x7fa1ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.343+0000 7fa1b67fc700 1 -- 172.21.15.64:0/1753439219 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa1b0086b90 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.343+0000 7fa1c5e1b700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1ac0692a0 0x7fa1ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.343+0000 7fa1c5e1b700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1ac0692a0 0x7fa1ac06b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009b30 tx=0x7fa1bc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:50.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.348+0000 7fa1b67fc700 1 -- 172.21.15.64:0/1753439219 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa1b004dbd0 con 0x7fa1c00fe2b0 2024-02-16T22:53:50.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:50.605+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa1a4005b10 con 0x7fa1c00fe2b0 2024-02-16T22:53:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.144+0000 7fa1b67fc700 1 -- 172.21.15.64:0/1753439219 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1b0056980 con 0x7fa1c00fe2b0 2024-02-16T22:53:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1ac0692a0 msgr2=0x7fa1ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1ac0692a0 0x7fa1ac06b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009b30 tx=0x7fa1bc009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c00fe2b0 msgr2=0x7fa1c010dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c00fe2b0 0x7fa1c010dee0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009fd0 tx=0x7fa1b000b040 comp rx=0 tx=0).stop 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 shutdown_connections 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa1ac0692a0 0x7fa1ac06b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00fb220 0x7fa1c010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 --2- 172.21.15.64:0/1753439219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c00fe2b0 0x7fa1c010dee0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 >> 172.21.15.64:0/1753439219 conn(0x7fa1c00f6ca0 msgr2=0x7fa1c01014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 shutdown_connections 2024-02-16T22:53:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:51.146+0000 7fa1c807f700 1 -- 172.21.15.64:0/1753439219 wait complete. 2024-02-16T22:53:51.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.bmcyuf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.bmcyuf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: Deploying daemon mds.cephfs.smithi064.bmcyuf on smithi064 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 1 op/s 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:boot 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 1 up:standby 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 1 up:standby 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1753439219' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T22:53:51.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:51 smithi064 ceph-mon[29226]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T22:53:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.bmcyuf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.bmcyuf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: Deploying daemon mds.cephfs.smithi064.bmcyuf on smithi064 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 1 op/s 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:boot 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 1 up:standby 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 1 up:standby 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1753439219' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T22:53:51.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:51 smithi192 ceph-mon[36254]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T22:53:52.275 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:53:52.287 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:53:52.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-02-16T22:53:52.443 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:52 smithi064 ceph-mon[29226]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-16T22:53:52.443 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:52 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 1 up:standby 2024-02-16T22:53:52.475 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:52 smithi192 ceph-mon[36254]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-16T22:53:52.475 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:52 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 1 up:standby 2024-02-16T22:53:52.873 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:53.242 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:53 smithi064 ceph-mon[29226]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 630 B/s wr, 3 op/s 2024-02-16T22:53:53.242 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:53.242 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:53.242 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:53.242 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.luiybj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:53:53.242 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.luiybj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:53:53.242 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:53 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:53.406 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:53 smithi192 ceph-mon[36254]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 630 B/s wr, 3 op/s 2024-02-16T22:53:53.406 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:53.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:53.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:53.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.luiybj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:53:53.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.luiybj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:53:53.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:53 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:53:54.415 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:54 smithi192 ceph-mon[36254]: Deploying daemon mds.cephfs.smithi192.luiybj on smithi192 2024-02-16T22:53:54.416 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:54 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:boot 2024-02-16T22:53:54.416 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:54 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 2 up:standby 2024-02-16T22:53:54.416 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:54 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T22:53:54.416 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:54 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:54.466 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:54 smithi064 ceph-mon[29226]: Deploying daemon mds.cephfs.smithi192.luiybj on smithi192 2024-02-16T22:53:54.467 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:54 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:boot 2024-02-16T22:53:54.467 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:54 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 2 up:standby 2024-02-16T22:53:54.467 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:54 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T22:53:54.467 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:54 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.478+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/4208394108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 msgr2=0x7fe2ec0fb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.478+0000 7fe2f2f9d700 1 --2- 172.21.15.64:0/4208394108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 0x7fe2ec0fb710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0004940 tx=0x7fe2e0009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.478+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/4208394108 shutdown_connections 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.478+0000 7fe2f2f9d700 1 --2- 172.21.15.64:0/4208394108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec0fbce0 0x7fe2ec102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.478+0000 7fe2f2f9d700 1 --2- 172.21.15.64:0/4208394108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 0x7fe2ec0fb710 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.478+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/4208394108 >> 172.21.15.64:0/4208394108 conn(0x7fe2ec0f6ca0 msgr2=0x7fe2ec0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.478+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/4208394108 shutdown_connections 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.478+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/4208394108 wait complete. 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f2f9d700 1 Processor -- start 2024-02-16T22:53:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f2f9d700 1 -- start start 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f2f9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 0x7fe2ec10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f2f9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec10ded0 0x7fe2ec10ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f2f9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec10e390 con 0x7fe2ec0fb300 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f2f9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec10e4d0 con 0x7fe2ec10ded0 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f0d39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 0x7fe2ec10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f0d39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 0x7fe2ec10d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53498/0 (socket says 172.21.15.64:53498) 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f0d39700 1 -- 172.21.15.64:0/1261347968 learned_addr learned my addr 172.21.15.64:0/1261347968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2ebfff700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec10ded0 0x7fe2ec10ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f0d39700 1 -- 172.21.15.64:0/1261347968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec10ded0 msgr2=0x7fe2ec10ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:54.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.479+0000 7fe2f0d39700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec10ded0 0x7fe2ec10ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:54.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.480+0000 7fe2f0d39700 1 -- 172.21.15.64:0/1261347968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0005040 con 0x7fe2ec0fb300 2024-02-16T22:53:54.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.480+0000 7fe2f0d39700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 0x7fe2ec10d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009150 tx=0x7fe2e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:54.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.480+0000 7fe2e97fa700 1 -- 172.21.15.64:0/1261347968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e000e040 con 0x7fe2ec0fb300 2024-02-16T22:53:54.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.480+0000 7fe2e97fa700 1 -- 172.21.15.64:0/1261347968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2e000bbf0 con 0x7fe2ec0fb300 2024-02-16T22:53:54.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.480+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec10f370 con 0x7fe2ec0fb300 2024-02-16T22:53:54.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.480+0000 7fe2e97fa700 1 -- 172.21.15.64:0/1261347968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e001c3f0 con 0x7fe2ec0fb300 2024-02-16T22:53:54.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.480+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec10f880 con 0x7fe2ec0fb300 2024-02-16T22:53:54.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.481+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec107ab0 con 0x7fe2ec0fb300 2024-02-16T22:53:54.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.481+0000 7fe2e97fa700 1 -- 172.21.15.64:0/1261347968 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe2e000bd60 con 0x7fe2ec0fb300 2024-02-16T22:53:54.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.482+0000 7fe2e97fa700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe2dc0692a0 0x7fe2dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:54.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.482+0000 7fe2e97fa700 1 -- 172.21.15.64:0/1261347968 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe2e0088560 con 0x7fe2ec0fb300 2024-02-16T22:53:54.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.482+0000 7fe2ebfff700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe2dc0692a0 0x7fe2dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:54.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.482+0000 7fe2ebfff700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe2dc0692a0 0x7fe2dc06b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009fd0 tx=0x7fe2d800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:54.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.486+0000 7fe2e97fa700 1 -- 172.21.15.64:0/1261347968 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe2e0058320 con 0x7fe2ec0fb300 2024-02-16T22:53:54.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:54.725+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2ec04bed0 con 0x7fe2ec0fb300 2024-02-16T22:53:55.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.166+0000 7fe2e97fa700 1 -- 172.21.15.64:0/1261347968 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v9) v1 ==== 168+0+0 (secure 0 0 0) 0x7fe2e0057eb0 con 0x7fe2ec0fb300 2024-02-16T22:53:55.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe2dc0692a0 msgr2=0x7fe2dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:55.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe2dc0692a0 0x7fe2dc06b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009fd0 tx=0x7fe2d800b040 comp rx=0 tx=0).stop 2024-02-16T22:53:55.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 msgr2=0x7fe2ec10d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 0x7fe2ec10d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009150 tx=0x7fe2e0009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 shutdown_connections 2024-02-16T22:53:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fe2dc0692a0 0x7fe2dc06b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2ec0fb300 0x7fe2ec10d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 --2- 172.21.15.64:0/1261347968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec10ded0 0x7fe2ec10ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 >> 172.21.15.64:0/1261347968 conn(0x7fe2ec0f6ca0 msgr2=0x7fe2ec100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 shutdown_connections 2024-02-16T22:53:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:55.168+0000 7fe2f2f9d700 1 -- 172.21.15.64:0/1261347968 wait complete. 2024-02-16T22:53:55.171 INFO:teuthology.orchestra.run.smithi064.stderr:inline data enabled 2024-02-16T22:53:55.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:55 smithi192 ceph-mon[36254]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-02-16T22:53:55.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:55 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1261347968' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-16T22:53:55.537 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:55 smithi064 ceph-mon[29226]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-02-16T22:53:55.537 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:55 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1261347968' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-16T22:53:55.867 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:53:55.877 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:53:55.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T22:53:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-02-16T22:53:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1261347968' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-02-16T22:53:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 2 up:standby 2024-02-16T22:53:56.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:56 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:53:56.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-02-16T22:53:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1261347968' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-02-16T22:53:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 2 up:standby 2024-02-16T22:53:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:56.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:56 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:53:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:53:57.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:57 smithi064 ceph-mon[29226]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-02-16T22:53:57.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:57 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:boot 2024-02-16T22:53:57.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:57 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] up:active 2024-02-16T22:53:57.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:57 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:53:57.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:57 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T22:53:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:57 smithi192 ceph-mon[36254]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-02-16T22:53:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:57 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:boot 2024-02-16T22:53:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:57 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] up:active 2024-02-16T22:53:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:57 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:53:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:57 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T22:53:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.541+0000 7fce79874700 1 -- 172.21.15.64:0/534762157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 msgr2=0x7fce740fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.541+0000 7fce79874700 1 --2- 172.21.15.64:0/534762157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 0x7fce740fd420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce68004940 tx=0x7fce68009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 -- 172.21.15.64:0/534762157 shutdown_connections 2024-02-16T22:53:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 --2- 172.21.15.64:0/534762157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 0x7fce740fd420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 --2- 172.21.15.64:0/534762157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce740fbe40 0x7fce740fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 -- 172.21.15.64:0/534762157 >> 172.21.15.64:0/534762157 conn(0x7fce740f7520 msgr2=0x7fce740f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 -- 172.21.15.64:0/534762157 shutdown_connections 2024-02-16T22:53:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 -- 172.21.15.64:0/534762157 wait complete. 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 Processor -- start 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 -- start start 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.542+0000 7fce79874700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce740fbe40 0x7fce74110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce79874700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 0x7fce74112b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce79874700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74066d50 con 0x7fce740fcfb0 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce79874700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74066e90 con 0x7fce740fbe40 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 0x7fce74112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 0x7fce74112b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53506/0 (socket says 172.21.15.64:53506) 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce727fc700 1 -- 172.21.15.64:0/2490370938 learned_addr learned my addr 172.21.15.64:0/2490370938 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce72ffd700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce740fbe40 0x7fce74110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce727fc700 1 -- 172.21.15.64:0/2490370938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce740fbe40 msgr2=0x7fce74110600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce727fc700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce740fbe40 0x7fce74110600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce727fc700 1 -- 172.21.15.64:0/2490370938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68005040 con 0x7fce740fcfb0 2024-02-16T22:53:58.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce727fc700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 0x7fce74112b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce680093e0 tx=0x7fce68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:58.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce5b7fe700 1 -- 172.21.15.64:0/2490370938 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6800e040 con 0x7fce740fcfb0 2024-02-16T22:53:58.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce5b7fe700 1 -- 172.21.15.64:0/2490370938 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce6800bcc0 con 0x7fce740fcfb0 2024-02-16T22:53:58.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce74113150 con 0x7fce740fcfb0 2024-02-16T22:53:58.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.543+0000 7fce5b7fe700 1 -- 172.21.15.64:0/2490370938 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6801c630 con 0x7fce740fcfb0 2024-02-16T22:53:58.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.544+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce74113690 con 0x7fce740fcfb0 2024-02-16T22:53:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.544+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7404bed0 con 0x7fce740fcfb0 2024-02-16T22:53:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.545+0000 7fce5b7fe700 1 -- 172.21.15.64:0/2490370938 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fce68013420 con 0x7fce740fcfb0 2024-02-16T22:53:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.546+0000 7fce5b7fe700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fce5c0692a0 0x7fce5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:53:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.546+0000 7fce5b7fe700 1 -- 172.21.15.64:0/2490370938 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fce68088420 con 0x7fce740fcfb0 2024-02-16T22:53:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.546+0000 7fce72ffd700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fce5c0692a0 0x7fce5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:53:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.546+0000 7fce72ffd700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fce5c0692a0 0x7fce5c06b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce740669d0 tx=0x7fce6400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:53:58.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.550+0000 7fce5b7fe700 1 -- 172.21.15.64:0/2490370938 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fce680547f0 con 0x7fce740fcfb0 2024-02-16T22:53:58.556 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:58.556 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:58.556 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:58 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:standby 2024-02-16T22:53:58.557 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:58 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:53:58.557 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:58.557 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:53:58.557 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:58.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:58.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:58.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:58 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:standby 2024-02-16T22:53:58.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:58 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:53:58.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:58.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:53:58.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:53:58.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.791+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce74104950 con 0x7fce740fcfb0 2024-02-16T22:53:58.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.792+0000 7fce5b7fe700 1 -- 172.21.15.64:0/2490370938 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 11 v11) v1 ==== 76+0+1782 (secure 0 0 0) 0x7fce68057e10 con 0x7fce740fcfb0 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:e11 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:53:58.793 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:53:58.794 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:53:58.795 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:53:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:58.795 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:53:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:53:58.795 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:53:58.795 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:53:58.795 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.793+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fce5c0692a0 msgr2=0x7fce5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.793+0000 7fce79874700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fce5c0692a0 0x7fce5c06b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce740669d0 tx=0x7fce6400b040 comp rx=0 tx=0).stop 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.793+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 msgr2=0x7fce74112b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.793+0000 7fce79874700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 0x7fce74112b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce680093e0 tx=0x7fce68009e30 comp rx=0 tx=0).stop 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.794+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 shutdown_connections 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.794+0000 7fce79874700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce740fbe40 0x7fce74110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.794+0000 7fce79874700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fce5c0692a0 0x7fce5c06b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.794+0000 7fce79874700 1 --2- 172.21.15.64:0/2490370938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce740fcfb0 0x7fce74112b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.794+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 >> 172.21.15.64:0/2490370938 conn(0x7fce740f7520 msgr2=0x7fce74100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.794+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 shutdown_connections 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:53:58.794+0000 7fce79874700 1 -- 172.21.15.64:0/2490370938 wait complete. 2024-02-16T22:53:58.796 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 11 2024-02-16T22:53:59.431 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:59 smithi064 ceph-mon[29226]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-02-16T22:53:59.432 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:59 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2490370938' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:53:59.432 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:59 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:standby 2024-02-16T22:53:59.432 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:53:59 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:53:59.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-16T22:53:59.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:59 smithi192 ceph-mon[36254]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-02-16T22:53:59.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:59 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2490370938' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:53:59.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:59 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:standby 2024-02-16T22:53:59.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:53:59 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:54:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:01.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:00 smithi064 ceph-mon[29226]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:standby 2024-02-16T22:54:01.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:00 smithi064 ceph-mon[29226]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:54:01.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:00 smithi064 ceph-mon[29226]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-02-16T22:54:01.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:00 smithi192 ceph-mon[36254]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:standby 2024-02-16T22:54:01.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:00 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:54:01.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:00 smithi192 ceph-mon[36254]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-02-16T22:54:02.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 -- 172.21.15.64:0/1656846835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 msgr2=0x7f21640fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:02.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 --2- 172.21.15.64:0/1656846835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 0x7f21640fd350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2154004940 tx=0x7f2154009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:02.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 -- 172.21.15.64:0/1656846835 shutdown_connections 2024-02-16T22:54:02.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 --2- 172.21.15.64:0/1656846835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21640fe1f0 0x7f21640fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:02.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 --2- 172.21.15.64:0/1656846835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 0x7f21640fd350 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:02.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 -- 172.21.15.64:0/1656846835 >> 172.21.15.64:0/1656846835 conn(0x7f21640f8680 msgr2=0x7f21640faad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:02.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 -- 172.21.15.64:0/1656846835 shutdown_connections 2024-02-16T22:54:02.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 -- 172.21.15.64:0/1656846835 wait complete. 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.114+0000 7f216881a700 1 Processor -- start 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216881a700 1 -- start start 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216881a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 0x7f2164071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216881a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21640fe1f0 0x7f21640702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216881a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2164070970 con 0x7f21640fcf40 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216881a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2164070ae0 con 0x7f21640fe1f0 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 0x7f2164071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 0x7f2164071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44830/0 (socket says 172.21.15.64:44830) 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216259c700 1 -- 172.21.15.64:0/20447362 learned_addr learned my addr 172.21.15.64:0/20447362 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f2161d9b700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21640fe1f0 0x7f21640702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216259c700 1 -- 172.21.15.64:0/20447362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21640fe1f0 msgr2=0x7f21640702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216259c700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21640fe1f0 0x7f21640702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216259c700 1 -- 172.21.15.64:0/20447362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154005040 con 0x7f21640fcf40 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216259c700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 0x7f2164071e60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21540093b0 tx=0x7f2154009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f2152ffd700 1 -- 172.21.15.64:0/20447362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215400e040 con 0x7f21640fcf40 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f2152ffd700 1 -- 172.21.15.64:0/20447362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f215400bcc0 con 0x7f21640fcf40 2024-02-16T22:54:02.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2164070d60 con 0x7f21640fcf40 2024-02-16T22:54:02.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f2152ffd700 1 -- 172.21.15.64:0/20447362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215401c630 con 0x7f21640fcf40 2024-02-16T22:54:02.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.115+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21641159e0 con 0x7f21640fcf40 2024-02-16T22:54:02.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.116+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216404bed0 con 0x7f21640fcf40 2024-02-16T22:54:02.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.117+0000 7f2152ffd700 1 -- 172.21.15.64:0/20447362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2154013980 con 0x7f21640fcf40 2024-02-16T22:54:02.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.117+0000 7f2152ffd700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f214c0692a0 0x7f214c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:02.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.117+0000 7f2152ffd700 1 -- 172.21.15.64:0/20447362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2154088500 con 0x7f21640fcf40 2024-02-16T22:54:02.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.117+0000 7f2161d9b700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f214c0692a0 0x7f214c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:02.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.118+0000 7f2161d9b700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f214c0692a0 0x7f214c06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2158009bf0 tx=0x7f21580091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:02.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.122+0000 7f2152ffd700 1 -- 172.21.15.64:0/20447362 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f21540582c0 con 0x7f21640fcf40 2024-02-16T22:54:02.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.376+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2164115bd0 con 0x7f21640fcf40 2024-02-16T22:54:02.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.376+0000 7f2152ffd700 1 -- 172.21.15.64:0/20447362 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4689 (secure 0 0 0) 0x7f2164115bd0 con 0x7f21640fcf40 2024-02-16T22:54:02.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f214c0692a0 msgr2=0x7f214c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f214c0692a0 0x7f214c06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2158009bf0 tx=0x7f21580091a0 comp rx=0 tx=0).stop 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 msgr2=0x7f2164071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 0x7f2164071e60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21540093b0 tx=0x7f2154009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 shutdown_connections 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f214c0692a0 0x7f214c06b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21640fcf40 0x7f2164071e60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 --2- 172.21.15.64:0/20447362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21640fe1f0 0x7f21640702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.378+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 >> 172.21.15.64:0/20447362 conn(0x7f21640f8680 msgr2=0x7f2164101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.379+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 shutdown_connections 2024-02-16T22:54:02.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:02.379+0000 7f216881a700 1 -- 172.21.15.64:0/20447362 wait complete. 2024-02-16T22:54:02.382 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:54:02.392 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:54:03.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-16T22:54:03.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:03 smithi192 ceph-mon[36254]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 2 op/s 2024-02-16T22:54:03.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:03 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/20447362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:54:03.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:03 smithi064 ceph-mon[29226]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 2 op/s 2024-02-16T22:54:03.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:03 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/20447362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:54:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:04.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:04 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:04.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:04 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:04.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:04 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:54:04.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:04 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:54:04.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:04 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:04.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:04 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:54:04.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:04 smithi192 ceph-mon[36254]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-16T22:54:04.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:04.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:04.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:54:04.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:54:04.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:04.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:04 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:54:04.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:04 smithi064 ceph-mon[29226]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-16T22:54:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.285+0000 7f47f8687700 1 -- 172.21.15.64:0/1836898463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fcfb0 msgr2=0x7f47f00fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.285+0000 7f47f8687700 1 --2- 172.21.15.64:0/1836898463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fcfb0 0x7f47f00fd420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f47e0004940 tx=0x7f47e0009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.285+0000 7f47f8687700 1 -- 172.21.15.64:0/1836898463 shutdown_connections 2024-02-16T22:54:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.285+0000 7f47f8687700 1 --2- 172.21.15.64:0/1836898463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fcfb0 0x7f47f00fd420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.285+0000 7f47f8687700 1 --2- 172.21.15.64:0/1836898463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f00fbe40 0x7f47f00fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.285+0000 7f47f8687700 1 -- 172.21.15.64:0/1836898463 >> 172.21.15.64:0/1836898463 conn(0x7f47f00f7500 msgr2=0x7f47f00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.285+0000 7f47f8687700 1 -- 172.21.15.64:0/1836898463 shutdown_connections 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.285+0000 7f47f8687700 1 -- 172.21.15.64:0/1836898463 wait complete. 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f8687700 1 Processor -- start 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f8687700 1 -- start start 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f8687700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fbe40 0x7f47f01129c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f8687700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f00fcfb0 0x7f47f0114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f8687700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0066be0 con 0x7f47f00fbe40 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f8687700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0066d20 con 0x7f47f00fcfb0 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f6423700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fbe40 0x7f47f01129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f6423700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fbe40 0x7f47f01129c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44854/0 (socket says 172.21.15.64:44854) 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f6423700 1 -- 172.21.15.64:0/3142768615 learned_addr learned my addr 172.21.15.64:0/3142768615 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.286+0000 7f47f5c22700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f00fcfb0 0x7f47f0114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47f5c22700 1 -- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fbe40 msgr2=0x7f47f01129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47f5c22700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fbe40 0x7f47f01129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47f5c22700 1 -- 172.21.15.64:0/3142768615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0005040 con 0x7f47f00fcfb0 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47f6423700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fbe40 0x7f47f01129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47f5c22700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f00fcfb0 0x7f47f0114f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47e0004890 tx=0x7f47e0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47e6ffd700 1 -- 172.21.15.64:0/3142768615 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47e0021070 con 0x7f47f00fcfb0 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47e6ffd700 1 -- 172.21.15.64:0/3142768615 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47e001c400 con 0x7f47f00fcfb0 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f0115450 con 0x7f47f00fcfb0 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47e6ffd700 1 -- 172.21.15.64:0/3142768615 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47e000e070 con 0x7f47f00fcfb0 2024-02-16T22:54:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.287+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f01157f0 con 0x7f47f00fcfb0 2024-02-16T22:54:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.288+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f004bed0 con 0x7f47f00fcfb0 2024-02-16T22:54:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.289+0000 7f47e6ffd700 1 -- 172.21.15.64:0/3142768615 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f47e0013af0 con 0x7f47f00fcfb0 2024-02-16T22:54:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.289+0000 7f47e6ffd700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f47dc0692a0 0x7f47dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.289+0000 7f47e6ffd700 1 -- 172.21.15.64:0/3142768615 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f47e0088300 con 0x7f47f00fcfb0 2024-02-16T22:54:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.289+0000 7f47f6423700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f47dc0692a0 0x7f47dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.290+0000 7f47f6423700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f47dc0692a0 0x7f47dc06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47ec004fd0 tx=0x7f47ec00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.294+0000 7f47e6ffd700 1 -- 172.21.15.64:0/3142768615 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f47e0054630 con 0x7f47f00fcfb0 2024-02-16T22:54:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.578+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f47f0104930 con 0x7f47f00fcfb0 2024-02-16T22:54:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.579+0000 7f47e6ffd700 1 -- 172.21.15.64:0/3142768615 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v13) v1 ==== 78+0+99 (secure 0 0 0) 0x7f47e0057c50 con 0x7f47f00fcfb0 2024-02-16T22:54:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f47dc0692a0 msgr2=0x7f47dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f47dc0692a0 0x7f47dc06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47ec004fd0 tx=0x7f47ec00b040 comp rx=0 tx=0).stop 2024-02-16T22:54:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f00fcfb0 msgr2=0x7f47f0114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f00fcfb0 0x7f47f0114f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47e0004890 tx=0x7f47e0009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 shutdown_connections 2024-02-16T22:54:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f47dc0692a0 0x7f47dc06b750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47f00fbe40 0x7f47f01129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 --2- 172.21.15.64:0/3142768615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f00fcfb0 0x7f47f0114f10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 >> 172.21.15.64:0/3142768615 conn(0x7f47f00f7500 msgr2=0x7f47f0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 shutdown_connections 2024-02-16T22:54:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:05.582+0000 7f47f8687700 1 -- 172.21.15.64:0/3142768615 wait complete. 2024-02-16T22:54:05.595 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:54:05.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:05 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3142768615' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-16T22:54:06.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:05 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3142768615' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-16T22:54:06.425 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-16T22:54:06.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T22:54:06.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.556+0000 7f12fa59d640 1 -- 172.21.15.64:0/1794575799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fc270 msgr2=0x7f12f40fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:06.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.556+0000 7f12fa59d640 1 --2- 172.21.15.64:0/1794575799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fc270 0x7f12f40fc680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f12e80048d0 tx=0x7f12e8009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.556+0000 7f12fa59d640 1 -- 172.21.15.64:0/1794575799 shutdown_connections 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.556+0000 7f12fa59d640 1 --2- 172.21.15.64:0/1794575799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12f40fd540 0x7f12f40fd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.556+0000 7f12fa59d640 1 --2- 172.21.15.64:0/1794575799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fc270 0x7f12f40fc680 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.556+0000 7f12fa59d640 1 -- 172.21.15.64:0/1794575799 >> 172.21.15.64:0/1794575799 conn(0x7f12f40f7a30 msgr2=0x7f12f40f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.556+0000 7f12fa59d640 1 -- 172.21.15.64:0/1794575799 shutdown_connections 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12fa59d640 1 -- 172.21.15.64:0/1794575799 wait complete. 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12fa59d640 1 Processor -- start 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12fa59d640 1 -- start start 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12fa59d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12f40fc270 0x7f12f410cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12fa59d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fd540 0x7f12f410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12fa59d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f410d6e0 con 0x7f12f40fd540 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12fa59d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f4112130 con 0x7f12f40fc270 2024-02-16T22:54:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fd540 0x7f12f410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fd540 0x7f12f410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44872/0 (socket says 172.21.15.64:44872) 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.557+0000 7f12f37fe640 1 -- 172.21.15.64:0/2112984323 learned_addr learned my addr 172.21.15.64:0/2112984323 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12f3fff640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12f40fc270 0x7f12f410cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12f37fe640 1 -- 172.21.15.64:0/2112984323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12f40fc270 msgr2=0x7f12f410cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12f37fe640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12f40fc270 0x7f12f410cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12f37fe640 1 -- 172.21.15.64:0/2112984323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8005040 con 0x7f12f40fd540 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12f37fe640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fd540 0x7f12f410d090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12e4002a50 tx=0x7f12e4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12f0ff9640 1 -- 172.21.15.64:0/2112984323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12e4007c30 con 0x7f12f40fd540 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12f0ff9640 1 -- 172.21.15.64:0/2112984323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12e400f040 con 0x7f12f40fd540 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f41122d0 con 0x7f12f40fd540 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12f0ff9640 1 -- 172.21.15.64:0/2112984323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12e400db50 con 0x7f12f40fd540 2024-02-16T22:54:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.558+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f41127c0 con 0x7f12f40fd540 2024-02-16T22:54:06.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.559+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f4104de0 con 0x7f12f40fd540 2024-02-16T22:54:06.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.560+0000 7f12f0ff9640 1 -- 172.21.15.64:0/2112984323 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f12e401c400 con 0x7f12f40fd540 2024-02-16T22:54:06.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.560+0000 7f12f0ff9640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12dc069210 0x7f12dc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:06.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.560+0000 7f12f0ff9640 1 -- 172.21.15.64:0/2112984323 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f12e4086a80 con 0x7f12f40fd540 2024-02-16T22:54:06.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.560+0000 7f12f3fff640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12dc069210 0x7f12dc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:06.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.560+0000 7f12f3fff640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12dc069210 0x7f12dc06b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f12e8003080 tx=0x7f12e8009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:06.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.564+0000 7f12f0ff9640 1 -- 172.21.15.64:0/2112984323 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f12e404cbd0 con 0x7f12f40fd540 2024-02-16T22:54:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.741+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12f4112b20 con 0x7f12f40fd540 2024-02-16T22:54:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.742+0000 7f12f0ff9640 1 -- 172.21.15.64:0/2112984323 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14256 (secure 0 0 0) 0x7f12e4052d30 con 0x7f12f40fd540 2024-02-16T22:54:06.743 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:54:06.743 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","created":"2024-02-16T22:46:56.779041+0000","modified":"2024-02-16T22:53:43.368860+0000","last_up_change":"2024-02-16T22:52:22.826789+0000","last_in_change":"2024-02-16T22:52:04.211979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:51:12.402108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T22:53:40.294617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T22:53:40.597634+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7a680d8c-0559-46c6-9bfc-95fca252a657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6803","nonce":3967835180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6805","nonce":3967835180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6809","nonce":3967835180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3967835180},{"type":"v1","addr":"172.21.15.64:6807","nonce":3967835180}]},"public_addr":"172.21.15.64:6803/3967835180","cluster_addr":"172.21.15.64:6805/3967835180","heartbeat_back_addr":"172.21.15.64:6809/3967835180","heartbeat_front_addr":"172.21.15.64:6807/3967835180","state":["exists","up"]},{"osd":1,"uuid":"21a8757d-9f77-4f46-83d3-ff904c9e50b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6811","nonce":2618370222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6813","nonce":2618370222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6817","nonce":2618370222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2618370222},{"type":"v1","addr":"172.21.15.64:6815","nonce":2618370222}]},"public_addr":"172.21.15.64:6811/2618370222","cluster_addr":"172.21.15.64:6813/2618370222","heartbeat_back_addr":"172.21.15.64:6817/2618370222","heartbeat_front_addr":"172.21.15.64:6815/2618370222","state":["exists","up"]},{"osd":2,"uuid":"3fcb2e12-54a0-46db-af11-d047cf25d50b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6819","nonce":2434652781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6821","nonce":2434652781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6825","nonce":2434652781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2434652781},{"type":"v1","addr":"172.21.15.64:6823","nonce":2434652781}]},"public_addr":"172.21.15.64:6819/2434652781","cluster_addr":"172.21.15.64:6821/2434652781","heartbeat_back_addr":"172.21.15.64:6825/2434652781","heartbeat_front_addr":"172.21.15.64:6823/2434652781","state":["exists","up"]},{"osd":3,"uuid":"a7a58c77-96fa-4835-9ea4-d16e6d995620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6801","nonce":1577479253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6803","nonce":1577479253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6807","nonce":1577479253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1577479253},{"type":"v1","addr":"172.21.15.192:6805","nonce":1577479253}]},"public_addr":"172.21.15.192:6801/1577479253","cluster_addr":"172.21.15.192:6803/1577479253","heartbeat_back_addr":"172.21.15.192:6807/1577479253","heartbeat_front_addr":"172.21.15.192:6805/1577479253","state":["exists","up"]},{"osd":4,"uuid":"04e094c5-2494-457a-a4a5-7b2742be9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6809","nonce":523969674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6811","nonce":523969674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6815","nonce":523969674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":523969674},{"type":"v1","addr":"172.21.15.192:6813","nonce":523969674}]},"public_addr":"172.21.15.192:6809/523969674","cluster_addr":"172.21.15.192:6811/523969674","heartbeat_back_addr":"172.21.15.192:6815/523969674","heartbeat_front_addr":"172.21.15.192:6813/523969674","state":["exists","up"]},{"osd":5,"uuid":"bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6817","nonce":610438408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6819","nonce":610438408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6823","nonce":610438408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":610438408},{"type":"v1","addr":"172.21.15.192:6821","nonce":610438408}]},"public_addr":"172.21.15.192:6817/610438408","cluster_addr":"172.21.15.192:6819/610438408","heartbeat_back_addr":"172.21.15.192:6823/610438408","heartbeat_front_addr":"172.21.15.192:6821/610438408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:14.481189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:42.243983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:09.715768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:32.358421+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:56.431975+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:52:21.204162+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1250745016":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/4290875509":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/4194870513":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1222682641":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/3827352044":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6801/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6800/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1538951056":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6800/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/3005970232":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1580822017":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6800/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1804571669":"2024-02-17T22:47:54.261948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12dc069210 msgr2=0x7f12dc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12dc069210 0x7f12dc06b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f12e8003080 tx=0x7f12e8009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fd540 msgr2=0x7f12f410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fd540 0x7f12f410d090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12e4002a50 tx=0x7f12e4004600 comp rx=0 tx=0).stop 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 shutdown_connections 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f12dc069210 0x7f12dc06b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12f40fd540 0x7f12f410d090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 --2- 172.21.15.64:0/2112984323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12f40fc270 0x7f12f410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 >> 172.21.15.64:0/2112984323 conn(0x7f12f40f7a30 msgr2=0x7f12f4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:06.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 shutdown_connections 2024-02-16T22:54:06.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.745+0000 7f12fa59d640 1 -- 172.21.15.64:0/2112984323 wait complete. 2024-02-16T22:54:06.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:51:12.402108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T22:53:40.294617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T22:53:40.597634+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-16T22:54:06.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T22:54:06.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.870+0000 7f827046e640 1 -- 172.21.15.64:0/1783957765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 msgr2=0x7f82680fb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:06.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.870+0000 7f827046e640 1 --2- 172.21.15.64:0/1783957765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 0x7f82680fb5b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f82640048d0 tx=0x7f8264009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 -- 172.21.15.64:0/1783957765 shutdown_connections 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 --2- 172.21.15.64:0/1783957765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82680fc470 0x7f82680fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 --2- 172.21.15.64:0/1783957765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 0x7f82680fb5b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 -- 172.21.15.64:0/1783957765 >> 172.21.15.64:0/1783957765 conn(0x7f82680f6960 msgr2=0x7f82680f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 -- 172.21.15.64:0/1783957765 shutdown_connections 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 -- 172.21.15.64:0/1783957765 wait complete. 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 Processor -- start 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 -- start start 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 0x7f826810cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82680fc470 0x7f826810d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826810d6f0 con 0x7f82680fb1a0 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f827046e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8268112140 con 0x7f82680fc470 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f826e20a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 0x7f826810cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f826e20a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 0x7f826810cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44884/0 (socket says 172.21.15.64:44884) 2024-02-16T22:54:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.871+0000 7f826e20a640 1 -- 172.21.15.64:0/1107927865 learned_addr learned my addr 172.21.15.64:0/1107927865 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f826e20a640 1 -- 172.21.15.64:0/1107927865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82680fc470 msgr2=0x7f826810d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f826e20a640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82680fc470 0x7f826810d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f826e20a640 1 -- 172.21.15.64:0/1107927865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258005040 con 0x7f82680fb1a0 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f826e20a640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 0x7f826810cb60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f826402bda0 tx=0x7f8264009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f825effd640 1 -- 172.21.15.64:0/1107927865 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82640094c0 con 0x7f82680fb1a0 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f825effd640 1 -- 172.21.15.64:0/1107927865 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8264009620 con 0x7f82680fb1a0 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8264005040 con 0x7f82680fb1a0 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f825effd640 1 -- 172.21.15.64:0/1107927865 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826403b640 con 0x7f82680fb1a0 2024-02-16T22:54:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.872+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82681125e0 con 0x7f82680fb1a0 2024-02-16T22:54:06.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.873+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8268103cf0 con 0x7f82680fb1a0 2024-02-16T22:54:06.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.874+0000 7f825effd640 1 -- 172.21.15.64:0/1107927865 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f826402f070 con 0x7f82680fb1a0 2024-02-16T22:54:06.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.874+0000 7f825effd640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8254069210 0x7f825406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:06.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.874+0000 7f825effd640 1 -- 172.21.15.64:0/1107927865 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f82640a9720 con 0x7f82680fb1a0 2024-02-16T22:54:06.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.874+0000 7f826da09640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8254069210 0x7f825406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:06.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.874+0000 7f826da09640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8254069210 0x7f825406b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8258007fd0 tx=0x7f8258009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:06.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:06.878+0000 7f825effd640 1 -- 172.21.15.64:0/1107927865 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8264075a50 con 0x7f82680fb1a0 2024-02-16T22:54:07.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.051+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8268112ab0 con 0x7f82680fb1a0 2024-02-16T22:54:07.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.052+0000 7f825effd640 1 -- 172.21.15.64:0/1107927865 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8264079070 con 0x7f82680fb1a0 2024-02-16T22:54:07.053 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8254069210 msgr2=0x7f825406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8254069210 0x7f825406b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8258007fd0 tx=0x7f8258009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 msgr2=0x7f826810cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 0x7f826810cb60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f826402bda0 tx=0x7f8264009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 shutdown_connections 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8254069210 0x7f825406b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82680fc470 0x7f826810d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 --2- 172.21.15.64:0/1107927865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82680fb1a0 0x7f826810cb60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 >> 172.21.15.64:0/1107927865 conn(0x7f82680f6960 msgr2=0x7f8268065690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:07.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 shutdown_connections 2024-02-16T22:54:07.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.054+0000 7f827046e640 1 -- 172.21.15.64:0/1107927865 wait complete. 2024-02-16T22:54:07.064 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:06 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:07.064 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:06 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:07.064 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:06 smithi064 ceph-mon[29226]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-16T22:54:07.064 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:06 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2112984323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:54:07.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 -- 172.21.15.64:0/828734637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 msgr2=0x7f30d40fa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 --2- 172.21.15.64:0/828734637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 0x7f30d40fa9e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f30c40048d0 tx=0x7f30c4009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 -- 172.21.15.64:0/828734637 shutdown_connections 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 --2- 172.21.15.64:0/828734637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30d40fd5e0 0x7f30d40fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 --2- 172.21.15.64:0/828734637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 0x7f30d40fa9e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 -- 172.21.15.64:0/828734637 >> 172.21.15.64:0/828734637 conn(0x7f30d40f5ff0 msgr2=0x7f30d40f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 -- 172.21.15.64:0/828734637 shutdown_connections 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 -- 172.21.15.64:0/828734637 wait complete. 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 Processor -- start 2024-02-16T22:54:07.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 -- start start 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 0x7f30d410ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30d40fd5e0 0x7f30d410f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d410f990 con 0x7f30d40fa5d0 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.168+0000 7f30da8ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d41143e0 con 0x7f30d40fd5e0 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 0x7f30d410ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 0x7f30d410ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44908/0 (socket says 172.21.15.64:44908) 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d3fff640 1 -- 172.21.15.64:0/3492972985 learned_addr learned my addr 172.21.15.64:0/3492972985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d37fe640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30d40fd5e0 0x7f30d410f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d3fff640 1 -- 172.21.15.64:0/3492972985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30d40fd5e0 msgr2=0x7f30d410f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d3fff640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30d40fd5e0 0x7f30d410f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d3fff640 1 -- 172.21.15.64:0/3492972985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c4005040 con 0x7f30d40fa5d0 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d37fe640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30d40fd5e0 0x7f30d410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d3fff640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 0x7f30d410ee00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f30c402bda0 tx=0x7f30c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d0ff9640 1 -- 172.21.15.64:0/3492972985 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30c40094c0 con 0x7f30d40fa5d0 2024-02-16T22:54:07.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d0ff9640 1 -- 172.21.15.64:0/3492972985 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30c4007cc0 con 0x7f30d40fa5d0 2024-02-16T22:54:07.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d4114520 con 0x7f30d40fa5d0 2024-02-16T22:54:07.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30d0ff9640 1 -- 172.21.15.64:0/3492972985 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30c403b400 con 0x7f30d40fa5d0 2024-02-16T22:54:07.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.169+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d4114a10 con 0x7f30d40fa5d0 2024-02-16T22:54:07.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.170+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d41015f0 con 0x7f30d40fa5d0 2024-02-16T22:54:07.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.171+0000 7f30d0ff9640 1 -- 172.21.15.64:0/3492972985 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f30c402f070 con 0x7f30d40fa5d0 2024-02-16T22:54:07.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.171+0000 7f30d0ff9640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30bc069210 0x7f30bc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.171+0000 7f30d0ff9640 1 -- 172.21.15.64:0/3492972985 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f30c40a9720 con 0x7f30d40fa5d0 2024-02-16T22:54:07.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.171+0000 7f30d37fe640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30bc069210 0x7f30bc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:07.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.172+0000 7f30d37fe640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30bc069210 0x7f30bc06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30d4110350 tx=0x7f30c800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:07.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.176+0000 7f30d0ff9640 1 -- 172.21.15.64:0/3492972985 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f30c40759d0 con 0x7f30d40fa5d0 2024-02-16T22:54:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:06 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:06 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:06 smithi192 ceph-mon[36254]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-16T22:54:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:06 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2112984323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:54:07.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.347+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f30d402a8e0 con 0x7f30d40fa5d0 2024-02-16T22:54:07.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.347+0000 7f30d0ff9640 1 -- 172.21.15.64:0/3492972985 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f30c4078ff0 con 0x7f30d40fa5d0 2024-02-16T22:54:07.349 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30bc069210 msgr2=0x7f30bc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30bc069210 0x7f30bc06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30d4110350 tx=0x7f30c800b040 comp rx=0 tx=0).stop 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 msgr2=0x7f30d410ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 0x7f30d410ee00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f30c402bda0 tx=0x7f30c4009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 shutdown_connections 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f30bc069210 0x7f30bc06b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30d40fd5e0 0x7f30d410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 --2- 172.21.15.64:0/3492972985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30d40fa5d0 0x7f30d410ee00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 >> 172.21.15.64:0/3492972985 conn(0x7f30d40f5ff0 msgr2=0x7f30d40fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 shutdown_connections 2024-02-16T22:54:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.349+0000 7f30da8ae640 1 -- 172.21.15.64:0/3492972985 wait complete. 2024-02-16T22:54:07.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T22:54:07.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 -- 172.21.15.64:0/3232156033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 msgr2=0x7f88d40fa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 --2- 172.21.15.64:0/3232156033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 0x7f88d40fa9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f88c40048d0 tx=0x7f88c4009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 -- 172.21.15.64:0/3232156033 shutdown_connections 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 --2- 172.21.15.64:0/3232156033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88d40fd660 0x7f88d40fdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 --2- 172.21.15.64:0/3232156033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 0x7f88d40fa9c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 -- 172.21.15.64:0/3232156033 >> 172.21.15.64:0/3232156033 conn(0x7f88d40f6090 msgr2=0x7f88d40f84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 -- 172.21.15.64:0/3232156033 shutdown_connections 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 -- 172.21.15.64:0/3232156033 wait complete. 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 Processor -- start 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 -- start start 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 0x7f88d410cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88d40fd660 0x7f88d410d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d410d780 con 0x7f88d40fa5b0 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d975d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4112190 con 0x7f88d40fd660 2024-02-16T22:54:07.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.462+0000 7f88d2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 0x7f88d410cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 0x7f88d410cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44922/0 (socket says 172.21.15.64:44922) 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d2ffd640 1 -- 172.21.15.64:0/3076849041 learned_addr learned my addr 172.21.15.64:0/3076849041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d27fc640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88d40fd660 0x7f88d410d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d2ffd640 1 -- 172.21.15.64:0/3076849041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88d40fd660 msgr2=0x7f88d410d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d2ffd640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88d40fd660 0x7f88d410d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d2ffd640 1 -- 172.21.15.64:0/3076849041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c4005040 con 0x7f88d40fa5b0 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d2ffd640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 0x7f88d410cbf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f88c402bda0 tx=0x7f88c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88bb7fe640 1 -- 172.21.15.64:0/3076849041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88c40092e0 con 0x7f88d40fa5b0 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88bb7fe640 1 -- 172.21.15.64:0/3076849041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88c403d450 con 0x7f88d40fa5b0 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d41122d0 con 0x7f88d40fa5b0 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88bb7fe640 1 -- 172.21.15.64:0/3076849041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88c403b640 con 0x7f88d40fa5b0 2024-02-16T22:54:07.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.463+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d41127a0 con 0x7f88d40fa5b0 2024-02-16T22:54:07.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.464+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d40644c0 con 0x7f88d40fa5b0 2024-02-16T22:54:07.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.464+0000 7f88bb7fe640 1 -- 172.21.15.64:0/3076849041 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f88c402f070 con 0x7f88d40fa5b0 2024-02-16T22:54:07.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.465+0000 7f88bb7fe640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88bc069210 0x7f88bc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.465+0000 7f88bb7fe640 1 -- 172.21.15.64:0/3076849041 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f88c40a9510 con 0x7f88d40fa5b0 2024-02-16T22:54:07.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.465+0000 7f88d27fc640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88bc069210 0x7f88bc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:07.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.465+0000 7f88d27fc640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88bc069210 0x7f88bc06b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009af0 tx=0x7f88c800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:07.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.469+0000 7f88bb7fe640 1 -- 172.21.15.64:0/3076849041 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f88c4075840 con 0x7f88d40fa5b0 2024-02-16T22:54:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.657+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f88d4112b80 con 0x7f88d40fa5b0 2024-02-16T22:54:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.658+0000 7f88bb7fe640 1 -- 172.21.15.64:0/3076849041 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f88c4078e60 con 0x7f88d40fa5b0 2024-02-16T22:54:07.659 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-02-16T22:54:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88bc069210 msgr2=0x7f88bc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88bc069210 0x7f88bc06b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009af0 tx=0x7f88c800b040 comp rx=0 tx=0).stop 2024-02-16T22:54:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 msgr2=0x7f88d410cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 0x7f88d410cbf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f88c402bda0 tx=0x7f88c4009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 shutdown_connections 2024-02-16T22:54:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88bc069210 0x7f88bc06b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88d40fd660 0x7f88d410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 --2- 172.21.15.64:0/3076849041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88d40fa5b0 0x7f88d410cbf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 >> 172.21.15.64:0/3076849041 conn(0x7f88d40f6090 msgr2=0x7f88d4100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 shutdown_connections 2024-02-16T22:54:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.660+0000 7f88d975d640 1 -- 172.21.15.64:0/3076849041 wait complete. 2024-02-16T22:54:07.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-16T22:54:07.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 -- 172.21.15.64:0/1561477467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fc450 msgr2=0x7f82240fc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 --2- 172.21.15.64:0/1561477467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fc450 0x7f82240fc8c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f820c0048d0 tx=0x7f820c009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 -- 172.21.15.64:0/1561477467 shutdown_connections 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 --2- 172.21.15.64:0/1561477467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fc450 0x7f82240fc8c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 --2- 172.21.15.64:0/1561477467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82240fb180 0x7f82240fb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 -- 172.21.15.64:0/1561477467 >> 172.21.15.64:0/1561477467 conn(0x7f82240f6940 msgr2=0x7f82240f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 -- 172.21.15.64:0/1561477467 shutdown_connections 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 -- 172.21.15.64:0/1561477467 wait complete. 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 Processor -- start 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.768+0000 7f8228d9d640 1 -- start start 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f8228d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fb180 0x7f822410caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f8228d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82240fc450 0x7f822410d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f8228d9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822410d680 con 0x7f82240fb180 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f8228d9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82241120d0 con 0x7f82240fc450 2024-02-16T22:54:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f822259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fb180 0x7f822410caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f822259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fb180 0x7f822410caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44946/0 (socket says 172.21.15.64:44946) 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f822259c640 1 -- 172.21.15.64:0/4288022648 learned_addr learned my addr 172.21.15.64:0/4288022648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f822259c640 1 -- 172.21.15.64:0/4288022648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82240fc450 msgr2=0x7f822410d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f822259c640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82240fc450 0x7f822410d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f822259c640 1 -- 172.21.15.64:0/4288022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c005040 con 0x7f82240fb180 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f822259c640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fb180 0x7f822410caf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f821400c810 tx=0x7f82140043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f821affd640 1 -- 172.21.15.64:0/4288022648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f821400a820 con 0x7f82240fb180 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f821affd640 1 -- 172.21.15.64:0/4288022648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f821400ae50 con 0x7f82240fb180 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8224112270 con 0x7f82240fb180 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f821affd640 1 -- 172.21.15.64:0/4288022648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82140115c0 con 0x7f82240fb180 2024-02-16T22:54:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.769+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8224112760 con 0x7f82240fb180 2024-02-16T22:54:07.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.770+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8224103cd0 con 0x7f82240fb180 2024-02-16T22:54:07.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.771+0000 7f821affd640 1 -- 172.21.15.64:0/4288022648 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f821400a980 con 0x7f82240fb180 2024-02-16T22:54:07.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.771+0000 7f821affd640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8210069210 0x7f821006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:07.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.771+0000 7f821affd640 1 -- 172.21.15.64:0/4288022648 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8214087760 con 0x7f82240fb180 2024-02-16T22:54:07.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.771+0000 7f821bfff640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8210069210 0x7f821006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:07.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.771+0000 7f821bfff640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8210069210 0x7f821006b6c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f820c034040 tx=0x7f820c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:07.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.776+0000 7f821affd640 1 -- 172.21.15.64:0/4288022648 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8214053a10 con 0x7f82240fb180 2024-02-16T22:54:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.993+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82241129e0 con 0x7f82240fb180 2024-02-16T22:54:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.994+0000 7f821affd640 1 -- 172.21.15.64:0/4288022648 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4689 (secure 0 0 0) 0x7f8214057030 con 0x7f82240fb180 2024-02-16T22:54:07.995 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:54:07.995 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T22:53:56.166293+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi064.tqesxx","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6827/3220035256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3220035256},{"type":"v1","addr":"172.21.15.64:6827","nonce":3220035256}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-16T22:54:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8210069210 msgr2=0x7f821006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8210069210 0x7f821006b6c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f820c034040 tx=0x7f820c009da0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fb180 msgr2=0x7f822410caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fb180 0x7f822410caf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f821400c810 tx=0x7f82140043c0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 shutdown_connections 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f8210069210 0x7f821006b6c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82240fc450 0x7f822410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 --2- 172.21.15.64:0/4288022648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240fb180 0x7f822410caf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.995+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 >> 172.21.15.64:0/4288022648 conn(0x7f82240f6940 msgr2=0x7f82240656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.996+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 shutdown_connections 2024-02-16T22:54:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:07.996+0000 7f8228d9d640 1 -- 172.21.15.64:0/4288022648 wait complete. 2024-02-16T22:54:08.006 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:07 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1107927865' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:54:08.006 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:07 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3492972985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:54:08.006 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:07 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3076849041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:54:08.007 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 10, 'max_mds': 1, 'flags': 18} 2024-02-16T22:54:08.007 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-02-16T22:54:08.031 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-02-16T22:54:08.031 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-02-16T22:54:08.031 INFO:tasks.ceph_fuse:client.0 config is {} 2024-02-16T22:54:08.031 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-16T22:54:08.031 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:08.031 INFO:tasks.ceph_fuse:client.1 config is {} 2024-02-16T22:54:08.031 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-16T22:54:08.031 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:08.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:08.032 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-02-16T22:54:08.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:08.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-02-16T22:54:08.114 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-02-16T22:54:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:54:08.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:08.116 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-02-16T22:54:08.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:08.136 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-02-16T22:54:08.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:07 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1107927865' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:54:08.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:07 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3492972985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:54:08.205 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:07 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3076849041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:54:08.213 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2024-02-16T22:54:08.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:54:08.215 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-02-16T22:54:08.215 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-16T22:54:08.653 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:09.104 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:08 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/4288022648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:54:09.104 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:08 smithi064 ceph-mon[29226]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:08 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/4288022648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:54:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:08 smithi192 ceph-mon[36254]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.982+0000 7f2049e6e700 1 -- 172.21.15.64:0/3309517575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 msgr2=0x7f20440fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.982+0000 7f2049e6e700 1 --2- 172.21.15.64:0/3309517575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 0x7f20440fe330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f202c004940 tx=0x7f202c009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.982+0000 7f2049e6e700 1 -- 172.21.15.64:0/3309517575 shutdown_connections 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.982+0000 7f2049e6e700 1 --2- 172.21.15.64:0/3309517575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20440fe900 0x7f2044106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.982+0000 7f2049e6e700 1 --2- 172.21.15.64:0/3309517575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 0x7f20440fe330 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.982+0000 7f2049e6e700 1 -- 172.21.15.64:0/3309517575 >> 172.21.15.64:0/3309517575 conn(0x7f20440f7a10 msgr2=0x7f20440f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.982+0000 7f2049e6e700 1 -- 172.21.15.64:0/3309517575 shutdown_connections 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.982+0000 7f2049e6e700 1 -- 172.21.15.64:0/3309517575 wait complete. 2024-02-16T22:54:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f2049e6e700 1 Processor -- start 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f2049e6e700 1 -- start start 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f2049e6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 0x7f2044113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f2049e6e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20440fe900 0x7f2044113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f2049e6e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2044114630 con 0x7f20440fdf60 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f2049e6e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204410d960 con 0x7f20440fe900 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f20437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 0x7f2044113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f20437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 0x7f2044113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36156/0 (socket says 172.21.15.64:36156) 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f20437fe700 1 -- 172.21.15.64:0/2150640116 learned_addr learned my addr 172.21.15.64:0/2150640116 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f2042ffd700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20440fe900 0x7f2044113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f20437fe700 1 -- 172.21.15.64:0/2150640116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20440fe900 msgr2=0x7f2044113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.983+0000 7f20437fe700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20440fe900 0x7f2044113fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.984+0000 7f20437fe700 1 -- 172.21.15.64:0/2150640116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c005040 con 0x7f20440fdf60 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.984+0000 7f20437fe700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 0x7f2044113aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f202c004890 tx=0x7f202c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.984+0000 7f203bfff700 1 -- 172.21.15.64:0/2150640116 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f202c00e040 con 0x7f20440fdf60 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.984+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204410dbe0 con 0x7f20440fdf60 2024-02-16T22:54:10.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.984+0000 7f203bfff700 1 -- 172.21.15.64:0/2150640116 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f202c004290 con 0x7f20440fdf60 2024-02-16T22:54:10.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.984+0000 7f203bfff700 1 -- 172.21.15.64:0/2150640116 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f202c01c630 con 0x7f20440fdf60 2024-02-16T22:54:10.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.984+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204410e120 con 0x7f20440fdf60 2024-02-16T22:54:10.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.985+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2044104550 con 0x7f20440fdf60 2024-02-16T22:54:10.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.985+0000 7f203bfff700 1 -- 172.21.15.64:0/2150640116 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f202c00bbf0 con 0x7f20440fdf60 2024-02-16T22:54:10.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.986+0000 7f203bfff700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f20300692a0 0x7f203006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:10.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.986+0000 7f203bfff700 1 -- 172.21.15.64:0/2150640116 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f202c0885e0 con 0x7f20440fdf60 2024-02-16T22:54:10.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.986+0000 7f2042ffd700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f20300692a0 0x7f203006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:10.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.986+0000 7f2042ffd700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f20300692a0 0x7f203006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f204410f2d0 tx=0x7f2034009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:10.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:10.990+0000 7f203bfff700 1 -- 172.21.15.64:0/2150640116 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f202c0583a0 con 0x7f20440fdf60 2024-02-16T22:54:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.242+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f204404bed0 con 0x7f20440fdf60 2024-02-16T22:54:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.242+0000 7f203bfff700 1 -- 172.21.15.64:0/2150640116 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+83 (secure 0 0 0) 0x7f202c057f30 con 0x7f20440fdf60 2024-02-16T22:54:11.243 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-16T22:54:11.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f20300692a0 msgr2=0x7f203006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:11.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f20300692a0 0x7f203006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f204410f2d0 tx=0x7f2034009040 comp rx=0 tx=0).stop 2024-02-16T22:54:11.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 msgr2=0x7f2044113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:11.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 0x7f2044113aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f202c004890 tx=0x7f202c009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:11.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 shutdown_connections 2024-02-16T22:54:11.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f20300692a0 0x7f203006b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:11.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440fdf60 0x7f2044113aa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:11.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 --2- 172.21.15.64:0/2150640116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20440fe900 0x7f2044113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:11.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.244+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 >> 172.21.15.64:0/2150640116 conn(0x7f20440f7a10 msgr2=0x7f20441016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:11.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.245+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 shutdown_connections 2024-02-16T22:54:11.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:11.245+0000 7f2049e6e700 1 -- 172.21.15.64:0/2150640116 wait complete. 2024-02-16T22:54:11.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:11 smithi192 ceph-mon[36254]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:11.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:11 smithi064 ceph-mon[29226]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:11.811 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-16T22:54:11.811 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-16T22:54:11.811 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-02-16T22:54:11.811 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-16T22:54:11.811 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:11.811 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-16T22:54:11.811 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-16T22:54:11.811 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-16T22:54:11.812 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-16T22:54:11.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:11.812 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-02-16T22:54:11.828 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:54:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-02-16T22:54:11.829 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-16T22:54:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-02-16T22:54:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-16T22:54:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42305sec preferred_lft 42305sec 2024-02-16T22:54:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-02-16T22:54:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:54:11.829 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-16T22:54:11.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:11.829 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-02-16T22:54:11.830 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-02-16T22:54:11.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-02-16T22:54:11.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-02-16T22:54:11.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-02-16T22:54:11.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-16T22:54:11.830 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-02-16T22:54:12.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.004 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-16T22:54:12.031 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-02-16T22:54:12.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.032 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-02-16T22:54:12.112 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-02-16T22:54:12.112 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-16T22:54:12.112 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-16T22:54:12.112 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-16T22:54:12.112 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-16T22:54:12.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.112 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-02-16T22:54:12.113 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-02-16T22:54:12.113 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-16T22:54:12.113 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-16T22:54:12.113 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-16T22:54:12.113 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-02-16T22:54:12.301 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:12 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2150640116' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T22:54:12.301 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:12 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:12.301 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:12 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:12.301 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:12 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:54:12.302 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:12 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:54:12.302 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:12 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:12.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.397 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-02-16T22:54:12.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.412 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-02-16T22:54:12.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.466 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-02-16T22:54:12.466 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-02-16T22:54:12.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:54:12.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-16T22:54:12.466 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-02-16T22:54:12.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:12 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2150640116' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T22:54:12.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:12 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:12.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:12 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:12.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:12 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:54:12.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:12 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:54:12.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:12 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:12.548 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-16T22:54:12.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.548 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-02-16T22:54:12.548 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-02-16T22:54:12.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-16T22:54:12.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-16T22:54:12.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-16T22:54:12.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-16T22:54:12.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-16T22:54:12.549 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-02-16T22:54:12.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.683 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-02-16T22:54:12.683 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-02-16T22:54:12.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-02-16T22:54:12.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-02-16T22:54:12.683 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-02-16T22:54:12.724 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-02-16T22:54:12.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T22:54:12.724 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:12.779 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-16T22:54:12.779 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T22:54:12.779 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:12.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-02-16T22:54:12.900 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-02-16T22:54:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-02-16T22:54:12.957 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/16bfdc90f39f909a437e23c37cc81b5ec1e577477c90d8e590cb8eb20b5035ee/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d41a81e94189018ef3fc6999531d69ecb3030a4a7bd4d9afa2f5b508c3cb61ce/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/ac7789dd3e171e25a03c1fb1c7dce65a7a2a80f14861bab54a665e8159cd3e82/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/9b6b73acb499526c88f9822798cd25b93e8fc3e0de9d44fb4abcc8d6e56342f5/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/dc34eb9207a9c0a2f2dc6dad456e669ed72723ab511164ee80fe21c55acb6f49/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/50e4928293e7c2042d8dbaf3cd3de219a417c24da4b50d8e8f15e21e87dd9868/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d4e7da1b6e3d659b2266acdd7af9adee3a5c2f1603106ab554dbe077761630dd/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7923dc85c55be68b5654a5c1f2a19d0a6948df89cb76a6255841597cc274ace1/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/02eb6f56841c753534243ecee8383aa0e767f85162e7c893510f349ab0db9326/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d2c266332181a146245dc03e73390ab21da86cbcb566030d4cd7dc7864cf9e8a/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/0c5576e99b459f98af5f6644903b9110b2fb5036018d50b93fd3c671857d4e2c/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/15623b6d1bba5192b43e3fc4a302c955b424ea837f20a673288c1c6ef74f3a09/merged 2024-02-16T22:54:12.958 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-02-16T22:54:12.959 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:54:12.959 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:54:12.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:12.959 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-02-16T22:54:13.011 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-16T22:54:13.011 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-02-16T22:54:13.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-02-16T22:54:13.080 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T22:54:13.136 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-02-16T22:54:13.137 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-02-16T22:54:13.138 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/16bfdc90f39f909a437e23c37cc81b5ec1e577477c90d8e590cb8eb20b5035ee/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d41a81e94189018ef3fc6999531d69ecb3030a4a7bd4d9afa2f5b508c3cb61ce/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/ac7789dd3e171e25a03c1fb1c7dce65a7a2a80f14861bab54a665e8159cd3e82/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/9b6b73acb499526c88f9822798cd25b93e8fc3e0de9d44fb4abcc8d6e56342f5/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/dc34eb9207a9c0a2f2dc6dad456e669ed72723ab511164ee80fe21c55acb6f49/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/50e4928293e7c2042d8dbaf3cd3de219a417c24da4b50d8e8f15e21e87dd9868/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d4e7da1b6e3d659b2266acdd7af9adee3a5c2f1603106ab554dbe077761630dd/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7923dc85c55be68b5654a5c1f2a19d0a6948df89cb76a6255841597cc274ace1/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/02eb6f56841c753534243ecee8383aa0e767f85162e7c893510f349ab0db9326/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d2c266332181a146245dc03e73390ab21da86cbcb566030d4cd7dc7864cf9e8a/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/0c5576e99b459f98af5f6644903b9110b2fb5036018d50b93fd3c671857d4e2c/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/15623b6d1bba5192b43e3fc4a302c955b424ea837f20a673288c1c6ef74f3a09/merged 2024-02-16T22:54:13.139 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-02-16T22:54:13.140 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:54:13.140 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:54:13.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:13.140 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-02-16T22:54:13.151 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2024-02-16T22:54:13.149+0000 7f6fd2abe300 -1 init, newargv = 0x55d8b52c20f0 newargc=15 2024-02-16T22:54:13.151 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[76752]: starting ceph client 2024-02-16T22:54:13.156 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[76752]: starting fuse 2024-02-16T22:54:13.165 INFO:teuthology.orchestra.run.smithi064.stdout:148 2024-02-16T22:54:13.165 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [148] 2024-02-16T22:54:13.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-02-16T22:54:13.165 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-02-16T22:54:13.165 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-02-16T22:54:13.165 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-16T22:54:13.166 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-02-16T22:54:13.167 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-02-16T22:54:13.167 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-02-16T22:54:13.167 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-16T22:54:13.167 DEBUG:teuthology.orchestra.run.smithi064:> 2024-02-16T22:54:13.167 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2024-02-16T22:54:13.167 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-02-16T22:54:13.286 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.76752.asok 2024-02-16T22:54:13.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-16T22:54:13.296 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76752.asok 2024-02-16T22:54:13.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:13.296 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76752.asok status 2024-02-16T22:54:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:54:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-02-16T22:54:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "b88fae66e6ad896087939a6bde358f026d85f561", 2024-02-16T22:54:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)", 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "76752", 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 14528, 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-02-16T22:54:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 14528 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3011212362 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3011212362 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.14528 192.168.0.1:0/3011212362", 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3011212362", 2024-02-16T22:54:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-02-16T22:54:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 10, 2024-02-16T22:54:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 46, 2024-02-16T22:54:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-02-16T22:54:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-02-16T22:54:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-02-16T22:54:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:54:13.441 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-16T22:54:13.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:13 smithi064 ceph-mon[29226]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:13.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:54:13.701 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:13.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:13 smithi192 ceph-mon[36254]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:13.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:54:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 -- 172.21.15.64:0/3585424373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 msgr2=0x7fb12c0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 --2- 172.21.15.64:0/3585424373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 0x7fb12c0fcf60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb120004940 tx=0x7fb120009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 -- 172.21.15.64:0/3585424373 shutdown_connections 2024-02-16T22:54:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 --2- 172.21.15.64:0/3585424373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 0x7fb12c0fcf60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 --2- 172.21.15.64:0/3585424373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c102f50 0x7fb12c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 -- 172.21.15.64:0/3585424373 >> 172.21.15.64:0/3585424373 conn(0x7fb12c0f8ae0 msgr2=0x7fb12c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 -- 172.21.15.64:0/3585424373 shutdown_connections 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 -- 172.21.15.64:0/3585424373 wait complete. 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 Processor -- start 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.273+0000 7fb1308ad700 1 -- start start 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb1308ad700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c102f50 0x7fb12c070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb1308ad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 0x7fb12c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb1308ad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c070e60 con 0x7fb12c1038f0 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb1308ad700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c070fd0 con 0x7fb12c102f50 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb129d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 0x7fb12c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb129d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 0x7fb12c0707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36170/0 (socket says 172.21.15.64:36170) 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb129d9b700 1 -- 172.21.15.64:0/2170111199 learned_addr learned my addr 172.21.15.64:0/2170111199 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb12a59c700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c102f50 0x7fb12c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb129d9b700 1 -- 172.21.15.64:0/2170111199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c102f50 msgr2=0x7fb12c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb129d9b700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c102f50 0x7fb12c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb129d9b700 1 -- 172.21.15.64:0/2170111199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120005040 con 0x7fb12c1038f0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb129d9b700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 0x7fb12c0707d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb1200090d0 tx=0x7fb120009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb11affd700 1 -- 172.21.15.64:0/2170111199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12000e040 con 0x7fb12c1038f0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb11affd700 1 -- 172.21.15.64:0/2170111199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb12000bbf0 con 0x7fb12c1038f0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12c119d00 con 0x7fb12c1038f0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb11affd700 1 -- 172.21.15.64:0/2170111199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12001c630 con 0x7fb12c1038f0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.274+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12c11a1f0 con 0x7fb12c1038f0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.275+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c0fc6e0 con 0x7fb12c1038f0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.276+0000 7fb11affd700 1 -- 172.21.15.64:0/2170111199 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb12000bd60 con 0x7fb12c1038f0 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.276+0000 7fb11affd700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb1140692a0 0x7fb11406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.276+0000 7fb11affd700 1 -- 172.21.15.64:0/2170111199 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb1200884c0 con 0x7fb12c1038f0 2024-02-16T22:54:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.276+0000 7fb12a59c700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb1140692a0 0x7fb11406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.277+0000 7fb12a59c700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb1140692a0 0x7fb11406b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb11c006fd0 tx=0x7fb11c008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.281+0000 7fb11affd700 1 -- 172.21.15.64:0/2170111199 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb1200547f0 con 0x7fb12c1038f0 2024-02-16T22:54:15.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.520+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb12c11a3e0 con 0x7fb12c1038f0 2024-02-16T22:54:15.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.520+0000 7fb11affd700 1 -- 172.21.15.64:0/2170111199 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+83 (secure 0 0 0) 0x7fb120013e60 con 0x7fb12c1038f0 2024-02-16T22:54:15.521 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-16T22:54:15.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb1140692a0 msgr2=0x7fb11406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:15.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb1140692a0 0x7fb11406b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb11c006fd0 tx=0x7fb11c008040 comp rx=0 tx=0).stop 2024-02-16T22:54:15.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 msgr2=0x7fb12c0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:15.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 0x7fb12c0707d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb1200090d0 tx=0x7fb120009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 shutdown_connections 2024-02-16T22:54:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c102f50 0x7fb12c070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb1140692a0 0x7fb11406b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 --2- 172.21.15.64:0/2170111199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb12c1038f0 0x7fb12c0707d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 >> 172.21.15.64:0/2170111199 conn(0x7fb12c0f8ae0 msgr2=0x7fb12c0fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 shutdown_connections 2024-02-16T22:54:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:15.522+0000 7fb1308ad700 1 -- 172.21.15.64:0/2170111199 wait complete. 2024-02-16T22:54:15.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:15 smithi064 ceph-mon[29226]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:15.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:15 smithi192 ceph-mon[36254]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:16.094 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-16T22:54:16.094 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-16T22:54:16.094 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-02-16T22:54:16.094 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-16T22:54:16.095 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:16.095 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-16T22:54:16.095 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-16T22:54:16.095 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-16T22:54:16.095 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-16T22:54:16.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.095 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-02-16T22:54:16.113 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-16T22:54:16.113 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-16T22:54:16.113 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-02-16T22:54:16.113 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:54:16.113 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-02-16T22:54:16.113 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:54:16.113 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42308sec preferred_lft 42308sec 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2024-02-16T22:54:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:54:16.115 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-16T22:54:16.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.115 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-02-16T22:54:16.115 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-02-16T22:54:16.115 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add name ceph-brx type bridge 2024-02-16T22:54:16.115 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr flush dev ceph-brx 2024-02-16T22:54:16.115 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set ceph-brx up 2024-02-16T22:54:16.115 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-16T22:54:16.115 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-02-16T22:54:16.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.427 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-16T22:54:16.453 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-02-16T22:54:16.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.455 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-02-16T22:54:16.543 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-02-16T22:54:16.544 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-16T22:54:16.544 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-16T22:54:16.544 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-16T22:54:16.544 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-16T22:54:16.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.544 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-02-16T22:54:16.544 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-02-16T22:54:16.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-16T22:54:16.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-16T22:54:16.545 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-16T22:54:16.545 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-02-16T22:54:16.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:16 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2170111199' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T22:54:16.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:16 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2170111199' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T22:54:16.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.775 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-02-16T22:54:16.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.831 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-02-16T22:54:16.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.886 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-02-16T22:54:16.886 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-02-16T22:54:16.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:54:16.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-16T22:54:16.886 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-02-16T22:54:16.968 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-16T22:54:16.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:16.969 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-02-16T22:54:16.969 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-02-16T22:54:16.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-16T22:54:16.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-16T22:54:16.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-16T22:54:16.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-16T22:54:16.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-16T22:54:16.969 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-02-16T22:54:17.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:17.103 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-02-16T22:54:17.103 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-02-16T22:54:17.103 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2024-02-16T22:54:17.103 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.0 master ceph-brx 2024-02-16T22:54:17.104 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-02-16T22:54:17.145 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-02-16T22:54:17.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T22:54:17.145 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:17.200 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-16T22:54:17.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T22:54:17.200 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:17.254 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-02-16T22:54:17.323 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T22:54:17.378 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-02-16T22:54:17.378 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-02-16T22:54:17.379 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/0 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay 2024-02-16T22:54:17.380 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/4f42ec6e3cc540e1d0ab7e2ce51cd5a5ac7b00273d2f10a54eedbc1bfb05b6bf/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/c5e3c81e8bad1421f3e7d32a8e110836db3ec0d734eaf86f674522a2ef247c76/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/ae8ce06f0adfc2b266d3cd35a356fdca464738d62e6b10f8d5671d63568ce01c/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/fc6395b3efbc26c1bdc4ea0191b1c84fddd77b9f2f45b816cc102de1106e4c5d/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/038fc680f92a8063d7367580bbfd2eb983c9864226100a0830495a8b58dfc8b2/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/c1976384ca5c8a0dc4334901e9a91f1a164ac8918761720ec5e840f447f10709/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/f6e8457a96dced2a1109daf78a11155b8d8e1b090b4d9feca0bd4d71b7826a07/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/51d7da6782d90c4da977eb1168a6e1a8f97d509d247f920f90c7b24d8ed4119c/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/5603d7ea07e4740de9fa3738b19e34a148b4e67fd4d3ad11a988d63a1a956387/merged 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:54:17.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:17.382 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-02-16T22:54:17.433 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-16T22:54:17.451 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-02-16T22:54:17.476 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-02-16T22:54:17.502 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T22:54:17.559 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-02-16T22:54:17.559 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-02-16T22:54:17.560 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/0 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/4f42ec6e3cc540e1d0ab7e2ce51cd5a5ac7b00273d2f10a54eedbc1bfb05b6bf/merged 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/c5e3c81e8bad1421f3e7d32a8e110836db3ec0d734eaf86f674522a2ef247c76/merged 2024-02-16T22:54:17.561 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/ae8ce06f0adfc2b266d3cd35a356fdca464738d62e6b10f8d5671d63568ce01c/merged 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/fc6395b3efbc26c1bdc4ea0191b1c84fddd77b9f2f45b816cc102de1106e4c5d/merged 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/038fc680f92a8063d7367580bbfd2eb983c9864226100a0830495a8b58dfc8b2/merged 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/c1976384ca5c8a0dc4334901e9a91f1a164ac8918761720ec5e840f447f10709/merged 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/f6e8457a96dced2a1109daf78a11155b8d8e1b090b4d9feca0bd4d71b7826a07/merged 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/51d7da6782d90c4da977eb1168a6e1a8f97d509d247f920f90c7b24d8ed4119c/merged 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/5603d7ea07e4740de9fa3738b19e34a148b4e67fd4d3ad11a988d63a1a956387/merged 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:54:17.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:17.562 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-02-16T22:54:17.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2024-02-16T22:54:17.574+0000 7f3aebc39300 -1 init, newargv = 0x563770259090 newargc=15 2024-02-16T22:54:17.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[61402]: starting ceph client 2024-02-16T22:54:17.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[61402]: starting fuse 2024-02-16T22:54:17.590 INFO:teuthology.orchestra.run.smithi192.stdout:115 2024-02-16T22:54:17.590 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-02-16T22:54:17.590 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2024-02-16T22:54:17.590 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2024-02-16T22:54:17.590 DEBUG:teuthology.orchestra.run.smithi192:> import re 2024-02-16T22:54:17.590 DEBUG:teuthology.orchestra.run.smithi192:> import os 2024-02-16T22:54:17.590 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2024-02-16T22:54:17.590 DEBUG:teuthology.orchestra.run.smithi192:> 2024-02-16T22:54:17.590 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2024-02-16T22:54:17.590 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> return f 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> 2024-02-16T22:54:17.591 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2024-02-16T22:54:17.592 DEBUG:teuthology.orchestra.run.smithi192:> ' 2024-02-16T22:54:17.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:17 smithi064 ceph-mon[29226]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:54:17.653 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:17 smithi192 ceph-mon[36254]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:54:17.709 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.61402.asok 2024-02-16T22:54:17.712 INFO:teuthology.orchestra.run.smithi192.stderr:2024-02-16T22:54:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-16T22:54:17.720 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61402.asok 2024-02-16T22:54:17.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:17.720 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61402.asok status 2024-02-16T22:54:17.851 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-02-16T22:54:17.851 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "b88fae66e6ad896087939a6bde358f026d85f561", 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)", 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "61402", 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 24327, 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2024-02-16T22:54:17.852 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 24327 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1356869742 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1356869742 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.24327 192.168.0.1:0/1356869742", 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/1356869742", 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2024-02-16T22:54:17.853 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 10, 2024-02-16T22:54:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 46, 2024-02-16T22:54:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2024-02-16T22:54:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2024-02-16T22:54:17.854 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2024-02-16T22:54:17.854 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-02-16T22:54:17.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:17.860 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-02-16T22:54:17.860 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:17.880 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-02-16T22:54:17.880 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:17.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:17.880 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:17.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:17.956 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2024-02-16T22:54:17.957 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:17.972 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2024-02-16T22:54:17.972 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:17.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:54:17.972 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:18.041 INFO:teuthology.run_tasks:Running task print... 2024-02-16T22:54:18.051 INFO:teuthology.task.print:**** done client 2024-02-16T22:54:18.051 INFO:teuthology.run_tasks:Running task parallel... 2024-02-16T22:54:18.099 INFO:teuthology.task.parallel:starting parallel... 2024-02-16T22:54:18.099 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-16T22:54:18.100 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T22:54:18.100 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:54:18.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-02-16T22:54:18.101 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-16T22:54:18.101 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-16T22:54:18.104 INFO:tasks.workunit:Pulling workunits from ref ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T22:54:18.104 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-16T22:54:18.104 INFO:tasks.workunit:timeout=3h 2024-02-16T22:54:18.104 INFO:tasks.workunit:cleanup=True 2024-02-16T22:54:18.105 DEBUG:teuthology.orchestra.run.smithi064:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:18.123 INFO:teuthology.orchestra.run.smithi064.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:18.123 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-16T22:54:18.124 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 94h/148d Inode: 1 Links: 2 2024-02-16T22:54:18.124 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:54:18.124 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-16T22:54:18.124 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-16T22:54:18.124 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-02-16 22:53:47.820432769 +0000 2024-02-16T22:54:18.124 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-02-16 22:54:17.954165438 +0000 2024-02-16T22:54:18.124 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-02-16T22:54:18.124 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-16T22:54:18.124 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-16T22:54:18.197 DEBUG:teuthology.orchestra.run.smithi192:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-02-16 22:54:18.191366813 +0000 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-02-16 22:54:18.191366813 +0000 2024-02-16T22:54:18.213 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2024-02-16T22:54:18.214 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-16T22:54:18.214 DEBUG:teuthology.orchestra.run.smithi192:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-16T22:54:18.284 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T22:54:18.285 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T22:54:18.300 INFO:tasks.workunit.client.0.smithi064.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-16T22:54:18.340 INFO:tasks.workunit.client.1.smithi192.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-16T22:54:18.379 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:19.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:18 smithi064 ceph-mon[29226]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:54:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:18 smithi192 ceph-mon[36254]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:54:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.908+0000 7f6bf4b20700 1 -- 172.21.15.64:0/4205686611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00fcfb0 msgr2=0x7f6bf00fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.908+0000 7f6bf4b20700 1 --2- 172.21.15.64:0/4205686611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00fcfb0 0x7f6bf00fd420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6be4004940 tx=0x7f6be4009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 -- 172.21.15.64:0/4205686611 shutdown_connections 2024-02-16T22:54:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 --2- 172.21.15.64:0/4205686611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00fcfb0 0x7f6bf00fd420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 --2- 172.21.15.64:0/4205686611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bf00fbe40 0x7f6bf00fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 -- 172.21.15.64:0/4205686611 >> 172.21.15.64:0/4205686611 conn(0x7f6bf00f7520 msgr2=0x7f6bf00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 -- 172.21.15.64:0/4205686611 shutdown_connections 2024-02-16T22:54:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 -- 172.21.15.64:0/4205686611 wait complete. 2024-02-16T22:54:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 Processor -- start 2024-02-16T22:54:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 -- start start 2024-02-16T22:54:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.909+0000 7f6bf4b20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00fbe40 0x7f6bf010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bf4b20700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bf00fcfb0 0x7f6bf010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bf4b20700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf010e480 con 0x7f6bf00fbe40 2024-02-16T22:54:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bf4b20700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf010ed20 con 0x7f6bf00fcfb0 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bee59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00fbe40 0x7f6bf010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bee59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00fbe40 0x7f6bf010d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36188/0 (socket says 172.21.15.64:36188) 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bee59c700 1 -- 172.21.15.64:0/3674666217 learned_addr learned my addr 172.21.15.64:0/3674666217 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bedd9b700 1 --2- 172.21.15.64:0/3674666217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bf00fcfb0 0x7f6bf010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bee59c700 1 -- 172.21.15.64:0/3674666217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bf00fcfb0 msgr2=0x7f6bf010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bee59c700 1 --2- 172.21.15.64:0/3674666217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bf00fcfb0 0x7f6bf010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bee59c700 1 -- 172.21.15.64:0/3674666217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be4005040 con 0x7f6bf00fbe40 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bee59c700 1 --2- 172.21.15.64:0/3674666217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00fbe40 0x7f6bf010d940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6be000c8c0 tx=0x7f6be00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bdeffd700 1 -- 172.21.15.64:0/3674666217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6be000a760 con 0x7f6bf00fbe40 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bf4b20700 1 -- 172.21.15.64:0/3674666217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf010f000 con 0x7f6bf00fbe40 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bdeffd700 1 -- 172.21.15.64:0/3674666217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6be000ad90 con 0x7f6bf00fbe40 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bdeffd700 1 -- 172.21.15.64:0/3674666217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6be00113a0 con 0x7f6bf00fbe40 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.910+0000 7f6bf4b20700 1 -- 172.21.15.64:0/3674666217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf010f540 con 0x7f6bf00fbe40 2024-02-16T22:54:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.911+0000 7f6bf4b20700 1 -- 172.21.15.64:0/3674666217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf004bed0 con 0x7f6bf00fbe40 2024-02-16T22:54:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.912+0000 7f6bdeffd700 1 -- 172.21.15.64:0/3674666217 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6be0012400 con 0x7f6bf00fbe40 2024-02-16T22:54:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.912+0000 7f6bdeffd700 1 --2- 172.21.15.64:0/3674666217 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6bd80692b0 0x7f6bd806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.912+0000 7f6bedd9b700 1 --2- 172.21.15.64:0/3674666217 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6bd80692b0 0x7f6bd806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.912+0000 7f6bdeffd700 1 -- 172.21.15.64:0/3674666217 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6be00871a0 con 0x7f6bf00fbe40 2024-02-16T22:54:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.913+0000 7f6bedd9b700 1 --2- 172.21.15.64:0/3674666217 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f6bd80692b0 0x7f6bd806b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6be40043d0 tx=0x7f6be4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:19.917+0000 7f6bdeffd700 1 -- 172.21.15.64:0/3674666217 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6be0056f60 con 0x7f6bf00fbe40 2024-02-16T22:54:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:20.110+0000 7f6bf4b20700 1 -- 172.21.15.64:0/3674666217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f6bf0104930 con 0x7f6bf00fbe40 2024-02-16T22:54:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:20.111+0000 7f6bdeffd700 1 -- 172.21.15.64:0/3674666217 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f6be0056af0 con 0x7f6bf00fbe40 2024-02-16T22:54:20.113 INFO:teuthology.orchestra.run.smithi064.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-02-16T22:54:20.790 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T22:54:20.791 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:54:20.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-16T22:54:21.101 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:21.462 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:21 smithi064 ceph-mon[29226]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:54:21.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:21 smithi192 ceph-mon[36254]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.844+0000 7f5e9580a700 1 -- 172.21.15.64:0/4293284625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e901000d0 msgr2=0x7f5e90102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.844+0000 7f5e9580a700 1 --2- 172.21.15.64:0/4293284625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e901000d0 0x7f5e90102530 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5e84004940 tx=0x7f5e84009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.844+0000 7f5e9580a700 1 -- 172.21.15.64:0/4293284625 shutdown_connections 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.844+0000 7f5e9580a700 1 --2- 172.21.15.64:0/4293284625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e901000d0 0x7f5e90102530 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.844+0000 7f5e9580a700 1 --2- 172.21.15.64:0/4293284625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e90066c70 0x7f5e900ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.844+0000 7f5e9580a700 1 -- 172.21.15.64:0/4293284625 >> 172.21.15.64:0/4293284625 conn(0x7f5e900f7510 msgr2=0x7f5e900f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.844+0000 7f5e9580a700 1 -- 172.21.15.64:0/4293284625 shutdown_connections 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.844+0000 7f5e9580a700 1 -- 172.21.15.64:0/4293284625 wait complete. 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e9580a700 1 Processor -- start 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e9580a700 1 -- start start 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e9580a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e90066c70 0x7f5e900fd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e9580a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e901000d0 0x7f5e900fba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e9580a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900fc000 con 0x7f5e90066c70 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e9580a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900fc170 con 0x7f5e901000d0 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e90066c70 0x7f5e900fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e90066c70 0x7f5e900fd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35962/0 (socket says 172.21.15.64:35962) 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e8effd700 1 -- 172.21.15.64:0/4119182013 learned_addr learned my addr 172.21.15.64:0/4119182013 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e8e7fc700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e901000d0 0x7f5e900fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e8effd700 1 -- 172.21.15.64:0/4119182013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e901000d0 msgr2=0x7f5e900fba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e8effd700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e901000d0 0x7f5e900fba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e8effd700 1 -- 172.21.15.64:0/4119182013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e84005040 con 0x7f5e90066c70 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e8effd700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e90066c70 0x7f5e900fd580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5e80002a50 tx=0x7f5e800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.845+0000 7f5e777fe700 1 -- 172.21.15.64:0/4119182013 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e80007cc0 con 0x7f5e90066c70 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.846+0000 7f5e777fe700 1 -- 172.21.15.64:0/4119182013 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e8000f040 con 0x7f5e90066c70 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.846+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e900fc450 con 0x7f5e90066c70 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.846+0000 7f5e777fe700 1 -- 172.21.15.64:0/4119182013 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e80009640 con 0x7f5e90066c70 2024-02-16T22:54:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.846+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e901159b0 con 0x7f5e90066c70 2024-02-16T22:54:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.846+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9004bed0 con 0x7f5e90066c70 2024-02-16T22:54:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.847+0000 7f5e777fe700 1 -- 172.21.15.64:0/4119182013 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5e800097a0 con 0x7f5e90066c70 2024-02-16T22:54:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.847+0000 7f5e777fe700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e780692a0 0x7f5e7806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.847+0000 7f5e777fe700 1 -- 172.21.15.64:0/4119182013 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5e80086fc0 con 0x7f5e90066c70 2024-02-16T22:54:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.847+0000 7f5e8e7fc700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e780692a0 0x7f5e7806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:22.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.848+0000 7f5e8e7fc700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e780692a0 0x7f5e7806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009fd0 tx=0x7f5e84009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:22.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:22.852+0000 7f5e777fe700 1 -- 172.21.15.64:0/4119182013 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5e80056d00 con 0x7f5e90066c70 2024-02-16T22:54:23.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.067+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f5e90105a20 con 0x7f5e90066c70 2024-02-16T22:54:23.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.067+0000 7f5e777fe700 1 -- 172.21.15.64:0/4119182013 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f5e80056890 con 0x7f5e90066c70 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e780692a0 msgr2=0x7f5e7806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e780692a0 0x7f5e7806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009fd0 tx=0x7f5e84009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e90066c70 msgr2=0x7f5e900fd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e90066c70 0x7f5e900fd580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5e80002a50 tx=0x7f5e800046b0 comp rx=0 tx=0).stop 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 shutdown_connections 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5e780692a0 0x7f5e7806b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e90066c70 0x7f5e900fd580 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 --2- 172.21.15.64:0/4119182013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e901000d0 0x7f5e900fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 >> 172.21.15.64:0/4119182013 conn(0x7f5e900f7510 msgr2=0x7f5e900f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 shutdown_connections 2024-02-16T22:54:23.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:23.069+0000 7f5e9580a700 1 -- 172.21.15.64:0/4119182013 wait complete. 2024-02-16T22:54:23.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:23 smithi064 ceph-mon[29226]: pgmap v196: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:54:23.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:23 smithi192 ceph-mon[36254]: pgmap v196: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:54:23.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-16T22:54:24.076 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:25.433 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:25 smithi064 ceph-mon[29226]: pgmap v197: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:54:25.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:25 smithi192 ceph-mon[36254]: pgmap v197: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:54:25.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.808+0000 7fb2469fb700 1 -- 172.21.15.64:0/3654076960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 msgr2=0x7fb2400ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:25.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.808+0000 7fb2469fb700 1 --2- 172.21.15.64:0/3654076960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 0x7fb2400ffe70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb230004940 tx=0x7fb230009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:25.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.808+0000 7fb2469fb700 1 -- 172.21.15.64:0/3654076960 shutdown_connections 2024-02-16T22:54:25.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.808+0000 7fb2469fb700 1 --2- 172.21.15.64:0/3654076960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2401003b0 0x7fb240102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:25.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.808+0000 7fb2469fb700 1 --2- 172.21.15.64:0/3654076960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 0x7fb2400ffe70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:25.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.808+0000 7fb2469fb700 1 -- 172.21.15.64:0/3654076960 >> 172.21.15.64:0/3654076960 conn(0x7fb2400f7510 msgr2=0x7fb2400f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:25.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.808+0000 7fb2469fb700 1 -- 172.21.15.64:0/3654076960 shutdown_connections 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.808+0000 7fb2469fb700 1 -- 172.21.15.64:0/3654076960 wait complete. 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb2469fb700 1 Processor -- start 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb2469fb700 1 -- start start 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb2469fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 0x7fb24010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb2469fb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2401003b0 0x7fb24010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb2469fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24010e450 con 0x7fb2400fda90 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb2469fb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24010ecf0 con 0x7fb2401003b0 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb23ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 0x7fb24010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb23ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 0x7fb24010d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35968/0 (socket says 172.21.15.64:35968) 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb23ffff700 1 -- 172.21.15.64:0/852573112 learned_addr learned my addr 172.21.15.64:0/852573112 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb23f7fe700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2401003b0 0x7fb24010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb23ffff700 1 -- 172.21.15.64:0/852573112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2401003b0 msgr2=0x7fb24010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:25.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb23ffff700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2401003b0 0x7fb24010de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:25.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.809+0000 7fb23ffff700 1 -- 172.21.15.64:0/852573112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230005040 con 0x7fb2400fda90 2024-02-16T22:54:25.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.810+0000 7fb23ffff700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 0x7fb24010d910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2300093b0 tx=0x7fb230009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:25.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.810+0000 7fb23cff9700 1 -- 172.21.15.64:0/852573112 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb23000e040 con 0x7fb2400fda90 2024-02-16T22:54:25.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.810+0000 7fb23cff9700 1 -- 172.21.15.64:0/852573112 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb23000bcc0 con 0x7fb2400fda90 2024-02-16T22:54:25.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.810+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24010ef70 con 0x7fb2400fda90 2024-02-16T22:54:25.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.810+0000 7fb23cff9700 1 -- 172.21.15.64:0/852573112 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb23001c630 con 0x7fb2400fda90 2024-02-16T22:54:25.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.810+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24010f4b0 con 0x7fb2400fda90 2024-02-16T22:54:25.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.811+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24004bed0 con 0x7fb2400fda90 2024-02-16T22:54:25.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.811+0000 7fb23cff9700 1 -- 172.21.15.64:0/852573112 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb230013980 con 0x7fb2400fda90 2024-02-16T22:54:25.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.812+0000 7fb23cff9700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb2280692a0 0x7fb22806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:25.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.812+0000 7fb23cff9700 1 -- 172.21.15.64:0/852573112 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb230088540 con 0x7fb2400fda90 2024-02-16T22:54:25.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.812+0000 7fb23f7fe700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb2280692a0 0x7fb22806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:25.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.812+0000 7fb23f7fe700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb2280692a0 0x7fb22806b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb234004f40 tx=0x7fb23400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:25.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:25.816+0000 7fb23cff9700 1 -- 172.21.15.64:0/852573112 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb230058300 con 0x7fb2400fda90 2024-02-16T22:54:26.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.020+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fb24010fb30 con 0x7fb2400fda90 2024-02-16T22:54:26.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.021+0000 7fb23cff9700 1 -- 172.21.15.64:0/852573112 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fb230057e90 con 0x7fb2400fda90 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb2280692a0 msgr2=0x7fb22806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb2280692a0 0x7fb22806b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb234004f40 tx=0x7fb23400b040 comp rx=0 tx=0).stop 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 msgr2=0x7fb24010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 0x7fb24010d910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2300093b0 tx=0x7fb230009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 shutdown_connections 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fb2280692a0 0x7fb22806b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2400fda90 0x7fb24010d910 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 --2- 172.21.15.64:0/852573112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2401003b0 0x7fb24010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.023+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 >> 172.21.15.64:0/852573112 conn(0x7fb2400f7510 msgr2=0x7fb2400f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.024+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 shutdown_connections 2024-02-16T22:54:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:26.024+0000 7fb2469fb700 1 -- 172.21.15.64:0/852573112 wait complete. 2024-02-16T22:54:26.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-16T22:54:26.993 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:27.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:27 smithi064 ceph-mon[29226]: pgmap v198: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T22:54:27.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:27 smithi192 ceph-mon[36254]: pgmap v198: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T22:54:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 -- 172.21.15.64:0/2566759929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 msgr2=0x7f91000fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 --2- 172.21.15.64:0/2566759929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 0x7f91000fd420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f90f0004940 tx=0x7f90f0009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 -- 172.21.15.64:0/2566759929 shutdown_connections 2024-02-16T22:54:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 --2- 172.21.15.64:0/2566759929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 0x7f91000fd420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 --2- 172.21.15.64:0/2566759929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91000fbe40 0x7f91000fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 -- 172.21.15.64:0/2566759929 >> 172.21.15.64:0/2566759929 conn(0x7f91000f7520 msgr2=0x7f91000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 -- 172.21.15.64:0/2566759929 shutdown_connections 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 -- 172.21.15.64:0/2566759929 wait complete. 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 Processor -- start 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.772+0000 7f91077e6700 1 -- start start 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f91077e6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91000fbe40 0x7f9100110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f91077e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 0x7f9100112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f91077e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100066d50 con 0x7f91000fcfb0 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f91077e6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100066e90 con 0x7f91000fbe40 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f9104d81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 0x7f9100112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f9105582700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91000fbe40 0x7f9100110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f9104d81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 0x7f9100112aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35992/0 (socket says 172.21.15.64:35992) 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f9104d81700 1 -- 172.21.15.64:0/3946402927 learned_addr learned my addr 172.21.15.64:0/3946402927 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f9104d81700 1 -- 172.21.15.64:0/3946402927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91000fbe40 msgr2=0x7f9100110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f9104d81700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91000fbe40 0x7f9100110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f9104d81700 1 -- 172.21.15.64:0/3946402927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0005040 con 0x7f91000fcfb0 2024-02-16T22:54:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f9104d81700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 0x7f9100112aa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007d40 tx=0x7f90f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f90f5ffb700 1 -- 172.21.15.64:0/3946402927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f0021070 con 0x7f91000fcfb0 2024-02-16T22:54:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f90f5ffb700 1 -- 172.21.15.64:0/3946402927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90f001c400 con 0x7f91000fcfb0 2024-02-16T22:54:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91001130a0 con 0x7f91000fcfb0 2024-02-16T22:54:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.773+0000 7f90f5ffb700 1 -- 172.21.15.64:0/3946402927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f000e070 con 0x7f91000fcfb0 2024-02-16T22:54:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.774+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9100113580 con 0x7f91000fcfb0 2024-02-16T22:54:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.775+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910004bed0 con 0x7f91000fcfb0 2024-02-16T22:54:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.775+0000 7f90f5ffb700 1 -- 172.21.15.64:0/3946402927 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f90f001c9e0 con 0x7f91000fcfb0 2024-02-16T22:54:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.775+0000 7f90f5ffb700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f90ec0692b0 0x7f90ec06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.776+0000 7f90f5ffb700 1 -- 172.21.15.64:0/3946402927 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f90f0088380 con 0x7f91000fcfb0 2024-02-16T22:54:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.776+0000 7f9105582700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f90ec0692b0 0x7f90ec06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:28.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.776+0000 7f9105582700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f90ec0692b0 0x7f90ec06b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91000669d0 tx=0x7f90fc00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:28.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.781+0000 7f90f5ffb700 1 -- 172.21.15.64:0/3946402927 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f90f00546b0 con 0x7f91000fcfb0 2024-02-16T22:54:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.978+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f9100104930 con 0x7f91000fcfb0 2024-02-16T22:54:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.978+0000 7f90f5ffb700 1 -- 172.21.15.64:0/3946402927 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9100104930 con 0x7f91000fcfb0 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f90ec0692b0 msgr2=0x7f90ec06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f90ec0692b0 0x7f90ec06b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91000669d0 tx=0x7f90fc00b040 comp rx=0 tx=0).stop 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 msgr2=0x7f9100112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 0x7f9100112aa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007d40 tx=0x7f90f0009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 shutdown_connections 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91000fbe40 0x7f9100110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f90ec0692b0 0x7f90ec06b760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 --2- 172.21.15.64:0/3946402927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91000fcfb0 0x7f9100112aa0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.980+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 >> 172.21.15.64:0/3946402927 conn(0x7f91000f7520 msgr2=0x7f9100100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.981+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 shutdown_connections 2024-02-16T22:54:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:28.981+0000 7f91077e6700 1 -- 172.21.15.64:0/3946402927 wait complete. 2024-02-16T22:54:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:28 smithi064 ceph-mon[29226]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-16T22:54:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:54:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:28 smithi192 ceph-mon[36254]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-16T22:54:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:54:29.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-16T22:54:29.877 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:31.500 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:31 smithi064 ceph-mon[29226]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:54:31.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.516+0000 7f3b78f9f700 1 -- 172.21.15.64:0/1542386717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 msgr2=0x7f3b740fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:31.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.516+0000 7f3b78f9f700 1 --2- 172.21.15.64:0/1542386717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 0x7f3b740fe700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b68004940 tx=0x7f3b68009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 -- 172.21.15.64:0/1542386717 shutdown_connections 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 --2- 172.21.15.64:0/1542386717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 0x7f3b740fe700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 --2- 172.21.15.64:0/1542386717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b740fb220 0x7f3b740fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 -- 172.21.15.64:0/1542386717 >> 172.21.15.64:0/1542386717 conn(0x7f3b740f6ca0 msgr2=0x7f3b740f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 -- 172.21.15.64:0/1542386717 shutdown_connections 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 -- 172.21.15.64:0/1542386717 wait complete. 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 Processor -- start 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 -- start start 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b740fb220 0x7f3b7410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 0x7f3b7410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7410e4c0 con 0x7f3b740fe2b0 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b78f9f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7410ed60 con 0x7f3b740fb220 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.517+0000 7f3b71d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 0x7f3b7410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b71d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 0x7f3b7410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48176/0 (socket says 172.21.15.64:48176) 2024-02-16T22:54:31.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b71d9b700 1 -- 172.21.15.64:0/3585546127 learned_addr learned my addr 172.21.15.64:0/3585546127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:31.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b7259c700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b740fb220 0x7f3b7410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:31.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b71d9b700 1 -- 172.21.15.64:0/3585546127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b740fb220 msgr2=0x7f3b7410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:31.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b71d9b700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b740fb220 0x7f3b7410d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:31.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b71d9b700 1 -- 172.21.15.64:0/3585546127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b68005040 con 0x7f3b740fe2b0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b71d9b700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 0x7f3b7410dec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009b30 tx=0x7f3b68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b62ffd700 1 -- 172.21.15.64:0/3585546127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b68021070 con 0x7f3b740fe2b0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b62ffd700 1 -- 172.21.15.64:0/3585546127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b6801c400 con 0x7f3b740fe2b0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7410efe0 con 0x7f3b740fe2b0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b62ffd700 1 -- 172.21.15.64:0/3585546127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b6800e070 con 0x7f3b740fe2b0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.518+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7410f4c0 con 0x7f3b740fe2b0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.519+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7404bed0 con 0x7f3b740fe2b0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.520+0000 7f3b62ffd700 1 -- 172.21.15.64:0/3585546127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3b6801ca00 con 0x7f3b740fe2b0 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.520+0000 7f3b62ffd700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f3b5c0692a0 0x7f3b5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.520+0000 7f3b62ffd700 1 -- 172.21.15.64:0/3585546127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3b68088230 con 0x7f3b740fe2b0 2024-02-16T22:54:31.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.520+0000 7f3b7259c700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f3b5c0692a0 0x7f3b5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:31.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.520+0000 7f3b7259c700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f3b5c0692a0 0x7f3b5c06b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009b80 tx=0x7f3b6400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.525+0000 7f3b62ffd700 1 -- 172.21.15.64:0/3585546127 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3b68058090 con 0x7f3b740fe2b0 2024-02-16T22:54:31.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:31 smithi192 ceph-mon[36254]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:54:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.741+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b7410f970 con 0x7f3b5c0692a0 2024-02-16T22:54:31.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.746+0000 7f3b62ffd700 1 -- 172.21.15.64:0/3585546127 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f3b7410f970 con 0x7f3b5c0692a0 2024-02-16T22:54:31.747 INFO:teuthology.orchestra.run.smithi064.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f3b5c0692a0 msgr2=0x7f3b5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f3b5c0692a0 0x7f3b5c06b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009b80 tx=0x7f3b6400b040 comp rx=0 tx=0).stop 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 msgr2=0x7f3b7410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 0x7f3b7410dec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009b30 tx=0x7f3b68009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 shutdown_connections 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b740fb220 0x7f3b7410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f3b5c0692a0 0x7f3b5c06b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 --2- 172.21.15.64:0/3585546127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740fe2b0 0x7f3b7410dec0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 >> 172.21.15.64:0/3585546127 conn(0x7f3b740f6ca0 msgr2=0x7f3b741014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 shutdown_connections 2024-02-16T22:54:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:31.748+0000 7f3b78f9f700 1 -- 172.21.15.64:0/3585546127 wait complete. 2024-02-16T22:54:32.352 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T22:54:32.352 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T22:54:32.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-02-16T22:54:32.643 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:32 smithi192 ceph-mon[36254]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:54:33.059 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:33.060 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:54:33.060 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:33.060 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:54:33.060 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T22:54:33.060 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:54:33.060 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:54:33.060 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:54:33.060 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:32 smithi064 ceph-mon[29226]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:54:34.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.472+0000 7fac36f28700 1 -- 172.21.15.64:0/1528702124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fcfb0 msgr2=0x7fac300fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:34.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.472+0000 7fac36f28700 1 --2- 172.21.15.64:0/1528702124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fcfb0 0x7fac300fd420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fac1c004940 tx=0x7fac1c009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.472+0000 7fac36f28700 1 -- 172.21.15.64:0/1528702124 shutdown_connections 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.472+0000 7fac36f28700 1 --2- 172.21.15.64:0/1528702124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fcfb0 0x7fac300fd420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.472+0000 7fac36f28700 1 --2- 172.21.15.64:0/1528702124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac300fbe40 0x7fac300fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.472+0000 7fac36f28700 1 -- 172.21.15.64:0/1528702124 >> 172.21.15.64:0/1528702124 conn(0x7fac300f7500 msgr2=0x7fac300f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.472+0000 7fac36f28700 1 -- 172.21.15.64:0/1528702124 shutdown_connections 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.472+0000 7fac36f28700 1 -- 172.21.15.64:0/1528702124 wait complete. 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac36f28700 1 Processor -- start 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac36f28700 1 -- start start 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac36f28700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fbe40 0x7fac30113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac36f28700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac300fcfb0 0x7fac30114410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac36f28700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac30114a10 con 0x7fac300fbe40 2024-02-16T22:54:34.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac36f28700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac301152b0 con 0x7fac300fcfb0 2024-02-16T22:54:34.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac34cc4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fbe40 0x7fac30113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:34.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac34cc4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fbe40 0x7fac30113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48192/0 (socket says 172.21.15.64:48192) 2024-02-16T22:54:34.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.473+0000 7fac34cc4700 1 -- 172.21.15.64:0/787647882 learned_addr learned my addr 172.21.15.64:0/787647882 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:34.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.474+0000 7fac2ffff700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac300fcfb0 0x7fac30114410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:34.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.474+0000 7fac34cc4700 1 -- 172.21.15.64:0/787647882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac300fcfb0 msgr2=0x7fac30114410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:34.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.474+0000 7fac34cc4700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac300fcfb0 0x7fac30114410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.474+0000 7fac34cc4700 1 -- 172.21.15.64:0/787647882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c005040 con 0x7fac300fbe40 2024-02-16T22:54:34.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.474+0000 7fac34cc4700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fbe40 0x7fac30113ed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac2400eb40 tx=0x7fac2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.474+0000 7fac2d7fa700 1 -- 172.21.15.64:0/787647882 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac24007cc0 con 0x7fac300fbe40 2024-02-16T22:54:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.474+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac30115530 con 0x7fac300fbe40 2024-02-16T22:54:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.474+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac30115a70 con 0x7fac300fbe40 2024-02-16T22:54:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.475+0000 7fac2d7fa700 1 -- 172.21.15.64:0/787647882 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac2400f040 con 0x7fac300fbe40 2024-02-16T22:54:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.475+0000 7fac2d7fa700 1 -- 172.21.15.64:0/787647882 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac2400ca10 con 0x7fac300fbe40 2024-02-16T22:54:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.476+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3004bed0 con 0x7fac300fbe40 2024-02-16T22:54:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.476+0000 7fac2d7fa700 1 -- 172.21.15.64:0/787647882 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fac2400cb70 con 0x7fac300fbe40 2024-02-16T22:54:34.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.477+0000 7fac2d7fa700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fac200692b0 0x7fac2006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:34.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.477+0000 7fac2d7fa700 1 -- 172.21.15.64:0/787647882 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fac24086c90 con 0x7fac300fbe40 2024-02-16T22:54:34.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.477+0000 7fac2ffff700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fac200692b0 0x7fac2006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:34.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.477+0000 7fac2ffff700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fac200692b0 0x7fac2006b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0136a0 tx=0x7fac1c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:34.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.482+0000 7fac2d7fa700 1 -- 172.21.15.64:0/787647882 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fac2404dbd0 con 0x7fac300fbe40 2024-02-16T22:54:34.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.778+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac300f9040 con 0x7fac200692b0 2024-02-16T22:54:34.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.779+0000 7fac2d7fa700 1 -- 172.21.15.64:0/787647882 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fac300f9040 con 0x7fac200692b0 2024-02-16T22:54:34.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fac200692b0 msgr2=0x7fac2006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:34.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fac200692b0 0x7fac2006b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0136a0 tx=0x7fac1c009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:34.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fbe40 msgr2=0x7fac30113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:34.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fbe40 0x7fac30113ed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac2400eb40 tx=0x7fac2400b040 comp rx=0 tx=0).stop 2024-02-16T22:54:34.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 shutdown_connections 2024-02-16T22:54:34.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fac200692b0 0x7fac2006b760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac300fbe40 0x7fac30113ed0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 --2- 172.21.15.64:0/787647882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac300fcfb0 0x7fac30114410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 >> 172.21.15.64:0/787647882 conn(0x7fac300f7500 msgr2=0x7fac300f7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:34.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 shutdown_connections 2024-02-16T22:54:34.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.782+0000 7fac36f28700 1 -- 172.21.15.64:0/787647882 wait complete. 2024-02-16T22:54:34.794 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:54:34.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 -- 172.21.15.64:0/2324300362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 msgr2=0x7f83900ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:34.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 --2- 172.21.15.64:0/2324300362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 0x7f83900ff580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8380004940 tx=0x7f8380009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:34.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 -- 172.21.15.64:0/2324300362 shutdown_connections 2024-02-16T22:54:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 --2- 172.21.15.64:0/2324300362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8390100380 0x7f83901007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 --2- 172.21.15.64:0/2324300362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 0x7f83900ff580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 -- 172.21.15.64:0/2324300362 >> 172.21.15.64:0/2324300362 conn(0x7f83900fa920 msgr2=0x7f83900fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 -- 172.21.15.64:0/2324300362 shutdown_connections 2024-02-16T22:54:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 -- 172.21.15.64:0/2324300362 wait complete. 2024-02-16T22:54:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.907+0000 7f8395b8a700 1 Processor -- start 2024-02-16T22:54:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f8395b8a700 1 -- start start 2024-02-16T22:54:34.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f8395b8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 0x7f839010fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:34.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f8395b8a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8390100380 0x7f8390110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:34.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f8395b8a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390110660 con 0x7f83900ff170 2024-02-16T22:54:34.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f8395b8a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390110f00 con 0x7f8390100380 2024-02-16T22:54:34.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f838f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 0x7f839010fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:34.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f838f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 0x7f839010fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48216/0 (socket says 172.21.15.64:48216) 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f838f7fe700 1 -- 172.21.15.64:0/3474210023 learned_addr learned my addr 172.21.15.64:0/3474210023 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f838effd700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8390100380 0x7f8390110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f838f7fe700 1 -- 172.21.15.64:0/3474210023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8390100380 msgr2=0x7f8390110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f838f7fe700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8390100380 0x7f8390110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.908+0000 7f838f7fe700 1 -- 172.21.15.64:0/3474210023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8380005040 con 0x7f83900ff170 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.909+0000 7f838f7fe700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 0x7f839010fb20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f838000b3b0 tx=0x7f8380009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.909+0000 7f8377fff700 1 -- 172.21.15.64:0/3474210023 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f838000e040 con 0x7f83900ff170 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.909+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8390111180 con 0x7f83900ff170 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.909+0000 7f8377fff700 1 -- 172.21.15.64:0/3474210023 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f838000bcc0 con 0x7f83900ff170 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.909+0000 7f8377fff700 1 -- 172.21.15.64:0/3474210023 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f838001c630 con 0x7f83900ff170 2024-02-16T22:54:34.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.909+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83901116c0 con 0x7f83900ff170 2024-02-16T22:54:34.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.911+0000 7f8377fff700 1 -- 172.21.15.64:0/3474210023 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8380013420 con 0x7f83900ff170 2024-02-16T22:54:34.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.911+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839004bed0 con 0x7f83900ff170 2024-02-16T22:54:34.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.911+0000 7f8377fff700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f83780692a0 0x7f837806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:34.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.912+0000 7f8377fff700 1 -- 172.21.15.64:0/3474210023 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8380088670 con 0x7f83900ff170 2024-02-16T22:54:34.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.912+0000 7f838effd700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f83780692a0 0x7f837806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:34.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.913+0000 7f838effd700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f83780692a0 0x7f837806b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8384009fd0 tx=0x7f838400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:34.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:34.920+0000 7f8377fff700 1 -- 172.21.15.64:0/3474210023 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8380058430 con 0x7f83900ff170 2024-02-16T22:54:35.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.180+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8390104cd0 con 0x7f83780692a0 2024-02-16T22:54:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.181+0000 7f8377fff700 1 -- 172.21.15.64:0/3474210023 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f8390104cd0 con 0x7f83780692a0 2024-02-16T22:54:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f83780692a0 msgr2=0x7f837806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f83780692a0 0x7f837806b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8384009fd0 tx=0x7f838400b040 comp rx=0 tx=0).stop 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 msgr2=0x7f839010fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 0x7f839010fb20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f838000b3b0 tx=0x7f8380009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 shutdown_connections 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f83780692a0 0x7f837806b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83900ff170 0x7f839010fb20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 --2- 172.21.15.64:0/3474210023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8390100380 0x7f8390110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.183+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 >> 172.21.15.64:0/3474210023 conn(0x7f83900fa920 msgr2=0x7f83901035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.184+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 shutdown_connections 2024-02-16T22:54:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.184+0000 7f8395b8a700 1 -- 172.21.15.64:0/3474210023 wait complete. 2024-02-16T22:54:35.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.304+0000 7f5aae3f4700 1 -- 172.21.15.64:0/4253556068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa81003a0 msgr2=0x7f5aa8102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:35.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.304+0000 7f5aae3f4700 1 --2- 172.21.15.64:0/4253556068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa81003a0 0x7f5aa8102780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c004940 tx=0x7f5a9c009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:35.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.304+0000 7f5aae3f4700 1 -- 172.21.15.64:0/4253556068 shutdown_connections 2024-02-16T22:54:35.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.304+0000 7f5aae3f4700 1 --2- 172.21.15.64:0/4253556068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa81003a0 0x7f5aa8102780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.304+0000 7f5aae3f4700 1 --2- 172.21.15.64:0/4253556068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa80fda80 0x7f5aa80ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.304+0000 7f5aae3f4700 1 -- 172.21.15.64:0/4253556068 >> 172.21.15.64:0/4253556068 conn(0x7f5aa80f74e0 msgr2=0x7f5aa80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:35.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.304+0000 7f5aae3f4700 1 -- 172.21.15.64:0/4253556068 shutdown_connections 2024-02-16T22:54:35.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.304+0000 7f5aae3f4700 1 -- 172.21.15.64:0/4253556068 wait complete. 2024-02-16T22:54:35.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.305+0000 7f5aae3f4700 1 Processor -- start 2024-02-16T22:54:35.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.305+0000 7f5aae3f4700 1 -- start start 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.305+0000 7f5aae3f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa80fda80 0x7f5aa810d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.305+0000 7f5aae3f4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa81003a0 0x7f5aa810de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.305+0000 7f5aae3f4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa810e410 con 0x7f5aa80fda80 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.305+0000 7f5aae3f4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa810ecb0 con 0x7f5aa81003a0 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa80fda80 0x7f5aa810d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa80fda80 0x7f5aa810d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48242/0 (socket says 172.21.15.64:48242) 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa7fff700 1 -- 172.21.15.64:0/2080831551 learned_addr learned my addr 172.21.15.64:0/2080831551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa77fe700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa81003a0 0x7f5aa810de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa7fff700 1 -- 172.21.15.64:0/2080831551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa81003a0 msgr2=0x7f5aa810de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:35.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa7fff700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa81003a0 0x7f5aa810de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa7fff700 1 -- 172.21.15.64:0/2080831551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c005040 con 0x7f5aa80fda80 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa7fff700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa80fda80 0x7f5aa810d8d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800eb70 tx=0x7f5a9800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa4ff9700 1 -- 172.21.15.64:0/2080831551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a98007cc0 con 0x7f5aa80fda80 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa810ef30 con 0x7f5aa80fda80 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa4ff9700 1 -- 172.21.15.64:0/2080831551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a9800f040 con 0x7f5aa80fda80 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aa4ff9700 1 -- 172.21.15.64:0/2080831551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a9800ca90 con 0x7f5aa80fda80 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.306+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa810f470 con 0x7f5aa80fda80 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.308+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa804bed0 con 0x7f5aa80fda80 2024-02-16T22:54:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.308+0000 7f5aa4ff9700 1 -- 172.21.15.64:0/2080831551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5a9800cbf0 con 0x7f5aa80fda80 2024-02-16T22:54:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.308+0000 7f5aa4ff9700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5a900692b0 0x7f5a9006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.308+0000 7f5aa4ff9700 1 -- 172.21.15.64:0/2080831551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5a98086d20 con 0x7f5aa80fda80 2024-02-16T22:54:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.308+0000 7f5aa77fe700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5a900692b0 0x7f5a9006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.309+0000 7f5aa77fe700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5a900692b0 0x7f5a9006b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c013740 tx=0x7f5a9c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.314+0000 7f5aa4ff9700 1 -- 172.21.15.64:0/2080831551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5a98056a60 con 0x7f5aa80fda80 2024-02-16T22:54:35.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:35 smithi192 ceph-mon[36254]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T22:54:35.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.588+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5aa810f870 con 0x7f5a900692b0 2024-02-16T22:54:35.590 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:35 smithi064 ceph-mon[29226]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T22:54:35.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.602+0000 7f5aa4ff9700 1 -- 172.21.15.64:0/2080831551 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f5aa810f870 con 0x7f5a900692b0 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (4m) 24s ago 6m 25.1M - 0.25.0 c8568f914cd2 829b066b8c26 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (6m) 24s ago 6m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (5m) 29s ago 5m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (4m) 24s ago 5m 86.0M - 9.4.7 954c08fa6188 529226ab33c5 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (43s) 24s ago 42s 14.7M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (48s) 24s ago 48s 14.5M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (40s) 29s ago 40s 13.5M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (45s) 29s ago 45s 13.5M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:9283 running (7m) 24s ago 7m 480M - 17.2.7-513-gb88fae66 870a0944ccb8 7189c74ed7fc 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (5m) 29s ago 5m 433M - 17.2.7-513-gb88fae66 870a0944ccb8 a017e081156a 2024-02-16T22:54:35.603 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (7m) 24s ago 7m 45.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (5m) 29s ago 5m 36.1M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (6m) 24s ago 6m 15.5M - 1.5.0 0da6a335fe13 c25d924fa381 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 29s ago 5m 16.1M - 1.5.0 0da6a335fe13 c32798cb2969 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (4m) 24s ago 4m 46.5M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (3m) 24s ago 3m 45.3M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (3m) 24s ago 3m 44.7M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (3m) 29s ago 3m 43.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (2m) 29s ago 2m 41.9M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (2m) 29s ago 2m 41.5M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:54:35.604 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (4m) 24s ago 5m 53.4M - 2.43.0 a07b618ecd1d dd20eff0542c 2024-02-16T22:54:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.604+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5a900692b0 msgr2=0x7f5a9006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.604+0000 7f5aae3f4700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5a900692b0 0x7f5a9006b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c013740 tx=0x7f5a9c009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa80fda80 msgr2=0x7f5aa810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa80fda80 0x7f5aa810d8d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800eb70 tx=0x7f5a9800b040 comp rx=0 tx=0).stop 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 shutdown_connections 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f5a900692b0 0x7f5a9006b760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa80fda80 0x7f5aa810d8d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 --2- 172.21.15.64:0/2080831551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa81003a0 0x7f5aa810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 >> 172.21.15.64:0/2080831551 conn(0x7f5aa80f74e0 msgr2=0x7f5aa80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 shutdown_connections 2024-02-16T22:54:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.605+0000 7f5aae3f4700 1 -- 172.21.15.64:0/2080831551 wait complete. 2024-02-16T22:54:35.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.725+0000 7fc95bda6700 1 -- 172.21.15.64:0/1706158041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fcfb0 msgr2=0x7fc9540fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:35.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.725+0000 7fc95bda6700 1 --2- 172.21.15.64:0/1706158041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fcfb0 0x7fc9540fd420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc944004940 tx=0x7fc944009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:35.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.725+0000 7fc95bda6700 1 -- 172.21.15.64:0/1706158041 shutdown_connections 2024-02-16T22:54:35.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.725+0000 7fc95bda6700 1 --2- 172.21.15.64:0/1706158041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fcfb0 0x7fc9540fd420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.725+0000 7fc95bda6700 1 --2- 172.21.15.64:0/1706158041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9540fbe40 0x7fc9540fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.725+0000 7fc95bda6700 1 -- 172.21.15.64:0/1706158041 >> 172.21.15.64:0/1706158041 conn(0x7fc9540f74e0 msgr2=0x7fc9540f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:35.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.725+0000 7fc95bda6700 1 -- 172.21.15.64:0/1706158041 shutdown_connections 2024-02-16T22:54:35.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.725+0000 7fc95bda6700 1 -- 172.21.15.64:0/1706158041 wait complete. 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc95bda6700 1 Processor -- start 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc95bda6700 1 -- start start 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc95bda6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fbe40 0x7fc95410c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc95bda6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9540fcfb0 0x7fc95410cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc95bda6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95410d3d0 con 0x7fc9540fbe40 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc95bda6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95410dc70 con 0x7fc9540fcfb0 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc959b42700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fbe40 0x7fc95410c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc959b42700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fbe40 0x7fc95410c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48250/0 (socket says 172.21.15.64:48250) 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc959b42700 1 -- 172.21.15.64:0/1662670035 learned_addr learned my addr 172.21.15.64:0/1662670035 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:35.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc959341700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9540fcfb0 0x7fc95410cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc959b42700 1 -- 172.21.15.64:0/1662670035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9540fcfb0 msgr2=0x7fc95410cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.726+0000 7fc959b42700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9540fcfb0 0x7fc95410cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.727+0000 7fc959b42700 1 -- 172.21.15.64:0/1662670035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944005040 con 0x7fc9540fbe40 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.727+0000 7fc959b42700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fbe40 0x7fc95410c890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc950002a50 tx=0x7fc9500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.727+0000 7fc94a7fc700 1 -- 172.21.15.64:0/1662670035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc950007cc0 con 0x7fc9540fbe40 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.727+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95410df50 con 0x7fc9540fbe40 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.727+0000 7fc94a7fc700 1 -- 172.21.15.64:0/1662670035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc95000f040 con 0x7fc9540fbe40 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.727+0000 7fc94a7fc700 1 -- 172.21.15.64:0/1662670035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc95000da70 con 0x7fc9540fbe40 2024-02-16T22:54:35.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.727+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95410e490 con 0x7fc9540fbe40 2024-02-16T22:54:35.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.728+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95404bed0 con 0x7fc9540fbe40 2024-02-16T22:54:35.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.728+0000 7fc94a7fc700 1 -- 172.21.15.64:0/1662670035 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc95000dc10 con 0x7fc9540fbe40 2024-02-16T22:54:35.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.729+0000 7fc94a7fc700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc9400692b0 0x7fc94006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:35.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.729+0000 7fc959341700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc9400692b0 0x7fc94006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:35.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.729+0000 7fc94a7fc700 1 -- 172.21.15.64:0/1662670035 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc950086820 con 0x7fc9540fbe40 2024-02-16T22:54:35.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.729+0000 7fc959341700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc9400692b0 0x7fc94006b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc9440043d0 tx=0x7fc944009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:35.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:35.737+0000 7fc94a7fc700 1 -- 172.21.15.64:0/1662670035 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc950056560 con 0x7fc9540fbe40 2024-02-16T22:54:36.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.093+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc954104950 con 0x7fc9540fbe40 2024-02-16T22:54:36.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.093+0000 7fc94a7fc700 1 -- 172.21.15.64:0/1662670035 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fc9500560f0 con 0x7fc9540fbe40 2024-02-16T22:54:36.094 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:54:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:54:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 14 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:54:36.095 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:54:36.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc9400692b0 msgr2=0x7fc94006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc9400692b0 0x7fc94006b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc9440043d0 tx=0x7fc944009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:36.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fbe40 msgr2=0x7fc95410c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fbe40 0x7fc95410c890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc950002a50 tx=0x7fc9500046b0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 shutdown_connections 2024-02-16T22:54:36.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fc9400692b0 0x7fc94006b760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9540fbe40 0x7fc95410c890 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 --2- 172.21.15.64:0/1662670035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9540fcfb0 0x7fc95410cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 >> 172.21.15.64:0/1662670035 conn(0x7fc9540f74e0 msgr2=0x7fc954100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:36.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 shutdown_connections 2024-02-16T22:54:36.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.096+0000 7fc95bda6700 1 -- 172.21.15.64:0/1662670035 wait complete. 2024-02-16T22:54:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.225+0000 7ff4096ba700 1 -- 172.21.15.64:0/3533073498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4040fbe40 msgr2=0x7ff4040fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.225+0000 7ff4096ba700 1 --2- 172.21.15.64:0/3533073498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4040fbe40 0x7ff4040fc250 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4004940 tx=0x7ff3f4009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.225+0000 7ff4096ba700 1 -- 172.21.15.64:0/3533073498 shutdown_connections 2024-02-16T22:54:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.225+0000 7ff4096ba700 1 --2- 172.21.15.64:0/3533073498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4040fcfb0 0x7ff4040fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.225+0000 7ff4096ba700 1 --2- 172.21.15.64:0/3533073498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4040fbe40 0x7ff4040fc250 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.225+0000 7ff4096ba700 1 -- 172.21.15.64:0/3533073498 >> 172.21.15.64:0/3533073498 conn(0x7ff4040f74e0 msgr2=0x7ff4040f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.225+0000 7ff4096ba700 1 -- 172.21.15.64:0/3533073498 shutdown_connections 2024-02-16T22:54:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.225+0000 7ff4096ba700 1 -- 172.21.15.64:0/3533073498 wait complete. 2024-02-16T22:54:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4096ba700 1 Processor -- start 2024-02-16T22:54:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4096ba700 1 -- start start 2024-02-16T22:54:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4096ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4040fbe40 0x7ff4040fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4096ba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4040fcfb0 0x7ff4040fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4096ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040fe980 con 0x7ff4040fbe40 2024-02-16T22:54:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4096ba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040feaf0 con 0x7ff4040fcfb0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4027fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4040fcfb0 0x7ff4040fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4027fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4040fcfb0 0x7ff4040fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:55790/0 (socket says 172.21.15.64:55790) 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff4027fc700 1 -- 172.21.15.64:0/561333220 learned_addr learned my addr 172.21.15.64:0/561333220 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.226+0000 7ff402ffd700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4040fbe40 0x7ff4040fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff4027fc700 1 -- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4040fbe40 msgr2=0x7ff4040fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff4027fc700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4040fbe40 0x7ff4040fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff4027fc700 1 -- 172.21.15.64:0/561333220 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4005040 con 0x7ff4040fcfb0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff4027fc700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4040fcfb0 0x7ff4040fe380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800eb40 tx=0x7ff3f800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff3eb7fe700 1 -- 172.21.15.64:0/561333220 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f8007cc0 con 0x7ff4040fcfb0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff3eb7fe700 1 -- 172.21.15.64:0/561333220 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3f800f040 con 0x7ff4040fcfb0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff3eb7fe700 1 -- 172.21.15.64:0/561333220 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f8009620 con 0x7ff4040fcfb0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4040fed70 con 0x7ff4040fcfb0 2024-02-16T22:54:36.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.227+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4041158e0 con 0x7ff4040fcfb0 2024-02-16T22:54:36.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.228+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40404bed0 con 0x7ff4040fcfb0 2024-02-16T22:54:36.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.229+0000 7ff3eb7fe700 1 -- 172.21.15.64:0/561333220 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff3f8009780 con 0x7ff4040fcfb0 2024-02-16T22:54:36.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.229+0000 7ff3eb7fe700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff3ec0692b0 0x7ff3ec06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:36.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.230+0000 7ff3eb7fe700 1 -- 172.21.15.64:0/561333220 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff3f8086ff0 con 0x7ff4040fcfb0 2024-02-16T22:54:36.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.230+0000 7ff402ffd700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff3ec0692b0 0x7ff3ec06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:36.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.230+0000 7ff402ffd700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff3ec0692b0 0x7ff3ec06b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400b0d0 tx=0x7ff3f4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.233+0000 7ff3eb7fe700 1 -- 172.21.15.64:0/561333220 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff3f8056e80 con 0x7ff4040fcfb0 2024-02-16T22:54:36.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:36 smithi064 ceph-mon[29226]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:36.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:36 smithi064 ceph-mon[29226]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:36.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:36 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1662670035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:54:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.517+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff404104930 con 0x7ff4040fcfb0 2024-02-16T22:54:36.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.517+0000 7ff3eb7fe700 1 -- 172.21.15.64:0/561333220 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7ff3f8056a10 con 0x7ff4040fcfb0 2024-02-16T22:54:36.518 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:54:36.518 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:54:36.518 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:54:36.518 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:54:36.518 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:54:36.519 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:54:36.520 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:54:36.521 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:54:36.521 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:54:36.521 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:54:36.521 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:54:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff3ec0692b0 msgr2=0x7ff3ec06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff3ec0692b0 0x7ff3ec06b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400b0d0 tx=0x7ff3f4009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4040fcfb0 msgr2=0x7ff4040fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4040fcfb0 0x7ff4040fe380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800eb40 tx=0x7ff3f800b040 comp rx=0 tx=0).stop 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 shutdown_connections 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff3ec0692b0 0x7ff3ec06b760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4040fbe40 0x7ff4040fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 --2- 172.21.15.64:0/561333220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4040fcfb0 0x7ff4040fe380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 >> 172.21.15.64:0/561333220 conn(0x7ff4040f74e0 msgr2=0x7ff4040656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 shutdown_connections 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.520+0000 7ff4096ba700 1 -- 172.21.15.64:0/561333220 wait complete. 2024-02-16T22:54:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:54:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:36 smithi192 ceph-mon[36254]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:36 smithi192 ceph-mon[36254]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:36 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1662670035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:54:36.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.643+0000 7fa29eea9700 1 -- 172.21.15.64:0/3657595846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 msgr2=0x7fa2980fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.643+0000 7fa29eea9700 1 --2- 172.21.15.64:0/3657595846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 0x7fa2980fe700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa284004940 tx=0x7fa284009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.643+0000 7fa29eea9700 1 -- 172.21.15.64:0/3657595846 shutdown_connections 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.643+0000 7fa29eea9700 1 --2- 172.21.15.64:0/3657595846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 0x7fa2980fe700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.643+0000 7fa29eea9700 1 --2- 172.21.15.64:0/3657595846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2980fb220 0x7fa2980fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.643+0000 7fa29eea9700 1 -- 172.21.15.64:0/3657595846 >> 172.21.15.64:0/3657595846 conn(0x7fa2980f6ca0 msgr2=0x7fa2980f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.643+0000 7fa29eea9700 1 -- 172.21.15.64:0/3657595846 shutdown_connections 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.643+0000 7fa29eea9700 1 -- 172.21.15.64:0/3657595846 wait complete. 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29eea9700 1 Processor -- start 2024-02-16T22:54:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29eea9700 1 -- start start 2024-02-16T22:54:36.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29eea9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2980fb220 0x7fa29810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:36.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29eea9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 0x7fa29810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29eea9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29810e490 con 0x7fa2980fe2b0 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29eea9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29810ed30 con 0x7fa2980fb220 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa297fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 0x7fa29810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29cc45700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2980fb220 0x7fa29810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29cc45700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2980fb220 0x7fa29810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:55800/0 (socket says 172.21.15.64:55800) 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa297fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 0x7fa29810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48292/0 (socket says 172.21.15.64:48292) 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.644+0000 7fa29cc45700 1 -- 172.21.15.64:0/782514256 learned_addr learned my addr 172.21.15.64:0/782514256 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa29cc45700 1 -- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 msgr2=0x7fa29810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa29cc45700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 0x7fa29810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa29cc45700 1 -- 172.21.15.64:0/782514256 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284005040 con 0x7fa2980fb220 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa297fff700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 0x7fa29810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa29cc45700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2980fb220 0x7fa29810d950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00eb70 tx=0x7fa28c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa2957fa700 1 -- 172.21.15.64:0/782514256 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa28c007cc0 con 0x7fa2980fb220 2024-02-16T22:54:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29810efb0 con 0x7fa2980fb220 2024-02-16T22:54:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29810f4f0 con 0x7fa2980fb220 2024-02-16T22:54:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa2957fa700 1 -- 172.21.15.64:0/782514256 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa28c00f040 con 0x7fa2980fb220 2024-02-16T22:54:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.645+0000 7fa2957fa700 1 -- 172.21.15.64:0/782514256 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa28c00cb20 con 0x7fa2980fb220 2024-02-16T22:54:36.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.646+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c0052e0 con 0x7fa2980fb220 2024-02-16T22:54:36.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.647+0000 7fa2957fa700 1 -- 172.21.15.64:0/782514256 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa28c00cc80 con 0x7fa2980fb220 2024-02-16T22:54:36.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.647+0000 7fa2957fa700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa2880692a0 0x7fa28806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:36.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.647+0000 7fa2957fa700 1 -- 172.21.15.64:0/782514256 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa28c086ba0 con 0x7fa2980fb220 2024-02-16T22:54:36.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.647+0000 7fa297fff700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa2880692a0 0x7fa28806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:36.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.648+0000 7fa297fff700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa2880692a0 0x7fa28806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa28400b0d0 tx=0x7fa284009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:36.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.654+0000 7fa2957fa700 1 -- 172.21.15.64:0/782514256 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa28c04dbd0 con 0x7fa2980fb220 2024-02-16T22:54:36.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.911+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27c000bf0 con 0x7fa2880692a0 2024-02-16T22:54:36.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.913+0000 7fa2957fa700 1 -- 172.21.15.64:0/782514256 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fa27c000bf0 con 0x7fa2880692a0 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [], 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "", 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 image", 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:54:36.914 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:54:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa2880692a0 msgr2=0x7fa28806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa2880692a0 0x7fa28806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa28400b0d0 tx=0x7fa284009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2980fb220 msgr2=0x7fa29810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2980fb220 0x7fa29810d950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00eb70 tx=0x7fa28c00b040 comp rx=0 tx=0).stop 2024-02-16T22:54:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 shutdown_connections 2024-02-16T22:54:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2980fb220 0x7fa29810d950 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fa2880692a0 0x7fa28806b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 --2- 172.21.15.64:0/782514256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2980fe2b0 0x7fa29810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 >> 172.21.15.64:0/782514256 conn(0x7fa2980f6ca0 msgr2=0x7fa2981014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 shutdown_connections 2024-02-16T22:54:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:36.915+0000 7fa29eea9700 1 -- 172.21.15.64:0/782514256 wait complete. 2024-02-16T22:54:37.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.042+0000 7f6077f72700 1 -- 172.21.15.64:0/2176020761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fd370 msgr2=0x7f60700fd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:37.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.042+0000 7f6077f72700 1 --2- 172.21.15.64:0/2176020761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fd370 0x7f60700fd780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f606c004940 tx=0x7f606c009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:37.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.042+0000 7f6077f72700 1 -- 172.21.15.64:0/2176020761 shutdown_connections 2024-02-16T22:54:37.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.042+0000 7f6077f72700 1 --2- 172.21.15.64:0/2176020761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60700fc0c0 0x7f60700fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:37.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.042+0000 7f6077f72700 1 --2- 172.21.15.64:0/2176020761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fd370 0x7f60700fd780 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:37.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.042+0000 7f6077f72700 1 -- 172.21.15.64:0/2176020761 >> 172.21.15.64:0/2176020761 conn(0x7f60700f6c00 msgr2=0x7f60700f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:37.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.042+0000 7f6077f72700 1 -- 172.21.15.64:0/2176020761 shutdown_connections 2024-02-16T22:54:37.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.042+0000 7f6077f72700 1 -- 172.21.15.64:0/2176020761 wait complete. 2024-02-16T22:54:37.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6077f72700 1 Processor -- start 2024-02-16T22:54:37.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6077f72700 1 -- start start 2024-02-16T22:54:37.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6077f72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fc0c0 0x7f607010b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:37.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6077f72700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60700fd370 0x7f607010bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:37.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6077f72700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010c050 con 0x7f60700fc0c0 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6077f72700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010c190 con 0x7f60700fd370 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6075d0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fc0c0 0x7f607010b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6075d0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fc0c0 0x7f607010b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48312/0 (socket says 172.21.15.64:48312) 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6075d0e700 1 -- 172.21.15.64:0/3846560753 learned_addr learned my addr 172.21.15.64:0/3846560753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f607550d700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60700fd370 0x7f607010bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6075d0e700 1 -- 172.21.15.64:0/3846560753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60700fd370 msgr2=0x7f607010bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6075d0e700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60700fd370 0x7f607010bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.043+0000 7f6075d0e700 1 -- 172.21.15.64:0/3846560753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c005040 con 0x7f60700fc0c0 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.044+0000 7f6075d0e700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fc0c0 0x7f607010b5d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f606c0043d0 tx=0x7f606c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.044+0000 7f60667fc700 1 -- 172.21.15.64:0/3846560753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f606c021070 con 0x7f60700fc0c0 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.044+0000 7f60667fc700 1 -- 172.21.15.64:0/3846560753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f606c0093f0 con 0x7f60700fc0c0 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.044+0000 7f60667fc700 1 -- 172.21.15.64:0/3846560753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f606c00e070 con 0x7f60700fc0c0 2024-02-16T22:54:37.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.044+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607010cac0 con 0x7f60700fc0c0 2024-02-16T22:54:37.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.044+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607010cf50 con 0x7f60700fc0c0 2024-02-16T22:54:37.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.045+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607004bed0 con 0x7f60700fc0c0 2024-02-16T22:54:37.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.046+0000 7f60667fc700 1 -- 172.21.15.64:0/3846560753 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f606c01c400 con 0x7f60700fc0c0 2024-02-16T22:54:37.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.046+0000 7f60667fc700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f605c069230 0x7f605c06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:54:37.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.046+0000 7f60667fc700 1 -- 172.21.15.64:0/3846560753 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f606c088530 con 0x7f60700fc0c0 2024-02-16T22:54:37.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.046+0000 7f607550d700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f605c069230 0x7f605c06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:54:37.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.047+0000 7f607550d700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f605c069230 0x7f605c06b6e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6060009b80 tx=0x7f606000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:54:37.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.051+0000 7f60667fc700 1 -- 172.21.15.64:0/3846560753 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f606c054860 con 0x7f60700fc0c0 2024-02-16T22:54:37.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.418+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60701048b0 con 0x7f60700fc0c0 2024-02-16T22:54:37.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.418+0000 7f60667fc700 1 -- 172.21.15.64:0/3846560753 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f606c057e80 con 0x7f60700fc0c0 2024-02-16T22:54:37.419 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:54:37.420 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:54:37.420 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:54:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f605c069230 msgr2=0x7f605c06b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f605c069230 0x7f605c06b6e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6060009b80 tx=0x7f606000b040 comp rx=0 tx=0).stop 2024-02-16T22:54:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fc0c0 msgr2=0x7f607010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:54:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fc0c0 0x7f607010b5d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f606c0043d0 tx=0x7f606c009e30 comp rx=0 tx=0).stop 2024-02-16T22:54:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 shutdown_connections 2024-02-16T22:54:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f605c069230 0x7f605c06b6e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60700fc0c0 0x7f607010b5d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:37.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 --2- 172.21.15.64:0/3846560753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60700fd370 0x7f607010bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:54:37.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 >> 172.21.15.64:0/3846560753 conn(0x7f60700f6c00 msgr2=0x7f60700fafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:54:37.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 shutdown_connections 2024-02-16T22:54:37.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:54:37.421+0000 7f6077f72700 1 -- 172.21.15.64:0/3846560753 wait complete. 2024-02-16T22:54:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:37 smithi192 ceph-mon[36254]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:37 smithi192 ceph-mon[36254]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T22:54:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:37 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/561333220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:54:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:37 smithi064 ceph-mon[29226]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:37 smithi064 ceph-mon[29226]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T22:54:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:37 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/561333220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:54:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:38 smithi192 ceph-mon[36254]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:38 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3846560753' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:54:38.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:38 smithi064 ceph-mon[29226]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:54:38.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:38 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3846560753' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:54:39.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:39 smithi064 ceph-mon[29226]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:54:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:39 smithi192 ceph-mon[36254]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:54:41.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:41 smithi192 ceph-mon[36254]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:54:41.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:41 smithi064 ceph-mon[29226]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:54:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:43 smithi192 ceph-mon[36254]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-16T22:54:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:43 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:54:43.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:43 smithi064 ceph-mon[29226]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-16T22:54:43.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:43 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:54:45.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:45 smithi192 ceph-mon[36254]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:45 smithi064 ceph-mon[29226]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:47.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:47 smithi192 ceph-mon[36254]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:47 smithi064 ceph-mon[29226]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:49.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:48 smithi064 ceph-mon[29226]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:49.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:48 smithi192 ceph-mon[36254]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:51 smithi192 ceph-mon[36254]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:51.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:51 smithi064 ceph-mon[29226]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:53 smithi192 ceph-mon[36254]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:53.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:53 smithi064 ceph-mon[29226]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:55.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:55 smithi192 ceph-mon[36254]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:55.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:55 smithi064 ceph-mon[29226]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:57 smithi192 ceph-mon[36254]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:57.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:57 smithi064 ceph-mon[29226]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:58 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:54:59.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:54:58 smithi064 ceph-mon[29226]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:54:59.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:58 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:54:59.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:54:58 smithi192 ceph-mon[36254]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:01.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:01 smithi192 ceph-mon[36254]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:01.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:01.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:01.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:01 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:01.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:01 smithi064 ceph-mon[29226]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:01.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:01.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:01.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:01 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:02.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:02 smithi192 ceph-mon[36254]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-16T22:55:02.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:02 smithi192 ceph-mon[36254]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T22:55:02.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:02 smithi192 ceph-mon[36254]: Upgrade: Need to upgrade myself (mgr.smithi064.kubmlk) 2024-02-16T22:55:02.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:02 smithi192 ceph-mon[36254]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi192 2024-02-16T22:55:02.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:02 smithi064 ceph-mon[29226]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-16T22:55:02.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:02 smithi064 ceph-mon[29226]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T22:55:02.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:02 smithi064 ceph-mon[29226]: Upgrade: Need to upgrade myself (mgr.smithi064.kubmlk) 2024-02-16T22:55:02.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:02 smithi064 ceph-mon[29226]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi192 2024-02-16T22:55:03.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:03 smithi192 ceph-mon[36254]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:03.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:03 smithi064 ceph-mon[29226]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:05.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:05 smithi192 ceph-mon[36254]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:05.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:05 smithi064 ceph-mon[29226]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:07.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:07 smithi192 ceph-mon[36254]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:07.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.563+0000 7f890a5a1700 1 -- 172.21.15.64:0/593935399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fbe40 msgr2=0x7f89040fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:07.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.563+0000 7f890a5a1700 1 --2- 172.21.15.64:0/593935399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fbe40 0x7f89040fc250 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f88f4004940 tx=0x7f88f4009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:07.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:07 smithi064 ceph-mon[29226]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.563+0000 7f890a5a1700 1 -- 172.21.15.64:0/593935399 shutdown_connections 2024-02-16T22:55:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.563+0000 7f890a5a1700 1 --2- 172.21.15.64:0/593935399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89040fcfb0 0x7f89040fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.563+0000 7f890a5a1700 1 --2- 172.21.15.64:0/593935399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fbe40 0x7f89040fc250 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.563+0000 7f890a5a1700 1 -- 172.21.15.64:0/593935399 >> 172.21.15.64:0/593935399 conn(0x7f89040f7520 msgr2=0x7f89040f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.563+0000 7f890a5a1700 1 -- 172.21.15.64:0/593935399 shutdown_connections 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.564+0000 7f890a5a1700 1 -- 172.21.15.64:0/593935399 wait complete. 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.564+0000 7f890a5a1700 1 Processor -- start 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.564+0000 7f890a5a1700 1 -- start start 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f890a5a1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89040fbe40 0x7f8904112570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f890a5a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fcfb0 0x7f8904114ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f890a5a1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890410e020 con 0x7f89040fcfb0 2024-02-16T22:55:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f890a5a1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8904115030 con 0x7f89040fbe40 2024-02-16T22:55:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f89037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fcfb0 0x7f8904114ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f89037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fcfb0 0x7f8904114ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35954/0 (socket says 172.21.15.64:35954) 2024-02-16T22:55:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f89037fe700 1 -- 172.21.15.64:0/4227058272 learned_addr learned my addr 172.21.15.64:0/4227058272 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f8903fff700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89040fbe40 0x7f8904112570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f89037fe700 1 -- 172.21.15.64:0/4227058272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89040fbe40 msgr2=0x7f8904112570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f89037fe700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89040fbe40 0x7f8904112570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.565+0000 7f89037fe700 1 -- 172.21.15.64:0/4227058272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4005040 con 0x7f89040fcfb0 2024-02-16T22:55:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.566+0000 7f89037fe700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fcfb0 0x7f8904114ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f88f800ed00 tx=0x7f88f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.566+0000 7f8900ff9700 1 -- 172.21.15.64:0/4227058272 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f8007ab0 con 0x7f89040fcfb0 2024-02-16T22:55:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.566+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89041152b0 con 0x7f89040fcfb0 2024-02-16T22:55:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.566+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89041157f0 con 0x7f89040fcfb0 2024-02-16T22:55:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.566+0000 7f8900ff9700 1 -- 172.21.15.64:0/4227058272 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88f800f040 con 0x7f89040fcfb0 2024-02-16T22:55:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.567+0000 7f8900ff9700 1 -- 172.21.15.64:0/4227058272 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f8009670 con 0x7f89040fcfb0 2024-02-16T22:55:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.567+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f00052e0 con 0x7f89040fcfb0 2024-02-16T22:55:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.568+0000 7f8900ff9700 1 -- 172.21.15.64:0/4227058272 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f88f80097d0 con 0x7f89040fcfb0 2024-02-16T22:55:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.569+0000 7f8900ff9700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88ec0690a0 0x7f88ec06b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:07.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.569+0000 7f8900ff9700 1 -- 172.21.15.64:0/4227058272 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f88f8086e80 con 0x7f89040fcfb0 2024-02-16T22:55:07.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.569+0000 7f8903fff700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88ec0690a0 0x7f88ec06b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:07.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.570+0000 7f8903fff700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88ec0690a0 0x7f88ec06b550 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f88f4000c00 tx=0x7f88f4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:07.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.577+0000 7f8900ff9700 1 -- 172.21.15.64:0/4227058272 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f88f8056c40 con 0x7f89040fcfb0 2024-02-16T22:55:07.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.935+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f88f0000bf0 con 0x7f88ec0690a0 2024-02-16T22:55:07.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.937+0000 7f8900ff9700 1 -- 172.21.15.64:0/4227058272 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f88f0000bf0 con 0x7f88ec0690a0 2024-02-16T22:55:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88ec0690a0 msgr2=0x7f88ec06b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88ec0690a0 0x7f88ec06b550 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f88f4000c00 tx=0x7f88f4009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fcfb0 msgr2=0x7f8904114ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fcfb0 0x7f8904114ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f88f800ed00 tx=0x7f88f800b040 comp rx=0 tx=0).stop 2024-02-16T22:55:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 shutdown_connections 2024-02-16T22:55:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89040fbe40 0x7f8904112570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f88ec0690a0 0x7f88ec06b550 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 --2- 172.21.15.64:0/4227058272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89040fcfb0 0x7f8904114ac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.939+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 >> 172.21.15.64:0/4227058272 conn(0x7f89040f7520 msgr2=0x7f8904100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.940+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 shutdown_connections 2024-02-16T22:55:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:07.940+0000 7f890a5a1700 1 -- 172.21.15.64:0/4227058272 wait complete. 2024-02-16T22:55:07.952 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:55:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.050+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2441867949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 msgr2=0x7f0fe40fd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.050+0000 7f0feb1ac700 1 --2- 172.21.15.64:0/2441867949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 0x7f0fe40fd920 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8004940 tx=0x7f0fd8009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.050+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2441867949 shutdown_connections 2024-02-16T22:55:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.050+0000 7f0feb1ac700 1 --2- 172.21.15.64:0/2441867949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe40fde60 0x7f0fe41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.050+0000 7f0feb1ac700 1 --2- 172.21.15.64:0/2441867949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 0x7f0fe40fd920 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.050+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2441867949 >> 172.21.15.64:0/2441867949 conn(0x7f0fe40f6ca0 msgr2=0x7f0fe40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.051+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2441867949 shutdown_connections 2024-02-16T22:55:08.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.051+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2441867949 wait complete. 2024-02-16T22:55:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.051+0000 7f0feb1ac700 1 Processor -- start 2024-02-16T22:55:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.051+0000 7f0feb1ac700 1 -- start start 2024-02-16T22:55:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0feb1ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 0x7f0fe40fe740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0feb1ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe40fde60 0x7f0fe40fec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0feb1ac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40ff2d0 con 0x7f0fe40fd510 2024-02-16T22:55:08.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0feb1ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40ffb70 con 0x7f0fe40fde60 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0fe3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe40fde60 0x7f0fe40fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0fe8f48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 0x7f0fe40fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0fe3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe40fde60 0x7f0fe40fec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:37222/0 (socket says 172.21.15.64:37222) 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0fe3fff700 1 -- 172.21.15.64:0/2351610564 learned_addr learned my addr 172.21.15.64:0/2351610564 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0fe3fff700 1 -- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 msgr2=0x7f0fe40fe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.052+0000 7f0fe3fff700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 0x7f0fe40fe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.053+0000 7f0fe3fff700 1 -- 172.21.15.64:0/2351610564 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd8005040 con 0x7f0fe40fde60 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.053+0000 7f0fe8f48700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 0x7f0fe40fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:55:08.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.053+0000 7f0fe3fff700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe40fde60 0x7f0fe40fec80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000eb40 tx=0x7f0fd000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:08.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.053+0000 7f0fe17fa700 1 -- 172.21.15.64:0/2351610564 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fd0007cc0 con 0x7f0fe40fde60 2024-02-16T22:55:08.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.053+0000 7f0fe17fa700 1 -- 172.21.15.64:0/2351610564 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fd000f040 con 0x7f0fe40fde60 2024-02-16T22:55:08.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.053+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe40ffdf0 con 0x7f0fe40fde60 2024-02-16T22:55:08.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.053+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe4100270 con 0x7f0fe40fde60 2024-02-16T22:55:08.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.054+0000 7f0fe17fa700 1 -- 172.21.15.64:0/2351610564 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fd000ca00 con 0x7f0fe40fde60 2024-02-16T22:55:08.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.055+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc80052e0 con 0x7f0fe40fde60 2024-02-16T22:55:08.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.055+0000 7f0fe17fa700 1 -- 172.21.15.64:0/2351610564 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0fd000cb60 con 0x7f0fe40fde60 2024-02-16T22:55:08.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.056+0000 7f0fe17fa700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f0fd40692a0 0x7f0fd406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.056+0000 7f0fe8f48700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f0fd40692a0 0x7f0fd406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.056+0000 7f0fe17fa700 1 -- 172.21.15.64:0/2351610564 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0fd0086c20 con 0x7f0fe40fde60 2024-02-16T22:55:08.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.057+0000 7f0fe8f48700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f0fd40692a0 0x7f0fd406b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009b30 tx=0x7f0fd8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:08.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.062+0000 7f0fe17fa700 1 -- 172.21.15.64:0/2351610564 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0fd004dbd0 con 0x7f0fe40fde60 2024-02-16T22:55:08.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.289+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fc8000bf0 con 0x7f0fd40692a0 2024-02-16T22:55:08.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.291+0000 7f0fe17fa700 1 -- 172.21.15.64:0/2351610564 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f0fc8000bf0 con 0x7f0fd40692a0 2024-02-16T22:55:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.292+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f0fd40692a0 msgr2=0x7f0fd406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.292+0000 7f0feb1ac700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f0fd40692a0 0x7f0fd406b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009b30 tx=0x7f0fd8009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe40fde60 msgr2=0x7f0fe40fec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe40fde60 0x7f0fe40fec80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000eb40 tx=0x7f0fd000b040 comp rx=0 tx=0).stop 2024-02-16T22:55:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 shutdown_connections 2024-02-16T22:55:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f0fd40692a0 0x7f0fd406b750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe40fd510 0x7f0fe40fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 --2- 172.21.15.64:0/2351610564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe40fde60 0x7f0fe40fec80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 >> 172.21.15.64:0/2351610564 conn(0x7f0fe40f6ca0 msgr2=0x7f0fe40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:08.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 shutdown_connections 2024-02-16T22:55:08.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.293+0000 7f0feb1ac700 1 -- 172.21.15.64:0/2351610564 wait complete. 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.423+0000 7fbe6a260700 1 -- 172.21.15.64:0/2990307423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fe0e0 msgr2=0x7fbe640fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.423+0000 7fbe6a260700 1 --2- 172.21.15.64:0/2990307423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fe0e0 0x7fbe640fe550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbe58004940 tx=0x7fbe58009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.424+0000 7fbe6a260700 1 -- 172.21.15.64:0/2990307423 shutdown_connections 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.424+0000 7fbe6a260700 1 --2- 172.21.15.64:0/2990307423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fe0e0 0x7fbe640fe550 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.424+0000 7fbe6a260700 1 --2- 172.21.15.64:0/2990307423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe640fcef0 0x7fbe640fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.424+0000 7fbe6a260700 1 -- 172.21.15.64:0/2990307423 >> 172.21.15.64:0/2990307423 conn(0x7fbe640f8620 msgr2=0x7fbe640faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.424+0000 7fbe6a260700 1 -- 172.21.15.64:0/2990307423 shutdown_connections 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.424+0000 7fbe6a260700 1 -- 172.21.15.64:0/2990307423 wait complete. 2024-02-16T22:55:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.424+0000 7fbe6a260700 1 Processor -- start 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.424+0000 7fbe6a260700 1 -- start start 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe6a260700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fcef0 0x7fbe64071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe6a260700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe640fe0e0 0x7fbe640702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe6a260700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640708e0 con 0x7fbe640fcef0 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe6a260700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64070a50 con 0x7fbe640fe0e0 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe62ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe640fe0e0 0x7fbe640702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe62ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe640fe0e0 0x7fbe640702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:37244/0 (socket says 172.21.15.64:37244) 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe62ffd700 1 -- 172.21.15.64:0/3235089562 learned_addr learned my addr 172.21.15.64:0/3235089562 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe637fe700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fcef0 0x7fbe64071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe62ffd700 1 -- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fcef0 msgr2=0x7fbe64071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe62ffd700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fcef0 0x7fbe64071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe62ffd700 1 -- 172.21.15.64:0/3235089562 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58005040 con 0x7fbe640fe0e0 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.425+0000 7fbe637fe700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fcef0 0x7fbe64071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.426+0000 7fbe62ffd700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe640fe0e0 0x7fbe640702e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe58000c00 tx=0x7fbe58009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.426+0000 7fbe4bfff700 1 -- 172.21.15.64:0/3235089562 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5800e070 con 0x7fbe640fe0e0 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.426+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe64070cd0 con 0x7fbe640fe0e0 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.426+0000 7fbe4bfff700 1 -- 172.21.15.64:0/3235089562 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe5801c400 con 0x7fbe640fe0e0 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.426+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe64115900 con 0x7fbe640fe0e0 2024-02-16T22:55:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.426+0000 7fbe4bfff700 1 -- 172.21.15.64:0/3235089562 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5800b650 con 0x7fbe640fe0e0 2024-02-16T22:55:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.427+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6404bed0 con 0x7fbe640fe0e0 2024-02-16T22:55:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.428+0000 7fbe4bfff700 1 -- 172.21.15.64:0/3235089562 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbe5801ca00 con 0x7fbe640fe0e0 2024-02-16T22:55:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.428+0000 7fbe4bfff700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbe4c0692b0 0x7fbe4c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.428+0000 7fbe4bfff700 1 -- 172.21.15.64:0/3235089562 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fbe580887a0 con 0x7fbe640fe0e0 2024-02-16T22:55:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.429+0000 7fbe637fe700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbe4c0692b0 0x7fbe4c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.429+0000 7fbe637fe700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbe4c0692b0 0x7fbe4c06b760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe64071bf0 tx=0x7fbe5400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:08.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.436+0000 7fbe4bfff700 1 -- 172.21.15.64:0/3235089562 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbe58058600 con 0x7fbe640fe0e0 2024-02-16T22:55:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.772+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe64102ac0 con 0x7fbe4c0692b0 2024-02-16T22:55:08.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.788+0000 7fbe4bfff700 1 -- 172.21.15.64:0/3235089562 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fbe64102ac0 con 0x7fbe4c0692b0 2024-02-16T22:55:08.789 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:55:08.789 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (5m) 57s ago 6m 25.1M - 0.25.0 c8568f914cd2 829b066b8c26 2024-02-16T22:55:08.789 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (6m) 57s ago 6m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (5m) 62s ago 5m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (5m) 57s ago 6m 86.0M - 9.4.7 954c08fa6188 529226ab33c5 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (76s) 57s ago 75s 14.7M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (81s) 57s ago 81s 14.5M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (74s) 62s ago 73s 13.5M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (79s) 62s ago 78s 13.5M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:9283 running (8m) 57s ago 8m 480M - 17.2.7-513-gb88fae66 870a0944ccb8 7189c74ed7fc 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (5m) 62s ago 5m 433M - 17.2.7-513-gb88fae66 870a0944ccb8 a017e081156a 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (8m) 57s ago 8m 45.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (5m) 62s ago 5m 36.1M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (6m) 57s ago 6m 15.5M - 1.5.0 0da6a335fe13 c25d924fa381 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 62s ago 5m 16.1M - 1.5.0 0da6a335fe13 c32798cb2969 2024-02-16T22:55:08.790 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (4m) 57s ago 4m 46.5M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:55:08.791 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (4m) 57s ago 4m 45.3M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:55:08.791 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (4m) 57s ago 4m 44.7M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:55:08.791 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (3m) 62s ago 3m 43.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:55:08.791 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (3m) 62s ago 3m 41.9M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:55:08.791 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (2m) 62s ago 2m 41.5M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:55:08.791 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (5m) 57s ago 6m 53.4M - 2.43.0 a07b618ecd1d dd20eff0542c 2024-02-16T22:55:08.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbe4c0692b0 msgr2=0x7fbe4c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbe4c0692b0 0x7fbe4c06b760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe64071bf0 tx=0x7fbe5400b040 comp rx=0 tx=0).stop 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe640fe0e0 msgr2=0x7fbe640702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe640fe0e0 0x7fbe640702e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe58000c00 tx=0x7fbe58009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 shutdown_connections 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbe4c0692b0 0x7fbe4c06b760 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe640fcef0 0x7fbe64071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 --2- 172.21.15.64:0/3235089562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe640fe0e0 0x7fbe640702e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.790+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 >> 172.21.15.64:0/3235089562 conn(0x7fbe640f8620 msgr2=0x7fbe641013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.791+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 shutdown_connections 2024-02-16T22:55:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.791+0000 7fbe6a260700 1 -- 172.21.15.64:0/3235089562 wait complete. 2024-02-16T22:55:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.923+0000 7f707bf76700 1 -- 172.21.15.64:0/1009682809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 msgr2=0x7f7074070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.923+0000 7f707bf76700 1 --2- 172.21.15.64:0/1009682809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 0x7f7074070320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7070004940 tx=0x7f7070009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.924+0000 7f707bf76700 1 -- 172.21.15.64:0/1009682809 shutdown_connections 2024-02-16T22:55:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.924+0000 7f707bf76700 1 --2- 172.21.15.64:0/1009682809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7074070860 0x7f7074070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.924+0000 7f707bf76700 1 --2- 172.21.15.64:0/1009682809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 0x7f7074070320 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.924+0000 7f707bf76700 1 -- 172.21.15.64:0/1009682809 >> 172.21.15.64:0/1009682809 conn(0x7f70740f8620 msgr2=0x7f70740faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.924+0000 7f707bf76700 1 -- 172.21.15.64:0/1009682809 shutdown_connections 2024-02-16T22:55:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.924+0000 7f707bf76700 1 -- 172.21.15.64:0/1009682809 wait complete. 2024-02-16T22:55:08.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.924+0000 7f707bf76700 1 Processor -- start 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.924+0000 7f707bf76700 1 -- start start 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f707bf76700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7074070860 0x7f707410fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f707bf76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 0x7f7074110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f707bf76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7074110600 con 0x7f7074071eb0 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f707bf76700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7074110ea0 con 0x7f7074070860 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f7079d12700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7074070860 0x7f707410fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f7079d12700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7074070860 0x7f707410fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:37268/0 (socket says 172.21.15.64:37268) 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f7079511700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 0x7f7074110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f7079511700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 0x7f7074110000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36010/0 (socket says 172.21.15.64:36010) 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f7079d12700 1 -- 172.21.15.64:0/1324751502 learned_addr learned my addr 172.21.15.64:0/1324751502 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f7079511700 1 -- 172.21.15.64:0/1324751502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7074070860 msgr2=0x7f707410fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f7079511700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7074070860 0x7f707410fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.925+0000 7f7079511700 1 -- 172.21.15.64:0/1324751502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070005040 con 0x7f7074071eb0 2024-02-16T22:55:08.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.926+0000 7f7079511700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 0x7f7074110000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7064002a50 tx=0x7f7064004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:08.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.926+0000 7f706a7fc700 1 -- 172.21.15.64:0/1324751502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7064007cc0 con 0x7f7074071eb0 2024-02-16T22:55:08.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.926+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7074111180 con 0x7f7074071eb0 2024-02-16T22:55:08.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.926+0000 7f706a7fc700 1 -- 172.21.15.64:0/1324751502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f706400f040 con 0x7f7074071eb0 2024-02-16T22:55:08.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.926+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70741116c0 con 0x7f7074071eb0 2024-02-16T22:55:08.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.926+0000 7f706a7fc700 1 -- 172.21.15.64:0/1324751502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7064009530 con 0x7f7074071eb0 2024-02-16T22:55:08.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.927+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707404bed0 con 0x7f7074071eb0 2024-02-16T22:55:08.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.928+0000 7f706a7fc700 1 -- 172.21.15.64:0/1324751502 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f70640097a0 con 0x7f7074071eb0 2024-02-16T22:55:08.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.928+0000 7f706a7fc700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f70600692a0 0x7f706006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:08.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.928+0000 7f706a7fc700 1 -- 172.21.15.64:0/1324751502 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f7064086dc0 con 0x7f7074071eb0 2024-02-16T22:55:08.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.928+0000 7f7079d12700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f70600692a0 0x7f706006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:08.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.929+0000 7f7079d12700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f70600692a0 0x7f706006b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7070004f90 tx=0x7f7070009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:08.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:08.933+0000 7f706a7fc700 1 -- 172.21.15.64:0/1324751502 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7064056b80 con 0x7f7074071eb0 2024-02-16T22:55:09.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:08 smithi064 ceph-mon[29226]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:09.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:08 smithi064 ceph-mon[29226]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:09.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:08 smithi064 ceph-mon[29226]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:09.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.211+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7074107c00 con 0x7f7074071eb0 2024-02-16T22:55:09.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.211+0000 7f706a7fc700 1 -- 172.21.15.64:0/1324751502 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f7064056710 con 0x7f7074071eb0 2024-02-16T22:55:09.212 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:55:09.213 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 14 2024-02-16T22:55:09.214 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:55:09.214 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:55:09.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f70600692a0 msgr2=0x7f706006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f70600692a0 0x7f706006b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7070004f90 tx=0x7f7070009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:09.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 msgr2=0x7f7074110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 0x7f7074110000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7064002a50 tx=0x7f7064004600 comp rx=0 tx=0).stop 2024-02-16T22:55:09.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 shutdown_connections 2024-02-16T22:55:09.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7074070860 0x7f707410fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f70600692a0 0x7f706006b750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 --2- 172.21.15.64:0/1324751502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7074071eb0 0x7f7074110000 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.213+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 >> 172.21.15.64:0/1324751502 conn(0x7f70740f8620 msgr2=0x7f7074102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:09.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.214+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 shutdown_connections 2024-02-16T22:55:09.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.214+0000 7f707bf76700 1 -- 172.21.15.64:0/1324751502 wait complete. 2024-02-16T22:55:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:08 smithi192 ceph-mon[36254]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:08 smithi192 ceph-mon[36254]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:08 smithi192 ceph-mon[36254]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:09.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.335+0000 7fbd699ec700 1 -- 172.21.15.64:0/3525630653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fcfb0 msgr2=0x7fbd640fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.335+0000 7fbd699ec700 1 --2- 172.21.15.64:0/3525630653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fcfb0 0x7fbd640fd420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbd58004940 tx=0x7fbd58009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:09.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.335+0000 7fbd699ec700 1 -- 172.21.15.64:0/3525630653 shutdown_connections 2024-02-16T22:55:09.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.335+0000 7fbd699ec700 1 --2- 172.21.15.64:0/3525630653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fcfb0 0x7fbd640fd420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.335+0000 7fbd699ec700 1 --2- 172.21.15.64:0/3525630653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd640fbe40 0x7fbd640fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.335+0000 7fbd699ec700 1 -- 172.21.15.64:0/3525630653 >> 172.21.15.64:0/3525630653 conn(0x7fbd640f74e0 msgr2=0x7fbd640f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:09.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.335+0000 7fbd699ec700 1 -- 172.21.15.64:0/3525630653 shutdown_connections 2024-02-16T22:55:09.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.335+0000 7fbd699ec700 1 -- 172.21.15.64:0/3525630653 wait complete. 2024-02-16T22:55:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd699ec700 1 Processor -- start 2024-02-16T22:55:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd699ec700 1 -- start start 2024-02-16T22:55:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd699ec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fbe40 0x7fbd6410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd699ec700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd640fcfb0 0x7fbd6410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd62ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fbe40 0x7fbd6410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd62ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fbe40 0x7fbd6410d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36032/0 (socket says 172.21.15.64:36032) 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd62ffd700 1 -- 172.21.15.64:0/406071653 learned_addr learned my addr 172.21.15.64:0/406071653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6410e420 con 0x7fbd640fbe40 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6410ecc0 con 0x7fbd640fcfb0 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.336+0000 7fbd627fc700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd640fcfb0 0x7fbd6410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd62ffd700 1 -- 172.21.15.64:0/406071653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd640fcfb0 msgr2=0x7fbd6410de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd62ffd700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd640fcfb0 0x7fbd6410de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd62ffd700 1 -- 172.21.15.64:0/406071653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd58005040 con 0x7fbd640fbe40 2024-02-16T22:55:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd62ffd700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fbe40 0x7fbd6410d8e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400eb40 tx=0x7fbd5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd4b7fe700 1 -- 172.21.15.64:0/406071653 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd54007cc0 con 0x7fbd640fbe40 2024-02-16T22:55:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd4b7fe700 1 -- 172.21.15.64:0/406071653 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd5400f040 con 0x7fbd640fbe40 2024-02-16T22:55:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6410ef40 con 0x7fbd640fbe40 2024-02-16T22:55:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd4b7fe700 1 -- 172.21.15.64:0/406071653 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd54007cc0 con 0x7fbd640fbe40 2024-02-16T22:55:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.337+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6410f430 con 0x7fbd640fbe40 2024-02-16T22:55:09.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.339+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6404bed0 con 0x7fbd640fbe40 2024-02-16T22:55:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.339+0000 7fbd4b7fe700 1 -- 172.21.15.64:0/406071653 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbd540090d0 con 0x7fbd640fbe40 2024-02-16T22:55:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.339+0000 7fbd4b7fe700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd4c069310 0x7fbd4c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.340+0000 7fbd4b7fe700 1 -- 172.21.15.64:0/406071653 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fbd54086970 con 0x7fbd640fbe40 2024-02-16T22:55:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.340+0000 7fbd627fc700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd4c069310 0x7fbd4c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.341+0000 7fbd627fc700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd4c069310 0x7fbd4c06b7c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009b30 tx=0x7fbd58009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:09.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.344+0000 7fbd4b7fe700 1 -- 172.21.15.64:0/406071653 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbd5404dbd0 con 0x7fbd640fbe40 2024-02-16T22:55:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.585+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd6410fb00 con 0x7fbd640fbe40 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.585+0000 7fbd4b7fe700 1 -- 172.21.15.64:0/406071653 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7fbd54019720 con 0x7fbd640fbe40 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:55:09.587 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:55:09.588 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:55:09.589 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.588+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd4c069310 msgr2=0x7fbd4c06b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.588+0000 7fbd699ec700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd4c069310 0x7fbd4c06b7c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009b30 tx=0x7fbd58009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.588+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fbe40 msgr2=0x7fbd6410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.588+0000 7fbd699ec700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fbe40 0x7fbd6410d8e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400eb40 tx=0x7fbd5400b040 comp rx=0 tx=0).stop 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.589+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 shutdown_connections 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.589+0000 7fbd699ec700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7fbd4c069310 0x7fbd4c06b7c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.589+0000 7fbd699ec700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd640fbe40 0x7fbd6410d8e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.589+0000 7fbd699ec700 1 --2- 172.21.15.64:0/406071653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd640fcfb0 0x7fbd6410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.589+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 >> 172.21.15.64:0/406071653 conn(0x7fbd640f74e0 msgr2=0x7fbd64100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.589+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 shutdown_connections 2024-02-16T22:55:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.589+0000 7fbd699ec700 1 -- 172.21.15.64:0/406071653 wait complete. 2024-02-16T22:55:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.698+0000 7f4f0140c700 1 -- 172.21.15.64:0/2088886182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fc160 msgr2=0x7f4efc0fc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.698+0000 7f4f0140c700 1 --2- 172.21.15.64:0/2088886182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fc160 0x7f4efc0fc5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0004940 tx=0x7f4ef0009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.698+0000 7f4f0140c700 1 -- 172.21.15.64:0/2088886182 shutdown_connections 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.698+0000 7f4f0140c700 1 --2- 172.21.15.64:0/2088886182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fc160 0x7f4efc0fc5d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.698+0000 7f4f0140c700 1 --2- 172.21.15.64:0/2088886182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4efc0fd4e0 0x7f4efc0fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.698+0000 7f4f0140c700 1 -- 172.21.15.64:0/2088886182 >> 172.21.15.64:0/2088886182 conn(0x7f4efc0f6ca0 msgr2=0x7f4efc0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.698+0000 7f4f0140c700 1 -- 172.21.15.64:0/2088886182 shutdown_connections 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.698+0000 7f4f0140c700 1 -- 172.21.15.64:0/2088886182 wait complete. 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4f0140c700 1 Processor -- start 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4f0140c700 1 -- start start 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4f0140c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4efc0fc160 0x7f4efc10e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4f0140c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fd4e0 0x7f4efc110980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:09.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4f0140c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0fbe10 con 0x7f4efc0fd4e0 2024-02-16T22:55:09.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4f0140c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0fbf50 con 0x7f4efc0fc160 2024-02-16T22:55:09.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4efaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4efc0fc160 0x7f4efc10e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:09.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4efaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4efc0fc160 0x7f4efc10e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:37306/0 (socket says 172.21.15.64:37306) 2024-02-16T22:55:09.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4efaffd700 1 -- 172.21.15.64:0/2196466894 learned_addr learned my addr 172.21.15.64:0/2196466894 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:09.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.699+0000 7f4efa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fd4e0 0x7f4efc110980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:09.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.700+0000 7f4efaffd700 1 -- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fd4e0 msgr2=0x7f4efc110980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.700+0000 7f4efaffd700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fd4e0 0x7f4efc110980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.700+0000 7f4efaffd700 1 -- 172.21.15.64:0/2196466894 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef0005040 con 0x7f4efc0fc160 2024-02-16T22:55:09.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.700+0000 7f4efa7fc700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fd4e0 0x7f4efc110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:55:09.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.700+0000 7f4efaffd700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4efc0fc160 0x7f4efc10e430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00eb40 tx=0x7f4eec00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:09.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.700+0000 7f4ee37fe700 1 -- 172.21.15.64:0/2196466894 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eec007cc0 con 0x7f4efc0fc160 2024-02-16T22:55:09.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.700+0000 7f4ee37fe700 1 -- 172.21.15.64:0/2196466894 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4eec00f040 con 0x7f4efc0fc160 2024-02-16T22:55:09.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.701+0000 7f4ee37fe700 1 -- 172.21.15.64:0/2196466894 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eec00ca10 con 0x7f4efc0fc160 2024-02-16T22:55:09.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.701+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc110f80 con 0x7f4efc0fc160 2024-02-16T22:55:09.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.701+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc111440 con 0x7f4efc0fc160 2024-02-16T22:55:09.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.702+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc04bed0 con 0x7f4efc0fc160 2024-02-16T22:55:09.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.703+0000 7f4ee37fe700 1 -- 172.21.15.64:0/2196466894 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4eec00cb70 con 0x7f4efc0fc160 2024-02-16T22:55:09.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.704+0000 7f4ee37fe700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f4ee40690e0 0x7f4ee406b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.704+0000 7f4efa7fc700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f4ee40690e0 0x7f4ee406b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.704+0000 7f4ee37fe700 1 -- 172.21.15.64:0/2196466894 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4eec086960 con 0x7f4efc0fc160 2024-02-16T22:55:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.705+0000 7f4efa7fc700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f4ee40690e0 0x7f4ee406b590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000b0d0 tx=0x7f4ef0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:09.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.708+0000 7f4ee37fe700 1 -- 172.21.15.64:0/2196466894 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4eec052bf0 con 0x7f4efc0fc160 2024-02-16T22:55:09.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.936+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4efc101ac0 con 0x7f4ee40690e0 2024-02-16T22:55:09.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.938+0000 7f4ee37fe700 1 -- 172.21.15.64:0/2196466894 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f4efc101ac0 con 0x7f4ee40690e0 2024-02-16T22:55:09.939 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:55:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:55:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:55:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:55:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [], 2024-02-16T22:55:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "0/21 daemons upgraded", 2024-02-16T22:55:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 image on host smithi192", 2024-02-16T22:55:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:55:09.940 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:55:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f4ee40690e0 msgr2=0x7f4ee406b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f4ee40690e0 0x7f4ee406b590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000b0d0 tx=0x7f4ef0009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4efc0fc160 msgr2=0x7f4efc10e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4efc0fc160 0x7f4efc10e430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00eb40 tx=0x7f4eec00b040 comp rx=0 tx=0).stop 2024-02-16T22:55:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 shutdown_connections 2024-02-16T22:55:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4efc0fc160 0x7f4efc10e430 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f4ee40690e0 0x7f4ee406b590 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 --2- 172.21.15.64:0/2196466894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc0fd4e0 0x7f4efc110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:09.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 >> 172.21.15.64:0/2196466894 conn(0x7f4efc0f6ca0 msgr2=0x7f4efc1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:09.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 shutdown_connections 2024-02-16T22:55:09.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:09.941+0000 7f4f0140c700 1 -- 172.21.15.64:0/2196466894 wait complete. 2024-02-16T22:55:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.097+0000 7f462262d700 1 -- 172.21.15.64:0/2661389395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 msgr2=0x7f461c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.097+0000 7f462262d700 1 --2- 172.21.15.64:0/2661389395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 0x7f461c0fd420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4610004940 tx=0x7f4610009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:10.098 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:09 smithi064 ceph-mon[29226]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:10.099 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:09 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1324751502' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:55:10.099 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:09 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/406071653' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.097+0000 7f462262d700 1 -- 172.21.15.64:0/2661389395 shutdown_connections 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.097+0000 7f462262d700 1 --2- 172.21.15.64:0/2661389395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 0x7f461c0fd420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.097+0000 7f462262d700 1 --2- 172.21.15.64:0/2661389395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c0fbe40 0x7f461c0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.097+0000 7f462262d700 1 -- 172.21.15.64:0/2661389395 >> 172.21.15.64:0/2661389395 conn(0x7f461c0f74e0 msgr2=0x7f461c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.097+0000 7f462262d700 1 -- 172.21.15.64:0/2661389395 shutdown_connections 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.097+0000 7f462262d700 1 -- 172.21.15.64:0/2661389395 wait complete. 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.098+0000 7f462262d700 1 Processor -- start 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.098+0000 7f462262d700 1 -- start start 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.098+0000 7f462262d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c0fbe40 0x7f461c10c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.098+0000 7f462262d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 0x7f461c10cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f462262d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c10d310 con 0x7f461c0fcfb0 2024-02-16T22:55:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f462262d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c10dbb0 con 0x7f461c0fbe40 2024-02-16T22:55:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 0x7f461c10cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c0fbe40 0x7f461c10c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 0x7f461c10cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36078/0 (socket says 172.21.15.64:36078) 2024-02-16T22:55:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461b7fe700 1 -- 172.21.15.64:0/538109958 learned_addr learned my addr 172.21.15.64:0/538109958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461bfff700 1 -- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 msgr2=0x7f461c10cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461bfff700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 0x7f461c10cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461bfff700 1 -- 172.21.15.64:0/538109958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610005040 con 0x7f461c0fbe40 2024-02-16T22:55:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461b7fe700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 0x7f461c10cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:55:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.099+0000 7f461bfff700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c0fbe40 0x7f461c10c7d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f460c009fd0 tx=0x7f460c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.100+0000 7f4618ff9700 1 -- 172.21.15.64:0/538109958 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f460c007ea0 con 0x7f461c0fbe40 2024-02-16T22:55:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.100+0000 7f4618ff9700 1 -- 172.21.15.64:0/538109958 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f460c00f040 con 0x7f461c0fbe40 2024-02-16T22:55:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.100+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c10de30 con 0x7f461c0fbe40 2024-02-16T22:55:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.100+0000 7f4618ff9700 1 -- 172.21.15.64:0/538109958 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f460c00cbf0 con 0x7f461c0fbe40 2024-02-16T22:55:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.100+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c10e290 con 0x7f461c0fbe40 2024-02-16T22:55:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.102+0000 7f4618ff9700 1 -- 172.21.15.64:0/538109958 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f460c018030 con 0x7f461c0fbe40 2024-02-16T22:55:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.102+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c04bed0 con 0x7f461c0fbe40 2024-02-16T22:55:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.102+0000 7f4618ff9700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46040692f0 0x7f460406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.102+0000 7f4618ff9700 1 -- 172.21.15.64:0/538109958 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f460c016070 con 0x7f461c0fbe40 2024-02-16T22:55:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.102+0000 7f461b7fe700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46040692f0 0x7f460406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.103+0000 7f461b7fe700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46040692f0 0x7f460406b7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f461000b0d0 tx=0x7f4610009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.106+0000 7f4618ff9700 1 -- 172.21.15.64:0/538109958 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f460c057010 con 0x7f461c0fbe40 2024-02-16T22:55:10.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:09 smithi192 ceph-mon[36254]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:10.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:09 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1324751502' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:55:10.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:09 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/406071653' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:55:10.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.515+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f461c10e8e0 con 0x7f461c0fbe40 2024-02-16T22:55:10.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.516+0000 7f4618ff9700 1 -- 172.21.15.64:0/538109958 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f460c056ba0 con 0x7f461c0fbe40 2024-02-16T22:55:10.517 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:55:10.517 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:55:10.517 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46040692f0 msgr2=0x7f460406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46040692f0 0x7f460406b7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f461000b0d0 tx=0x7f4610009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c0fbe40 msgr2=0x7f461c10c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c0fbe40 0x7f461c10c7d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f460c009fd0 tx=0x7f460c00b040 comp rx=0 tx=0).stop 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 shutdown_connections 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c0fbe40 0x7f461c10c7d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f46040692f0 0x7f460406b7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 --2- 172.21.15.64:0/538109958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461c0fcfb0 0x7f461c10cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.518+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 >> 172.21.15.64:0/538109958 conn(0x7f461c0f74e0 msgr2=0x7f461c100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.519+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 shutdown_connections 2024-02-16T22:55:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:10.519+0000 7f462262d700 1 -- 172.21.15.64:0/538109958 wait complete. 2024-02-16T22:55:11.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:10 smithi064 ceph-mon[29226]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:11.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:10 smithi064 ceph-mon[29226]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:11.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:10 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/538109958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:55:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:10 smithi192 ceph-mon[36254]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:10 smithi192 ceph-mon[36254]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:10 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/538109958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr:Note: switching to 'ed7ab725e1a43615009dfb74a42bf33d80081bc8'. 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr:state without impacting any branches by switching back to a branch. 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-16T22:55:11.315 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr: git switch -c 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr:Or undo this operation with: 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr: git switch - 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-02-16T22:55:11.316 INFO:tasks.workunit.client.1.smithi192.stderr:HEAD is now at ed7ab725e1a qa: test fuse/kclient for mds upgrade seq 2024-02-16T22:55:11.320 DEBUG:teuthology.orchestra.run.smithi192:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-16T22:55:11.385 INFO:tasks.workunit.client.1.smithi192.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-16T22:55:11.387 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-16T22:55:11.387 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-16T22:55:13.151 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-16T22:55:13.225 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-16T22:55:13.266 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-16T22:55:13.267 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-16T22:55:13.267 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-16T22:55:13.308 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-16T22:55:13.311 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-02-16T22:55:13.312 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-16T22:55:13.367 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-16T22:55:13.369 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-16T22:55:13.369 DEBUG:teuthology.orchestra.run.smithi192:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed7ab725e1a43615009dfb74a42bf33d80081bc8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-16T22:55:13.433 INFO:tasks.workunit.client.1.smithi192.stderr:+ mkdir -p fsstress 2024-02-16T22:55:13.436 INFO:tasks.workunit.client.1.smithi192.stderr:+ pushd fsstress 2024-02-16T22:55:13.437 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T22:55:13.437 INFO:tasks.workunit.client.1.smithi192.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-16T22:55:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:13 smithi192 ceph-mon[36254]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:13.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:13 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:55:13.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:13 smithi064 ceph-mon[29226]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:13.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:13 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr:Note: switching to 'ed7ab725e1a43615009dfb74a42bf33d80081bc8'. 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr:state without impacting any branches by switching back to a branch. 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr: git switch -c 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-02-16T22:55:14.827 INFO:tasks.workunit.client.0.smithi064.stderr:Or undo this operation with: 2024-02-16T22:55:14.828 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-02-16T22:55:14.828 INFO:tasks.workunit.client.0.smithi064.stderr: git switch - 2024-02-16T22:55:14.828 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-02-16T22:55:14.828 INFO:tasks.workunit.client.0.smithi064.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-16T22:55:14.828 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-02-16T22:55:14.828 INFO:tasks.workunit.client.0.smithi064.stderr:HEAD is now at ed7ab725e1a qa: test fuse/kclient for mds upgrade seq 2024-02-16T22:55:14.831 INFO:tasks.workunit.client.1.smithi192.stderr:+ tar xzf ltp-full.tgz 2024-02-16T22:55:14.832 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-16T22:55:14.867 INFO:tasks.workunit.client.0.smithi064.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-16T22:55:14.869 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-16T22:55:14.869 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-16T22:55:15.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:15 smithi192 ceph-mon[36254]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:15.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:15 smithi064 ceph-mon[29226]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:55:15.801 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-16T22:55:15.870 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-16T22:55:15.911 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-16T22:55:15.912 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-16T22:55:15.912 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-16T22:55:15.951 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-16T22:55:15.954 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T22:55:15.954 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-16T22:55:16.011 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-16T22:55:16.012 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-16T22:55:16.012 DEBUG:teuthology.orchestra.run.smithi064:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed7ab725e1a43615009dfb74a42bf33d80081bc8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-16T22:55:16.074 INFO:tasks.workunit.client.0.smithi064.stderr:+ mkdir -p fsstress 2024-02-16T22:55:16.077 INFO:tasks.workunit.client.0.smithi064.stderr:+ pushd fsstress 2024-02-16T22:55:16.078 INFO:tasks.workunit.client.0.smithi064.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T22:55:16.078 INFO:tasks.workunit.client.0.smithi064.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-16T22:55:17.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:17 smithi192 ceph-mon[36254]: pgmap v223: 65 pgs: 65 active+clean; 6.9 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 539 KiB/s wr, 0 op/s 2024-02-16T22:55:17.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:17 smithi064 ceph-mon[29226]: pgmap v223: 65 pgs: 65 active+clean; 6.9 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 539 KiB/s wr, 0 op/s 2024-02-16T22:55:17.849 INFO:tasks.workunit.client.0.smithi064.stderr:+ tar xzf ltp-full.tgz 2024-02-16T22:55:19.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:18 smithi064 ceph-mon[29226]: pgmap v224: 65 pgs: 65 active+clean; 7.5 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 597 KiB/s wr, 9 op/s 2024-02-16T22:55:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:18 smithi192 ceph-mon[36254]: pgmap v224: 65 pgs: 65 active+clean; 7.5 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 597 KiB/s wr, 9 op/s 2024-02-16T22:55:21.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:21 smithi192 ceph-mon[36254]: pgmap v225: 65 pgs: 65 active+clean; 18 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 41 op/s 2024-02-16T22:55:21.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:21 smithi064 ceph-mon[29226]: pgmap v225: 65 pgs: 65 active+clean; 18 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 41 op/s 2024-02-16T22:55:23.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:23 smithi192 ceph-mon[36254]: pgmap v226: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 70 op/s 2024-02-16T22:55:23.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:23 smithi064 ceph-mon[29226]: pgmap v226: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 70 op/s 2024-02-16T22:55:25.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:25 smithi192 ceph-mon[36254]: pgmap v227: 65 pgs: 65 active+clean; 33 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.7 MiB/s wr, 111 op/s 2024-02-16T22:55:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:25 smithi064 ceph-mon[29226]: pgmap v227: 65 pgs: 65 active+clean; 33 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.7 MiB/s wr, 111 op/s 2024-02-16T22:55:27.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:27 smithi064 ceph-mon[29226]: pgmap v228: 65 pgs: 65 active+clean; 42 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 134 op/s 2024-02-16T22:55:27.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:27 smithi192 ceph-mon[36254]: pgmap v228: 65 pgs: 65 active+clean; 42 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 134 op/s 2024-02-16T22:55:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:28 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:55:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:28 smithi064 ceph-mon[29226]: pgmap v229: 65 pgs: 65 active+clean; 50 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 156 op/s 2024-02-16T22:55:29.153 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:28 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:55:29.153 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:28 smithi192 ceph-mon[36254]: pgmap v229: 65 pgs: 65 active+clean; 50 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 156 op/s 2024-02-16T22:55:31.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:31 smithi192 ceph-mon[36254]: pgmap v230: 65 pgs: 65 active+clean; 55 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 182 op/s 2024-02-16T22:55:31.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:31 smithi064 ceph-mon[29226]: pgmap v230: 65 pgs: 65 active+clean; 55 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 182 op/s 2024-02-16T22:55:33.399 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:33 smithi192 ceph-mon[36254]: pgmap v231: 65 pgs: 65 active+clean; 76 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.9 MiB/s wr, 185 op/s 2024-02-16T22:55:33.399 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:33.399 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:55:33.399 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:55:33.399 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:33 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:55:33.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:33 smithi064 ceph-mon[29226]: pgmap v231: 65 pgs: 65 active+clean; 76 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.9 MiB/s wr, 185 op/s 2024-02-16T22:55:33.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:33.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:55:33.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:55:33.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:33 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:55:34.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:34 smithi192 ceph-mon[36254]: Upgrade: Updating mgr.smithi192.pauvih 2024-02-16T22:55:34.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:34 smithi192 ceph-mon[36254]: Deploying daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:55:34.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:34 smithi064 ceph-mon[29226]: Upgrade: Updating mgr.smithi192.pauvih 2024-02-16T22:55:34.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:34 smithi064 ceph-mon[29226]: Deploying daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:55:35.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:35 smithi192 ceph-mon[36254]: pgmap v232: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 207 op/s 2024-02-16T22:55:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:35 smithi064 ceph-mon[29226]: pgmap v232: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 207 op/s 2024-02-16T22:55:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:37 smithi192 ceph-mon[36254]: pgmap v233: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 192 op/s 2024-02-16T22:55:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:37 smithi064 ceph-mon[29226]: pgmap v233: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 192 op/s 2024-02-16T22:55:39.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:38 smithi064 ceph-mon[29226]: pgmap v234: 65 pgs: 65 active+clean; 96 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 193 op/s 2024-02-16T22:55:39.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:38 smithi192 ceph-mon[36254]: pgmap v234: 65 pgs: 65 active+clean; 96 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 193 op/s 2024-02-16T22:55:40.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.667+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1569113949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 msgr2=0x7f90b00fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:40.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.667+0000 7f90b7b7a700 1 --2- 172.21.15.64:0/1569113949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 0x7f90b00fd430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004940 tx=0x7f90a0009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.667+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1569113949 shutdown_connections 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.667+0000 7f90b7b7a700 1 --2- 172.21.15.64:0/1569113949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 0x7f90b00fd430 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.667+0000 7f90b7b7a700 1 --2- 172.21.15.64:0/1569113949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90b00fbe50 0x7f90b00fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.667+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1569113949 >> 172.21.15.64:0/1569113949 conn(0x7f90b00f7510 msgr2=0x7f90b00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.667+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1569113949 shutdown_connections 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.667+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1569113949 wait complete. 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b7b7a700 1 Processor -- start 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b7b7a700 1 -- start start 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b7b7a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90b00fbe50 0x7f90b010c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b7b7a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 0x7f90b010ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b7b7a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b010d2f0 con 0x7f90b00fcfc0 2024-02-16T22:55:40.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b7b7a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b010db90 con 0x7f90b00fbe50 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b5115700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 0x7f90b010ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b5115700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 0x7f90b010ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52796/0 (socket says 172.21.15.64:52796) 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b5115700 1 -- 172.21.15.64:0/1038641829 learned_addr learned my addr 172.21.15.64:0/1038641829 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.668+0000 7f90b5916700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90b00fbe50 0x7f90b010c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90b5916700 1 -- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 msgr2=0x7f90b010ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90b5916700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 0x7f90b010ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90b5916700 1 -- 172.21.15.64:0/1038641829 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0005040 con 0x7f90b00fbe50 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90b5115700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 0x7f90b010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90b5916700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90b00fbe50 0x7f90b010c7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009fd0 tx=0x7f90ac00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90a67fc700 1 -- 172.21.15.64:0/1038641829 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ac004e90 con 0x7f90b00fbe50 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90a67fc700 1 -- 172.21.15.64:0/1038641829 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90ac00f040 con 0x7f90b00fbe50 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b010de10 con 0x7f90b00fbe50 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90a67fc700 1 -- 172.21.15.64:0/1038641829 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ac00ca50 con 0x7f90b00fbe50 2024-02-16T22:55:40.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.669+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b010e350 con 0x7f90b00fbe50 2024-02-16T22:55:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.670+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b004bed0 con 0x7f90b00fbe50 2024-02-16T22:55:40.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.671+0000 7f90a67fc700 1 -- 172.21.15.64:0/1038641829 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f90ac007cc0 con 0x7f90b00fbe50 2024-02-16T22:55:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.672+0000 7f90a67fc700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f909c0692b0 0x7f909c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.672+0000 7f90a67fc700 1 -- 172.21.15.64:0/1038641829 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f90ac086b20 con 0x7f90b00fbe50 2024-02-16T22:55:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.672+0000 7f90b5115700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f909c0692b0 0x7f909c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:40.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.672+0000 7f90b5115700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f909c0692b0 0x7f909c06b760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f90a0013740 tx=0x7f90a0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:40.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.676+0000 7f90a67fc700 1 -- 172.21.15.64:0/1038641829 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f90ac04dbd0 con 0x7f90b00fbe50 2024-02-16T22:55:40.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.995+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90b010e7c0 con 0x7f909c0692b0 2024-02-16T22:55:40.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.996+0000 7f90a67fc700 1 -- 172.21.15.64:0/1038641829 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f90b010e7c0 con 0x7f909c0692b0 2024-02-16T22:55:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f909c0692b0 msgr2=0x7f909c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f909c0692b0 0x7f909c06b760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f90a0013740 tx=0x7f90a0009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90b00fbe50 msgr2=0x7f90b010c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90b00fbe50 0x7f90b010c7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009fd0 tx=0x7f90ac00b040 comp rx=0 tx=0).stop 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 shutdown_connections 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90b00fbe50 0x7f90b010c7b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f909c0692b0 0x7f909c06b760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 --2- 172.21.15.64:0/1038641829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90b00fcfc0 0x7f90b010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.998+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 >> 172.21.15.64:0/1038641829 conn(0x7f90b00f7510 msgr2=0x7f90b0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.999+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 shutdown_connections 2024-02-16T22:55:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:40.999+0000 7f90b7b7a700 1 -- 172.21.15.64:0/1038641829 wait complete. 2024-02-16T22:55:41.010 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:55:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.135+0000 7ff0e938c700 1 -- 172.21.15.64:0/1119985281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 msgr2=0x7ff0e40fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.135+0000 7ff0e938c700 1 --2- 172.21.15.64:0/1119985281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 0x7ff0e40fc260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4004940 tx=0x7ff0d4009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.136+0000 7ff0e938c700 1 -- 172.21.15.64:0/1119985281 shutdown_connections 2024-02-16T22:55:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.136+0000 7ff0e938c700 1 --2- 172.21.15.64:0/1119985281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0e40fcfc0 0x7ff0e40fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.136+0000 7ff0e938c700 1 --2- 172.21.15.64:0/1119985281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 0x7ff0e40fc260 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.136+0000 7ff0e938c700 1 -- 172.21.15.64:0/1119985281 >> 172.21.15.64:0/1119985281 conn(0x7ff0e40f7510 msgr2=0x7ff0e40f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.136+0000 7ff0e938c700 1 -- 172.21.15.64:0/1119985281 shutdown_connections 2024-02-16T22:55:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.136+0000 7ff0e938c700 1 -- 172.21.15.64:0/1119985281 wait complete. 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.136+0000 7ff0e938c700 1 Processor -- start 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.136+0000 7ff0e938c700 1 -- start start 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e938c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 0x7ff0e41105e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 0x7ff0e41105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 0x7ff0e41105e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52814/0 (socket says 172.21.15.64:52814) 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e938c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0e40fcfc0 0x7ff0e4112b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e2ffd700 1 -- 172.21.15.64:0/1749045054 learned_addr learned my addr 172.21.15.64:0/1749045054 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e938c700 1 -- 172.21.15.64:0/1749045054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4066d10 con 0x7ff0e40fbe50 2024-02-16T22:55:41.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e938c700 1 -- 172.21.15.64:0/1749045054 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4066e50 con 0x7ff0e40fcfc0 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e27fc700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0e40fcfc0 0x7ff0e4112b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e2ffd700 1 -- 172.21.15.64:0/1749045054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0e40fcfc0 msgr2=0x7ff0e4112b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e2ffd700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0e40fcfc0 0x7ff0e4112b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.137+0000 7ff0e2ffd700 1 -- 172.21.15.64:0/1749045054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d4005040 con 0x7ff0e40fbe50 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.138+0000 7ff0e2ffd700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 0x7ff0e41105e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4000c00 tx=0x7ff0d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.138+0000 7ff0cb7fe700 1 -- 172.21.15.64:0/1749045054 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0d4021070 con 0x7ff0e40fbe50 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.138+0000 7ff0e938c700 1 -- 172.21.15.64:0/1749045054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e4113130 con 0x7ff0e40fbe50 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.138+0000 7ff0e938c700 1 -- 172.21.15.64:0/1749045054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e41135c0 con 0x7ff0e40fbe50 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.138+0000 7ff0cb7fe700 1 -- 172.21.15.64:0/1749045054 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0d401c400 con 0x7ff0e40fbe50 2024-02-16T22:55:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.138+0000 7ff0cb7fe700 1 -- 172.21.15.64:0/1749045054 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0d400e070 con 0x7ff0e40fbe50 2024-02-16T22:55:41.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.139+0000 7ff0c97fa700 1 -- 172.21.15.64:0/1749045054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c40052b0 con 0x7ff0e40fbe50 2024-02-16T22:55:41.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.140+0000 7ff0cb7fe700 1 -- 172.21.15.64:0/1749045054 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff0d401ca50 con 0x7ff0e40fbe50 2024-02-16T22:55:41.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.140+0000 7ff0cb7fe700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0cc0692a0 0x7ff0cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:41.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.140+0000 7ff0cb7fe700 1 -- 172.21.15.64:0/1749045054 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff0d4088290 con 0x7ff0e40fbe50 2024-02-16T22:55:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.140+0000 7ff0e27fc700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0cc0692a0 0x7ff0cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.141+0000 7ff0e27fc700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0cc0692a0 0x7ff0cc06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009b80 tx=0x7ff0d800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.146+0000 7ff0cb7fe700 1 -- 172.21.15.64:0/1749045054 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff0d4054660 con 0x7ff0e40fbe50 2024-02-16T22:55:41.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:41 smithi064 ceph-mon[29226]: pgmap v235: 65 pgs: 65 active+clean; 102 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 201 op/s 2024-02-16T22:55:41.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.466+0000 7ff0c97fa700 1 -- 172.21.15.64:0/1749045054 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0c4000bc0 con 0x7ff0cc0692a0 2024-02-16T22:55:41.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.467+0000 7ff0cb7fe700 1 -- 172.21.15.64:0/1749045054 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7ff0c4000bc0 con 0x7ff0cc0692a0 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.470+0000 7ff0c97fa700 1 -- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0cc0692a0 msgr2=0x7ff0cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.470+0000 7ff0c97fa700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0cc0692a0 0x7ff0cc06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009b80 tx=0x7ff0d800b040 comp rx=0 tx=0).stop 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.470+0000 7ff0c97fa700 1 -- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 msgr2=0x7ff0e41105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.470+0000 7ff0c97fa700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 0x7ff0e41105e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4000c00 tx=0x7ff0d4009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.470+0000 7ff0c97fa700 1 -- 172.21.15.64:0/1749045054 shutdown_connections 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.470+0000 7ff0c97fa700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff0cc0692a0 0x7ff0cc06b750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.470+0000 7ff0c97fa700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e40fbe50 0x7ff0e41105e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.470+0000 7ff0c97fa700 1 --2- 172.21.15.64:0/1749045054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0e40fcfc0 0x7ff0e4112b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.471+0000 7ff0c97fa700 1 -- 172.21.15.64:0/1749045054 >> 172.21.15.64:0/1749045054 conn(0x7ff0e40f7510 msgr2=0x7ff0e4100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.471+0000 7ff0c97fa700 1 -- 172.21.15.64:0/1749045054 shutdown_connections 2024-02-16T22:55:41.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.471+0000 7ff0c97fa700 1 -- 172.21.15.64:0/1749045054 wait complete. 2024-02-16T22:55:41.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:41 smithi192 ceph-mon[36254]: pgmap v235: 65 pgs: 65 active+clean; 102 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 201 op/s 2024-02-16T22:55:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.586+0000 7ff58a2a3700 1 -- 172.21.15.64:0/4184424031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5841003a0 msgr2=0x7ff584102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.586+0000 7ff58a2a3700 1 --2- 172.21.15.64:0/4184424031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5841003a0 0x7ff584102780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff578004940 tx=0x7ff578009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.586+0000 7ff58a2a3700 1 -- 172.21.15.64:0/4184424031 shutdown_connections 2024-02-16T22:55:41.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.586+0000 7ff58a2a3700 1 --2- 172.21.15.64:0/4184424031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5841003a0 0x7ff584102780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.586+0000 7ff58a2a3700 1 --2- 172.21.15.64:0/4184424031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5840fda80 0x7ff5840ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.586+0000 7ff58a2a3700 1 -- 172.21.15.64:0/4184424031 >> 172.21.15.64:0/4184424031 conn(0x7ff5840f7500 msgr2=0x7ff5840f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.586+0000 7ff58a2a3700 1 -- 172.21.15.64:0/4184424031 shutdown_connections 2024-02-16T22:55:41.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.586+0000 7ff58a2a3700 1 -- 172.21.15.64:0/4184424031 wait complete. 2024-02-16T22:55:41.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.587+0000 7ff58a2a3700 1 Processor -- start 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.587+0000 7ff58a2a3700 1 -- start start 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.587+0000 7ff58a2a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5840fda80 0x7ff58410d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.587+0000 7ff58a2a3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5841003a0 0x7ff58410de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.587+0000 7ff58a2a3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58410e400 con 0x7ff5840fda80 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.587+0000 7ff58a2a3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58410eca0 con 0x7ff5841003a0 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.587+0000 7ff583fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5840fda80 0x7ff58410d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff583fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5840fda80 0x7ff58410d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52834/0 (socket says 172.21.15.64:52834) 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff583fff700 1 -- 172.21.15.64:0/2361270218 learned_addr learned my addr 172.21.15.64:0/2361270218 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff5837fe700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5841003a0 0x7ff58410de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff5837fe700 1 -- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5840fda80 msgr2=0x7ff58410d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff5837fe700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5840fda80 0x7ff58410d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff5837fe700 1 -- 172.21.15.64:0/2361270218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff578005040 con 0x7ff5841003a0 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff583fff700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5840fda80 0x7ff58410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:55:41.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff5837fe700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5841003a0 0x7ff58410de00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff578004890 tx=0x7ff578009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff580ff9700 1 -- 172.21.15.64:0/2361270218 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff578021070 con 0x7ff5841003a0 2024-02-16T22:55:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff580ff9700 1 -- 172.21.15.64:0/2361270218 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff57801c400 con 0x7ff5841003a0 2024-02-16T22:55:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58410ef20 con 0x7ff5841003a0 2024-02-16T22:55:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.588+0000 7ff580ff9700 1 -- 172.21.15.64:0/2361270218 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff57800e070 con 0x7ff5841003a0 2024-02-16T22:55:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.589+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58410f460 con 0x7ff5841003a0 2024-02-16T22:55:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.590+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff584107a50 con 0x7ff5841003a0 2024-02-16T22:55:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.590+0000 7ff580ff9700 1 -- 172.21.15.64:0/2361270218 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff578013af0 con 0x7ff5841003a0 2024-02-16T22:55:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.590+0000 7ff580ff9700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff56c0692a0 0x7ff56c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.591+0000 7ff580ff9700 1 -- 172.21.15.64:0/2361270218 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff578088210 con 0x7ff5841003a0 2024-02-16T22:55:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.591+0000 7ff583fff700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff56c0692a0 0x7ff56c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:41.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.591+0000 7ff583fff700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff56c0692a0 0x7ff56c06b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff5740041d0 tx=0x7ff57400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:41.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.595+0000 7ff580ff9700 1 -- 172.21.15.64:0/2361270218 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff578058070 con 0x7ff5841003a0 2024-02-16T22:55:41.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.870+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff58402b310 con 0x7ff56c0692a0 2024-02-16T22:55:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.884+0000 7ff580ff9700 1 -- 172.21.15.64:0/2361270218 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7ff58402b310 con 0x7ff56c0692a0 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (5m) 90s ago 7m 25.1M - 0.25.0 c8568f914cd2 829b066b8c26 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (7m) 90s ago 7m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (6m) 96s ago 6m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (5m) 90s ago 7m 86.0M - 9.4.7 954c08fa6188 529226ab33c5 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (109s) 90s ago 109s 14.7M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (114s) 90s ago 114s 14.5M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (107s) 96s ago 106s 13.5M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (112s) 96s ago 111s 13.5M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:9283 running (8m) 90s ago 8m 480M - 17.2.7-513-gb88fae66 870a0944ccb8 7189c74ed7fc 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (6m) 96s ago 6m 433M - 17.2.7-513-gb88fae66 870a0944ccb8 a017e081156a 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (8m) 90s ago 8m 45.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (6m) 96s ago 6m 36.1M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (7m) 90s ago 7m 15.5M - 1.5.0 0da6a335fe13 c25d924fa381 2024-02-16T22:55:41.885 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 96s ago 6m 16.1M - 1.5.0 0da6a335fe13 c32798cb2969 2024-02-16T22:55:41.886 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (5m) 90s ago 5m 46.5M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:55:41.886 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (5m) 90s ago 5m 45.3M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:55:41.886 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (4m) 90s ago 4m 44.7M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:55:41.886 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (4m) 96s ago 4m 43.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:55:41.886 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (3m) 96s ago 3m 41.9M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:55:41.886 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (3m) 96s ago 3m 41.5M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:55:41.886 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (5m) 90s ago 6m 53.4M - 2.43.0 a07b618ecd1d dd20eff0542c 2024-02-16T22:55:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.886+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff56c0692a0 msgr2=0x7ff56c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.886+0000 7ff58a2a3700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff56c0692a0 0x7ff56c06b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff5740041d0 tx=0x7ff57400b040 comp rx=0 tx=0).stop 2024-02-16T22:55:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.886+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5841003a0 msgr2=0x7ff58410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.886+0000 7ff58a2a3700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5841003a0 0x7ff58410de00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff578004890 tx=0x7ff578009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.887+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 shutdown_connections 2024-02-16T22:55:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.887+0000 7ff58a2a3700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7ff56c0692a0 0x7ff56c06b750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.887+0000 7ff58a2a3700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5840fda80 0x7ff58410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.887+0000 7ff58a2a3700 1 --2- 172.21.15.64:0/2361270218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5841003a0 0x7ff58410de00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:41.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.887+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 >> 172.21.15.64:0/2361270218 conn(0x7ff5840f7500 msgr2=0x7ff5840f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:41.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.887+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 shutdown_connections 2024-02-16T22:55:41.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:41.887+0000 7ff58a2a3700 1 -- 172.21.15.64:0/2361270218 wait complete. 2024-02-16T22:55:42.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.014+0000 7f5172dd5700 1 -- 172.21.15.64:0/3258901861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 msgr2=0x7f516c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.014+0000 7f5172dd5700 1 --2- 172.21.15.64:0/3258901861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 0x7f516c102780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5158004940 tx=0x7f5158009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.014+0000 7f5172dd5700 1 -- 172.21.15.64:0/3258901861 shutdown_connections 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.014+0000 7f5172dd5700 1 --2- 172.21.15.64:0/3258901861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 0x7f516c102780 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.014+0000 7f5172dd5700 1 --2- 172.21.15.64:0/3258901861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c0fda80 0x7f516c0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.014+0000 7f5172dd5700 1 -- 172.21.15.64:0/3258901861 >> 172.21.15.64:0/3258901861 conn(0x7f516c0f74e0 msgr2=0x7f516c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.014+0000 7f5172dd5700 1 -- 172.21.15.64:0/3258901861 shutdown_connections 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.014+0000 7f5172dd5700 1 -- 172.21.15.64:0/3258901861 wait complete. 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5172dd5700 1 Processor -- start 2024-02-16T22:55:42.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5172dd5700 1 -- start start 2024-02-16T22:55:42.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5172dd5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c0fda80 0x7f516c0fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5172dd5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 0x7f516c0fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5172dd5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0fbf90 con 0x7f516c1003a0 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5172dd5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0fc100 con 0x7f516c0fda80 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5170b71700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c0fda80 0x7f516c0fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5170b71700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c0fda80 0x7f516c0fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:40564/0 (socket says 172.21.15.64:40564) 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.015+0000 7f5170b71700 1 -- 172.21.15.64:0/1322065072 learned_addr learned my addr 172.21.15.64:0/1322065072 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.016+0000 7f516bfff700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 0x7f516c0fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.016+0000 7f516bfff700 1 -- 172.21.15.64:0/1322065072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c0fda80 msgr2=0x7f516c0fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.016+0000 7f516bfff700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c0fda80 0x7f516c0fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.016+0000 7f516bfff700 1 -- 172.21.15.64:0/1322065072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5158005040 con 0x7f516c1003a0 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.016+0000 7f516bfff700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 0x7f516c0fb990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5158000c00 tx=0x7f5158009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.016+0000 7f51697fa700 1 -- 172.21.15.64:0/1322065072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515800e040 con 0x7f516c1003a0 2024-02-16T22:55:42.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.017+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c0fc380 con 0x7f516c1003a0 2024-02-16T22:55:42.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.017+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c115970 con 0x7f516c1003a0 2024-02-16T22:55:42.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.017+0000 7f51697fa700 1 -- 172.21.15.64:0/1322065072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f515800bbf0 con 0x7f516c1003a0 2024-02-16T22:55:42.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.017+0000 7f51697fa700 1 -- 172.21.15.64:0/1322065072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5158013420 con 0x7f516c1003a0 2024-02-16T22:55:42.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.018+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51500052e0 con 0x7f516c1003a0 2024-02-16T22:55:42.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.020+0000 7f51697fa700 1 -- 172.21.15.64:0/1322065072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f515800bd60 con 0x7f516c1003a0 2024-02-16T22:55:42.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.020+0000 7f51697fa700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f515c0692a0 0x7f515c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:42.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.020+0000 7f51697fa700 1 -- 172.21.15.64:0/1322065072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5158087ce0 con 0x7f516c1003a0 2024-02-16T22:55:42.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.021+0000 7f5170b71700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f515c0692a0 0x7f515c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:42.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.021+0000 7f5170b71700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f515c0692a0 0x7f515c06b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f516c0fd2a0 tx=0x7f5160009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:42.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.025+0000 7f51697fa700 1 -- 172.21.15.64:0/1322065072 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5158057b50 con 0x7f516c1003a0 2024-02-16T22:55:42.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.429+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5150005da0 con 0x7f516c1003a0 2024-02-16T22:55:42.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.430+0000 7f51697fa700 1 -- 172.21.15.64:0/1322065072 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f51580576e0 con 0x7f516c1003a0 2024-02-16T22:55:42.431 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:55:42.431 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:55:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:55:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:55:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:55:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:55:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:55:42.432 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:55:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:55:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:55:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:55:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:55:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:55:42.433 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:55:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 14 2024-02-16T22:55:42.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:55:42.434 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.433+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f515c0692a0 msgr2=0x7f515c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.433+0000 7f5172dd5700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f515c0692a0 0x7f515c06b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f516c0fd2a0 tx=0x7f5160009040 comp rx=0 tx=0).stop 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.433+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 msgr2=0x7f516c0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.433+0000 7f5172dd5700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 0x7f516c0fb990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5158000c00 tx=0x7f5158009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.434+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 shutdown_connections 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.434+0000 7f5172dd5700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c0fda80 0x7f516c0fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.434+0000 7f5172dd5700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f515c0692a0 0x7f515c06b750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.434+0000 7f5172dd5700 1 --2- 172.21.15.64:0/1322065072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f516c1003a0 0x7f516c0fb990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.434+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 >> 172.21.15.64:0/1322065072 conn(0x7f516c0f74e0 msgr2=0x7f516c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.435+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 shutdown_connections 2024-02-16T22:55:42.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.435+0000 7f5172dd5700 1 -- 172.21.15.64:0/1322065072 wait complete. 2024-02-16T22:55:42.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:42 smithi192 ceph-mon[36254]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:42.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.587+0000 7faaa0348700 1 -- 172.21.15.64:0/1910221088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98066c10 msgr2=0x7faa980ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:42.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.587+0000 7faaa0348700 1 --2- 172.21.15.64:0/1910221088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98066c10 0x7faa980ffb50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faa94004940 tx=0x7faa94009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:42.589 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:42 smithi064 ceph-mon[29226]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.587+0000 7faaa0348700 1 -- 172.21.15.64:0/1910221088 shutdown_connections 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.587+0000 7faaa0348700 1 --2- 172.21.15.64:0/1910221088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa98100120 0x7faa98102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.587+0000 7faaa0348700 1 --2- 172.21.15.64:0/1910221088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98066c10 0x7faa980ffb50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.587+0000 7faaa0348700 1 -- 172.21.15.64:0/1910221088 >> 172.21.15.64:0/1910221088 conn(0x7faa980f7500 msgr2=0x7faa980f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.587+0000 7faaa0348700 1 -- 172.21.15.64:0/1910221088 shutdown_connections 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.587+0000 7faaa0348700 1 -- 172.21.15.64:0/1910221088 wait complete. 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.588+0000 7faaa0348700 1 Processor -- start 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.588+0000 7faaa0348700 1 -- start start 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.588+0000 7faaa0348700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa98066c10 0x7faa9810fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.588+0000 7faaa0348700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98100120 0x7faa98110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:42.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.588+0000 7faaa0348700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa98110640 con 0x7faa98100120 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.588+0000 7faaa0348700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa98110ee0 con 0x7faa98066c10 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.588+0000 7faa9d8e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98100120 0x7faa98110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa9d8e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98100120 0x7faa98110040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52878/0 (socket says 172.21.15.64:52878) 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa9e0e4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa98066c10 0x7faa9810fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa9d8e3700 1 -- 172.21.15.64:0/2899808119 learned_addr learned my addr 172.21.15.64:0/2899808119 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa9e0e4700 1 -- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98100120 msgr2=0x7faa98110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa9e0e4700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98100120 0x7faa98110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa9e0e4700 1 -- 172.21.15.64:0/2899808119 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa94005040 con 0x7faa98066c10 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa9d8e3700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98100120 0x7faa98110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa9e0e4700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa98066c10 0x7faa9810fb00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa94004890 tx=0x7faa94009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa8effd700 1 -- 172.21.15.64:0/2899808119 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa94021070 con 0x7faa98066c10 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa98111160 con 0x7faa98066c10 2024-02-16T22:55:42.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa8effd700 1 -- 172.21.15.64:0/2899808119 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa9401c400 con 0x7faa98066c10 2024-02-16T22:55:42.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faa8effd700 1 -- 172.21.15.64:0/2899808119 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9400e070 con 0x7faa98066c10 2024-02-16T22:55:42.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.589+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa981116a0 con 0x7faa98066c10 2024-02-16T22:55:42.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.591+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa980f9080 con 0x7faa98066c10 2024-02-16T22:55:42.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.591+0000 7faa8effd700 1 -- 172.21.15.64:0/2899808119 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7faa9401ca70 con 0x7faa98066c10 2024-02-16T22:55:42.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.592+0000 7faa8effd700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7faa840692a0 0x7faa8406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:42.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.592+0000 7faa8effd700 1 -- 172.21.15.64:0/2899808119 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7faa940883a0 con 0x7faa98066c10 2024-02-16T22:55:42.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.592+0000 7faa9d8e3700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7faa840692a0 0x7faa8406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:42.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.592+0000 7faa9d8e3700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7faa840692a0 0x7faa8406b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa88009b80 tx=0x7faa8800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:42.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.596+0000 7faa8effd700 1 -- 172.21.15.64:0/2899808119 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7faa940580c0 con 0x7faa98066c10 2024-02-16T22:55:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.881+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faa9802b2d0 con 0x7faa98066c10 2024-02-16T22:55:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.881+0000 7faa8effd700 1 -- 172.21.15.64:0/2899808119 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7faa94004490 con 0x7faa98066c10 2024-02-16T22:55:42.882 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:55:42.882 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:55:42.883 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:55:42.884 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:55:42.885 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:55:42.886 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:55:42.886 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:55:42.886 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:55:42.886 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.883+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7faa840692a0 msgr2=0x7faa8406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.883+0000 7faaa0348700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7faa840692a0 0x7faa8406b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa88009b80 tx=0x7faa8800b040 comp rx=0 tx=0).stop 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.883+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa98066c10 msgr2=0x7faa9810fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.883+0000 7faaa0348700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa98066c10 0x7faa9810fb00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa94004890 tx=0x7faa94009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.884+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 shutdown_connections 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.884+0000 7faaa0348700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa98066c10 0x7faa9810fb00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.884+0000 7faaa0348700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7faa840692a0 0x7faa8406b750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.884+0000 7faaa0348700 1 --2- 172.21.15.64:0/2899808119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa98100120 0x7faa98110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.884+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 >> 172.21.15.64:0/2899808119 conn(0x7faa980f7500 msgr2=0x7faa98100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.884+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 shutdown_connections 2024-02-16T22:55:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:42.884+0000 7faaa0348700 1 -- 172.21.15.64:0/2899808119 wait complete. 2024-02-16T22:55:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:55:43.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.023+0000 7f545d9a2700 1 -- 172.21.15.64:0/321512857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54581000c0 msgr2=0x7f5458102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:43.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.023+0000 7f545d9a2700 1 --2- 172.21.15.64:0/321512857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54581000c0 0x7f5458102520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f544c004940 tx=0x7f544c009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:43.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.023+0000 7f545d9a2700 1 -- 172.21.15.64:0/321512857 shutdown_connections 2024-02-16T22:55:43.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.023+0000 7f545d9a2700 1 --2- 172.21.15.64:0/321512857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54581000c0 0x7f5458102520 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.023+0000 7f545d9a2700 1 --2- 172.21.15.64:0/321512857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54580663e0 0x7f54580ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.023+0000 7f545d9a2700 1 -- 172.21.15.64:0/321512857 >> 172.21.15.64:0/321512857 conn(0x7f54580f74e0 msgr2=0x7f54580f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:43.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.023+0000 7f545d9a2700 1 -- 172.21.15.64:0/321512857 shutdown_connections 2024-02-16T22:55:43.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.023+0000 7f545d9a2700 1 -- 172.21.15.64:0/321512857 wait complete. 2024-02-16T22:55:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f545d9a2700 1 Processor -- start 2024-02-16T22:55:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f545d9a2700 1 -- start start 2024-02-16T22:55:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f545d9a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580663e0 0x7f545810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f545d9a2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54581000c0 0x7f545810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f545d9a2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545810e4b0 con 0x7f54580663e0 2024-02-16T22:55:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f545d9a2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545810ed50 con 0x7f54581000c0 2024-02-16T22:55:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f5456ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580663e0 0x7f545810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f5456ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580663e0 0x7f545810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52902/0 (socket says 172.21.15.64:52902) 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f5456ffd700 1 -- 172.21.15.64:0/2464721362 learned_addr learned my addr 172.21.15.64:0/2464721362 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.024+0000 7f54567fc700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54581000c0 0x7f545810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f5456ffd700 1 -- 172.21.15.64:0/2464721362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54581000c0 msgr2=0x7f545810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f5456ffd700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54581000c0 0x7f545810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f5456ffd700 1 -- 172.21.15.64:0/2464721362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c005040 con 0x7f54580663e0 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f5456ffd700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580663e0 0x7f545810d970 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5448002a50 tx=0x7f54480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f543f7fe700 1 -- 172.21.15.64:0/2464721362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5448007cc0 con 0x7f54580663e0 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f543f7fe700 1 -- 172.21.15.64:0/2464721362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f544800f040 con 0x7f54580663e0 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f543f7fe700 1 -- 172.21.15.64:0/2464721362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f544800da30 con 0x7f54580663e0 2024-02-16T22:55:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545810f030 con 0x7f54580663e0 2024-02-16T22:55:43.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.025+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545810f4b0 con 0x7f54580663e0 2024-02-16T22:55:43.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.026+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545804bed0 con 0x7f54580663e0 2024-02-16T22:55:43.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.027+0000 7f543f7fe700 1 -- 172.21.15.64:0/2464721362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f544800db90 con 0x7f54580663e0 2024-02-16T22:55:43.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.028+0000 7f543f7fe700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f54400692a0 0x7f544006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:43.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.028+0000 7f543f7fe700 1 -- 172.21.15.64:0/2464721362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5448086e60 con 0x7f54580663e0 2024-02-16T22:55:43.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.028+0000 7f54567fc700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f54400692a0 0x7f544006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:43.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.028+0000 7f54567fc700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f54400692a0 0x7f544006b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f544c004890 tx=0x7f544c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:43.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.032+0000 7f543f7fe700 1 -- 172.21.15.64:0/2464721362 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5448056c20 con 0x7f54580663e0 2024-02-16T22:55:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.305+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 --> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f545810fa10 con 0x7f54400692a0 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.306+0000 7f543f7fe700 1 -- 172.21.15.64:0/2464721362 <== mgr.14208 v2:172.21.15.64:6800/700343021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f545810fa10 con 0x7f54400692a0 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [], 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "0/21 daemons upgraded", 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:55:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:55:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f54400692a0 msgr2=0x7f544006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f54400692a0 0x7f544006b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f544c004890 tx=0x7f544c009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580663e0 msgr2=0x7f545810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580663e0 0x7f545810d970 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5448002a50 tx=0x7f54480046b0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 shutdown_connections 2024-02-16T22:55:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f54400692a0 0x7f544006b750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580663e0 0x7f545810d970 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 --2- 172.21.15.64:0/2464721362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54581000c0 0x7f545810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 >> 172.21.15.64:0/2464721362 conn(0x7f54580f74e0 msgr2=0x7f54580f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:43.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 shutdown_connections 2024-02-16T22:55:43.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.309+0000 7f545d9a2700 1 -- 172.21.15.64:0/2464721362 wait complete. 2024-02-16T22:55:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.442+0000 7f77c74ae700 1 -- 172.21.15.64:0/2533614487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 msgr2=0x7f77c00fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.442+0000 7f77c74ae700 1 --2- 172.21.15.64:0/2533614487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 0x7f77c00fd320 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f77bc004940 tx=0x7f77bc009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.442+0000 7f77c74ae700 1 -- 172.21.15.64:0/2533614487 shutdown_connections 2024-02-16T22:55:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.442+0000 7f77c74ae700 1 --2- 172.21.15.64:0/2533614487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c00fe0e0 0x7f77c00fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.442+0000 7f77c74ae700 1 --2- 172.21.15.64:0/2533614487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 0x7f77c00fd320 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.442+0000 7f77c74ae700 1 -- 172.21.15.64:0/2533614487 >> 172.21.15.64:0/2533614487 conn(0x7f77c00f8620 msgr2=0x7f77c00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.442+0000 7f77c74ae700 1 -- 172.21.15.64:0/2533614487 shutdown_connections 2024-02-16T22:55:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.443+0000 7f77c74ae700 1 -- 172.21.15.64:0/2533614487 wait complete. 2024-02-16T22:55:43.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.443+0000 7f77c74ae700 1 Processor -- start 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.443+0000 7f77c74ae700 1 -- start start 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.443+0000 7f77c74ae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 0x7f77c0071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.443+0000 7f77c74ae700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c00fe0e0 0x7f77c00702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.443+0000 7f77c74ae700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c00708e0 con 0x7f77c00fcef0 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.443+0000 7f77c74ae700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c0070a50 con 0x7f77c00fe0e0 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c524a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 0x7f77c0071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c524a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 0x7f77c0071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52924/0 (socket says 172.21.15.64:52924) 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c524a700 1 -- 172.21.15.64:0/406227274 learned_addr learned my addr 172.21.15.64:0/406227274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c4a49700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c00fe0e0 0x7f77c00702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c524a700 1 -- 172.21.15.64:0/406227274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c00fe0e0 msgr2=0x7f77c00702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c524a700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c00fe0e0 0x7f77c00702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c524a700 1 -- 172.21.15.64:0/406227274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc005040 con 0x7f77c00fcef0 2024-02-16T22:55:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c4a49700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c00fe0e0 0x7f77c00702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:55:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.444+0000 7f77c524a700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 0x7f77c0071e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0043d0 tx=0x7f77bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.445+0000 7f77b5ffb700 1 -- 172.21.15.64:0/406227274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77bc00e070 con 0x7f77c00fcef0 2024-02-16T22:55:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.445+0000 7f77b5ffb700 1 -- 172.21.15.64:0/406227274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77bc01c400 con 0x7f77c00fcef0 2024-02-16T22:55:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.445+0000 7f77b5ffb700 1 -- 172.21.15.64:0/406227274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77bc00b650 con 0x7f77c00fcef0 2024-02-16T22:55:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.445+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c0070cd0 con 0x7f77c00fcef0 2024-02-16T22:55:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.445+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c0115920 con 0x7f77c00fcef0 2024-02-16T22:55:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.446+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c004bed0 con 0x7f77c00fcef0 2024-02-16T22:55:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.447+0000 7f77b5ffb700 1 -- 172.21.15.64:0/406227274 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f77bc01ca50 con 0x7f77c00fcef0 2024-02-16T22:55:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.447+0000 7f77b5ffb700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f77ac069370 0x7f77ac06b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:55:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.447+0000 7f77b5ffb700 1 -- 172.21.15.64:0/406227274 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f77bc0886f0 con 0x7f77c00fcef0 2024-02-16T22:55:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.447+0000 7f77c4a49700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f77ac069370 0x7f77ac06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:55:43.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.448+0000 7f77c4a49700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f77ac069370 0x7f77ac06b820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009b80 tx=0x7f77b000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:55:43.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.453+0000 7f77b5ffb700 1 -- 172.21.15.64:0/406227274 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f77bc058550 con 0x7f77c00fcef0 2024-02-16T22:55:43.454 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:43 smithi064 ceph-mon[29226]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:43.454 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:43 smithi064 ceph-mon[29226]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:43.454 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:43 smithi064 ceph-mon[29226]: pgmap v236: 65 pgs: 65 active+clean; 106 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 187 op/s 2024-02-16T22:55:43.455 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:43 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1322065072' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:55:43.455 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:43 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2899808119' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:55:43.455 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:43 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:55:43.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:43 smithi192 ceph-mon[36254]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:43 smithi192 ceph-mon[36254]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:43 smithi192 ceph-mon[36254]: pgmap v236: 65 pgs: 65 active+clean; 106 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 187 op/s 2024-02-16T22:55:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:43 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1322065072' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:55:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:43 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2899808119' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:55:43.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:43 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:55:43.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.799+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f77c0115cf0 con 0x7f77c00fcef0 2024-02-16T22:55:43.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.800+0000 7f77b5ffb700 1 -- 172.21.15.64:0/406227274 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f77bc0580e0 con 0x7f77c00fcef0 2024-02-16T22:55:43.801 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:55:43.801 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:55:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f77ac069370 msgr2=0x7f77ac06b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f77ac069370 0x7f77ac06b820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009b80 tx=0x7f77b000b040 comp rx=0 tx=0).stop 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 msgr2=0x7f77c0071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 0x7f77c0071e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0043d0 tx=0x7f77bc009e30 comp rx=0 tx=0).stop 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 shutdown_connections 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:6800/700343021,v1:172.21.15.64:6801/700343021] conn(0x7f77ac069370 0x7f77ac06b820 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c00fcef0 0x7f77c0071e60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 --2- 172.21.15.64:0/406227274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c00fe0e0 0x7f77c00702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:55:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.802+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 >> 172.21.15.64:0/406227274 conn(0x7f77c00f8620 msgr2=0x7f77c01013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:55:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.803+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 shutdown_connections 2024-02-16T22:55:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:55:43.803+0000 7f77c74ae700 1 -- 172.21.15.64:0/406227274 wait complete. 2024-02-16T22:55:44.337 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:44 smithi192 ceph-mon[36254]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:44.337 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:44 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/406227274' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:55:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:44 smithi064 ceph-mon[29226]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:55:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:44 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/406227274' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:55:45.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:45 smithi192 ceph-mon[36254]: pgmap v237: 65 pgs: 65 active+clean; 115 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 186 op/s 2024-02-16T22:55:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:45 smithi064 ceph-mon[29226]: pgmap v237: 65 pgs: 65 active+clean; 115 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 186 op/s 2024-02-16T22:55:47.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:47 smithi192 ceph-mon[36254]: pgmap v238: 65 pgs: 65 active+clean; 126 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 164 op/s 2024-02-16T22:55:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:47 smithi064 ceph-mon[29226]: pgmap v238: 65 pgs: 65 active+clean; 126 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 164 op/s 2024-02-16T22:55:48.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:48.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:48.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:48 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:48.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:48.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:48.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:48 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:49.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:49 smithi192 ceph-mon[36254]: pgmap v239: 65 pgs: 65 active+clean; 129 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 174 op/s 2024-02-16T22:55:49.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:49 smithi064 ceph-mon[29226]: pgmap v239: 65 pgs: 65 active+clean; 129 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 174 op/s 2024-02-16T22:55:51.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:51 smithi192 ceph-mon[36254]: pgmap v240: 65 pgs: 65 active+clean; 142 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.9 MiB/s wr, 217 op/s 2024-02-16T22:55:51.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:51.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:51.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:51.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:51 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:51.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:51 smithi064 ceph-mon[29226]: pgmap v240: 65 pgs: 65 active+clean; 142 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.9 MiB/s wr, 217 op/s 2024-02-16T22:55:51.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:51.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:51.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:51.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:51 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:53.504 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:53 smithi192 ceph-mon[36254]: pgmap v241: 65 pgs: 65 active+clean; 152 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.3 MiB/s wr, 234 op/s 2024-02-16T22:55:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:53 smithi064 ceph-mon[29226]: pgmap v241: 65 pgs: 65 active+clean; 152 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.3 MiB/s wr, 234 op/s 2024-02-16T22:55:55.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: pgmap v242: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 294 op/s 2024-02-16T22:55:55.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr fail", "who": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "mgr fail", "who": "smithi064.kubmlk"}]': finished 2024-02-16T22:55:55.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:55 smithi064 ceph-mon[29226]: mgrmap e20: smithi192.pauvih(active, starting, since 0.0104028s) 2024-02-16T22:55:55.717 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: pgmap v242: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 294 op/s 2024-02-16T22:55:55.717 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:55.717 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:55.717 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:55:55.717 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:55:55.717 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' 2024-02-16T22:55:55.718 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:55.718 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr fail", "who": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:55:55.718 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T22:55:55.718 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: from='mgr.14208 172.21.15.64:0/1819574479' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "mgr fail", "who": "smithi064.kubmlk"}]': finished 2024-02-16T22:55:55.718 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:55 smithi192 ceph-mon[36254]: mgrmap e20: smithi192.pauvih(active, starting, since 0.0104028s) 2024-02-16T22:55:56.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: Active manager daemon smithi192.pauvih restarted 2024-02-16T22:55:56.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: Activating manager daemon smithi192.pauvih 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: mgrmap e21: smithi192.pauvih(active, starting, since 0.00704674s) 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr metadata", "who": "smithi192.pauvih", "id": "smithi192.pauvih"}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:55:56.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: Manager daemon smithi192.pauvih is now available 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.pauvih/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:55:56.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.pauvih/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:55:56.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.pauvih/trash_purge_schedule"}]: dispatch 2024-02-16T22:55:56.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:56 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.pauvih/trash_purge_schedule"}]: dispatch 2024-02-16T22:55:56.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: Active manager daemon smithi192.pauvih restarted 2024-02-16T22:55:56.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: Activating manager daemon smithi192.pauvih 2024-02-16T22:55:56.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:55:56.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T22:55:56.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: mgrmap e21: smithi192.pauvih(active, starting, since 0.00704674s) 2024-02-16T22:55:56.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:55:56.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr metadata", "who": "smithi192.pauvih", "id": "smithi192.pauvih"}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:55:56.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: Manager daemon smithi192.pauvih is now available 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:55:56.871 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.pauvih/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:55:56.872 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.pauvih/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:55:56.872 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.pauvih/trash_purge_schedule"}]: dispatch 2024-02-16T22:55:56.872 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:56 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.pauvih/trash_purge_schedule"}]: dispatch 2024-02-16T22:55:57.898 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:57 smithi064 ceph-mon[29226]: Deploying cephadm binary to smithi192 2024-02-16T22:55:57.898 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:57 smithi064 ceph-mon[29226]: Deploying cephadm binary to smithi064 2024-02-16T22:55:57.898 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:57 smithi064 ceph-mon[29226]: mgrmap e22: smithi192.pauvih(active, since 1.01097s) 2024-02-16T22:55:57.898 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:57 smithi064 ceph-mon[29226]: pgmap v3: 65 pgs: 65 active+clean; 181 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:55:57.898 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:57 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:55:57.898 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:57 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:55:57.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:57 smithi192 ceph-mon[36254]: Deploying cephadm binary to smithi192 2024-02-16T22:55:57.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:57 smithi192 ceph-mon[36254]: Deploying cephadm binary to smithi064 2024-02-16T22:55:57.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:57 smithi192 ceph-mon[36254]: mgrmap e22: smithi192.pauvih(active, since 1.01097s) 2024-02-16T22:55:57.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:57 smithi192 ceph-mon[36254]: pgmap v3: 65 pgs: 65 active+clean; 181 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:55:57.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:57 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:55:57.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:57 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:55:59.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:58 smithi064 ceph-mon[29226]: pgmap v4: 65 pgs: 65 active+clean; 181 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:55:59.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:58 smithi064 ceph-mon[29226]: mgrmap e23: smithi192.pauvih(active, since 3s) 2024-02-16T22:55:59.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:58 smithi192 ceph-mon[36254]: pgmap v4: 65 pgs: 65 active+clean; 181 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:55:59.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:58 smithi192 ceph-mon[36254]: mgrmap e23: smithi192.pauvih(active, since 3s) 2024-02-16T22:56:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:59 smithi064 ceph-mon[29226]: [16/Feb/2024:22:55:58] ENGINE Bus STARTING 2024-02-16T22:56:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:59 smithi064 ceph-mon[29226]: [16/Feb/2024:22:55:58] ENGINE Serving on http://172.21.15.192:8765 2024-02-16T22:56:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:59 smithi064 ceph-mon[29226]: [16/Feb/2024:22:55:58] ENGINE Serving on https://172.21.15.192:7150 2024-02-16T22:56:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:55:59 smithi064 ceph-mon[29226]: [16/Feb/2024:22:55:58] ENGINE Bus STARTED 2024-02-16T22:56:00.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:59 smithi192 ceph-mon[36254]: [16/Feb/2024:22:55:58] ENGINE Bus STARTING 2024-02-16T22:56:00.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:59 smithi192 ceph-mon[36254]: [16/Feb/2024:22:55:58] ENGINE Serving on http://172.21.15.192:8765 2024-02-16T22:56:00.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:59 smithi192 ceph-mon[36254]: [16/Feb/2024:22:55:58] ENGINE Serving on https://172.21.15.192:7150 2024-02-16T22:56:00.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:55:59 smithi192 ceph-mon[36254]: [16/Feb/2024:22:55:58] ENGINE Bus STARTED 2024-02-16T22:56:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:00 smithi064 ceph-mon[29226]: pgmap v5: 65 pgs: 65 active+clean; 181 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:56:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:00 smithi064 ceph-mon[29226]: mgrmap e24: smithi192.pauvih(active, since 4s) 2024-02-16T22:56:01.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:00 smithi192 ceph-mon[36254]: pgmap v5: 65 pgs: 65 active+clean; 181 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:56:01.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:00 smithi192 ceph-mon[36254]: mgrmap e24: smithi192.pauvih(active, since 4s) 2024-02-16T22:56:02.646 INFO:tasks.workunit.client.1.smithi192.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-16T22:56:02.650 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T22:56:02.651 INFO:tasks.workunit.client.1.smithi192.stderr:+ make 2024-02-16T22:56:03.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:02 smithi064 ceph-mon[29226]: pgmap v6: 65 pgs: 65 active+clean; 183 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 370 KiB/s wr, 39 op/s 2024-02-16T22:56:03.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:02 smithi192 ceph-mon[36254]: pgmap v6: 65 pgs: 65 active+clean; 183 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 370 KiB/s wr, 39 op/s 2024-02-16T22:56:03.549 INFO:tasks.workunit.client.1.smithi192.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-16T22:56:04.592 INFO:tasks.workunit.client.0.smithi064.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-16T22:56:04.596 INFO:tasks.workunit.client.0.smithi064.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T22:56:04.596 INFO:tasks.workunit.client.0.smithi064.stderr:+ make 2024-02-16T22:56:04.790 INFO:tasks.workunit.client.0.smithi064.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-16T22:56:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:04 smithi192 ceph-mon[36254]: pgmap v7: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 3.5 MiB/s wr, 257 op/s 2024-02-16T22:56:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:04 smithi192 ceph-mon[36254]: Standby manager daemon smithi064.kubmlk started 2024-02-16T22:56:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:04 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.64:0/4287949702' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi064.kubmlk/crt"}]: dispatch 2024-02-16T22:56:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:04 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.64:0/4287949702' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:56:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:04 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.64:0/4287949702' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi064.kubmlk/key"}]: dispatch 2024-02-16T22:56:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:04 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.64:0/4287949702' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:56:05.114 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:04 smithi064 ceph-mon[29226]: pgmap v7: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 3.5 MiB/s wr, 257 op/s 2024-02-16T22:56:05.114 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:04 smithi064 ceph-mon[29226]: Standby manager daemon smithi064.kubmlk started 2024-02-16T22:56:05.114 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:04 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.64:0/4287949702' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi064.kubmlk/crt"}]: dispatch 2024-02-16T22:56:05.115 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:04 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.64:0/4287949702' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:56:05.115 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:04 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.64:0/4287949702' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi064.kubmlk/key"}]: dispatch 2024-02-16T22:56:05.115 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:04 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.64:0/4287949702' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:56:05.512 INFO:tasks.workunit.client.0.smithi064.stderr:++ readlink -f fsstress 2024-02-16T22:56:05.514 INFO:tasks.workunit.client.0.smithi064.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-16T22:56:05.515 INFO:tasks.workunit.client.0.smithi064.stderr:+ popd 2024-02-16T22:56:05.516 INFO:tasks.workunit.client.0.smithi064.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T22:56:05.516 INFO:tasks.workunit.client.0.smithi064.stderr:+ popd 2024-02-16T22:56:05.517 INFO:tasks.workunit.client.0.smithi064.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-16T22:56:05.517 INFO:tasks.workunit.client.0.smithi064.stderr:++ mktemp -d -p . 2024-02-16T22:56:05.520 INFO:tasks.workunit.client.0.smithi064.stderr:+ T=./tmp.5LFff9q55c 2024-02-16T22:56:05.520 INFO:tasks.workunit.client.0.smithi064.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.5LFff9q55c -l 1 -n 1000 -p 10 -v 2024-02-16T22:56:05.525 INFO:tasks.workunit.client.0.smithi064.stdout:seed = 1707601195 2024-02-16T22:56:05.529 INFO:tasks.workunit.client.0.smithi064.stdout:0/0: unlink - no file 2024-02-16T22:56:05.529 INFO:tasks.workunit.client.0.smithi064.stdout:0/1: dread - no filename 2024-02-16T22:56:05.529 INFO:tasks.workunit.client.0.smithi064.stdout:0/2: rename - no filename 2024-02-16T22:56:05.530 INFO:tasks.workunit.client.0.smithi064.stdout:1/0: dwrite - no filename 2024-02-16T22:56:05.530 INFO:tasks.workunit.client.0.smithi064.stdout:1/1: truncate - no filename 2024-02-16T22:56:05.530 INFO:tasks.workunit.client.0.smithi064.stdout:1/2: dread - no filename 2024-02-16T22:56:05.530 INFO:tasks.workunit.client.0.smithi064.stdout:1/3: write - no filename 2024-02-16T22:56:05.530 INFO:tasks.workunit.client.0.smithi064.stdout:1/4: fsync - no filename 2024-02-16T22:56:05.532 INFO:tasks.workunit.client.0.smithi064.stdout:2/0: dwrite - no filename 2024-02-16T22:56:05.532 INFO:tasks.workunit.client.0.smithi064.stdout:2/1: stat - no entries 2024-02-16T22:56:05.532 INFO:tasks.workunit.client.0.smithi064.stdout:0/3: symlink l0 0 2024-02-16T22:56:05.534 INFO:tasks.workunit.client.0.smithi064.stdout:3/0: rename - no filename 2024-02-16T22:56:05.534 INFO:tasks.workunit.client.0.smithi064.stdout:3/1: rmdir - no directory 2024-02-16T22:56:05.535 INFO:tasks.workunit.client.0.smithi064.stdout:1/5: symlink l0 0 2024-02-16T22:56:05.536 INFO:tasks.workunit.client.0.smithi064.stdout:2/2: creat f0 x:0 0 0 2024-02-16T22:56:05.537 INFO:tasks.workunit.client.0.smithi064.stdout:2/3: truncate f0 1034266 0 2024-02-16T22:56:05.538 INFO:tasks.workunit.client.0.smithi064.stdout:4/0: stat - no entries 2024-02-16T22:56:05.539 INFO:tasks.workunit.client.0.smithi064.stdout:2/4: dread f0 [0,4194304] 0 2024-02-16T22:56:05.540 INFO:tasks.workunit.client.0.smithi064.stdout:1/6: rename l0 to l1 0 2024-02-16T22:56:05.540 INFO:tasks.workunit.client.0.smithi064.stdout:1/7: fsync - no filename 2024-02-16T22:56:05.540 INFO:tasks.workunit.client.0.smithi064.stdout:1/8: fdatasync - no filename 2024-02-16T22:56:05.540 INFO:tasks.workunit.client.0.smithi064.stdout:1/9: truncate - no filename 2024-02-16T22:56:05.541 INFO:tasks.workunit.client.0.smithi064.stdout:3/2: symlink l0 0 2024-02-16T22:56:05.541 INFO:tasks.workunit.client.0.smithi064.stdout:3/3: write - no filename 2024-02-16T22:56:05.542 INFO:tasks.workunit.client.0.smithi064.stdout:5/0: rmdir - no directory 2024-02-16T22:56:05.543 INFO:tasks.workunit.client.0.smithi064.stdout:5/1: chown . 6282699 1 2024-02-16T22:56:05.543 INFO:tasks.workunit.client.0.smithi064.stdout:5/2: write - no filename 2024-02-16T22:56:05.543 INFO:tasks.workunit.client.0.smithi064.stdout:5/3: write - no filename 2024-02-16T22:56:05.543 INFO:tasks.workunit.client.0.smithi064.stdout:5/4: rmdir - no directory 2024-02-16T22:56:05.543 INFO:tasks.workunit.client.0.smithi064.stdout:5/5: chown . 288478 1 2024-02-16T22:56:05.543 INFO:tasks.workunit.client.0.smithi064.stdout:5/6: truncate - no filename 2024-02-16T22:56:05.544 INFO:tasks.workunit.client.0.smithi064.stdout:1/10: mkdir d2 0 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:6/0: write - no filename 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:6/1: read - no filename 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:6/2: dread - no filename 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:6/3: truncate - no filename 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:6/4: dwrite - no filename 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:4/1: getdents . 0 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:4/2: dread - no filename 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:4/3: fsync - no filename 2024-02-16T22:56:05.549 INFO:tasks.workunit.client.0.smithi064.stdout:4/4: write - no filename 2024-02-16T22:56:05.553 INFO:tasks.workunit.client.0.smithi064.stdout:7/0: dread - no filename 2024-02-16T22:56:05.554 INFO:tasks.workunit.client.0.smithi064.stdout:7/1: truncate - no filename 2024-02-16T22:56:05.554 INFO:tasks.workunit.client.0.smithi064.stdout:7/2: rename - no filename 2024-02-16T22:56:05.554 INFO:tasks.workunit.client.0.smithi064.stdout:7/3: read - no filename 2024-02-16T22:56:05.554 INFO:tasks.workunit.client.0.smithi064.stdout:7/4: dread - no filename 2024-02-16T22:56:05.555 INFO:tasks.workunit.client.0.smithi064.stdout:5/7: creat f0 x:0 0 0 2024-02-16T22:56:05.556 INFO:tasks.workunit.client.0.smithi064.stdout:1/11: creat d2/f3 x:0 0 0 2024-02-16T22:56:05.557 INFO:tasks.workunit.client.0.smithi064.stdout:3/4: link l0 l1 0 2024-02-16T22:56:05.557 INFO:tasks.workunit.client.0.smithi064.stdout:1/12: truncate d2/f3 60811 0 2024-02-16T22:56:05.559 INFO:tasks.workunit.client.0.smithi064.stdout:6/5: mkdir d0 0 2024-02-16T22:56:05.562 INFO:tasks.workunit.client.0.smithi064.stdout:6/6: rename d0 to d0/d1 22 2024-02-16T22:56:05.562 INFO:tasks.workunit.client.0.smithi064.stdout:4/5: creat f0 x:0 0 0 2024-02-16T22:56:05.562 INFO:tasks.workunit.client.0.smithi064.stdout:6/7: rename d0 to d0/d2 22 2024-02-16T22:56:05.562 INFO:tasks.workunit.client.0.smithi064.stdout:6/8: unlink - no file 2024-02-16T22:56:05.564 INFO:tasks.workunit.client.0.smithi064.stdout:9/0: stat - no entries 2024-02-16T22:56:05.564 INFO:tasks.workunit.client.0.smithi064.stdout:9/1: dwrite - no filename 2024-02-16T22:56:05.566 INFO:tasks.workunit.client.0.smithi064.stdout:7/5: creat f0 x:0 0 0 2024-02-16T22:56:05.567 INFO:tasks.workunit.client.0.smithi064.stdout:4/6: creat f1 x:0 0 0 2024-02-16T22:56:05.568 INFO:tasks.workunit.client.0.smithi064.stdout:3/5: chown l0 2 1 2024-02-16T22:56:05.568 INFO:tasks.workunit.client.0.smithi064.stdout:4/7: dread - f0 zero size 2024-02-16T22:56:05.569 INFO:tasks.workunit.client.0.smithi064.stdout:8/0: symlink l0 0 2024-02-16T22:56:05.572 INFO:tasks.workunit.client.0.smithi064.stdout:6/9: getdents d0 0 2024-02-16T22:56:05.572 INFO:tasks.workunit.client.0.smithi064.stdout:6/10: fsync - no filename 2024-02-16T22:56:05.572 INFO:tasks.workunit.client.0.smithi064.stdout:6/11: dwrite - no filename 2024-02-16T22:56:05.573 INFO:tasks.workunit.client.0.smithi064.stdout:6/12: rename d0 to d0/d3 22 2024-02-16T22:56:05.573 INFO:tasks.workunit.client.0.smithi064.stdout:6/13: dwrite - no filename 2024-02-16T22:56:05.573 INFO:tasks.workunit.client.0.smithi064.stdout:6/14: truncate - no filename 2024-02-16T22:56:05.573 INFO:tasks.workunit.client.0.smithi064.stdout:6/15: dwrite - no filename 2024-02-16T22:56:05.573 INFO:tasks.workunit.client.0.smithi064.stdout:6/16: dwrite - no filename 2024-02-16T22:56:05.573 INFO:tasks.workunit.client.0.smithi064.stdout:6/17: write - no filename 2024-02-16T22:56:05.573 INFO:tasks.workunit.client.0.smithi064.stdout:4/8: mknod c2 0 2024-02-16T22:56:05.573 INFO:tasks.workunit.client.0.smithi064.stdout:6/18: chown d0 379815630 1 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:4/9: chown f1 0 1 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:6/19: chown d0 29 1 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:6/20: write - no filename 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:6/21: dwrite - no filename 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:9/2: creat f0 x:0 0 0 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:6/22: chown d0 13292 1 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:6/23: dread - no filename 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:6/24: dread - no filename 2024-02-16T22:56:05.574 INFO:tasks.workunit.client.0.smithi064.stdout:4/10: read - f1 zero size 2024-02-16T22:56:05.575 INFO:tasks.workunit.client.0.smithi064.stdout:9/3: write f0 [428902,45614] 0 2024-02-16T22:56:05.575 INFO:tasks.workunit.client.0.smithi064.stdout:6/25: rename d0 to d0/d4 22 2024-02-16T22:56:05.575 INFO:tasks.workunit.client.0.smithi064.stdout:6/26: dread - no filename 2024-02-16T22:56:05.575 INFO:tasks.workunit.client.0.smithi064.stdout:6/27: write - no filename 2024-02-16T22:56:05.575 INFO:tasks.workunit.client.0.smithi064.stdout:6/28: write - no filename 2024-02-16T22:56:05.575 INFO:tasks.workunit.client.0.smithi064.stdout:8/1: creat f1 x:0 0 0 2024-02-16T22:56:05.576 INFO:tasks.workunit.client.0.smithi064.stdout:6/29: chown d0 33 1 2024-02-16T22:56:05.576 INFO:tasks.workunit.client.0.smithi064.stdout:6/30: dwrite - no filename 2024-02-16T22:56:05.576 INFO:tasks.workunit.client.0.smithi064.stdout:3/6: chown l0 3934 1 2024-02-16T22:56:05.578 INFO:tasks.workunit.client.0.smithi064.stdout:9/4: rename f0 to f1 0 2024-02-16T22:56:05.580 INFO:tasks.workunit.client.0.smithi064.stdout:8/2: creat f2 x:0 0 0 2024-02-16T22:56:05.581 INFO:tasks.workunit.client.0.smithi064.stdout:8/3: fdatasync f2 0 2024-02-16T22:56:05.583 INFO:tasks.workunit.client.0.smithi064.stdout:4/11: dwrite f0 [0,4194304] 0 2024-02-16T22:56:05.585 INFO:tasks.workunit.client.0.smithi064.stdout:6/31: mkdir d0/d5 0 2024-02-16T22:56:05.585 INFO:tasks.workunit.client.0.smithi064.stdout:6/32: truncate - no filename 2024-02-16T22:56:05.587 INFO:tasks.workunit.client.0.smithi064.stdout:3/7: chown l1 261668086 1 2024-02-16T22:56:05.587 INFO:tasks.workunit.client.0.smithi064.stdout:4/12: truncate f1 172338 0 2024-02-16T22:56:05.590 INFO:tasks.workunit.client.0.smithi064.stdout:6/33: rmdir d0 39 2024-02-16T22:56:05.590 INFO:tasks.workunit.client.0.smithi064.stdout:6/34: write - no filename 2024-02-16T22:56:05.590 INFO:tasks.workunit.client.0.smithi064.stdout:6/35: link - no file 2024-02-16T22:56:05.590 INFO:tasks.workunit.client.0.smithi064.stdout:6/36: link - no file 2024-02-16T22:56:05.592 INFO:tasks.workunit.client.0.smithi064.stdout:4/13: creat f3 x:0 0 0 2024-02-16T22:56:05.593 INFO:tasks.workunit.client.0.smithi064.stdout:3/8: stat l0 0 2024-02-16T22:56:05.596 INFO:tasks.workunit.client.0.smithi064.stdout:4/14: mknod c4 0 2024-02-16T22:56:05.597 INFO:tasks.workunit.client.0.smithi064.stdout:9/5: dwrite f1 [0,4194304] 0 2024-02-16T22:56:05.598 INFO:tasks.workunit.client.0.smithi064.stdout:8/4: dwrite f2 [0,4194304] 0 2024-02-16T22:56:05.600 INFO:tasks.workunit.client.0.smithi064.stdout:9/6: read f1 [3529008,102658] 0 2024-02-16T22:56:05.600 INFO:tasks.workunit.client.0.smithi064.stdout:4/15: dread f1 [0,4194304] 0 2024-02-16T22:56:05.606 INFO:tasks.workunit.client.0.smithi064.stdout:8/5: creat f3 x:0 0 0 2024-02-16T22:56:05.606 INFO:tasks.workunit.client.0.smithi064.stdout:8/6: rmdir - no directory 2024-02-16T22:56:05.607 INFO:tasks.workunit.client.0.smithi064.stdout:3/9: link l1 l2 0 2024-02-16T22:56:05.607 INFO:tasks.workunit.client.0.smithi064.stdout:9/7: dread f1 [0,4194304] 0 2024-02-16T22:56:05.608 INFO:tasks.workunit.client.0.smithi064.stdout:4/16: creat f5 x:0 0 0 2024-02-16T22:56:05.609 INFO:tasks.workunit.client.0.smithi064.stdout:8/7: mknod c4 0 2024-02-16T22:56:05.611 INFO:tasks.workunit.client.0.smithi064.stdout:4/17: symlink l6 0 2024-02-16T22:56:05.619 INFO:tasks.workunit.client.0.smithi064.stdout:9/8: dwrite f1 [0,4194304] 0 2024-02-16T22:56:05.621 INFO:tasks.workunit.client.0.smithi064.stdout:4/18: dwrite f3 [0,4194304] 0 2024-02-16T22:56:05.632 INFO:tasks.workunit.client.0.smithi064.stdout:4/19: dwrite f0 [0,4194304] 0 2024-02-16T22:56:05.645 INFO:tasks.workunit.client.0.smithi064.stdout:2/5: fsync f0 0 2024-02-16T22:56:05.646 INFO:tasks.workunit.client.0.smithi064.stdout:2/6: symlink l1 0 2024-02-16T22:56:05.647 INFO:tasks.workunit.client.0.smithi064.stdout:2/7: dread f0 [0,4194304] 0 2024-02-16T22:56:05.649 INFO:tasks.workunit.client.0.smithi064.stdout:2/8: mknod c2 0 2024-02-16T22:56:05.649 INFO:tasks.workunit.client.0.smithi064.stdout:2/9: readlink l1 0 2024-02-16T22:56:05.649 INFO:tasks.workunit.client.0.smithi064.stdout:2/10: read f0 [89550,101881] 0 2024-02-16T22:56:05.650 INFO:tasks.workunit.client.0.smithi064.stdout:1/13: getdents d2 0 2024-02-16T22:56:05.650 INFO:tasks.workunit.client.0.smithi064.stdout:2/11: truncate f0 1439280 0 2024-02-16T22:56:05.650 INFO:tasks.workunit.client.0.smithi064.stdout:1/14: read d2/f3 [308,97626] 0 2024-02-16T22:56:05.770 INFO:tasks.workunit.client.0.smithi064.stdout:9/9: fdatasync f1 0 2024-02-16T22:56:05.771 INFO:tasks.workunit.client.0.smithi064.stdout:2/12: dwrite f0 [0,4194304] 0 2024-02-16T22:56:05.772 INFO:tasks.workunit.client.0.smithi064.stdout:6/37: chown d0 212663 1 2024-02-16T22:56:05.772 INFO:tasks.workunit.client.0.smithi064.stdout:6/38: dwrite - no filename 2024-02-16T22:56:05.774 INFO:tasks.workunit.client.0.smithi064.stdout:1/15: symlink d2/l4 0 2024-02-16T22:56:05.777 INFO:tasks.workunit.client.0.smithi064.stdout:7/6: fsync f0 0 2024-02-16T22:56:05.777 INFO:tasks.workunit.client.0.smithi064.stdout:2/13: dread f0 [0,4194304] 0 2024-02-16T22:56:05.778 INFO:tasks.workunit.client.0.smithi064.stdout:9/10: mknod c2 0 2024-02-16T22:56:05.781 INFO:tasks.workunit.client.0.smithi064.stdout:6/39: mkdir d0/d6 0 2024-02-16T22:56:05.781 INFO:tasks.workunit.client.0.smithi064.stdout:6/40: dwrite - no filename 2024-02-16T22:56:05.781 INFO:tasks.workunit.client.0.smithi064.stdout:6/41: read - no filename 2024-02-16T22:56:05.782 INFO:tasks.workunit.client.0.smithi064.stdout:6/42: dread - no filename 2024-02-16T22:56:05.782 INFO:tasks.workunit.client.0.smithi064.stdout:6/43: truncate - no filename 2024-02-16T22:56:05.784 INFO:tasks.workunit.client.0.smithi064.stdout:7/7: symlink l1 0 2024-02-16T22:56:05.784 INFO:tasks.workunit.client.0.smithi064.stdout:7/8: dread - f0 zero size 2024-02-16T22:56:05.784 INFO:tasks.workunit.client.0.smithi064.stdout:9/11: fsync f1 0 2024-02-16T22:56:05.785 INFO:tasks.workunit.client.0.smithi064.stdout:7/9: dread - f0 zero size 2024-02-16T22:56:05.785 INFO:tasks.workunit.client.0.smithi064.stdout:7/10: truncate f0 789086 0 2024-02-16T22:56:05.791 INFO:tasks.workunit.client.0.smithi064.stdout:9/12: mkdir d3 0 2024-02-16T22:56:05.791 INFO:tasks.workunit.client.0.smithi064.stdout:7/11: mknod c2 0 2024-02-16T22:56:05.792 INFO:tasks.workunit.client.0.smithi064.stdout:6/44: creat d0/d5/f7 x:0 0 0 2024-02-16T22:56:05.794 INFO:tasks.workunit.client.0.smithi064.stdout:7/12: creat f3 x:0 0 0 2024-02-16T22:56:05.795 INFO:tasks.workunit.client.0.smithi064.stdout:7/13: write f0 [1243089,28795] 0 2024-02-16T22:56:05.795 INFO:tasks.workunit.client.0.smithi064.stdout:9/13: mknod d3/c4 0 2024-02-16T22:56:05.796 INFO:tasks.workunit.client.0.smithi064.stdout:7/14: write f0 [1583841,36963] 0 2024-02-16T22:56:05.798 INFO:tasks.workunit.client.0.smithi064.stdout:7/15: fsync f0 0 2024-02-16T22:56:05.798 INFO:tasks.workunit.client.0.smithi064.stdout:6/45: creat d0/f8 x:0 0 0 2024-02-16T22:56:05.799 INFO:tasks.workunit.client.0.smithi064.stdout:6/46: stat d0/d6 0 2024-02-16T22:56:05.799 INFO:tasks.workunit.client.0.smithi064.stdout:8/8: getdents . 0 2024-02-16T22:56:05.799 INFO:tasks.workunit.client.0.smithi064.stdout:7/16: chown c2 6 1 2024-02-16T22:56:05.799 INFO:tasks.workunit.client.0.smithi064.stdout:8/9: dread - f1 zero size 2024-02-16T22:56:05.799 INFO:tasks.workunit.client.0.smithi064.stdout:6/47: dread - d0/d5/f7 zero size 2024-02-16T22:56:05.800 INFO:tasks.workunit.client.0.smithi064.stdout:6/48: fdatasync d0/d5/f7 0 2024-02-16T22:56:05.803 INFO:tasks.workunit.client.0.smithi064.stdout:4/20: truncate f0 2603777 0 2024-02-16T22:56:05.804 INFO:tasks.workunit.client.0.smithi064.stdout:8/10: mknod c5 0 2024-02-16T22:56:05.805 INFO:tasks.workunit.client.0.smithi064.stdout:8/11: write f3 [123579,101741] 0 2024-02-16T22:56:05.807 INFO:tasks.workunit.client.0.smithi064.stdout:8/12: stat c5 0 2024-02-16T22:56:05.808 INFO:tasks.workunit.client.0.smithi064.stdout:9/14: link d3/c4 d3/c5 0 2024-02-16T22:56:05.808 INFO:tasks.workunit.client.0.smithi064.stdout:8/13: dread - f1 zero size 2024-02-16T22:56:05.809 INFO:tasks.workunit.client.0.smithi064.stdout:6/49: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:56:05.811 INFO:tasks.workunit.client.0.smithi064.stdout:2/14: truncate f0 2208703 0 2024-02-16T22:56:05.814 INFO:tasks.workunit.client.0.smithi064.stdout:9/15: symlink d3/l6 0 2024-02-16T22:56:05.818 INFO:tasks.workunit.client.0.smithi064.stdout:8/14: dwrite f3 [0,4194304] 0 2024-02-16T22:56:05.819 INFO:tasks.workunit.client.0.smithi064.stdout:9/16: creat d3/f7 x:0 0 0 2024-02-16T22:56:05.820 INFO:tasks.workunit.client.0.smithi064.stdout:8/15: truncate f1 615578 0 2024-02-16T22:56:05.820 INFO:tasks.workunit.client.0.smithi064.stdout:7/17: fsync f3 0 2024-02-16T22:56:05.821 INFO:tasks.workunit.client.0.smithi064.stdout:7/18: fdatasync f3 0 2024-02-16T22:56:05.824 INFO:tasks.workunit.client.0.smithi064.stdout:6/50: getdents d0/d5 0 2024-02-16T22:56:05.826 INFO:tasks.workunit.client.0.smithi064.stdout:6/51: fdatasync d0/d5/f7 0 2024-02-16T22:56:05.827 INFO:tasks.workunit.client.0.smithi064.stdout:6/52: chown d0/d5 802769 1 2024-02-16T22:56:05.827 INFO:tasks.workunit.client.0.smithi064.stdout:7/19: rename l1 to l4 0 2024-02-16T22:56:05.829 INFO:tasks.workunit.client.0.smithi064.stdout:7/20: dread - f3 zero size 2024-02-16T22:56:05.829 INFO:tasks.workunit.client.0.smithi064.stdout:4/21: read f0 [1829369,42751] 0 2024-02-16T22:56:05.831 INFO:tasks.workunit.client.0.smithi064.stdout:9/17: dwrite d3/f7 [0,4194304] 0 2024-02-16T22:56:05.832 INFO:tasks.workunit.client.0.smithi064.stdout:7/21: write f3 [254974,72061] 0 2024-02-16T22:56:05.832 INFO:tasks.workunit.client.0.smithi064.stdout:4/22: mknod c7 0 2024-02-16T22:56:05.834 INFO:tasks.workunit.client.0.smithi064.stdout:2/15: write f0 [507911,119699] 0 2024-02-16T22:56:05.836 INFO:tasks.workunit.client.0.smithi064.stdout:6/53: dread d0/f8 [0,4194304] 0 2024-02-16T22:56:05.837 INFO:tasks.workunit.client.0.smithi064.stdout:4/23: symlink l8 0 2024-02-16T22:56:05.838 INFO:tasks.workunit.client.0.smithi064.stdout:6/54: dread - d0/d5/f7 zero size 2024-02-16T22:56:05.838 INFO:tasks.workunit.client.0.smithi064.stdout:6/55: readlink - no filename 2024-02-16T22:56:05.842 INFO:tasks.workunit.client.0.smithi064.stdout:9/18: dread d3/f7 [0,4194304] 0 2024-02-16T22:56:05.856 INFO:tasks.workunit.client.0.smithi064.stdout:6/56: fsync d0/d5/f7 0 2024-02-16T22:56:05.856 INFO:tasks.workunit.client.0.smithi064.stdout:2/16: dread f0 [0,4194304] 0 2024-02-16T22:56:05.856 INFO:tasks.workunit.client.0.smithi064.stdout:8/16: write f3 [4981854,10629] 0 2024-02-16T22:56:05.860 INFO:tasks.workunit.client.0.smithi064.stdout:8/17: chown f2 922016396 1 2024-02-16T22:56:05.862 INFO:tasks.workunit.client.0.smithi064.stdout:8/18: write f3 [5007824,8116] 0 2024-02-16T22:56:05.864 INFO:tasks.workunit.client.0.smithi064.stdout:1/16: dwrite d2/f3 [0,4194304] 0 2024-02-16T22:56:05.868 INFO:tasks.workunit.client.0.smithi064.stdout:1/17: write d2/f3 [688226,38527] 0 2024-02-16T22:56:05.870 INFO:tasks.workunit.client.0.smithi064.stdout:4/24: dwrite f0 [0,4194304] 0 2024-02-16T22:56:05.872 INFO:tasks.workunit.client.0.smithi064.stdout:1/18: chown d2/f3 2 1 2024-02-16T22:56:05.872 INFO:tasks.workunit.client.0.smithi064.stdout:4/25: chown f0 910 1 2024-02-16T22:56:05.874 INFO:tasks.workunit.client.0.smithi064.stdout:6/57: dwrite d0/d5/f7 [0,4194304] 0 2024-02-16T22:56:05.877 INFO:tasks.workunit.client.0.smithi064.stdout:8/19: dread f2 [0,4194304] 0 2024-02-16T22:56:05.878 INFO:tasks.workunit.client.0.smithi064.stdout:8/20: fdatasync f1 0 2024-02-16T22:56:05.878 INFO:tasks.workunit.client.0.smithi064.stdout:8/21: rmdir - no directory 2024-02-16T22:56:05.887 INFO:tasks.workunit.client.0.smithi064.stdout:8/22: dwrite f2 [0,4194304] 0 2024-02-16T22:56:05.887 INFO:tasks.workunit.client.0.smithi064.stdout:8/23: chown f1 13542 1 2024-02-16T22:56:05.891 INFO:tasks.workunit.client.0.smithi064.stdout:8/24: dread f2 [0,4194304] 0 2024-02-16T22:56:05.891 INFO:tasks.workunit.client.0.smithi064.stdout:8/25: fsync f1 0 2024-02-16T22:56:05.895 INFO:tasks.workunit.client.0.smithi064.stdout:8/26: dread f2 [0,4194304] 0 2024-02-16T22:56:05.896 INFO:tasks.workunit.client.0.smithi064.stdout:8/27: write f1 [263887,122325] 0 2024-02-16T22:56:05.973 INFO:tasks.workunit.client.0.smithi064.stdout:9/19: symlink d3/l8 0 2024-02-16T22:56:05.978 INFO:tasks.workunit.client.0.smithi064.stdout:4/26: dread f1 [0,4194304] 0 2024-02-16T22:56:05.979 INFO:tasks.workunit.client.0.smithi064.stdout:6/58: symlink d0/l9 0 2024-02-16T22:56:05.981 INFO:tasks.workunit.client.0.smithi064.stdout:2/17: dwrite f0 [0,4194304] 0 2024-02-16T22:56:05.985 INFO:tasks.workunit.client.0.smithi064.stdout:9/20: unlink d3/f7 0 2024-02-16T22:56:05.986 INFO:tasks.workunit.client.0.smithi064.stdout:2/18: dread f0 [0,4194304] 0 2024-02-16T22:56:05.987 INFO:tasks.workunit.client.0.smithi064.stdout:2/19: truncate f0 4787069 0 2024-02-16T22:56:05.990 INFO:tasks.workunit.client.0.smithi064.stdout:4/27: symlink l9 0 2024-02-16T22:56:05.991 INFO:tasks.workunit.client.0.smithi064.stdout:4/28: readlink l6 0 2024-02-16T22:56:05.991 INFO:tasks.workunit.client.0.smithi064.stdout:2/20: dread f0 [0,4194304] 0 2024-02-16T22:56:05.995 INFO:tasks.workunit.client.0.smithi064.stdout:9/21: mkdir d3/d9 0 2024-02-16T22:56:05.999 INFO:tasks.workunit.client.0.smithi064.stdout:9/22: mknod d3/ca 0 2024-02-16T22:56:06.003 INFO:tasks.workunit.client.0.smithi064.stdout:2/21: getdents . 0 2024-02-16T22:56:06.003 INFO:tasks.workunit.client.0.smithi064.stdout:9/23: dread f1 [0,4194304] 0 2024-02-16T22:56:06.005 INFO:tasks.workunit.client.0.smithi064.stdout:9/24: symlink d3/d9/lb 0 2024-02-16T22:56:06.006 INFO:tasks.workunit.client.0.smithi064.stdout:9/25: write f1 [958123,67281] 0 2024-02-16T22:56:06.008 INFO:tasks.workunit.client.0.smithi064.stdout:9/26: read f1 [3689398,8792] 0 2024-02-16T22:56:06.009 INFO:tasks.workunit.client.0.smithi064.stdout:2/22: dwrite f0 [0,4194304] 0 2024-02-16T22:56:06.011 INFO:tasks.workunit.client.0.smithi064.stdout:9/27: creat d3/fc x:0 0 0 2024-02-16T22:56:06.012 INFO:tasks.workunit.client.0.smithi064.stdout:2/23: symlink l3 0 2024-02-16T22:56:06.017 INFO:tasks.workunit.client.0.smithi064.stdout:2/24: dread f0 [0,4194304] 0 2024-02-16T22:56:06.019 INFO:tasks.workunit.client.0.smithi064.stdout:2/25: symlink l4 0 2024-02-16T22:56:06.067 INFO:tasks.workunit.client.0.smithi064.stdout:1/19: truncate d2/f3 3336439 0 2024-02-16T22:56:06.068 INFO:tasks.workunit.client.0.smithi064.stdout:1/20: chown d2/l4 6542978 1 2024-02-16T22:56:06.068 INFO:tasks.workunit.client.0.smithi064.stdout:8/28: truncate f3 4735257 0 2024-02-16T22:56:06.070 INFO:tasks.workunit.client.0.smithi064.stdout:6/59: dwrite d0/d5/f7 [4194304,4194304] 0 2024-02-16T22:56:06.075 INFO:tasks.workunit.client.0.smithi064.stdout:9/28: getdents d3/d9 0 2024-02-16T22:56:06.076 INFO:tasks.workunit.client.0.smithi064.stdout:6/60: mkdir d0/d6/da 0 2024-02-16T22:56:06.078 INFO:tasks.workunit.client.0.smithi064.stdout:9/29: mknod d3/d9/cd 0 2024-02-16T22:56:06.080 INFO:tasks.workunit.client.0.smithi064.stdout:4/29: dwrite f1 [0,4194304] 0 2024-02-16T22:56:06.080 INFO:tasks.workunit.client.0.smithi064.stdout:9/30: write d3/fc [358029,113697] 0 2024-02-16T22:56:06.081 INFO:tasks.workunit.client.0.smithi064.stdout:6/61: symlink d0/d5/lb 0 2024-02-16T22:56:06.082 INFO:tasks.workunit.client.0.smithi064.stdout:6/62: chown d0/l9 215588338 1 2024-02-16T22:56:06.088 INFO:tasks.workunit.client.0.smithi064.stdout:8/29: write f2 [4072687,59763] 0 2024-02-16T22:56:06.091 INFO:tasks.workunit.client.0.smithi064.stdout:6/63: dread d0/d5/f7 [0,4194304] 0 2024-02-16T22:56:06.095 INFO:tasks.workunit.client.0.smithi064.stdout:4/30: dwrite f5 [0,4194304] 0 2024-02-16T22:56:06.095 INFO:tasks.workunit.client.0.smithi064.stdout:8/30: link c5 c6 0 2024-02-16T22:56:06.095 INFO:tasks.workunit.client.0.smithi064.stdout:6/64: creat d0/d6/fc x:0 0 0 2024-02-16T22:56:06.096 INFO:tasks.workunit.client.0.smithi064.stdout:4/31: mknod ca 0 2024-02-16T22:56:06.099 INFO:tasks.workunit.client.0.smithi064.stdout:6/65: unlink d0/d6/fc 0 2024-02-16T22:56:06.102 INFO:tasks.workunit.client.0.smithi064.stdout:1/21: write d2/f3 [3458210,96394] 0 2024-02-16T22:56:06.102 INFO:tasks.workunit.client.1.smithi192.stderr:++ readlink -f fsstress 2024-02-16T22:56:06.104 INFO:tasks.workunit.client.0.smithi064.stdout:6/66: mknod d0/d6/cd 0 2024-02-16T22:56:06.105 INFO:tasks.workunit.client.0.smithi064.stdout:4/32: dwrite f5 [0,4194304] 0 2024-02-16T22:56:06.105 INFO:tasks.workunit.client.1.smithi192.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-16T22:56:06.105 INFO:tasks.workunit.client.1.smithi192.stderr:+ popd 2024-02-16T22:56:06.106 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T22:56:06.106 INFO:tasks.workunit.client.1.smithi192.stderr:+ popd 2024-02-16T22:56:06.106 INFO:tasks.workunit.client.0.smithi064.stdout:9/31: truncate f1 3972640 0 2024-02-16T22:56:06.107 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-16T22:56:06.107 INFO:tasks.workunit.client.1.smithi192.stderr:++ mktemp -d -p . 2024-02-16T22:56:06.111 INFO:tasks.workunit.client.0.smithi064.stdout:6/67: write d0/d5/f7 [7294431,59822] 0 2024-02-16T22:56:06.112 INFO:tasks.workunit.client.1.smithi192.stderr:+ T=./tmp.elFSIvtpwP 2024-02-16T22:56:06.112 INFO:tasks.workunit.client.1.smithi192.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.elFSIvtpwP -l 1 -n 1000 -p 10 -v 2024-02-16T22:56:06.113 INFO:tasks.workunit.client.0.smithi064.stdout:4/33: dwrite f1 [0,4194304] 0 2024-02-16T22:56:06.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:05 smithi064 ceph-mon[29226]: mgrmap e25: smithi192.pauvih(active, since 9s), standbys: smithi064.kubmlk 2024-02-16T22:56:06.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:05 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:56:06.119 INFO:tasks.workunit.client.0.smithi064.stdout:4/34: creat fb x:0 0 0 2024-02-16T22:56:06.119 INFO:tasks.workunit.client.0.smithi064.stdout:4/35: rmdir - no directory 2024-02-16T22:56:06.119 INFO:tasks.workunit.client.0.smithi064.stdout:4/36: readlink l6 0 2024-02-16T22:56:06.120 INFO:tasks.workunit.client.0.smithi064.stdout:1/22: rename d2/l4 to d2/l5 0 2024-02-16T22:56:06.121 INFO:tasks.workunit.client.0.smithi064.stdout:1/23: chown d2/f3 41 1 2024-02-16T22:56:06.122 INFO:tasks.workunit.client.1.smithi192.stdout:seed = 1708001811 2024-02-16T22:56:06.123 INFO:tasks.workunit.client.0.smithi064.stdout:8/31: chown f3 36009741 1 2024-02-16T22:56:06.123 INFO:tasks.workunit.client.0.smithi064.stdout:6/68: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:56:06.124 INFO:tasks.workunit.client.0.smithi064.stdout:6/69: fsync d0/d5/f7 0 2024-02-16T22:56:06.125 INFO:tasks.workunit.client.0.smithi064.stdout:4/37: mkdir dc 0 2024-02-16T22:56:06.125 INFO:tasks.workunit.client.0.smithi064.stdout:4/38: chown c4 6147 1 2024-02-16T22:56:06.126 INFO:tasks.workunit.client.0.smithi064.stdout:9/32: getdents d3/d9 0 2024-02-16T22:56:06.139 INFO:tasks.workunit.client.0.smithi064.stdout:9/33: read d3/fc [261159,42249] 0 2024-02-16T22:56:06.142 INFO:tasks.workunit.client.0.smithi064.stdout:8/32: dwrite f1 [0,4194304] 0 2024-02-16T22:56:06.144 INFO:tasks.workunit.client.0.smithi064.stdout:8/33: write f2 [4037133,95097] 0 2024-02-16T22:56:06.144 INFO:tasks.workunit.client.1.smithi192.stdout:0/0: chown . 1 1 2024-02-16T22:56:06.144 INFO:tasks.workunit.client.1.smithi192.stdout:0/1: dwrite - no filename 2024-02-16T22:56:06.144 INFO:tasks.workunit.client.1.smithi192.stdout:0/2: rename - no filename 2024-02-16T22:56:06.144 INFO:tasks.workunit.client.1.smithi192.stdout:0/3: rename - no filename 2024-02-16T22:56:06.149 INFO:tasks.workunit.client.1.smithi192.stdout:1/0: dread - no filename 2024-02-16T22:56:06.149 INFO:tasks.workunit.client.1.smithi192.stdout:1/1: chown . 2 1 2024-02-16T22:56:06.149 INFO:tasks.workunit.client.1.smithi192.stdout:1/2: rmdir - no directory 2024-02-16T22:56:06.151 INFO:tasks.workunit.client.1.smithi192.stdout:2/0: read - no filename 2024-02-16T22:56:06.151 INFO:tasks.workunit.client.1.smithi192.stdout:2/1: chown . 172 1 2024-02-16T22:56:06.151 INFO:tasks.workunit.client.1.smithi192.stdout:2/2: read - no filename 2024-02-16T22:56:06.151 INFO:tasks.workunit.client.1.smithi192.stdout:2/3: dwrite - no filename 2024-02-16T22:56:06.151 INFO:tasks.workunit.client.1.smithi192.stdout:2/4: dwrite - no filename 2024-02-16T22:56:06.152 INFO:tasks.workunit.client.0.smithi064.stdout:1/24: mknod d2/c6 0 2024-02-16T22:56:06.152 INFO:tasks.workunit.client.1.smithi192.stdout:0/4: getdents . 0 2024-02-16T22:56:06.154 INFO:tasks.workunit.client.1.smithi192.stdout:3/0: dwrite - no filename 2024-02-16T22:56:06.154 INFO:tasks.workunit.client.1.smithi192.stdout:3/1: fsync - no filename 2024-02-16T22:56:06.154 INFO:tasks.workunit.client.1.smithi192.stdout:3/2: dwrite - no filename 2024-02-16T22:56:06.154 INFO:tasks.workunit.client.1.smithi192.stdout:3/3: rename - no filename 2024-02-16T22:56:06.154 INFO:tasks.workunit.client.0.smithi064.stdout:6/70: fdatasync d0/f8 0 2024-02-16T22:56:06.154 INFO:tasks.workunit.client.0.smithi064.stdout:8/34: fsync f3 0 2024-02-16T22:56:06.155 INFO:tasks.workunit.client.1.smithi192.stdout:1/3: getdents . 0 2024-02-16T22:56:06.157 INFO:tasks.workunit.client.0.smithi064.stdout:6/71: dread d0/d5/f7 [0,4194304] 0 2024-02-16T22:56:06.158 INFO:tasks.workunit.client.0.smithi064.stdout:4/39: mkdir dc/dd 0 2024-02-16T22:56:06.159 INFO:tasks.workunit.client.0.smithi064.stdout:4/40: chown c7 1 1 2024-02-16T22:56:06.159 INFO:tasks.workunit.client.0.smithi064.stdout:4/41: chown l8 456 1 2024-02-16T22:56:06.159 INFO:tasks.workunit.client.0.smithi064.stdout:4/42: stat f3 0 2024-02-16T22:56:06.160 INFO:tasks.workunit.client.1.smithi192.stdout:2/5: symlink l0 0 2024-02-16T22:56:06.160 INFO:tasks.workunit.client.1.smithi192.stdout:2/6: dwrite - no filename 2024-02-16T22:56:06.160 INFO:tasks.workunit.client.1.smithi192.stdout:2/7: rmdir - no directory 2024-02-16T22:56:06.160 INFO:tasks.workunit.client.0.smithi064.stdout:1/25: stat l1 0 2024-02-16T22:56:06.165 INFO:tasks.workunit.client.0.smithi064.stdout:8/35: symlink l7 0 2024-02-16T22:56:06.165 INFO:tasks.workunit.client.0.smithi064.stdout:8/36: read f1 [1884541,16229] 0 2024-02-16T22:56:06.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/4: mkdir d0 0 2024-02-16T22:56:06.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/5: chown d0 0 1 2024-02-16T22:56:06.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/6: dwrite - no filename 2024-02-16T22:56:06.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/7: read - no filename 2024-02-16T22:56:06.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/8: write - no filename 2024-02-16T22:56:06.169 INFO:tasks.workunit.client.1.smithi192.stdout:2/8: creat f1 x:0 0 0 2024-02-16T22:56:06.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/4: symlink l0 0 2024-02-16T22:56:06.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/5: write - no filename 2024-02-16T22:56:06.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/6: write - no filename 2024-02-16T22:56:06.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/7: read - no filename 2024-02-16T22:56:06.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/8: read - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/0: read - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/1: truncate - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/2: rename - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/3: dwrite - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/4: write - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/5: write - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/6: unlink - no file 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/7: chown . 1825 1 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/8: write - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/9: readlink - no filename 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/10: rmdir - no directory 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/11: chown . 579788 1 2024-02-16T22:56:06.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/12: chown . 779821 1 2024-02-16T22:56:06.174 INFO:tasks.workunit.client.0.smithi064.stdout:8/37: creat f8 x:0 0 0 2024-02-16T22:56:06.175 INFO:tasks.workunit.client.0.smithi064.stdout:6/72: link d0/d5/f7 d0/d6/da/fe 0 2024-02-16T22:56:06.178 INFO:tasks.workunit.client.0.smithi064.stdout:6/73: write d0/f8 [3290915,125176] 0 2024-02-16T22:56:06.179 INFO:tasks.workunit.client.1.smithi192.stdout:4/0: creat f0 x:0 0 0 2024-02-16T22:56:06.180 INFO:tasks.workunit.client.0.smithi064.stdout:1/26: symlink d2/l7 0 2024-02-16T22:56:06.180 INFO:tasks.workunit.client.0.smithi064.stdout:8/38: dwrite f8 [0,4194304] 0 2024-02-16T22:56:06.182 INFO:tasks.workunit.client.0.smithi064.stdout:9/34: dread f1 [0,4194304] 0 2024-02-16T22:56:06.182 INFO:tasks.workunit.client.0.smithi064.stdout:9/35: readlink d3/l8 0 2024-02-16T22:56:06.184 INFO:tasks.workunit.client.1.smithi192.stdout:1/9: mknod c1 0 2024-02-16T22:56:06.187 INFO:tasks.workunit.client.1.smithi192.stdout:7/0: unlink - no file 2024-02-16T22:56:06.187 INFO:tasks.workunit.client.1.smithi192.stdout:7/1: dread - no filename 2024-02-16T22:56:06.189 INFO:tasks.workunit.client.0.smithi064.stdout:6/74: mkdir d0/d5/df 0 2024-02-16T22:56:06.190 INFO:tasks.workunit.client.0.smithi064.stdout:6/75: read d0/d5/f7 [1224640,39836] 0 2024-02-16T22:56:06.190 INFO:tasks.workunit.client.1.smithi192.stdout:5/0: symlink l0 0 2024-02-16T22:56:06.190 INFO:tasks.workunit.client.1.smithi192.stdout:5/1: dread - no filename 2024-02-16T22:56:06.190 INFO:tasks.workunit.client.1.smithi192.stdout:5/2: read - no filename 2024-02-16T22:56:06.190 INFO:tasks.workunit.client.1.smithi192.stdout:5/3: dwrite - no filename 2024-02-16T22:56:06.191 INFO:tasks.workunit.client.0.smithi064.stdout:8/39: creat f9 x:0 0 0 2024-02-16T22:56:06.191 INFO:tasks.workunit.client.0.smithi064.stdout:8/40: stat f8 0 2024-02-16T22:56:06.191 INFO:tasks.workunit.client.0.smithi064.stdout:9/36: mknod d3/ce 0 2024-02-16T22:56:06.193 INFO:tasks.workunit.client.1.smithi192.stdout:4/1: mkdir d1 0 2024-02-16T22:56:06.194 INFO:tasks.workunit.client.0.smithi064.stdout:1/27: mknod d2/c8 0 2024-02-16T22:56:06.194 INFO:tasks.workunit.client.0.smithi064.stdout:1/28: readlink l1 0 2024-02-16T22:56:06.195 INFO:tasks.workunit.client.1.smithi192.stdout:2/9: link f1 f2 0 2024-02-16T22:56:06.198 INFO:tasks.workunit.client.1.smithi192.stdout:8/0: rename - no filename 2024-02-16T22:56:06.198 INFO:tasks.workunit.client.0.smithi064.stdout:6/76: fdatasync d0/d6/da/fe 0 2024-02-16T22:56:06.199 INFO:tasks.workunit.client.0.smithi064.stdout:6/77: write d0/f8 [3503045,103563] 0 2024-02-16T22:56:06.199 INFO:tasks.workunit.client.0.smithi064.stdout:6/78: chown d0/d5/lb 5 1 2024-02-16T22:56:06.210 INFO:tasks.workunit.client.0.smithi064.stdout:6/79: mkdir d0/d6/d10 0 2024-02-16T22:56:06.212 INFO:tasks.workunit.client.0.smithi064.stdout:6/80: unlink d0/d6/da/fe 0 2024-02-16T22:56:06.213 INFO:tasks.workunit.client.0.smithi064.stdout:1/29: creat d2/f9 x:0 0 0 2024-02-16T22:56:06.214 INFO:tasks.workunit.client.0.smithi064.stdout:1/30: read d2/f3 [1168603,6184] 0 2024-02-16T22:56:06.216 INFO:tasks.workunit.client.0.smithi064.stdout:1/31: mkdir d2/da 0 2024-02-16T22:56:06.217 INFO:tasks.workunit.client.0.smithi064.stdout:1/32: symlink d2/lb 0 2024-02-16T22:56:06.218 INFO:tasks.workunit.client.0.smithi064.stdout:1/33: write d2/f9 [422982,53149] 0 2024-02-16T22:56:06.221 INFO:tasks.workunit.client.0.smithi064.stdout:1/34: rmdir d2/da 0 2024-02-16T22:56:06.222 INFO:tasks.workunit.client.0.smithi064.stdout:1/35: unlink d2/c8 0 2024-02-16T22:56:06.223 INFO:tasks.workunit.client.0.smithi064.stdout:1/36: truncate d2/f9 982242 0 2024-02-16T22:56:06.225 INFO:tasks.workunit.client.0.smithi064.stdout:1/37: symlink d2/lc 0 2024-02-16T22:56:06.231 INFO:tasks.workunit.client.0.smithi064.stdout:1/38: dwrite d2/f3 [0,4194304] 0 2024-02-16T22:56:06.232 INFO:tasks.workunit.client.0.smithi064.stdout:1/39: chown d2 0 1 2024-02-16T22:56:06.233 INFO:tasks.workunit.client.0.smithi064.stdout:1/40: creat d2/fd x:0 0 0 2024-02-16T22:56:06.234 INFO:tasks.workunit.client.0.smithi064.stdout:1/41: chown d2/c6 14 1 2024-02-16T22:56:06.235 INFO:tasks.workunit.client.0.smithi064.stdout:1/42: mknod d2/ce 0 2024-02-16T22:56:06.237 INFO:tasks.workunit.client.0.smithi064.stdout:1/43: mkdir d2/df 0 2024-02-16T22:56:06.237 INFO:tasks.workunit.client.0.smithi064.stdout:1/44: readlink l1 0 2024-02-16T22:56:06.238 INFO:tasks.workunit.client.0.smithi064.stdout:1/45: symlink d2/l10 0 2024-02-16T22:56:06.244 INFO:tasks.workunit.client.0.smithi064.stdout:1/46: dwrite d2/fd [0,4194304] 0 2024-02-16T22:56:06.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:05 smithi192 ceph-mon[36254]: mgrmap e25: smithi192.pauvih(active, since 9s), standbys: smithi064.kubmlk 2024-02-16T22:56:06.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:05 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:56:06.380 INFO:tasks.workunit.client.1.smithi192.stdout:6/13: mknod c0 0 2024-02-16T22:56:06.380 INFO:tasks.workunit.client.1.smithi192.stdout:6/14: dwrite - no filename 2024-02-16T22:56:06.380 INFO:tasks.workunit.client.1.smithi192.stdout:4/2: dwrite f0 [0,4194304] 0 2024-02-16T22:56:06.380 INFO:tasks.workunit.client.1.smithi192.stdout:3/9: rmdir d0 0 2024-02-16T22:56:06.380 INFO:tasks.workunit.client.1.smithi192.stdout:2/10: rename l0 to l3 0 2024-02-16T22:56:06.381 INFO:tasks.workunit.client.1.smithi192.stdout:7/2: mknod c0 0 2024-02-16T22:56:06.381 INFO:tasks.workunit.client.1.smithi192.stdout:7/3: dread - no filename 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/0: dwrite - no filename 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/1: rmdir - no directory 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/2: read - no filename 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/3: write - no filename 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/4: stat - no entries 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/5: dread - no filename 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/6: chown . 16920450 1 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:7/4: chown c0 1844350 1 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:7/5: dwrite - no filename 2024-02-16T22:56:06.382 INFO:tasks.workunit.client.1.smithi192.stdout:7/6: dread - no filename 2024-02-16T22:56:06.394 INFO:tasks.workunit.client.0.smithi064.stdout:8/41: truncate f8 2473691 0 2024-02-16T22:56:06.394 INFO:tasks.workunit.client.0.smithi064.stdout:8/42: rmdir - no directory 2024-02-16T22:56:06.395 INFO:tasks.workunit.client.0.smithi064.stdout:9/37: dwrite d3/fc [0,4194304] 0 2024-02-16T22:56:06.395 INFO:tasks.workunit.client.0.smithi064.stdout:9/38: chown d3 80707 1 2024-02-16T22:56:06.398 INFO:tasks.workunit.client.0.smithi064.stdout:8/43: creat fa x:0 0 0 2024-02-16T22:56:06.399 INFO:tasks.workunit.client.0.smithi064.stdout:8/44: chown f9 173 1 2024-02-16T22:56:06.400 INFO:tasks.workunit.client.0.smithi064.stdout:9/39: creat d3/d9/ff x:0 0 0 2024-02-16T22:56:06.440 INFO:tasks.workunit.client.0.smithi064.stdout:8/45: write f3 [503700,11312] 0 2024-02-16T22:56:06.475 INFO:tasks.workunit.client.0.smithi064.stdout:8/46: fsync f8 0 2024-02-16T22:56:06.478 INFO:tasks.workunit.client.0.smithi064.stdout:8/47: chown c5 0 1 2024-02-16T22:56:06.484 INFO:tasks.workunit.client.0.smithi064.stdout:8/48: dwrite f9 [0,4194304] 0 2024-02-16T22:56:06.484 INFO:tasks.workunit.client.0.smithi064.stdout:8/49: rmdir - no directory 2024-02-16T22:56:06.513 INFO:tasks.workunit.client.0.smithi064.stdout:8/50: stat f8 0 2024-02-16T22:56:06.516 INFO:tasks.workunit.client.0.smithi064.stdout:8/51: creat fb x:0 0 0 2024-02-16T22:56:06.518 INFO:tasks.workunit.client.0.smithi064.stdout:8/52: mkdir dc 0 2024-02-16T22:56:06.555 INFO:tasks.workunit.client.1.smithi192.stdout:7/7: mkdir d1 0 2024-02-16T22:56:06.556 INFO:tasks.workunit.client.1.smithi192.stdout:7/8: stat d1 0 2024-02-16T22:56:06.556 INFO:tasks.workunit.client.1.smithi192.stdout:4/3: mknod d1/c2 0 2024-02-16T22:56:06.556 INFO:tasks.workunit.client.1.smithi192.stdout:4/4: fsync f0 0 2024-02-16T22:56:06.557 INFO:tasks.workunit.client.1.smithi192.stdout:4/5: stat f0 0 2024-02-16T22:56:06.557 INFO:tasks.workunit.client.1.smithi192.stdout:9/7: creat f0 x:0 0 0 2024-02-16T22:56:06.557 INFO:tasks.workunit.client.1.smithi192.stdout:9/8: fdatasync f0 0 2024-02-16T22:56:06.557 INFO:tasks.workunit.client.1.smithi192.stdout:5/4: getdents . 0 2024-02-16T22:56:06.560 INFO:tasks.workunit.client.1.smithi192.stdout:4/6: creat d1/f3 x:0 0 0 2024-02-16T22:56:06.561 INFO:tasks.workunit.client.1.smithi192.stdout:6/15: getdents . 0 2024-02-16T22:56:06.561 INFO:tasks.workunit.client.1.smithi192.stdout:4/7: fsync f0 0 2024-02-16T22:56:06.562 INFO:tasks.workunit.client.1.smithi192.stdout:7/9: mkdir d1/d2 0 2024-02-16T22:56:06.564 INFO:tasks.workunit.client.1.smithi192.stdout:6/16: creat f1 x:0 0 0 2024-02-16T22:56:06.566 INFO:tasks.workunit.client.1.smithi192.stdout:4/8: creat d1/f4 x:0 0 0 2024-02-16T22:56:06.566 INFO:tasks.workunit.client.1.smithi192.stdout:4/9: readlink - no filename 2024-02-16T22:56:06.566 INFO:tasks.workunit.client.1.smithi192.stdout:5/5: link l0 l1 0 2024-02-16T22:56:06.566 INFO:tasks.workunit.client.1.smithi192.stdout:5/6: dread - no filename 2024-02-16T22:56:06.566 INFO:tasks.workunit.client.1.smithi192.stdout:9/9: dwrite f0 [0,4194304] 0 2024-02-16T22:56:06.567 INFO:tasks.workunit.client.1.smithi192.stdout:7/10: symlink d1/d2/l3 0 2024-02-16T22:56:06.567 INFO:tasks.workunit.client.1.smithi192.stdout:7/11: write - no filename 2024-02-16T22:56:06.567 INFO:tasks.workunit.client.1.smithi192.stdout:9/10: stat f0 0 2024-02-16T22:56:06.567 INFO:tasks.workunit.client.1.smithi192.stdout:9/11: rmdir - no directory 2024-02-16T22:56:06.569 INFO:tasks.workunit.client.1.smithi192.stdout:5/7: creat f2 x:0 0 0 2024-02-16T22:56:06.570 INFO:tasks.workunit.client.1.smithi192.stdout:4/10: creat d1/f5 x:0 0 0 2024-02-16T22:56:06.571 INFO:tasks.workunit.client.1.smithi192.stdout:7/12: creat d1/d2/f4 x:0 0 0 2024-02-16T22:56:06.575 INFO:tasks.workunit.client.1.smithi192.stdout:5/8: rename l1 to l3 0 2024-02-16T22:56:06.577 INFO:tasks.workunit.client.1.smithi192.stdout:7/13: symlink d1/d2/l5 0 2024-02-16T22:56:06.577 INFO:tasks.workunit.client.1.smithi192.stdout:5/9: read - f2 zero size 2024-02-16T22:56:06.579 INFO:tasks.workunit.client.1.smithi192.stdout:6/17: dwrite f1 [0,4194304] 0 2024-02-16T22:56:06.580 INFO:tasks.workunit.client.1.smithi192.stdout:7/14: read - d1/d2/f4 zero size 2024-02-16T22:56:06.580 INFO:tasks.workunit.client.1.smithi192.stdout:6/18: chown c0 215692 1 2024-02-16T22:56:06.581 INFO:tasks.workunit.client.1.smithi192.stdout:9/12: dwrite f0 [0,4194304] 0 2024-02-16T22:56:06.584 INFO:tasks.workunit.client.1.smithi192.stdout:6/19: creat f2 x:0 0 0 2024-02-16T22:56:06.588 INFO:tasks.workunit.client.1.smithi192.stdout:6/20: mknod c3 0 2024-02-16T22:56:06.590 INFO:tasks.workunit.client.1.smithi192.stdout:9/13: dread f0 [0,4194304] 0 2024-02-16T22:56:06.592 INFO:tasks.workunit.client.1.smithi192.stdout:7/15: dwrite d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:06.592 INFO:tasks.workunit.client.1.smithi192.stdout:7/16: write d1/d2/f4 [367870,35150] 0 2024-02-16T22:56:06.593 INFO:tasks.workunit.client.1.smithi192.stdout:9/14: creat f1 x:0 0 0 2024-02-16T22:56:06.596 INFO:tasks.workunit.client.1.smithi192.stdout:9/15: creat f2 x:0 0 0 2024-02-16T22:56:06.616 INFO:tasks.workunit.client.1.smithi192.stdout:2/11: write f2 [484889,61983] 0 2024-02-16T22:56:06.616 INFO:tasks.workunit.client.1.smithi192.stdout:9/16: symlink l3 0 2024-02-16T22:56:06.616 INFO:tasks.workunit.client.1.smithi192.stdout:9/17: dread - f2 zero size 2024-02-16T22:56:06.618 INFO:tasks.workunit.client.1.smithi192.stdout:4/11: getdents d1 0 2024-02-16T22:56:06.619 INFO:tasks.workunit.client.1.smithi192.stdout:6/21: fdatasync f1 0 2024-02-16T22:56:06.619 INFO:tasks.workunit.client.1.smithi192.stdout:9/18: fsync f0 0 2024-02-16T22:56:06.626 INFO:tasks.workunit.client.1.smithi192.stdout:4/12: truncate d1/f4 671929 0 2024-02-16T22:56:06.627 INFO:tasks.workunit.client.1.smithi192.stdout:2/12: getdents . 0 2024-02-16T22:56:06.627 INFO:tasks.workunit.client.1.smithi192.stdout:5/10: readlink l3 0 2024-02-16T22:56:06.627 INFO:tasks.workunit.client.1.smithi192.stdout:2/13: readlink l3 0 2024-02-16T22:56:06.628 INFO:tasks.workunit.client.1.smithi192.stdout:5/11: fsync f2 0 2024-02-16T22:56:06.628 INFO:tasks.workunit.client.1.smithi192.stdout:7/17: truncate d1/d2/f4 1344666 0 2024-02-16T22:56:06.629 INFO:tasks.workunit.client.1.smithi192.stdout:9/19: mknod c4 0 2024-02-16T22:56:06.630 INFO:tasks.workunit.client.1.smithi192.stdout:5/12: write f2 [155497,101406] 0 2024-02-16T22:56:06.631 INFO:tasks.workunit.client.1.smithi192.stdout:7/18: stat d1/d2/l3 0 2024-02-16T22:56:06.631 INFO:tasks.workunit.client.1.smithi192.stdout:4/13: getdents d1 0 2024-02-16T22:56:06.631 INFO:tasks.workunit.client.1.smithi192.stdout:9/20: mknod c5 0 2024-02-16T22:56:06.632 INFO:tasks.workunit.client.1.smithi192.stdout:2/14: dread f1 [0,4194304] 0 2024-02-16T22:56:06.634 INFO:tasks.workunit.client.1.smithi192.stdout:5/13: dread f2 [0,4194304] 0 2024-02-16T22:56:06.635 INFO:tasks.workunit.client.1.smithi192.stdout:2/15: creat f4 x:0 0 0 2024-02-16T22:56:06.636 INFO:tasks.workunit.client.1.smithi192.stdout:9/21: write f2 [455973,59686] 0 2024-02-16T22:56:06.636 INFO:tasks.workunit.client.1.smithi192.stdout:9/22: rmdir - no directory 2024-02-16T22:56:06.637 INFO:tasks.workunit.client.1.smithi192.stdout:7/19: truncate d1/d2/f4 1471888 0 2024-02-16T22:56:06.637 INFO:tasks.workunit.client.1.smithi192.stdout:6/22: dwrite f2 [0,4194304] 0 2024-02-16T22:56:06.641 INFO:tasks.workunit.client.1.smithi192.stdout:7/20: creat d1/d2/f6 x:0 0 0 2024-02-16T22:56:06.642 INFO:tasks.workunit.client.1.smithi192.stdout:6/23: write f2 [671495,14605] 0 2024-02-16T22:56:06.646 INFO:tasks.workunit.client.1.smithi192.stdout:7/21: chown d1/d2/f4 743 1 2024-02-16T22:56:06.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/24: chown f1 12991701 1 2024-02-16T22:56:06.646 INFO:tasks.workunit.client.1.smithi192.stdout:4/14: dread f0 [0,4194304] 0 2024-02-16T22:56:06.650 INFO:tasks.workunit.client.1.smithi192.stdout:7/22: symlink d1/l7 0 2024-02-16T22:56:06.650 INFO:tasks.workunit.client.1.smithi192.stdout:6/25: chown c3 2872270 1 2024-02-16T22:56:06.654 INFO:tasks.workunit.client.1.smithi192.stdout:6/26: chown c0 609 1 2024-02-16T22:56:06.655 INFO:tasks.workunit.client.1.smithi192.stdout:9/23: dwrite f2 [0,4194304] 0 2024-02-16T22:56:06.658 INFO:tasks.workunit.client.1.smithi192.stdout:2/16: dwrite f4 [0,4194304] 0 2024-02-16T22:56:06.658 INFO:tasks.workunit.client.1.smithi192.stdout:5/14: dwrite f2 [0,4194304] 0 2024-02-16T22:56:06.661 INFO:tasks.workunit.client.1.smithi192.stdout:2/17: readlink l3 0 2024-02-16T22:56:06.661 INFO:tasks.workunit.client.1.smithi192.stdout:9/24: mknod c6 0 2024-02-16T22:56:06.666 INFO:tasks.workunit.client.1.smithi192.stdout:2/18: creat f5 x:0 0 0 2024-02-16T22:56:06.666 INFO:tasks.workunit.client.1.smithi192.stdout:9/25: readlink l3 0 2024-02-16T22:56:06.666 INFO:tasks.workunit.client.1.smithi192.stdout:9/26: rmdir - no directory 2024-02-16T22:56:06.670 INFO:tasks.workunit.client.1.smithi192.stdout:2/19: mknod c6 0 2024-02-16T22:56:06.671 INFO:tasks.workunit.client.1.smithi192.stdout:9/27: mkdir d7 0 2024-02-16T22:56:06.674 INFO:tasks.workunit.client.1.smithi192.stdout:4/15: dwrite d1/f5 [0,4194304] 0 2024-02-16T22:56:06.674 INFO:tasks.workunit.client.1.smithi192.stdout:2/20: mknod c7 0 2024-02-16T22:56:06.675 INFO:tasks.workunit.client.1.smithi192.stdout:6/27: truncate f2 3280100 0 2024-02-16T22:56:06.676 INFO:tasks.workunit.client.1.smithi192.stdout:7/23: dwrite d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:06.677 INFO:tasks.workunit.client.1.smithi192.stdout:9/28: creat d7/f8 x:0 0 0 2024-02-16T22:56:06.679 INFO:tasks.workunit.client.1.smithi192.stdout:4/16: symlink d1/l6 0 2024-02-16T22:56:06.681 INFO:tasks.workunit.client.1.smithi192.stdout:6/28: unlink c0 0 2024-02-16T22:56:06.682 INFO:tasks.workunit.client.1.smithi192.stdout:7/24: dread d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:06.684 INFO:tasks.workunit.client.1.smithi192.stdout:2/21: link f2 f8 0 2024-02-16T22:56:06.684 INFO:tasks.workunit.client.1.smithi192.stdout:7/25: read d1/d2/f4 [821128,64850] 0 2024-02-16T22:56:06.685 INFO:tasks.workunit.client.1.smithi192.stdout:9/29: symlink d7/l9 0 2024-02-16T22:56:06.686 INFO:tasks.workunit.client.1.smithi192.stdout:5/15: fsync f2 0 2024-02-16T22:56:06.687 INFO:tasks.workunit.client.1.smithi192.stdout:4/17: mknod d1/c7 0 2024-02-16T22:56:06.687 INFO:tasks.workunit.client.1.smithi192.stdout:9/30: write d7/f8 [995959,14863] 0 2024-02-16T22:56:06.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/29: dwrite f1 [0,4194304] 0 2024-02-16T22:56:06.693 INFO:tasks.workunit.client.1.smithi192.stdout:7/26: symlink d1/d2/l8 0 2024-02-16T22:56:06.693 INFO:tasks.workunit.client.1.smithi192.stdout:5/16: unlink l3 0 2024-02-16T22:56:06.693 INFO:tasks.workunit.client.1.smithi192.stdout:2/22: write f2 [1353220,47695] 0 2024-02-16T22:56:06.696 INFO:tasks.workunit.client.1.smithi192.stdout:9/31: write f2 [5204328,8068] 0 2024-02-16T22:56:06.696 INFO:tasks.workunit.client.1.smithi192.stdout:2/23: truncate f4 5215545 0 2024-02-16T22:56:06.696 INFO:tasks.workunit.client.1.smithi192.stdout:4/18: mkdir d1/d8 0 2024-02-16T22:56:06.699 INFO:tasks.workunit.client.1.smithi192.stdout:4/19: write d1/f5 [4044936,116383] 0 2024-02-16T22:56:06.700 INFO:tasks.workunit.client.1.smithi192.stdout:2/24: mknod c9 0 2024-02-16T22:56:06.703 INFO:tasks.workunit.client.1.smithi192.stdout:2/25: creat fa x:0 0 0 2024-02-16T22:56:06.703 INFO:tasks.workunit.client.1.smithi192.stdout:9/32: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:06.707 INFO:tasks.workunit.client.1.smithi192.stdout:4/20: rename f0 to d1/d8/f9 0 2024-02-16T22:56:06.707 INFO:tasks.workunit.client.1.smithi192.stdout:2/26: creat fb x:0 0 0 2024-02-16T22:56:06.707 INFO:tasks.workunit.client.1.smithi192.stdout:9/33: fsync d7/f8 0 2024-02-16T22:56:06.708 INFO:tasks.workunit.client.1.smithi192.stdout:5/17: dwrite f2 [0,4194304] 0 2024-02-16T22:56:06.708 INFO:tasks.workunit.client.1.smithi192.stdout:2/27: dread - f5 zero size 2024-02-16T22:56:06.709 INFO:tasks.workunit.client.1.smithi192.stdout:6/30: truncate f1 3677686 0 2024-02-16T22:56:06.711 INFO:tasks.workunit.client.1.smithi192.stdout:7/27: dwrite d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:06.712 INFO:tasks.workunit.client.1.smithi192.stdout:9/34: mknod d7/ca 0 2024-02-16T22:56:06.715 INFO:tasks.workunit.client.1.smithi192.stdout:2/28: rename c6 to cc 0 2024-02-16T22:56:06.715 INFO:tasks.workunit.client.1.smithi192.stdout:7/28: symlink d1/l9 0 2024-02-16T22:56:06.716 INFO:tasks.workunit.client.1.smithi192.stdout:2/29: chown f8 2689213 1 2024-02-16T22:56:06.717 INFO:tasks.workunit.client.1.smithi192.stdout:9/35: symlink d7/lb 0 2024-02-16T22:56:06.717 INFO:tasks.workunit.client.1.smithi192.stdout:2/30: chown c9 173271773 1 2024-02-16T22:56:06.718 INFO:tasks.workunit.client.1.smithi192.stdout:4/21: dread d1/d8/f9 [0,4194304] 0 2024-02-16T22:56:06.719 INFO:tasks.workunit.client.1.smithi192.stdout:5/18: dwrite f2 [0,4194304] 0 2024-02-16T22:56:06.720 INFO:tasks.workunit.client.1.smithi192.stdout:2/31: write f8 [1081001,50898] 0 2024-02-16T22:56:06.723 INFO:tasks.workunit.client.1.smithi192.stdout:6/31: link f1 f4 0 2024-02-16T22:56:06.724 INFO:tasks.workunit.client.1.smithi192.stdout:9/36: dread f0 [0,4194304] 0 2024-02-16T22:56:06.724 INFO:tasks.workunit.client.1.smithi192.stdout:9/37: fsync f0 0 2024-02-16T22:56:06.725 INFO:tasks.workunit.client.1.smithi192.stdout:5/19: dread f2 [0,4194304] 0 2024-02-16T22:56:06.726 INFO:tasks.workunit.client.1.smithi192.stdout:9/38: fsync f0 0 2024-02-16T22:56:06.727 INFO:tasks.workunit.client.1.smithi192.stdout:6/32: symlink l5 0 2024-02-16T22:56:06.729 INFO:tasks.workunit.client.1.smithi192.stdout:2/32: link fb fd 0 2024-02-16T22:56:06.731 INFO:tasks.workunit.client.1.smithi192.stdout:5/20: mknod c4 0 2024-02-16T22:56:06.733 INFO:tasks.workunit.client.1.smithi192.stdout:9/39: rename c5 to d7/cc 0 2024-02-16T22:56:06.734 INFO:tasks.workunit.client.1.smithi192.stdout:9/40: chown f2 61943525 1 2024-02-16T22:56:06.734 INFO:tasks.workunit.client.1.smithi192.stdout:6/33: creat f6 x:0 0 0 2024-02-16T22:56:06.735 INFO:tasks.workunit.client.1.smithi192.stdout:5/21: creat f5 x:0 0 0 2024-02-16T22:56:06.737 INFO:tasks.workunit.client.1.smithi192.stdout:5/22: chown f2 359 1 2024-02-16T22:56:06.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/22: getdents d1/d8 0 2024-02-16T22:56:06.737 INFO:tasks.workunit.client.1.smithi192.stdout:2/33: dwrite f5 [0,4194304] 0 2024-02-16T22:56:06.737 INFO:tasks.workunit.client.1.smithi192.stdout:5/23: read - f5 zero size 2024-02-16T22:56:06.739 INFO:tasks.workunit.client.1.smithi192.stdout:6/34: write f1 [1546530,51951] 0 2024-02-16T22:56:06.739 INFO:tasks.workunit.client.1.smithi192.stdout:7/29: getdents d1 0 2024-02-16T22:56:06.740 INFO:tasks.workunit.client.1.smithi192.stdout:9/41: creat d7/fd x:0 0 0 2024-02-16T22:56:06.741 INFO:tasks.workunit.client.1.smithi192.stdout:2/34: creat fe x:0 0 0 2024-02-16T22:56:06.741 INFO:tasks.workunit.client.1.smithi192.stdout:9/42: fsync f2 0 2024-02-16T22:56:06.742 INFO:tasks.workunit.client.1.smithi192.stdout:4/23: creat d1/d8/fa x:0 0 0 2024-02-16T22:56:06.743 INFO:tasks.workunit.client.1.smithi192.stdout:5/24: mkdir d6 0 2024-02-16T22:56:06.744 INFO:tasks.workunit.client.1.smithi192.stdout:6/35: mknod c7 0 2024-02-16T22:56:06.746 INFO:tasks.workunit.client.1.smithi192.stdout:2/35: dread f8 [0,4194304] 0 2024-02-16T22:56:06.746 INFO:tasks.workunit.client.1.smithi192.stdout:2/36: rmdir - no directory 2024-02-16T22:56:06.749 INFO:tasks.workunit.client.1.smithi192.stdout:4/24: creat d1/d8/fb x:0 0 0 2024-02-16T22:56:06.751 INFO:tasks.workunit.client.1.smithi192.stdout:5/25: unlink f5 0 2024-02-16T22:56:06.752 INFO:tasks.workunit.client.1.smithi192.stdout:6/36: symlink l8 0 2024-02-16T22:56:06.756 INFO:tasks.workunit.client.1.smithi192.stdout:2/37: symlink lf 0 2024-02-16T22:56:06.756 INFO:tasks.workunit.client.1.smithi192.stdout:2/38: chown c7 2342 1 2024-02-16T22:56:06.759 INFO:tasks.workunit.client.1.smithi192.stdout:5/26: chown l0 14361140 1 2024-02-16T22:56:06.760 INFO:tasks.workunit.client.1.smithi192.stdout:9/43: chown d7/cc 163389 1 2024-02-16T22:56:06.760 INFO:tasks.workunit.client.1.smithi192.stdout:7/30: link c0 d1/d2/ca 0 2024-02-16T22:56:06.761 INFO:tasks.workunit.client.1.smithi192.stdout:9/44: chown d7/l9 567 1 2024-02-16T22:56:06.761 INFO:tasks.workunit.client.1.smithi192.stdout:2/39: creat f10 x:0 0 0 2024-02-16T22:56:06.762 INFO:tasks.workunit.client.1.smithi192.stdout:6/37: link f6 f9 0 2024-02-16T22:56:06.764 INFO:tasks.workunit.client.1.smithi192.stdout:4/25: dwrite d1/d8/f9 [0,4194304] 0 2024-02-16T22:56:06.764 INFO:tasks.workunit.client.1.smithi192.stdout:5/27: write f2 [3448733,50318] 0 2024-02-16T22:56:06.764 INFO:tasks.workunit.client.1.smithi192.stdout:7/31: mknod d1/cb 0 2024-02-16T22:56:06.764 INFO:tasks.workunit.client.1.smithi192.stdout:2/40: mknod c11 0 2024-02-16T22:56:06.765 INFO:tasks.workunit.client.1.smithi192.stdout:9/45: rename d7/fd to d7/fe 0 2024-02-16T22:56:06.768 INFO:tasks.workunit.client.1.smithi192.stdout:7/32: symlink d1/d2/lc 0 2024-02-16T22:56:06.769 INFO:tasks.workunit.client.1.smithi192.stdout:7/33: stat d1/l9 0 2024-02-16T22:56:06.770 INFO:tasks.workunit.client.1.smithi192.stdout:9/46: truncate d7/fe 142734 0 2024-02-16T22:56:06.770 INFO:tasks.workunit.client.1.smithi192.stdout:4/26: rename d1/d8/fa to d1/d8/fc 0 2024-02-16T22:56:06.772 INFO:tasks.workunit.client.1.smithi192.stdout:7/34: rmdir d1 39 2024-02-16T22:56:06.773 INFO:tasks.workunit.client.1.smithi192.stdout:5/28: dread f2 [0,4194304] 0 2024-02-16T22:56:06.774 INFO:tasks.workunit.client.1.smithi192.stdout:4/27: write d1/f4 [1376304,61937] 0 2024-02-16T22:56:06.774 INFO:tasks.workunit.client.1.smithi192.stdout:2/41: dread f4 [0,4194304] 0 2024-02-16T22:56:06.775 INFO:tasks.workunit.client.1.smithi192.stdout:4/28: dread - d1/d8/fc zero size 2024-02-16T22:56:06.775 INFO:tasks.workunit.client.1.smithi192.stdout:9/47: mkdir d7/df 0 2024-02-16T22:56:06.779 INFO:tasks.workunit.client.1.smithi192.stdout:6/38: getdents . 0 2024-02-16T22:56:06.781 INFO:tasks.workunit.client.1.smithi192.stdout:7/35: creat d1/d2/fd x:0 0 0 2024-02-16T22:56:06.782 INFO:tasks.workunit.client.1.smithi192.stdout:4/29: write d1/d8/fc [419593,8365] 0 2024-02-16T22:56:06.784 INFO:tasks.workunit.client.1.smithi192.stdout:6/39: truncate f2 35357 0 2024-02-16T22:56:06.785 INFO:tasks.workunit.client.1.smithi192.stdout:2/42: link c9 c12 0 2024-02-16T22:56:06.785 INFO:tasks.workunit.client.1.smithi192.stdout:9/48: dwrite d7/f8 [0,4194304] 0 2024-02-16T22:56:06.786 INFO:tasks.workunit.client.1.smithi192.stdout:6/40: dread - f6 zero size 2024-02-16T22:56:06.786 INFO:tasks.workunit.client.1.smithi192.stdout:2/43: read - f10 zero size 2024-02-16T22:56:06.788 INFO:tasks.workunit.client.1.smithi192.stdout:7/36: mkdir d1/d2/de 0 2024-02-16T22:56:06.788 INFO:tasks.workunit.client.1.smithi192.stdout:6/41: chown c3 51781217 1 2024-02-16T22:56:06.788 INFO:tasks.workunit.client.1.smithi192.stdout:6/42: rmdir - no directory 2024-02-16T22:56:06.788 INFO:tasks.workunit.client.1.smithi192.stdout:4/30: rename d1/f3 to d1/fd 0 2024-02-16T22:56:06.790 INFO:tasks.workunit.client.1.smithi192.stdout:6/43: stat f1 0 2024-02-16T22:56:06.791 INFO:tasks.workunit.client.1.smithi192.stdout:5/29: dwrite f2 [0,4194304] 0 2024-02-16T22:56:06.791 INFO:tasks.workunit.client.1.smithi192.stdout:9/49: creat d7/f10 x:0 0 0 2024-02-16T22:56:06.791 INFO:tasks.workunit.client.1.smithi192.stdout:2/44: fdatasync fb 0 2024-02-16T22:56:06.793 INFO:tasks.workunit.client.1.smithi192.stdout:9/50: dread - f1 zero size 2024-02-16T22:56:06.794 INFO:tasks.workunit.client.1.smithi192.stdout:6/44: mkdir da 0 2024-02-16T22:56:06.794 INFO:tasks.workunit.client.1.smithi192.stdout:4/31: read d1/d8/fc [25391,59359] 0 2024-02-16T22:56:06.796 INFO:tasks.workunit.client.1.smithi192.stdout:7/37: dread d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:06.796 INFO:tasks.workunit.client.1.smithi192.stdout:9/51: write f2 [1082392,57495] 0 2024-02-16T22:56:06.800 INFO:tasks.workunit.client.1.smithi192.stdout:2/45: dread f1 [0,4194304] 0 2024-02-16T22:56:06.800 INFO:tasks.workunit.client.1.smithi192.stdout:4/32: creat d1/d8/fe x:0 0 0 2024-02-16T22:56:06.803 INFO:tasks.workunit.client.1.smithi192.stdout:7/38: write d1/d2/f4 [45815,75764] 0 2024-02-16T22:56:06.804 INFO:tasks.workunit.client.1.smithi192.stdout:5/30: symlink d6/l7 0 2024-02-16T22:56:06.805 INFO:tasks.workunit.client.1.smithi192.stdout:2/46: mknod c13 0 2024-02-16T22:56:06.809 INFO:tasks.workunit.client.1.smithi192.stdout:6/45: dread f2 [0,4194304] 0 2024-02-16T22:56:06.810 INFO:tasks.workunit.client.1.smithi192.stdout:5/31: dread f2 [0,4194304] 0 2024-02-16T22:56:06.810 INFO:tasks.workunit.client.1.smithi192.stdout:7/39: dwrite d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:06.810 INFO:tasks.workunit.client.1.smithi192.stdout:6/46: dread - f9 zero size 2024-02-16T22:56:06.812 INFO:tasks.workunit.client.1.smithi192.stdout:2/47: rename c12 to c14 0 2024-02-16T22:56:06.812 INFO:tasks.workunit.client.1.smithi192.stdout:2/48: write fe [924522,85103] 0 2024-02-16T22:56:06.814 INFO:tasks.workunit.client.1.smithi192.stdout:5/32: fdatasync f2 0 2024-02-16T22:56:06.816 INFO:tasks.workunit.client.1.smithi192.stdout:9/52: fsync d7/fe 0 2024-02-16T22:56:06.817 INFO:tasks.workunit.client.1.smithi192.stdout:5/33: mkdir d6/d8 0 2024-02-16T22:56:06.818 INFO:tasks.workunit.client.1.smithi192.stdout:9/53: chown c4 206868655 1 2024-02-16T22:56:06.818 INFO:tasks.workunit.client.1.smithi192.stdout:5/34: chown d6 0 1 2024-02-16T22:56:06.820 INFO:tasks.workunit.client.1.smithi192.stdout:5/35: fdatasync f2 0 2024-02-16T22:56:06.821 INFO:tasks.workunit.client.1.smithi192.stdout:9/54: mknod d7/c11 0 2024-02-16T22:56:06.821 INFO:tasks.workunit.client.1.smithi192.stdout:6/47: write f9 [578300,20575] 0 2024-02-16T22:56:06.821 INFO:tasks.workunit.client.1.smithi192.stdout:7/40: dwrite d1/d2/fd [0,4194304] 0 2024-02-16T22:56:06.824 INFO:tasks.workunit.client.1.smithi192.stdout:5/36: write f2 [3417565,4962] 0 2024-02-16T22:56:06.824 INFO:tasks.workunit.client.1.smithi192.stdout:9/55: write f0 [730826,86661] 0 2024-02-16T22:56:06.826 INFO:tasks.workunit.client.1.smithi192.stdout:2/49: dwrite fe [0,4194304] 0 2024-02-16T22:56:06.827 INFO:tasks.workunit.client.1.smithi192.stdout:2/50: readlink lf 0 2024-02-16T22:56:06.831 INFO:tasks.workunit.client.1.smithi192.stdout:5/37: symlink d6/d8/l9 0 2024-02-16T22:56:06.833 INFO:tasks.workunit.client.1.smithi192.stdout:2/51: chown lf 6868836 1 2024-02-16T22:56:06.835 INFO:tasks.workunit.client.1.smithi192.stdout:5/38: symlink d6/d8/la 0 2024-02-16T22:56:06.836 INFO:tasks.workunit.client.1.smithi192.stdout:2/52: chown c9 9626 1 2024-02-16T22:56:06.836 INFO:tasks.workunit.client.1.smithi192.stdout:9/56: getdents d7 0 2024-02-16T22:56:06.838 INFO:tasks.workunit.client.1.smithi192.stdout:2/53: stat lf 0 2024-02-16T22:56:06.838 INFO:tasks.workunit.client.1.smithi192.stdout:6/48: dwrite f6 [0,4194304] 0 2024-02-16T22:56:06.838 INFO:tasks.workunit.client.1.smithi192.stdout:5/39: chown d6/d8/l9 0 1 2024-02-16T22:56:06.839 INFO:tasks.workunit.client.1.smithi192.stdout:7/41: dwrite d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:06.839 INFO:tasks.workunit.client.1.smithi192.stdout:9/57: creat d7/f12 x:0 0 0 2024-02-16T22:56:06.840 INFO:tasks.workunit.client.1.smithi192.stdout:2/54: symlink l15 0 2024-02-16T22:56:06.841 INFO:tasks.workunit.client.1.smithi192.stdout:5/40: write f2 [3047015,121964] 0 2024-02-16T22:56:06.842 INFO:tasks.workunit.client.1.smithi192.stdout:6/49: rename f4 to da/fb 0 2024-02-16T22:56:06.843 INFO:tasks.workunit.client.1.smithi192.stdout:6/50: chown c7 34 1 2024-02-16T22:56:06.844 INFO:tasks.workunit.client.1.smithi192.stdout:7/42: mknod d1/d2/cf 0 2024-02-16T22:56:06.846 INFO:tasks.workunit.client.1.smithi192.stdout:5/41: mknod d6/cb 0 2024-02-16T22:56:06.847 INFO:tasks.workunit.client.1.smithi192.stdout:6/51: mkdir da/dc 0 2024-02-16T22:56:06.850 INFO:tasks.workunit.client.1.smithi192.stdout:2/55: dwrite fa [0,4194304] 0 2024-02-16T22:56:06.850 INFO:tasks.workunit.client.1.smithi192.stdout:9/58: link f0 d7/df/f13 0 2024-02-16T22:56:06.851 INFO:tasks.workunit.client.1.smithi192.stdout:9/59: dread - f1 zero size 2024-02-16T22:56:06.853 INFO:tasks.workunit.client.1.smithi192.stdout:6/52: symlink da/ld 0 2024-02-16T22:56:06.855 INFO:tasks.workunit.client.1.smithi192.stdout:9/60: readlink d7/l9 0 2024-02-16T22:56:06.855 INFO:tasks.workunit.client.1.smithi192.stdout:7/43: symlink d1/d2/de/l10 0 2024-02-16T22:56:06.856 INFO:tasks.workunit.client.1.smithi192.stdout:6/53: chown f2 33053424 1 2024-02-16T22:56:06.857 INFO:tasks.workunit.client.1.smithi192.stdout:6/54: read f1 [1625594,23852] 0 2024-02-16T22:56:06.858 INFO:tasks.workunit.client.1.smithi192.stdout:5/42: dwrite f2 [4194304,4194304] 0 2024-02-16T22:56:06.861 INFO:tasks.workunit.client.1.smithi192.stdout:2/56: dread f5 [0,4194304] 0 2024-02-16T22:56:06.862 INFO:tasks.workunit.client.1.smithi192.stdout:2/57: chown c13 15389 1 2024-02-16T22:56:06.866 INFO:tasks.workunit.client.1.smithi192.stdout:6/55: write f1 [3762831,40090] 0 2024-02-16T22:56:06.868 INFO:tasks.workunit.client.1.smithi192.stdout:7/44: dwrite d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:06.868 INFO:tasks.workunit.client.1.smithi192.stdout:9/61: creat d7/df/f14 x:0 0 0 2024-02-16T22:56:06.869 INFO:tasks.workunit.client.1.smithi192.stdout:9/62: fsync f1 0 2024-02-16T22:56:06.871 INFO:tasks.workunit.client.1.smithi192.stdout:2/58: mknod c16 0 2024-02-16T22:56:06.876 INFO:tasks.workunit.client.1.smithi192.stdout:9/63: dwrite f1 [0,4194304] 0 2024-02-16T22:56:06.881 INFO:tasks.workunit.client.1.smithi192.stdout:7/45: creat d1/f11 x:0 0 0 2024-02-16T22:56:06.882 INFO:tasks.workunit.client.1.smithi192.stdout:2/59: symlink l17 0 2024-02-16T22:56:06.886 INFO:tasks.workunit.client.1.smithi192.stdout:9/64: mkdir d7/d15 0 2024-02-16T22:56:06.888 INFO:tasks.workunit.client.1.smithi192.stdout:6/56: mkdir da/dc/de 0 2024-02-16T22:56:06.890 INFO:tasks.workunit.client.1.smithi192.stdout:2/60: chown cc 83801789 1 2024-02-16T22:56:06.891 INFO:tasks.workunit.client.1.smithi192.stdout:9/65: creat d7/df/f16 x:0 0 0 2024-02-16T22:56:06.891 INFO:tasks.workunit.client.1.smithi192.stdout:9/66: chown d7/cc 259816205 1 2024-02-16T22:56:06.894 INFO:tasks.workunit.client.1.smithi192.stdout:2/61: mkdir d18 0 2024-02-16T22:56:06.895 INFO:tasks.workunit.client.1.smithi192.stdout:9/67: dread d7/df/f13 [0,4194304] 0 2024-02-16T22:56:06.897 INFO:tasks.workunit.client.1.smithi192.stdout:5/43: truncate f2 1016098 0 2024-02-16T22:56:06.902 INFO:tasks.workunit.client.1.smithi192.stdout:2/62: unlink l3 0 2024-02-16T22:56:06.907 INFO:tasks.workunit.client.1.smithi192.stdout:9/68: symlink d7/l17 0 2024-02-16T22:56:06.911 INFO:tasks.workunit.client.1.smithi192.stdout:9/69: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:06.911 INFO:tasks.workunit.client.1.smithi192.stdout:6/57: rmdir da/dc/de 0 2024-02-16T22:56:06.913 INFO:tasks.workunit.client.1.smithi192.stdout:6/58: dread da/fb [0,4194304] 0 2024-02-16T22:56:06.927 INFO:tasks.workunit.client.1.smithi192.stdout:6/59: rename da/dc to da/df 0 2024-02-16T22:56:06.929 INFO:tasks.workunit.client.1.smithi192.stdout:5/44: getdents d6/d8 0 2024-02-16T22:56:06.931 INFO:tasks.workunit.client.1.smithi192.stdout:6/60: unlink f1 0 2024-02-16T22:56:06.932 INFO:tasks.workunit.client.1.smithi192.stdout:2/63: dwrite f8 [0,4194304] 0 2024-02-16T22:56:06.932 INFO:tasks.workunit.client.1.smithi192.stdout:5/45: fsync f2 0 2024-02-16T22:56:06.933 INFO:tasks.workunit.client.1.smithi192.stdout:9/70: dwrite f0 [4194304,4194304] 0 2024-02-16T22:56:06.934 INFO:tasks.workunit.client.1.smithi192.stdout:9/71: readlink d7/l17 0 2024-02-16T22:56:06.937 INFO:tasks.workunit.client.1.smithi192.stdout:6/61: mknod da/c10 0 2024-02-16T22:56:06.946 INFO:tasks.workunit.client.1.smithi192.stdout:6/62: creat da/df/f11 x:0 0 0 2024-02-16T22:56:07.003 INFO:tasks.workunit.client.1.smithi192.stdout:0/5: sync 2024-02-16T22:56:07.020 INFO:tasks.workunit.client.1.smithi192.stdout:5/46: link f2 d6/fc 0 2024-02-16T22:56:07.062 INFO:tasks.workunit.client.1.smithi192.stdout:2/64: rmdir d18 0 2024-02-16T22:56:07.062 INFO:tasks.workunit.client.1.smithi192.stdout:2/65: chown fd 574317218 1 2024-02-16T22:56:07.069 INFO:tasks.workunit.client.1.smithi192.stdout:5/47: write d6/fc [1456801,70439] 0 2024-02-16T22:56:07.073 INFO:tasks.workunit.client.1.smithi192.stdout:2/66: creat f19 x:0 0 0 2024-02-16T22:56:07.074 INFO:tasks.workunit.client.1.smithi192.stdout:9/72: truncate f2 2944859 0 2024-02-16T22:56:07.074 INFO:tasks.workunit.client.1.smithi192.stdout:0/6: creat f0 x:0 0 0 2024-02-16T22:56:07.076 INFO:tasks.workunit.client.1.smithi192.stdout:5/48: creat d6/fd x:0 0 0 2024-02-16T22:56:07.080 INFO:tasks.workunit.client.1.smithi192.stdout:9/73: symlink d7/df/l18 0 2024-02-16T22:56:07.080 INFO:tasks.workunit.client.1.smithi192.stdout:6/63: getdents da 0 2024-02-16T22:56:07.080 INFO:tasks.workunit.client.1.smithi192.stdout:0/7: dwrite f0 [0,4194304] 0 2024-02-16T22:56:07.108 INFO:tasks.workunit.client.1.smithi192.stdout:0/8: write f0 [3218329,36346] 0 2024-02-16T22:56:07.110 INFO:tasks.workunit.client.1.smithi192.stdout:9/74: mkdir d7/d19 0 2024-02-16T22:56:07.115 INFO:tasks.workunit.client.1.smithi192.stdout:0/9: dwrite f0 [0,4194304] 0 2024-02-16T22:56:07.117 INFO:tasks.workunit.client.1.smithi192.stdout:5/49: link c4 d6/ce 0 2024-02-16T22:56:07.124 INFO:tasks.workunit.client.1.smithi192.stdout:0/10: mkdir d1 0 2024-02-16T22:56:07.126 INFO:tasks.workunit.client.1.smithi192.stdout:5/50: rmdir d6/d8 39 2024-02-16T22:56:07.127 INFO:tasks.workunit.client.1.smithi192.stdout:0/11: dread f0 [0,4194304] 0 2024-02-16T22:56:07.128 INFO:tasks.workunit.client.1.smithi192.stdout:0/12: chown f0 766394236 1 2024-02-16T22:56:07.129 INFO:tasks.workunit.client.1.smithi192.stdout:9/75: symlink d7/d19/l1a 0 2024-02-16T22:56:07.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/76: fsync d7/df/f16 0 2024-02-16T22:56:07.134 INFO:tasks.workunit.client.1.smithi192.stdout:5/51: chown d6/d8 1 1 2024-02-16T22:56:07.137 INFO:tasks.workunit.client.1.smithi192.stdout:6/64: getdents da 0 2024-02-16T22:56:07.138 INFO:tasks.workunit.client.1.smithi192.stdout:6/65: chown f6 6138200 1 2024-02-16T22:56:07.141 INFO:tasks.workunit.client.1.smithi192.stdout:0/13: mkdir d1/d2 0 2024-02-16T22:56:07.142 INFO:tasks.workunit.client.1.smithi192.stdout:9/77: getdents d7/d15 0 2024-02-16T22:56:07.142 INFO:tasks.workunit.client.1.smithi192.stdout:9/78: fdatasync d7/df/f13 0 2024-02-16T22:56:07.143 INFO:tasks.workunit.client.1.smithi192.stdout:9/79: stat f1 0 2024-02-16T22:56:07.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/66: mkdir da/df/d12 0 2024-02-16T22:56:07.145 INFO:tasks.workunit.client.1.smithi192.stdout:5/52: symlink d6/lf 0 2024-02-16T22:56:07.145 INFO:tasks.workunit.client.1.smithi192.stdout:5/53: chown d6 256461037 1 2024-02-16T22:56:07.147 INFO:tasks.workunit.client.1.smithi192.stdout:0/14: mkdir d1/d3 0 2024-02-16T22:56:07.154 INFO:tasks.workunit.client.1.smithi192.stdout:0/15: rename f0 to d1/d2/f4 0 2024-02-16T22:56:07.154 INFO:tasks.workunit.client.1.smithi192.stdout:0/16: readlink - no filename 2024-02-16T22:56:07.154 INFO:tasks.workunit.client.1.smithi192.stdout:0/17: chown d1 336 1 2024-02-16T22:56:07.158 INFO:tasks.workunit.client.1.smithi192.stdout:6/67: link f9 da/df/d12/f13 0 2024-02-16T22:56:07.162 INFO:tasks.workunit.client.1.smithi192.stdout:0/18: creat d1/f5 x:0 0 0 2024-02-16T22:56:07.162 INFO:tasks.workunit.client.1.smithi192.stdout:9/80: link d7/ca d7/c1b 0 2024-02-16T22:56:07.168 INFO:tasks.workunit.client.1.smithi192.stdout:1/10: sync 2024-02-16T22:56:07.168 INFO:tasks.workunit.client.1.smithi192.stdout:7/46: sync 2024-02-16T22:56:07.168 INFO:tasks.workunit.client.1.smithi192.stdout:1/11: dread - no filename 2024-02-16T22:56:07.168 INFO:tasks.workunit.client.1.smithi192.stdout:3/10: sync 2024-02-16T22:56:07.168 INFO:tasks.workunit.client.1.smithi192.stdout:1/12: dread - no filename 2024-02-16T22:56:07.168 INFO:tasks.workunit.client.1.smithi192.stdout:1/13: dread - no filename 2024-02-16T22:56:07.168 INFO:tasks.workunit.client.1.smithi192.stdout:3/11: dwrite - no filename 2024-02-16T22:56:07.169 INFO:tasks.workunit.client.1.smithi192.stdout:1/14: dread - no filename 2024-02-16T22:56:07.169 INFO:tasks.workunit.client.1.smithi192.stdout:3/12: rename - no filename 2024-02-16T22:56:07.169 INFO:tasks.workunit.client.1.smithi192.stdout:8/1: sync 2024-02-16T22:56:07.169 INFO:tasks.workunit.client.1.smithi192.stdout:3/13: stat - no entries 2024-02-16T22:56:07.169 INFO:tasks.workunit.client.1.smithi192.stdout:8/2: write - no filename 2024-02-16T22:56:07.169 INFO:tasks.workunit.client.1.smithi192.stdout:4/33: sync 2024-02-16T22:56:07.169 INFO:tasks.workunit.client.1.smithi192.stdout:7/47: chown d1/d2/l5 2035 1 2024-02-16T22:56:07.169 INFO:tasks.workunit.client.1.smithi192.stdout:9/81: unlink d7/f10 0 2024-02-16T22:56:07.173 INFO:tasks.workunit.client.1.smithi192.stdout:0/19: mkdir d1/d3/d6 0 2024-02-16T22:56:07.177 INFO:tasks.workunit.client.1.smithi192.stdout:1/15: rename c1 to c2 0 2024-02-16T22:56:07.179 INFO:tasks.workunit.client.1.smithi192.stdout:0/20: dwrite d1/f5 [0,4194304] 0 2024-02-16T22:56:07.179 INFO:tasks.workunit.client.1.smithi192.stdout:4/34: rename d1/l6 to d1/lf 0 2024-02-16T22:56:07.180 INFO:tasks.workunit.client.1.smithi192.stdout:5/54: getdents d6 0 2024-02-16T22:56:07.181 INFO:tasks.workunit.client.1.smithi192.stdout:7/48: creat d1/f12 x:0 0 0 2024-02-16T22:56:07.183 INFO:tasks.workunit.client.1.smithi192.stdout:9/82: symlink d7/l1c 0 2024-02-16T22:56:07.183 INFO:tasks.workunit.client.1.smithi192.stdout:5/55: dread d6/fc [0,4194304] 0 2024-02-16T22:56:07.184 INFO:tasks.workunit.client.1.smithi192.stdout:8/3: mknod c0 0 2024-02-16T22:56:07.184 INFO:tasks.workunit.client.1.smithi192.stdout:8/4: dread - no filename 2024-02-16T22:56:07.184 INFO:tasks.workunit.client.1.smithi192.stdout:8/5: dread - no filename 2024-02-16T22:56:07.186 INFO:tasks.workunit.client.1.smithi192.stdout:5/56: read f2 [673799,44749] 0 2024-02-16T22:56:07.187 INFO:tasks.workunit.client.1.smithi192.stdout:5/57: readlink l0 0 2024-02-16T22:56:07.188 INFO:tasks.workunit.client.1.smithi192.stdout:1/16: mkdir d3 0 2024-02-16T22:56:07.190 INFO:tasks.workunit.client.1.smithi192.stdout:7/49: dwrite d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:07.192 INFO:tasks.workunit.client.1.smithi192.stdout:7/50: read - d1/f12 zero size 2024-02-16T22:56:07.198 INFO:tasks.workunit.client.1.smithi192.stdout:4/35: dread d1/d8/fc [0,4194304] 0 2024-02-16T22:56:07.198 INFO:tasks.workunit.client.1.smithi192.stdout:6/68: write f6 [4640265,16416] 0 2024-02-16T22:56:07.199 INFO:tasks.workunit.client.1.smithi192.stdout:5/58: dwrite d6/fc [0,4194304] 0 2024-02-16T22:56:07.199 INFO:tasks.workunit.client.1.smithi192.stdout:8/6: symlink l1 0 2024-02-16T22:56:07.199 INFO:tasks.workunit.client.1.smithi192.stdout:8/7: fdatasync - no filename 2024-02-16T22:56:07.199 INFO:tasks.workunit.client.1.smithi192.stdout:8/8: write - no filename 2024-02-16T22:56:07.199 INFO:tasks.workunit.client.1.smithi192.stdout:6/69: chown da/ld 70 1 2024-02-16T22:56:07.206 INFO:tasks.workunit.client.1.smithi192.stdout:7/51: mknod d1/d2/de/c13 0 2024-02-16T22:56:07.207 INFO:tasks.workunit.client.1.smithi192.stdout:0/21: fsync d1/f5 0 2024-02-16T22:56:07.209 INFO:tasks.workunit.client.1.smithi192.stdout:9/83: creat d7/d15/f1d x:0 0 0 2024-02-16T22:56:07.214 INFO:tasks.workunit.client.1.smithi192.stdout:4/36: rename d1/c7 to d1/d8/c10 0 2024-02-16T22:56:07.217 INFO:tasks.workunit.client.1.smithi192.stdout:1/17: creat d3/f4 x:0 0 0 2024-02-16T22:56:07.220 INFO:tasks.workunit.client.1.smithi192.stdout:6/70: symlink da/l14 0 2024-02-16T22:56:07.222 INFO:tasks.workunit.client.1.smithi192.stdout:7/52: unlink d1/cb 0 2024-02-16T22:56:07.224 INFO:tasks.workunit.client.1.smithi192.stdout:7/53: chown d1/d2/l3 5539 1 2024-02-16T22:56:07.224 INFO:tasks.workunit.client.1.smithi192.stdout:5/59: dwrite d6/fc [0,4194304] 0 2024-02-16T22:56:07.225 INFO:tasks.workunit.client.1.smithi192.stdout:0/22: creat d1/f7 x:0 0 0 2024-02-16T22:56:07.229 INFO:tasks.workunit.client.1.smithi192.stdout:4/37: mkdir d1/d8/d11 0 2024-02-16T22:56:07.231 INFO:tasks.workunit.client.1.smithi192.stdout:4/38: truncate d1/d8/fe 86766 0 2024-02-16T22:56:07.232 INFO:tasks.workunit.client.1.smithi192.stdout:1/18: stat c2 0 2024-02-16T22:56:07.232 INFO:tasks.workunit.client.1.smithi192.stdout:5/60: dwrite d6/fc [0,4194304] 0 2024-02-16T22:56:07.233 INFO:tasks.workunit.client.1.smithi192.stdout:5/61: fdatasync d6/fd 0 2024-02-16T22:56:07.236 INFO:tasks.workunit.client.1.smithi192.stdout:5/62: stat d6/d8/la 0 2024-02-16T22:56:07.236 INFO:tasks.workunit.client.1.smithi192.stdout:7/54: mknod d1/c14 0 2024-02-16T22:56:07.239 INFO:tasks.workunit.client.1.smithi192.stdout:5/63: chown d6/cb 15789 1 2024-02-16T22:56:07.242 INFO:tasks.workunit.client.1.smithi192.stdout:7/55: truncate d1/f11 275624 0 2024-02-16T22:56:07.246 INFO:tasks.workunit.client.1.smithi192.stdout:4/39: dread d1/d8/f9 [0,4194304] 0 2024-02-16T22:56:07.246 INFO:tasks.workunit.client.1.smithi192.stdout:9/84: link d7/l1c d7/l1e 0 2024-02-16T22:56:07.248 INFO:tasks.workunit.client.1.smithi192.stdout:6/71: dwrite f2 [0,4194304] 0 2024-02-16T22:56:07.249 INFO:tasks.workunit.client.1.smithi192.stdout:1/19: dwrite d3/f4 [0,4194304] 0 2024-02-16T22:56:07.250 INFO:tasks.workunit.client.1.smithi192.stdout:8/9: getdents . 0 2024-02-16T22:56:07.251 INFO:tasks.workunit.client.1.smithi192.stdout:8/10: readlink l1 0 2024-02-16T22:56:07.251 INFO:tasks.workunit.client.1.smithi192.stdout:1/20: chown l0 603 1 2024-02-16T22:56:07.252 INFO:tasks.workunit.client.1.smithi192.stdout:8/11: chown c0 8675786 1 2024-02-16T22:56:07.252 INFO:tasks.workunit.client.1.smithi192.stdout:7/56: mknod d1/d2/c15 0 2024-02-16T22:56:07.252 INFO:tasks.workunit.client.1.smithi192.stdout:0/23: dwrite d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:07.253 INFO:tasks.workunit.client.1.smithi192.stdout:0/24: fsync d1/f5 0 2024-02-16T22:56:07.254 INFO:tasks.workunit.client.1.smithi192.stdout:0/25: chown d1/f5 97 1 2024-02-16T22:56:07.257 INFO:tasks.workunit.client.1.smithi192.stdout:2/67: sync 2024-02-16T22:56:07.258 INFO:tasks.workunit.client.1.smithi192.stdout:2/68: read f5 [2376691,15779] 0 2024-02-16T22:56:07.262 INFO:tasks.workunit.client.1.smithi192.stdout:9/85: symlink d7/df/l1f 0 2024-02-16T22:56:07.267 INFO:tasks.workunit.client.1.smithi192.stdout:1/21: symlink d3/l5 0 2024-02-16T22:56:07.271 INFO:tasks.workunit.client.1.smithi192.stdout:8/12: creat f2 x:0 0 0 2024-02-16T22:56:07.271 INFO:tasks.workunit.client.1.smithi192.stdout:8/13: chown f2 19121367 1 2024-02-16T22:56:07.273 INFO:tasks.workunit.client.1.smithi192.stdout:7/57: rename d1/d2/l3 to d1/d2/de/l16 0 2024-02-16T22:56:07.275 INFO:tasks.workunit.client.1.smithi192.stdout:5/64: link d6/fd d6/f10 0 2024-02-16T22:56:07.279 INFO:tasks.workunit.client.1.smithi192.stdout:8/14: dwrite f2 [0,4194304] 0 2024-02-16T22:56:07.280 INFO:tasks.workunit.client.1.smithi192.stdout:4/40: link d1/f5 d1/f12 0 2024-02-16T22:56:07.281 INFO:tasks.workunit.client.1.smithi192.stdout:4/41: write d1/d8/fb [399273,126646] 0 2024-02-16T22:56:07.283 INFO:tasks.workunit.client.1.smithi192.stdout:2/69: rename c14 to c1a 0 2024-02-16T22:56:07.283 INFO:tasks.workunit.client.1.smithi192.stdout:2/70: fsync f1 0 2024-02-16T22:56:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:06 smithi192 ceph-mon[36254]: pgmap v8: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 2.8 MiB/s wr, 200 op/s 2024-02-16T22:56:07.285 INFO:tasks.workunit.client.1.smithi192.stdout:4/42: dread d1/f4 [0,4194304] 0 2024-02-16T22:56:07.287 INFO:tasks.workunit.client.1.smithi192.stdout:9/86: unlink d7/d19/l1a 0 2024-02-16T22:56:07.290 INFO:tasks.workunit.client.1.smithi192.stdout:1/22: mknod d3/c6 0 2024-02-16T22:56:07.292 INFO:tasks.workunit.client.1.smithi192.stdout:7/58: rename d1/l7 to d1/d2/l17 0 2024-02-16T22:56:07.295 INFO:tasks.workunit.client.1.smithi192.stdout:5/65: stat d6/d8 0 2024-02-16T22:56:07.299 INFO:tasks.workunit.client.1.smithi192.stdout:8/15: rename c0 to c3 0 2024-02-16T22:56:07.307 INFO:tasks.workunit.client.1.smithi192.stdout:4/43: write d1/d8/f9 [2491383,16851] 0 2024-02-16T22:56:07.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/23: rename d3/l5 to d3/l7 0 2024-02-16T22:56:07.313 INFO:tasks.workunit.client.1.smithi192.stdout:0/26: rmdir d1/d3/d6 0 2024-02-16T22:56:07.313 INFO:tasks.workunit.client.1.smithi192.stdout:3/14: sync 2024-02-16T22:56:07.320 INFO:tasks.workunit.client.1.smithi192.stdout:6/72: mkdir da/df/d12/d15 0 2024-02-16T22:56:07.320 INFO:tasks.workunit.client.1.smithi192.stdout:4/44: fsync d1/f4 0 2024-02-16T22:56:07.320 INFO:tasks.workunit.client.1.smithi192.stdout:4/45: chown d1/f5 118 1 2024-02-16T22:56:07.320 INFO:tasks.workunit.client.1.smithi192.stdout:9/87: truncate f2 1011426 0 2024-02-16T22:56:07.320 INFO:tasks.workunit.client.1.smithi192.stdout:9/88: truncate d7/d15/f1d 986474 0 2024-02-16T22:56:07.321 INFO:tasks.workunit.client.1.smithi192.stdout:1/24: rename d3/c6 to d3/c8 0 2024-02-16T22:56:07.321 INFO:tasks.workunit.client.1.smithi192.stdout:9/89: dread - d7/df/f14 zero size 2024-02-16T22:56:07.322 INFO:tasks.workunit.client.1.smithi192.stdout:3/15: symlink l1 0 2024-02-16T22:56:07.322 INFO:tasks.workunit.client.1.smithi192.stdout:0/27: creat d1/f8 x:0 0 0 2024-02-16T22:56:07.322 INFO:tasks.workunit.client.1.smithi192.stdout:0/28: readlink - no filename 2024-02-16T22:56:07.323 INFO:tasks.workunit.client.1.smithi192.stdout:0/29: write d1/f5 [3940946,86331] 0 2024-02-16T22:56:07.324 INFO:tasks.workunit.client.1.smithi192.stdout:6/73: creat da/df/f16 x:0 0 0 2024-02-16T22:56:07.326 INFO:tasks.workunit.client.1.smithi192.stdout:4/46: mknod d1/c13 0 2024-02-16T22:56:07.327 INFO:tasks.workunit.client.1.smithi192.stdout:5/66: creat d6/f11 x:0 0 0 2024-02-16T22:56:07.329 INFO:tasks.workunit.client.1.smithi192.stdout:9/90: rename d7/df/f13 to d7/d15/f20 0 2024-02-16T22:56:07.330 INFO:tasks.workunit.client.1.smithi192.stdout:2/71: getdents . 0 2024-02-16T22:56:07.332 INFO:tasks.workunit.client.1.smithi192.stdout:8/16: unlink c3 0 2024-02-16T22:56:07.336 INFO:tasks.workunit.client.1.smithi192.stdout:9/91: creat d7/d19/f21 x:0 0 0 2024-02-16T22:56:07.336 INFO:tasks.workunit.client.1.smithi192.stdout:9/92: chown d7/df/l1f 210782 1 2024-02-16T22:56:07.337 INFO:tasks.workunit.client.1.smithi192.stdout:2/72: creat f1b x:0 0 0 2024-02-16T22:56:07.337 INFO:tasks.workunit.client.1.smithi192.stdout:9/93: write d7/d19/f21 [521265,60563] 0 2024-02-16T22:56:07.338 INFO:tasks.workunit.client.1.smithi192.stdout:2/73: chown f8 16 1 2024-02-16T22:56:07.338 INFO:tasks.workunit.client.1.smithi192.stdout:4/47: creat d1/d8/d11/f14 x:0 0 0 2024-02-16T22:56:07.339 INFO:tasks.workunit.client.1.smithi192.stdout:4/48: fsync d1/d8/fe 0 2024-02-16T22:56:07.339 INFO:tasks.workunit.client.1.smithi192.stdout:4/49: chown d1/d8/fb 23004793 1 2024-02-16T22:56:07.340 INFO:tasks.workunit.client.1.smithi192.stdout:5/67: mkdir d6/d12 0 2024-02-16T22:56:07.341 INFO:tasks.workunit.client.1.smithi192.stdout:1/25: rename d3/l7 to d3/l9 0 2024-02-16T22:56:07.342 INFO:tasks.workunit.client.1.smithi192.stdout:9/94: creat d7/df/f22 x:0 0 0 2024-02-16T22:56:07.347 INFO:tasks.workunit.client.1.smithi192.stdout:1/26: dwrite d3/f4 [0,4194304] 0 2024-02-16T22:56:07.347 INFO:tasks.workunit.client.1.smithi192.stdout:4/50: mkdir d1/d15 0 2024-02-16T22:56:07.348 INFO:tasks.workunit.client.1.smithi192.stdout:0/30: rmdir d1 39 2024-02-16T22:56:07.355 INFO:tasks.workunit.client.1.smithi192.stdout:5/68: getdents d6/d12 0 2024-02-16T22:56:07.356 INFO:tasks.workunit.client.1.smithi192.stdout:9/95: dwrite d7/d15/f20 [4194304,4194304] 0 2024-02-16T22:56:07.357 INFO:tasks.workunit.client.1.smithi192.stdout:9/96: dread - d7/df/f22 zero size 2024-02-16T22:56:07.361 INFO:tasks.workunit.client.1.smithi192.stdout:1/27: rmdir d3 39 2024-02-16T22:56:07.362 INFO:tasks.workunit.client.1.smithi192.stdout:2/74: getdents . 0 2024-02-16T22:56:07.363 INFO:tasks.workunit.client.1.smithi192.stdout:9/97: creat d7/df/f23 x:0 0 0 2024-02-16T22:56:07.366 INFO:tasks.workunit.client.1.smithi192.stdout:8/17: truncate f2 3968716 0 2024-02-16T22:56:07.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:06 smithi064 ceph-mon[29226]: pgmap v8: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 2.8 MiB/s wr, 200 op/s 2024-02-16T22:56:07.370 INFO:tasks.workunit.client.1.smithi192.stdout:2/75: symlink l1c 0 2024-02-16T22:56:07.372 INFO:tasks.workunit.client.1.smithi192.stdout:8/18: mknod c4 0 2024-02-16T22:56:07.374 INFO:tasks.workunit.client.1.smithi192.stdout:2/76: truncate fd 952472 0 2024-02-16T22:56:07.374 INFO:tasks.workunit.client.1.smithi192.stdout:8/19: mkdir d5 0 2024-02-16T22:56:07.376 INFO:tasks.workunit.client.1.smithi192.stdout:8/20: chown c4 3063 1 2024-02-16T22:56:07.377 INFO:tasks.workunit.client.1.smithi192.stdout:1/28: dwrite d3/f4 [4194304,4194304] 0 2024-02-16T22:56:07.379 INFO:tasks.workunit.client.1.smithi192.stdout:4/51: dwrite d1/f12 [0,4194304] 0 2024-02-16T22:56:07.385 INFO:tasks.workunit.client.1.smithi192.stdout:5/69: write d6/fd [212382,80520] 0 2024-02-16T22:56:07.393 INFO:tasks.workunit.client.1.smithi192.stdout:0/31: truncate d1/f5 364504 0 2024-02-16T22:56:07.394 INFO:tasks.workunit.client.1.smithi192.stdout:2/77: getdents . 0 2024-02-16T22:56:07.395 INFO:tasks.workunit.client.1.smithi192.stdout:9/98: truncate d7/d15/f1d 183347 0 2024-02-16T22:56:07.395 INFO:tasks.workunit.client.1.smithi192.stdout:4/52: write d1/d8/fb [1043555,32693] 0 2024-02-16T22:56:07.398 INFO:tasks.workunit.client.1.smithi192.stdout:1/29: dwrite d3/f4 [0,4194304] 0 2024-02-16T22:56:07.401 INFO:tasks.workunit.client.1.smithi192.stdout:0/32: dread d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:07.401 INFO:tasks.workunit.client.1.smithi192.stdout:0/33: readlink - no filename 2024-02-16T22:56:07.401 INFO:tasks.workunit.client.1.smithi192.stdout:0/34: fdatasync d1/f8 0 2024-02-16T22:56:07.402 INFO:tasks.workunit.client.1.smithi192.stdout:0/35: dread - d1/f8 zero size 2024-02-16T22:56:07.535 INFO:tasks.workunit.client.1.smithi192.stdout:7/59: sync 2024-02-16T22:56:07.540 INFO:tasks.workunit.client.1.smithi192.stdout:7/60: dwrite d1/f11 [0,4194304] 0 2024-02-16T22:56:07.543 INFO:tasks.workunit.client.1.smithi192.stdout:7/61: dread d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:07.549 INFO:tasks.workunit.client.1.smithi192.stdout:7/62: dwrite d1/f11 [0,4194304] 0 2024-02-16T22:56:07.592 INFO:tasks.workunit.client.1.smithi192.stdout:3/16: sync 2024-02-16T22:56:07.592 INFO:tasks.workunit.client.1.smithi192.stdout:6/74: sync 2024-02-16T22:56:07.593 INFO:tasks.workunit.client.1.smithi192.stdout:6/75: stat l5 0 2024-02-16T22:56:07.593 INFO:tasks.workunit.client.1.smithi192.stdout:3/17: chown l1 27 1 2024-02-16T22:56:07.640 INFO:tasks.workunit.client.1.smithi192.stdout:2/78: symlink l1d 0 2024-02-16T22:56:07.641 INFO:tasks.workunit.client.1.smithi192.stdout:2/79: write f19 [883694,115836] 0 2024-02-16T22:56:07.642 INFO:tasks.workunit.client.1.smithi192.stdout:9/99: creat d7/df/f24 x:0 0 0 2024-02-16T22:56:07.643 INFO:tasks.workunit.client.1.smithi192.stdout:4/53: mkdir d1/d8/d16 0 2024-02-16T22:56:07.652 INFO:tasks.workunit.client.1.smithi192.stdout:7/63: rename d1/d2/c15 to d1/d2/c18 0 2024-02-16T22:56:07.652 INFO:tasks.workunit.client.1.smithi192.stdout:7/64: chown d1 3059 1 2024-02-16T22:56:07.652 INFO:tasks.workunit.client.1.smithi192.stdout:7/65: chown d1/f12 2 1 2024-02-16T22:56:07.652 INFO:tasks.workunit.client.1.smithi192.stdout:7/66: truncate d1/d2/f4 4754405 0 2024-02-16T22:56:07.655 INFO:tasks.workunit.client.1.smithi192.stdout:5/70: rename d6/ce to d6/d8/c13 0 2024-02-16T22:56:07.655 INFO:tasks.workunit.client.1.smithi192.stdout:4/54: rename d1/d8 to d1/d8/d16/d17 22 2024-02-16T22:56:07.655 INFO:tasks.workunit.client.1.smithi192.stdout:4/55: fsync d1/d8/d11/f14 0 2024-02-16T22:56:07.656 INFO:tasks.workunit.client.1.smithi192.stdout:2/80: symlink l1e 0 2024-02-16T22:56:07.659 INFO:tasks.workunit.client.1.smithi192.stdout:1/30: rename d3/l9 to d3/la 0 2024-02-16T22:56:07.662 INFO:tasks.workunit.client.1.smithi192.stdout:2/81: dwrite f19 [0,4194304] 0 2024-02-16T22:56:07.663 INFO:tasks.workunit.client.1.smithi192.stdout:2/82: read f2 [1455768,81021] 0 2024-02-16T22:56:07.663 INFO:tasks.workunit.client.1.smithi192.stdout:2/83: stat c16 0 2024-02-16T22:56:07.664 INFO:tasks.workunit.client.1.smithi192.stdout:6/76: link da/df/d12/f13 da/df/d12/f17 0 2024-02-16T22:56:07.665 INFO:tasks.workunit.client.1.smithi192.stdout:1/31: dread d3/f4 [0,4194304] 0 2024-02-16T22:56:07.666 INFO:tasks.workunit.client.1.smithi192.stdout:4/56: creat d1/f18 x:0 0 0 2024-02-16T22:56:07.670 INFO:tasks.workunit.client.1.smithi192.stdout:7/67: dread d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:07.670 INFO:tasks.workunit.client.1.smithi192.stdout:3/18: link l1 l2 0 2024-02-16T22:56:07.670 INFO:tasks.workunit.client.1.smithi192.stdout:3/19: dread - no filename 2024-02-16T22:56:07.670 INFO:tasks.workunit.client.1.smithi192.stdout:3/20: fdatasync - no filename 2024-02-16T22:56:07.670 INFO:tasks.workunit.client.1.smithi192.stdout:3/21: read - no filename 2024-02-16T22:56:07.673 INFO:tasks.workunit.client.1.smithi192.stdout:6/77: write f6 [750587,47405] 0 2024-02-16T22:56:07.673 INFO:tasks.workunit.client.1.smithi192.stdout:5/71: mknod d6/c14 0 2024-02-16T22:56:07.674 INFO:tasks.workunit.client.1.smithi192.stdout:2/84: dread fb [0,4194304] 0 2024-02-16T22:56:07.676 INFO:tasks.workunit.client.1.smithi192.stdout:4/57: dwrite d1/f12 [0,4194304] 0 2024-02-16T22:56:07.676 INFO:tasks.workunit.client.1.smithi192.stdout:9/100: link d7/l9 d7/d19/l25 0 2024-02-16T22:56:07.677 INFO:tasks.workunit.client.1.smithi192.stdout:2/85: dread fb [0,4194304] 0 2024-02-16T22:56:07.678 INFO:tasks.workunit.client.1.smithi192.stdout:7/68: dread d1/d2/fd [0,4194304] 0 2024-02-16T22:56:07.680 INFO:tasks.workunit.client.1.smithi192.stdout:3/22: creat f3 x:0 0 0 2024-02-16T22:56:07.680 INFO:tasks.workunit.client.1.smithi192.stdout:3/23: read - f3 zero size 2024-02-16T22:56:07.683 INFO:tasks.workunit.client.1.smithi192.stdout:1/32: link d3/f4 d3/fb 0 2024-02-16T22:56:07.684 INFO:tasks.workunit.client.1.smithi192.stdout:6/78: mkdir da/df/d18 0 2024-02-16T22:56:07.686 INFO:tasks.workunit.client.1.smithi192.stdout:7/69: dread d1/d2/fd [0,4194304] 0 2024-02-16T22:56:07.687 INFO:tasks.workunit.client.1.smithi192.stdout:7/70: fdatasync d1/f11 0 2024-02-16T22:56:07.688 INFO:tasks.workunit.client.1.smithi192.stdout:4/58: rename d1/d8/d11/f14 to d1/f19 0 2024-02-16T22:56:07.689 INFO:tasks.workunit.client.1.smithi192.stdout:3/24: dwrite f3 [0,4194304] 0 2024-02-16T22:56:07.689 INFO:tasks.workunit.client.1.smithi192.stdout:7/71: write d1/d2/f4 [4198006,52088] 0 2024-02-16T22:56:07.690 INFO:tasks.workunit.client.1.smithi192.stdout:2/86: mknod c1f 0 2024-02-16T22:56:07.691 INFO:tasks.workunit.client.1.smithi192.stdout:4/59: chown d1/d8/d16 1500658 1 2024-02-16T22:56:07.693 INFO:tasks.workunit.client.1.smithi192.stdout:1/33: creat d3/fc x:0 0 0 2024-02-16T22:56:07.694 INFO:tasks.workunit.client.1.smithi192.stdout:5/72: mkdir d6/d12/d15 0 2024-02-16T22:56:07.694 INFO:tasks.workunit.client.1.smithi192.stdout:5/73: fsync d6/f11 0 2024-02-16T22:56:07.695 INFO:tasks.workunit.client.1.smithi192.stdout:5/74: dread - d6/f11 zero size 2024-02-16T22:56:07.699 INFO:tasks.workunit.client.1.smithi192.stdout:6/79: rename da/df/f16 to da/df/d18/f19 0 2024-02-16T22:56:07.700 INFO:tasks.workunit.client.1.smithi192.stdout:6/80: write da/df/d12/f13 [5582852,20952] 0 2024-02-16T22:56:07.701 INFO:tasks.workunit.client.1.smithi192.stdout:7/72: mkdir d1/d19 0 2024-02-16T22:56:07.702 INFO:tasks.workunit.client.1.smithi192.stdout:2/87: mknod c20 0 2024-02-16T22:56:07.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/34: symlink d3/ld 0 2024-02-16T22:56:07.706 INFO:tasks.workunit.client.1.smithi192.stdout:3/25: chown l1 1248 1 2024-02-16T22:56:07.707 INFO:tasks.workunit.client.1.smithi192.stdout:6/81: dwrite f6 [0,4194304] 0 2024-02-16T22:56:07.708 INFO:tasks.workunit.client.1.smithi192.stdout:4/60: creat d1/d8/d16/f1a x:0 0 0 2024-02-16T22:56:07.710 INFO:tasks.workunit.client.1.smithi192.stdout:2/88: rename fd to f21 0 2024-02-16T22:56:07.710 INFO:tasks.workunit.client.1.smithi192.stdout:2/89: write f1b [195640,99505] 0 2024-02-16T22:56:07.711 INFO:tasks.workunit.client.1.smithi192.stdout:7/73: rename d1/d2/l5 to d1/d2/l1a 0 2024-02-16T22:56:07.714 INFO:tasks.workunit.client.1.smithi192.stdout:7/74: dread d1/d2/fd [0,4194304] 0 2024-02-16T22:56:07.716 INFO:tasks.workunit.client.1.smithi192.stdout:3/26: unlink l1 0 2024-02-16T22:56:07.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/61: mknod d1/d15/c1b 0 2024-02-16T22:56:07.719 INFO:tasks.workunit.client.1.smithi192.stdout:7/75: write d1/d2/fd [1922256,2238] 0 2024-02-16T22:56:07.723 INFO:tasks.workunit.client.1.smithi192.stdout:4/62: rename d1/f12 to d1/f1c 0 2024-02-16T22:56:07.723 INFO:tasks.workunit.client.1.smithi192.stdout:4/63: dread - d1/d8/d16/f1a zero size 2024-02-16T22:56:07.724 INFO:tasks.workunit.client.1.smithi192.stdout:7/76: rmdir d1/d2/de 39 2024-02-16T22:56:07.726 INFO:tasks.workunit.client.1.smithi192.stdout:4/64: creat d1/d8/d11/f1d x:0 0 0 2024-02-16T22:56:07.727 INFO:tasks.workunit.client.1.smithi192.stdout:4/65: write d1/f1c [2908948,116336] 0 2024-02-16T22:56:07.728 INFO:tasks.workunit.client.1.smithi192.stdout:4/66: write d1/f4 [1556787,31191] 0 2024-02-16T22:56:07.731 INFO:tasks.workunit.client.1.smithi192.stdout:4/67: truncate d1/f18 477196 0 2024-02-16T22:56:07.733 INFO:tasks.workunit.client.1.smithi192.stdout:0/36: dread d1/f5 [0,4194304] 0 2024-02-16T22:56:07.737 INFO:tasks.workunit.client.1.smithi192.stdout:7/77: chown d1/d2/c18 611759 1 2024-02-16T22:56:07.738 INFO:tasks.workunit.client.1.smithi192.stdout:4/68: dwrite d1/f4 [0,4194304] 0 2024-02-16T22:56:07.746 INFO:tasks.workunit.client.1.smithi192.stdout:7/78: creat d1/d2/f1b x:0 0 0 2024-02-16T22:56:07.746 INFO:tasks.workunit.client.1.smithi192.stdout:0/37: dwrite d1/f8 [0,4194304] 0 2024-02-16T22:56:07.747 INFO:tasks.workunit.client.1.smithi192.stdout:5/75: rmdir d6 39 2024-02-16T22:56:07.748 INFO:tasks.workunit.client.1.smithi192.stdout:3/27: write f3 [4861073,25498] 0 2024-02-16T22:56:07.748 INFO:tasks.workunit.client.1.smithi192.stdout:5/76: fsync f2 0 2024-02-16T22:56:07.752 INFO:tasks.workunit.client.1.smithi192.stdout:4/69: dwrite d1/d8/fb [0,4194304] 0 2024-02-16T22:56:07.753 INFO:tasks.workunit.client.1.smithi192.stdout:3/28: creat f4 x:0 0 0 2024-02-16T22:56:07.757 INFO:tasks.workunit.client.1.smithi192.stdout:7/79: dwrite d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:07.757 INFO:tasks.workunit.client.1.smithi192.stdout:5/77: symlink d6/d8/l16 0 2024-02-16T22:56:07.763 INFO:tasks.workunit.client.1.smithi192.stdout:2/90: write fb [539213,111381] 0 2024-02-16T22:56:07.766 INFO:tasks.workunit.client.1.smithi192.stdout:7/80: chown d1/d2/de/l10 26540612 1 2024-02-16T22:56:07.769 INFO:tasks.workunit.client.1.smithi192.stdout:2/91: dwrite fa [0,4194304] 0 2024-02-16T22:56:07.772 INFO:tasks.workunit.client.1.smithi192.stdout:2/92: write fb [1644996,44145] 0 2024-02-16T22:56:07.775 INFO:tasks.workunit.client.1.smithi192.stdout:0/38: getdents d1/d3 0 2024-02-16T22:56:07.777 INFO:tasks.workunit.client.1.smithi192.stdout:7/81: symlink d1/l1c 0 2024-02-16T22:56:07.780 INFO:tasks.workunit.client.1.smithi192.stdout:3/29: mknod c5 0 2024-02-16T22:56:07.782 INFO:tasks.workunit.client.1.smithi192.stdout:0/39: mknod d1/d3/c9 0 2024-02-16T22:56:07.783 INFO:tasks.workunit.client.1.smithi192.stdout:9/101: dwrite d7/d15/f1d [0,4194304] 0 2024-02-16T22:56:07.788 INFO:tasks.workunit.client.1.smithi192.stdout:5/78: link d6/d8/c13 d6/d12/d15/c17 0 2024-02-16T22:56:07.792 INFO:tasks.workunit.client.1.smithi192.stdout:8/21: dwrite f2 [0,4194304] 0 2024-02-16T22:56:07.794 INFO:tasks.workunit.client.1.smithi192.stdout:2/93: dwrite fb [0,4194304] 0 2024-02-16T22:56:07.799 INFO:tasks.workunit.client.1.smithi192.stdout:9/102: mknod d7/d15/c26 0 2024-02-16T22:56:07.802 INFO:tasks.workunit.client.1.smithi192.stdout:4/70: dwrite d1/d8/fc [0,4194304] 0 2024-02-16T22:56:07.803 INFO:tasks.workunit.client.1.smithi192.stdout:7/82: dwrite d1/d2/fd [0,4194304] 0 2024-02-16T22:56:07.804 INFO:tasks.workunit.client.1.smithi192.stdout:0/40: write d1/f5 [753998,90048] 0 2024-02-16T22:56:07.804 INFO:tasks.workunit.client.1.smithi192.stdout:9/103: fdatasync d7/f12 0 2024-02-16T22:56:07.807 INFO:tasks.workunit.client.1.smithi192.stdout:2/94: write f1b [1075884,72575] 0 2024-02-16T22:56:07.808 INFO:tasks.workunit.client.1.smithi192.stdout:3/30: dread f3 [0,4194304] 0 2024-02-16T22:56:07.812 INFO:tasks.workunit.client.1.smithi192.stdout:0/41: write d1/f5 [1202037,23698] 0 2024-02-16T22:56:07.812 INFO:tasks.workunit.client.1.smithi192.stdout:2/95: fdatasync f4 0 2024-02-16T22:56:07.814 INFO:tasks.workunit.client.1.smithi192.stdout:3/31: fdatasync f4 0 2024-02-16T22:56:07.814 INFO:tasks.workunit.client.1.smithi192.stdout:7/83: unlink d1/d2/fd 0 2024-02-16T22:56:07.817 INFO:tasks.workunit.client.1.smithi192.stdout:7/84: chown d1/d2/cf 107 1 2024-02-16T22:56:07.817 INFO:tasks.workunit.client.1.smithi192.stdout:9/104: creat d7/d15/f27 x:0 0 0 2024-02-16T22:56:07.817 INFO:tasks.workunit.client.1.smithi192.stdout:8/22: dwrite f2 [4194304,4194304] 0 2024-02-16T22:56:07.819 INFO:tasks.workunit.client.1.smithi192.stdout:7/85: read - d1/d2/f1b zero size 2024-02-16T22:56:07.819 INFO:tasks.workunit.client.1.smithi192.stdout:2/96: mkdir d22 0 2024-02-16T22:56:07.820 INFO:tasks.workunit.client.1.smithi192.stdout:0/42: creat d1/fa x:0 0 0 2024-02-16T22:56:07.822 INFO:tasks.workunit.client.1.smithi192.stdout:4/71: dwrite d1/f18 [0,4194304] 0 2024-02-16T22:56:07.824 INFO:tasks.workunit.client.1.smithi192.stdout:2/97: write f10 [60324,111697] 0 2024-02-16T22:56:07.825 INFO:tasks.workunit.client.1.smithi192.stdout:9/105: chown d7/c1b 0 1 2024-02-16T22:56:07.825 INFO:tasks.workunit.client.1.smithi192.stdout:3/32: mknod c6 0 2024-02-16T22:56:07.826 INFO:tasks.workunit.client.1.smithi192.stdout:8/23: symlink d5/l6 0 2024-02-16T22:56:07.827 INFO:tasks.workunit.client.1.smithi192.stdout:8/24: stat d5 0 2024-02-16T22:56:07.829 INFO:tasks.workunit.client.1.smithi192.stdout:0/43: mknod d1/d2/cb 0 2024-02-16T22:56:07.831 INFO:tasks.workunit.client.1.smithi192.stdout:9/106: mkdir d7/d19/d28 0 2024-02-16T22:56:07.833 INFO:tasks.workunit.client.1.smithi192.stdout:7/86: creat d1/d19/f1d x:0 0 0 2024-02-16T22:56:07.834 INFO:tasks.workunit.client.1.smithi192.stdout:2/98: dwrite f8 [0,4194304] 0 2024-02-16T22:56:07.836 INFO:tasks.workunit.client.1.smithi192.stdout:0/44: creat d1/d3/fc x:0 0 0 2024-02-16T22:56:07.836 INFO:tasks.workunit.client.1.smithi192.stdout:8/25: dread f2 [4194304,4194304] 0 2024-02-16T22:56:07.838 INFO:tasks.workunit.client.1.smithi192.stdout:3/33: symlink l7 0 2024-02-16T22:56:07.840 INFO:tasks.workunit.client.1.smithi192.stdout:8/26: rename d5/l6 to d5/l7 0 2024-02-16T22:56:07.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/99: mkdir d22/d23 0 2024-02-16T22:56:07.844 INFO:tasks.workunit.client.1.smithi192.stdout:7/87: creat d1/d2/f1e x:0 0 0 2024-02-16T22:56:07.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/100: chown fe 3035 1 2024-02-16T22:56:07.845 INFO:tasks.workunit.client.1.smithi192.stdout:8/27: write f2 [3844729,125521] 0 2024-02-16T22:56:07.845 INFO:tasks.workunit.client.1.smithi192.stdout:9/107: getdents d7/d15 0 2024-02-16T22:56:07.845 INFO:tasks.workunit.client.1.smithi192.stdout:0/45: dwrite d1/f5 [0,4194304] 0 2024-02-16T22:56:07.846 INFO:tasks.workunit.client.1.smithi192.stdout:7/88: stat d1/l1c 0 2024-02-16T22:56:07.846 INFO:tasks.workunit.client.1.smithi192.stdout:2/101: chown l1d 881058 1 2024-02-16T22:56:07.848 INFO:tasks.workunit.client.1.smithi192.stdout:4/72: truncate d1/d8/fc 1165890 0 2024-02-16T22:56:07.849 INFO:tasks.workunit.client.1.smithi192.stdout:2/102: chown c11 7147 1 2024-02-16T22:56:07.850 INFO:tasks.workunit.client.1.smithi192.stdout:0/46: creat d1/fd x:0 0 0 2024-02-16T22:56:07.851 INFO:tasks.workunit.client.1.smithi192.stdout:7/89: write d1/d2/f4 [418643,32172] 0 2024-02-16T22:56:07.852 INFO:tasks.workunit.client.1.smithi192.stdout:9/108: mknod d7/d19/c29 0 2024-02-16T22:56:07.852 INFO:tasks.workunit.client.1.smithi192.stdout:0/47: stat d1/fd 0 2024-02-16T22:56:07.854 INFO:tasks.workunit.client.1.smithi192.stdout:9/109: truncate d7/fe 360961 0 2024-02-16T22:56:07.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/34: dwrite f4 [0,4194304] 0 2024-02-16T22:56:07.856 INFO:tasks.workunit.client.1.smithi192.stdout:9/110: write d7/d15/f27 [569016,35131] 0 2024-02-16T22:56:07.859 INFO:tasks.workunit.client.1.smithi192.stdout:4/73: creat d1/d8/f1e x:0 0 0 2024-02-16T22:56:07.859 INFO:tasks.workunit.client.1.smithi192.stdout:9/111: write d7/df/f22 [479546,94973] 0 2024-02-16T22:56:07.859 INFO:tasks.workunit.client.1.smithi192.stdout:8/28: chown d5/l7 870807486 1 2024-02-16T22:56:07.860 INFO:tasks.workunit.client.1.smithi192.stdout:7/90: mkdir d1/d1f 0 2024-02-16T22:56:07.864 INFO:tasks.workunit.client.1.smithi192.stdout:8/29: creat d5/f8 x:0 0 0 2024-02-16T22:56:07.868 INFO:tasks.workunit.client.1.smithi192.stdout:2/103: rmdir d22/d23 0 2024-02-16T22:56:07.868 INFO:tasks.workunit.client.1.smithi192.stdout:9/112: dwrite f1 [0,4194304] 0 2024-02-16T22:56:07.868 INFO:tasks.workunit.client.1.smithi192.stdout:9/113: stat d7/l17 0 2024-02-16T22:56:07.869 INFO:tasks.workunit.client.1.smithi192.stdout:9/114: dread - d7/f12 zero size 2024-02-16T22:56:07.873 INFO:tasks.workunit.client.1.smithi192.stdout:3/35: link f4 f8 0 2024-02-16T22:56:07.874 INFO:tasks.workunit.client.1.smithi192.stdout:3/36: chown l7 411954 1 2024-02-16T22:56:07.875 INFO:tasks.workunit.client.1.smithi192.stdout:8/30: mkdir d5/d9 0 2024-02-16T22:56:07.877 INFO:tasks.workunit.client.1.smithi192.stdout:2/104: write f5 [4888650,56632] 0 2024-02-16T22:56:07.879 INFO:tasks.workunit.client.1.smithi192.stdout:4/74: link d1/d8/d11/f1d d1/d8/d16/f1f 0 2024-02-16T22:56:07.880 INFO:tasks.workunit.client.1.smithi192.stdout:2/105: dread f21 [0,4194304] 0 2024-02-16T22:56:07.880 INFO:tasks.workunit.client.1.smithi192.stdout:9/115: chown d7/d19/l25 15090 1 2024-02-16T22:56:07.881 INFO:tasks.workunit.client.1.smithi192.stdout:9/116: chown d7/c1b 6014 1 2024-02-16T22:56:07.881 INFO:tasks.workunit.client.1.smithi192.stdout:9/117: fdatasync d7/df/f16 0 2024-02-16T22:56:07.886 INFO:tasks.workunit.client.1.smithi192.stdout:2/106: dread fe [0,4194304] 0 2024-02-16T22:56:07.887 INFO:tasks.workunit.client.1.smithi192.stdout:4/75: write d1/d8/d16/f1f [846176,121724] 0 2024-02-16T22:56:07.890 INFO:tasks.workunit.client.1.smithi192.stdout:9/118: mknod d7/d15/c2a 0 2024-02-16T22:56:07.891 INFO:tasks.workunit.client.1.smithi192.stdout:3/37: dwrite f3 [4194304,4194304] 0 2024-02-16T22:56:07.895 INFO:tasks.workunit.client.1.smithi192.stdout:4/76: mknod d1/d8/d11/c20 0 2024-02-16T22:56:07.896 INFO:tasks.workunit.client.1.smithi192.stdout:7/91: dwrite d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:07.913 INFO:tasks.workunit.client.1.smithi192.stdout:8/31: truncate f2 6889179 0 2024-02-16T22:56:07.919 INFO:tasks.workunit.client.1.smithi192.stdout:3/38: creat f9 x:0 0 0 2024-02-16T22:56:07.920 INFO:tasks.workunit.client.1.smithi192.stdout:8/32: dwrite d5/f8 [0,4194304] 0 2024-02-16T22:56:07.920 INFO:tasks.workunit.client.1.smithi192.stdout:4/77: rmdir d1/d8/d11 39 2024-02-16T22:56:07.921 INFO:tasks.workunit.client.1.smithi192.stdout:4/78: dread d1/d8/fe [0,4194304] 0 2024-02-16T22:56:07.925 INFO:tasks.workunit.client.1.smithi192.stdout:2/107: truncate f5 4847865 0 2024-02-16T22:56:07.926 INFO:tasks.workunit.client.1.smithi192.stdout:2/108: write f19 [3095046,85917] 0 2024-02-16T22:56:07.927 INFO:tasks.workunit.client.1.smithi192.stdout:2/109: readlink l17 0 2024-02-16T22:56:07.927 INFO:tasks.workunit.client.1.smithi192.stdout:7/92: getdents d1/d19 0 2024-02-16T22:56:07.928 INFO:tasks.workunit.client.1.smithi192.stdout:2/110: stat f1 0 2024-02-16T22:56:07.928 INFO:tasks.workunit.client.1.smithi192.stdout:2/111: chown l15 13596957 1 2024-02-16T22:56:07.929 INFO:tasks.workunit.client.1.smithi192.stdout:8/33: dwrite d5/f8 [0,4194304] 0 2024-02-16T22:56:07.932 INFO:tasks.workunit.client.1.smithi192.stdout:9/119: write f2 [19063,127421] 0 2024-02-16T22:56:07.938 INFO:tasks.workunit.client.1.smithi192.stdout:3/39: mkdir da 0 2024-02-16T22:56:07.938 INFO:tasks.workunit.client.1.smithi192.stdout:2/112: dread f19 [0,4194304] 0 2024-02-16T22:56:07.939 INFO:tasks.workunit.client.1.smithi192.stdout:2/113: chown f2 9 1 2024-02-16T22:56:07.939 INFO:tasks.workunit.client.1.smithi192.stdout:2/114: chown l1d 57 1 2024-02-16T22:56:07.941 INFO:tasks.workunit.client.1.smithi192.stdout:9/120: dwrite d7/f12 [0,4194304] 0 2024-02-16T22:56:07.941 INFO:tasks.workunit.client.1.smithi192.stdout:8/34: symlink d5/la 0 2024-02-16T22:56:07.946 INFO:tasks.workunit.client.1.smithi192.stdout:7/93: getdents d1/d1f 0 2024-02-16T22:56:07.946 INFO:tasks.workunit.client.1.smithi192.stdout:9/121: dread d7/d19/f21 [0,4194304] 0 2024-02-16T22:56:07.950 INFO:tasks.workunit.client.1.smithi192.stdout:3/40: dwrite f8 [0,4194304] 0 2024-02-16T22:56:07.951 INFO:tasks.workunit.client.1.smithi192.stdout:3/41: read f3 [1061587,28177] 0 2024-02-16T22:56:07.952 INFO:tasks.workunit.client.1.smithi192.stdout:2/115: truncate fb 114515 0 2024-02-16T22:56:07.952 INFO:tasks.workunit.client.1.smithi192.stdout:3/42: write f4 [779890,90689] 0 2024-02-16T22:56:07.954 INFO:tasks.workunit.client.1.smithi192.stdout:8/35: mkdir d5/db 0 2024-02-16T22:56:07.958 INFO:tasks.workunit.client.1.smithi192.stdout:7/94: creat d1/d2/de/f20 x:0 0 0 2024-02-16T22:56:07.961 INFO:tasks.workunit.client.1.smithi192.stdout:9/122: symlink d7/d19/l2b 0 2024-02-16T22:56:07.970 INFO:tasks.workunit.client.1.smithi192.stdout:7/95: mknod d1/c21 0 2024-02-16T22:56:07.972 INFO:tasks.workunit.client.1.smithi192.stdout:9/123: fdatasync d7/f8 0 2024-02-16T22:56:07.974 INFO:tasks.workunit.client.1.smithi192.stdout:7/96: dread d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:07.975 INFO:tasks.workunit.client.1.smithi192.stdout:3/43: symlink da/lb 0 2024-02-16T22:56:07.978 INFO:tasks.workunit.client.1.smithi192.stdout:9/124: dwrite f2 [0,4194304] 0 2024-02-16T22:56:07.979 INFO:tasks.workunit.client.1.smithi192.stdout:1/35: sync 2024-02-16T22:56:07.979 INFO:tasks.workunit.client.1.smithi192.stdout:9/125: chown d7/d19/f21 13741 1 2024-02-16T22:56:07.982 INFO:tasks.workunit.client.1.smithi192.stdout:7/97: readlink d1/d2/de/l16 0 2024-02-16T22:56:07.987 INFO:tasks.workunit.client.1.smithi192.stdout:1/36: chown d3/c8 45 1 2024-02-16T22:56:07.987 INFO:tasks.workunit.client.1.smithi192.stdout:1/37: chown d3/c8 0 1 2024-02-16T22:56:07.987 INFO:tasks.workunit.client.1.smithi192.stdout:1/38: dread - d3/fc zero size 2024-02-16T22:56:07.990 INFO:tasks.workunit.client.1.smithi192.stdout:9/126: rename d7/df/f14 to d7/d19/d28/f2c 0 2024-02-16T22:56:07.991 INFO:tasks.workunit.client.1.smithi192.stdout:9/127: dread - d7/df/f16 zero size 2024-02-16T22:56:07.991 INFO:tasks.workunit.client.1.smithi192.stdout:1/39: dread d3/fb [0,4194304] 0 2024-02-16T22:56:07.991 INFO:tasks.workunit.client.1.smithi192.stdout:1/40: dread - d3/fc zero size 2024-02-16T22:56:08.045 INFO:tasks.workunit.client.1.smithi192.stdout:7/98: creat d1/d1f/f22 x:0 0 0 2024-02-16T22:56:08.046 INFO:tasks.workunit.client.1.smithi192.stdout:9/128: dwrite d7/d15/f27 [0,4194304] 0 2024-02-16T22:56:08.046 INFO:tasks.workunit.client.1.smithi192.stdout:3/44: truncate f8 1169483 0 2024-02-16T22:56:08.047 INFO:tasks.workunit.client.1.smithi192.stdout:1/41: write d3/f4 [8268446,81352] 0 2024-02-16T22:56:08.047 INFO:tasks.workunit.client.1.smithi192.stdout:3/45: fsync f9 0 2024-02-16T22:56:08.051 INFO:tasks.workunit.client.1.smithi192.stdout:1/42: dread d3/fb [4194304,4194304] 0 2024-02-16T22:56:08.058 INFO:tasks.workunit.client.1.smithi192.stdout:1/43: mkdir d3/de 0 2024-02-16T22:56:08.061 INFO:tasks.workunit.client.1.smithi192.stdout:7/99: creat d1/d2/f23 x:0 0 0 2024-02-16T22:56:08.064 INFO:tasks.workunit.client.1.smithi192.stdout:9/129: link d7/d19/l2b d7/d19/l2d 0 2024-02-16T22:56:08.065 INFO:tasks.workunit.client.1.smithi192.stdout:1/44: creat d3/de/ff x:0 0 0 2024-02-16T22:56:08.067 INFO:tasks.workunit.client.1.smithi192.stdout:9/130: creat d7/f2e x:0 0 0 2024-02-16T22:56:08.068 INFO:tasks.workunit.client.1.smithi192.stdout:7/100: dwrite d1/d2/f23 [0,4194304] 0 2024-02-16T22:56:08.069 INFO:tasks.workunit.client.1.smithi192.stdout:1/45: mknod d3/c10 0 2024-02-16T22:56:08.071 INFO:tasks.workunit.client.1.smithi192.stdout:7/101: truncate d1/d2/f6 4289513 0 2024-02-16T22:56:08.072 INFO:tasks.workunit.client.1.smithi192.stdout:1/46: chown d3/la 5337442 1 2024-02-16T22:56:08.073 INFO:tasks.workunit.client.1.smithi192.stdout:7/102: chown d1/d2/de/f20 15568061 1 2024-02-16T22:56:08.077 INFO:tasks.workunit.client.1.smithi192.stdout:9/131: dwrite d7/df/f22 [0,4194304] 0 2024-02-16T22:56:08.080 INFO:tasks.workunit.client.1.smithi192.stdout:7/103: dwrite d1/f11 [0,4194304] 0 2024-02-16T22:56:08.080 INFO:tasks.workunit.client.1.smithi192.stdout:9/132: mkdir d7/df/d2f 0 2024-02-16T22:56:08.085 INFO:tasks.workunit.client.1.smithi192.stdout:6/82: sync 2024-02-16T22:56:08.087 INFO:tasks.workunit.client.1.smithi192.stdout:9/133: mknod d7/d15/c30 0 2024-02-16T22:56:08.088 INFO:tasks.workunit.client.1.smithi192.stdout:9/134: dread d7/fe [0,4194304] 0 2024-02-16T22:56:08.092 INFO:tasks.workunit.client.1.smithi192.stdout:7/104: unlink d1/d2/ca 0 2024-02-16T22:56:08.094 INFO:tasks.workunit.client.1.smithi192.stdout:6/83: rename c3 to da/df/d12/d15/c1a 0 2024-02-16T22:56:08.098 INFO:tasks.workunit.client.1.smithi192.stdout:9/135: creat d7/f31 x:0 0 0 2024-02-16T22:56:08.099 INFO:tasks.workunit.client.1.smithi192.stdout:9/136: chown d7/d15 177260753 1 2024-02-16T22:56:08.100 INFO:tasks.workunit.client.1.smithi192.stdout:7/105: dwrite d1/d2/f1b [0,4194304] 0 2024-02-16T22:56:08.100 INFO:tasks.workunit.client.1.smithi192.stdout:6/84: mkdir da/df/d12/d15/d1b 0 2024-02-16T22:56:08.101 INFO:tasks.workunit.client.1.smithi192.stdout:7/106: fdatasync d1/d2/f23 0 2024-02-16T22:56:08.105 INFO:tasks.workunit.client.1.smithi192.stdout:9/137: unlink d7/df/l18 0 2024-02-16T22:56:08.113 INFO:tasks.workunit.client.1.smithi192.stdout:9/138: creat d7/d15/f32 x:0 0 0 2024-02-16T22:56:08.114 INFO:tasks.workunit.client.1.smithi192.stdout:7/107: dread d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:08.124 INFO:tasks.workunit.client.1.smithi192.stdout:7/108: getdents d1/d2/de 0 2024-02-16T22:56:08.131 INFO:tasks.workunit.client.1.smithi192.stdout:7/109: dwrite d1/d19/f1d [0,4194304] 0 2024-02-16T22:56:08.131 INFO:tasks.workunit.client.1.smithi192.stdout:7/110: readlink d1/l1c 0 2024-02-16T22:56:08.140 INFO:tasks.workunit.client.1.smithi192.stdout:3/46: truncate f8 459249 0 2024-02-16T22:56:08.140 INFO:tasks.workunit.client.1.smithi192.stdout:3/47: creat da/fc x:0 0 0 2024-02-16T22:56:08.141 INFO:tasks.workunit.client.1.smithi192.stdout:3/48: mkdir da/dd 0 2024-02-16T22:56:08.148 INFO:tasks.workunit.client.1.smithi192.stdout:9/139: write f2 [4699375,53084] 0 2024-02-16T22:56:08.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/49: dwrite da/fc [0,4194304] 0 2024-02-16T22:56:08.151 INFO:tasks.workunit.client.1.smithi192.stdout:7/111: truncate d1/d19/f1d 1014641 0 2024-02-16T22:56:08.153 INFO:tasks.workunit.client.1.smithi192.stdout:9/140: rename d7/d15/f1d to d7/d15/f33 0 2024-02-16T22:56:08.153 INFO:tasks.workunit.client.1.smithi192.stdout:6/85: dwrite da/fb [0,4194304] 0 2024-02-16T22:56:08.156 INFO:tasks.workunit.client.1.smithi192.stdout:3/50: creat da/fe x:0 0 0 2024-02-16T22:56:08.156 INFO:tasks.workunit.client.1.smithi192.stdout:7/112: rmdir d1/d19 39 2024-02-16T22:56:08.157 INFO:tasks.workunit.client.1.smithi192.stdout:6/86: fdatasync da/df/d18/f19 0 2024-02-16T22:56:08.157 INFO:tasks.workunit.client.1.smithi192.stdout:7/113: dread - d1/d2/f1e zero size 2024-02-16T22:56:08.158 INFO:tasks.workunit.client.1.smithi192.stdout:6/87: write f9 [4924457,24622] 0 2024-02-16T22:56:08.158 INFO:tasks.workunit.client.1.smithi192.stdout:9/141: creat d7/df/d2f/f34 x:0 0 0 2024-02-16T22:56:08.162 INFO:tasks.workunit.client.1.smithi192.stdout:6/88: unlink l8 0 2024-02-16T22:56:08.162 INFO:tasks.workunit.client.1.smithi192.stdout:9/142: symlink d7/l35 0 2024-02-16T22:56:08.164 INFO:tasks.workunit.client.1.smithi192.stdout:6/89: unlink da/df/d18/f19 0 2024-02-16T22:56:08.164 INFO:tasks.workunit.client.1.smithi192.stdout:6/90: fdatasync f2 0 2024-02-16T22:56:08.166 INFO:tasks.workunit.client.1.smithi192.stdout:6/91: creat da/df/f1c x:0 0 0 2024-02-16T22:56:08.167 INFO:tasks.workunit.client.1.smithi192.stdout:7/114: write d1/d19/f1d [1670836,93624] 0 2024-02-16T22:56:08.169 INFO:tasks.workunit.client.1.smithi192.stdout:3/51: stat f4 0 2024-02-16T22:56:08.169 INFO:tasks.workunit.client.1.smithi192.stdout:7/115: creat d1/d2/de/f24 x:0 0 0 2024-02-16T22:56:08.170 INFO:tasks.workunit.client.1.smithi192.stdout:3/52: mkdir da/df 0 2024-02-16T22:56:08.171 INFO:tasks.workunit.client.1.smithi192.stdout:3/53: fdatasync f9 0 2024-02-16T22:56:08.173 INFO:tasks.workunit.client.1.smithi192.stdout:3/54: dread f4 [0,4194304] 0 2024-02-16T22:56:08.175 INFO:tasks.workunit.client.1.smithi192.stdout:6/92: dwrite da/df/f1c [0,4194304] 0 2024-02-16T22:56:08.176 INFO:tasks.workunit.client.1.smithi192.stdout:7/116: rename d1/d1f/f22 to d1/d2/f25 0 2024-02-16T22:56:08.177 INFO:tasks.workunit.client.1.smithi192.stdout:3/55: write da/fc [2428084,121772] 0 2024-02-16T22:56:08.180 INFO:tasks.workunit.client.1.smithi192.stdout:9/143: truncate d7/df/f22 852266 0 2024-02-16T22:56:08.181 INFO:tasks.workunit.client.1.smithi192.stdout:9/144: write f0 [7282939,57726] 0 2024-02-16T22:56:08.181 INFO:tasks.workunit.client.1.smithi192.stdout:3/56: mknod da/c10 0 2024-02-16T22:56:08.181 INFO:tasks.workunit.client.1.smithi192.stdout:9/145: read - d7/df/f16 zero size 2024-02-16T22:56:08.184 INFO:tasks.workunit.client.1.smithi192.stdout:3/57: getdents da/dd 0 2024-02-16T22:56:08.187 INFO:tasks.workunit.client.1.smithi192.stdout:3/58: mkdir da/d11 0 2024-02-16T22:56:08.190 INFO:tasks.workunit.client.1.smithi192.stdout:9/146: write d7/d19/f21 [1004166,116554] 0 2024-02-16T22:56:08.191 INFO:tasks.workunit.client.1.smithi192.stdout:9/147: read f1 [3155779,57352] 0 2024-02-16T22:56:08.191 INFO:tasks.workunit.client.1.smithi192.stdout:3/59: rmdir da/dd 0 2024-02-16T22:56:08.193 INFO:tasks.workunit.client.1.smithi192.stdout:9/148: unlink c4 0 2024-02-16T22:56:08.194 INFO:tasks.workunit.client.1.smithi192.stdout:3/60: creat da/f12 x:0 0 0 2024-02-16T22:56:08.197 INFO:tasks.workunit.client.1.smithi192.stdout:5/79: sync 2024-02-16T22:56:08.197 INFO:tasks.workunit.client.1.smithi192.stdout:0/48: sync 2024-02-16T22:56:08.197 INFO:tasks.workunit.client.1.smithi192.stdout:4/79: sync 2024-02-16T22:56:08.198 INFO:tasks.workunit.client.1.smithi192.stdout:5/80: stat d6 0 2024-02-16T22:56:08.198 INFO:tasks.workunit.client.1.smithi192.stdout:4/80: fdatasync d1/d8/f1e 0 2024-02-16T22:56:08.198 INFO:tasks.workunit.client.1.smithi192.stdout:6/93: fsync da/df/f1c 0 2024-02-16T22:56:08.198 INFO:tasks.workunit.client.1.smithi192.stdout:0/49: dread - d1/f7 zero size 2024-02-16T22:56:08.199 INFO:tasks.workunit.client.1.smithi192.stdout:0/50: read d1/f5 [2888731,48489] 0 2024-02-16T22:56:08.200 INFO:tasks.workunit.client.1.smithi192.stdout:6/94: creat da/df/f1d x:0 0 0 2024-02-16T22:56:08.202 INFO:tasks.workunit.client.1.smithi192.stdout:4/81: dread d1/f5 [0,4194304] 0 2024-02-16T22:56:08.261 INFO:tasks.workunit.client.1.smithi192.stdout:2/116: sync 2024-02-16T22:56:08.261 INFO:tasks.workunit.client.1.smithi192.stdout:8/36: sync 2024-02-16T22:56:08.263 INFO:tasks.workunit.client.1.smithi192.stdout:8/37: dread d5/f8 [0,4194304] 0 2024-02-16T22:56:08.266 INFO:tasks.workunit.client.1.smithi192.stdout:8/38: dread d5/f8 [0,4194304] 0 2024-02-16T22:56:08.295 INFO:tasks.workunit.client.1.smithi192.stdout:5/81: truncate d6/fc 1984656 0 2024-02-16T22:56:08.295 INFO:tasks.workunit.client.1.smithi192.stdout:0/51: mkdir d1/d3/de 0 2024-02-16T22:56:08.297 INFO:tasks.workunit.client.1.smithi192.stdout:4/82: rename d1/c13 to d1/d8/d16/c21 0 2024-02-16T22:56:08.299 INFO:tasks.workunit.client.1.smithi192.stdout:2/117: unlink l15 0 2024-02-16T22:56:08.299 INFO:tasks.workunit.client.1.smithi192.stdout:3/61: write f4 [363133,125982] 0 2024-02-16T22:56:08.300 INFO:tasks.workunit.client.1.smithi192.stdout:3/62: truncate f9 271658 0 2024-02-16T22:56:08.301 INFO:tasks.workunit.client.1.smithi192.stdout:9/149: truncate d7/d15/f27 1341111 0 2024-02-16T22:56:08.301 INFO:tasks.workunit.client.1.smithi192.stdout:1/47: sync 2024-02-16T22:56:08.301 INFO:tasks.workunit.client.1.smithi192.stdout:7/117: sync 2024-02-16T22:56:08.302 INFO:tasks.workunit.client.1.smithi192.stdout:9/150: chown d7/c11 0 1 2024-02-16T22:56:08.302 INFO:tasks.workunit.client.1.smithi192.stdout:5/82: creat d6/d12/f18 x:0 0 0 2024-02-16T22:56:08.303 INFO:tasks.workunit.client.1.smithi192.stdout:8/39: dread f2 [4194304,4194304] 0 2024-02-16T22:56:08.303 INFO:tasks.workunit.client.1.smithi192.stdout:9/151: chown d7/d15/f20 0 1 2024-02-16T22:56:08.303 INFO:tasks.workunit.client.1.smithi192.stdout:5/83: chown d6/f11 3404 1 2024-02-16T22:56:08.303 INFO:tasks.workunit.client.1.smithi192.stdout:6/95: link f6 da/df/d12/d15/d1b/f1e 0 2024-02-16T22:56:08.306 INFO:tasks.workunit.client.1.smithi192.stdout:1/48: symlink d3/l11 0 2024-02-16T22:56:08.307 INFO:tasks.workunit.client.1.smithi192.stdout:7/118: symlink d1/d1f/l26 0 2024-02-16T22:56:08.308 INFO:tasks.workunit.client.1.smithi192.stdout:2/118: dread f19 [0,4194304] 0 2024-02-16T22:56:08.309 INFO:tasks.workunit.client.1.smithi192.stdout:7/119: write d1/d2/de/f20 [1013663,13848] 0 2024-02-16T22:56:08.309 INFO:tasks.workunit.client.1.smithi192.stdout:2/119: chown c16 47387 1 2024-02-16T22:56:08.312 INFO:tasks.workunit.client.1.smithi192.stdout:8/40: dread d5/f8 [0,4194304] 0 2024-02-16T22:56:08.314 INFO:tasks.workunit.client.1.smithi192.stdout:1/49: unlink d3/la 0 2024-02-16T22:56:08.315 INFO:tasks.workunit.client.1.smithi192.stdout:1/50: write d3/de/ff [566031,34014] 0 2024-02-16T22:56:08.316 INFO:tasks.workunit.client.1.smithi192.stdout:1/51: write d3/fc [272711,127441] 0 2024-02-16T22:56:08.319 INFO:tasks.workunit.client.1.smithi192.stdout:0/52: link d1/d2/cb d1/d3/cf 0 2024-02-16T22:56:08.322 INFO:tasks.workunit.client.1.smithi192.stdout:3/63: dwrite f3 [0,4194304] 0 2024-02-16T22:56:08.323 INFO:tasks.workunit.client.1.smithi192.stdout:6/96: creat da/f1f x:0 0 0 2024-02-16T22:56:08.324 INFO:tasks.workunit.client.1.smithi192.stdout:1/52: symlink d3/de/l12 0 2024-02-16T22:56:08.326 INFO:tasks.workunit.client.1.smithi192.stdout:1/53: chown d3/de 648 1 2024-02-16T22:56:08.328 INFO:tasks.workunit.client.1.smithi192.stdout:5/84: link c4 d6/d12/c19 0 2024-02-16T22:56:08.329 INFO:tasks.workunit.client.1.smithi192.stdout:5/85: write d6/f10 [1312907,79048] 0 2024-02-16T22:56:08.329 INFO:tasks.workunit.client.1.smithi192.stdout:2/120: dwrite f4 [0,4194304] 0 2024-02-16T22:56:08.329 INFO:tasks.workunit.client.1.smithi192.stdout:6/97: unlink f2 0 2024-02-16T22:56:08.330 INFO:tasks.workunit.client.1.smithi192.stdout:5/86: fdatasync d6/f11 0 2024-02-16T22:56:08.333 INFO:tasks.workunit.client.1.smithi192.stdout:9/152: getdents d7 0 2024-02-16T22:56:08.333 INFO:tasks.workunit.client.1.smithi192.stdout:8/41: link l1 d5/d9/lc 0 2024-02-16T22:56:08.334 INFO:tasks.workunit.client.1.smithi192.stdout:0/53: creat d1/d3/de/f10 x:0 0 0 2024-02-16T22:56:08.334 INFO:tasks.workunit.client.1.smithi192.stdout:8/42: stat c4 0 2024-02-16T22:56:08.334 INFO:tasks.workunit.client.1.smithi192.stdout:9/153: readlink d7/l9 0 2024-02-16T22:56:08.334 INFO:tasks.workunit.client.1.smithi192.stdout:7/120: truncate d1/d2/de/f20 681480 0 2024-02-16T22:56:08.335 INFO:tasks.workunit.client.1.smithi192.stdout:1/54: unlink d3/c8 0 2024-02-16T22:56:08.335 INFO:tasks.workunit.client.1.smithi192.stdout:3/64: symlink da/d11/l13 0 2024-02-16T22:56:08.336 INFO:tasks.workunit.client.1.smithi192.stdout:9/154: chown d7/d15/c30 383927993 1 2024-02-16T22:56:08.336 INFO:tasks.workunit.client.1.smithi192.stdout:0/54: truncate d1/d3/fc 741632 0 2024-02-16T22:56:08.337 INFO:tasks.workunit.client.1.smithi192.stdout:3/65: fsync f8 0 2024-02-16T22:56:08.337 INFO:tasks.workunit.client.1.smithi192.stdout:1/55: fsync d3/fc 0 2024-02-16T22:56:08.338 INFO:tasks.workunit.client.1.smithi192.stdout:5/87: mknod d6/d12/c1a 0 2024-02-16T22:56:08.339 INFO:tasks.workunit.client.1.smithi192.stdout:1/56: chown c2 14203368 1 2024-02-16T22:56:08.339 INFO:tasks.workunit.client.1.smithi192.stdout:6/98: write da/df/f1c [2636508,46192] 0 2024-02-16T22:56:08.340 INFO:tasks.workunit.client.1.smithi192.stdout:9/155: mkdir d7/d15/d36 0 2024-02-16T22:56:08.344 INFO:tasks.workunit.client.1.smithi192.stdout:2/121: unlink fb 0 2024-02-16T22:56:08.344 INFO:tasks.workunit.client.1.smithi192.stdout:9/156: dread - d7/df/f23 zero size 2024-02-16T22:56:08.344 INFO:tasks.workunit.client.1.smithi192.stdout:1/57: truncate d3/de/ff 1235204 0 2024-02-16T22:56:08.344 INFO:tasks.workunit.client.1.smithi192.stdout:8/43: mkdir d5/db/dd 0 2024-02-16T22:56:08.345 INFO:tasks.workunit.client.1.smithi192.stdout:2/122: mkdir d22/d24 0 2024-02-16T22:56:08.345 INFO:tasks.workunit.client.1.smithi192.stdout:7/121: rename d1/l9 to d1/l27 0 2024-02-16T22:56:08.347 INFO:tasks.workunit.client.1.smithi192.stdout:8/44: mkdir d5/db/de 0 2024-02-16T22:56:08.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/58: truncate d3/fb 6566530 0 2024-02-16T22:56:08.349 INFO:tasks.workunit.client.1.smithi192.stdout:3/66: dwrite f9 [0,4194304] 0 2024-02-16T22:56:08.349 INFO:tasks.workunit.client.1.smithi192.stdout:6/99: dread da/df/f1c [0,4194304] 0 2024-02-16T22:56:08.351 INFO:tasks.workunit.client.0.smithi064.stdout:0/4: sync 2024-02-16T22:56:08.351 INFO:tasks.workunit.client.1.smithi192.stdout:2/123: mkdir d22/d25 0 2024-02-16T22:56:08.353 INFO:tasks.workunit.client.1.smithi192.stdout:3/67: dread f8 [0,4194304] 0 2024-02-16T22:56:08.353 INFO:tasks.workunit.client.1.smithi192.stdout:0/55: dread d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:08.356 INFO:tasks.workunit.client.0.smithi064.stdout:0/5: mknod c1 0 2024-02-16T22:56:08.356 INFO:tasks.workunit.client.0.smithi064.stdout:0/6: dwrite - no filename 2024-02-16T22:56:08.356 INFO:tasks.workunit.client.0.smithi064.stdout:0/7: read - no filename 2024-02-16T22:56:08.356 INFO:tasks.workunit.client.0.smithi064.stdout:0/8: dwrite - no filename 2024-02-16T22:56:08.357 INFO:tasks.workunit.client.0.smithi064.stdout:0/9: chown c1 10 1 2024-02-16T22:56:08.357 INFO:tasks.workunit.client.0.smithi064.stdout:0/10: dread - no filename 2024-02-16T22:56:08.357 INFO:tasks.workunit.client.0.smithi064.stdout:0/11: chown l0 15632550 1 2024-02-16T22:56:08.357 INFO:tasks.workunit.client.1.smithi192.stdout:0/56: write d1/f8 [862394,69415] 0 2024-02-16T22:56:08.358 INFO:tasks.workunit.client.1.smithi192.stdout:3/68: truncate da/f12 27951 0 2024-02-16T22:56:08.360 INFO:tasks.workunit.client.0.smithi064.stdout:0/12: symlink l2 0 2024-02-16T22:56:08.361 INFO:tasks.workunit.client.1.smithi192.stdout:1/59: dwrite d3/de/ff [0,4194304] 0 2024-02-16T22:56:08.361 INFO:tasks.workunit.client.0.smithi064.stdout:0/13: readlink l0 0 2024-02-16T22:56:08.361 INFO:tasks.workunit.client.1.smithi192.stdout:1/60: readlink d3/l11 0 2024-02-16T22:56:08.362 INFO:tasks.workunit.client.1.smithi192.stdout:5/88: getdents d6/d12 0 2024-02-16T22:56:08.362 INFO:tasks.workunit.client.1.smithi192.stdout:5/89: read - d6/f11 zero size 2024-02-16T22:56:08.363 INFO:tasks.workunit.client.1.smithi192.stdout:6/100: symlink da/df/d18/l20 0 2024-02-16T22:56:08.364 INFO:tasks.workunit.client.1.smithi192.stdout:5/90: dread - d6/f11 zero size 2024-02-16T22:56:08.365 INFO:tasks.workunit.client.1.smithi192.stdout:8/45: mkdir d5/db/de/df 0 2024-02-16T22:56:08.367 INFO:tasks.workunit.client.1.smithi192.stdout:2/124: mknod d22/d24/c26 0 2024-02-16T22:56:08.368 INFO:tasks.workunit.client.1.smithi192.stdout:1/61: dread d3/de/ff [0,4194304] 0 2024-02-16T22:56:08.368 INFO:tasks.workunit.client.1.smithi192.stdout:3/69: rename da/fc to da/f14 0 2024-02-16T22:56:08.369 INFO:tasks.workunit.client.1.smithi192.stdout:0/57: truncate d1/d2/f4 2661574 0 2024-02-16T22:56:08.372 INFO:tasks.workunit.client.1.smithi192.stdout:6/101: write da/fb [2722705,25244] 0 2024-02-16T22:56:08.376 INFO:tasks.workunit.client.1.smithi192.stdout:9/157: write d7/df/f22 [337720,8425] 0 2024-02-16T22:56:08.376 INFO:tasks.workunit.client.1.smithi192.stdout:8/46: dread d5/f8 [0,4194304] 0 2024-02-16T22:56:08.377 INFO:tasks.workunit.client.1.smithi192.stdout:3/70: mkdir da/d15 0 2024-02-16T22:56:08.379 INFO:tasks.workunit.client.1.smithi192.stdout:1/62: rename l0 to d3/de/l13 0 2024-02-16T22:56:08.379 INFO:tasks.workunit.client.1.smithi192.stdout:5/91: dwrite d6/d12/f18 [0,4194304] 0 2024-02-16T22:56:08.384 INFO:tasks.workunit.client.1.smithi192.stdout:8/47: symlink d5/l10 0 2024-02-16T22:56:08.387 INFO:tasks.workunit.client.1.smithi192.stdout:6/102: write da/df/f1c [3998441,122395] 0 2024-02-16T22:56:08.388 INFO:tasks.workunit.client.1.smithi192.stdout:9/158: rename d7/df/d2f to d7/d19/d37 0 2024-02-16T22:56:08.390 INFO:tasks.workunit.client.1.smithi192.stdout:7/122: dwrite d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:08.392 INFO:tasks.workunit.client.1.smithi192.stdout:2/125: dwrite fa [0,4194304] 0 2024-02-16T22:56:08.400 INFO:tasks.workunit.client.1.smithi192.stdout:1/63: mknod d3/de/c14 0 2024-02-16T22:56:08.400 INFO:tasks.workunit.client.1.smithi192.stdout:5/92: dread f2 [0,4194304] 0 2024-02-16T22:56:08.401 INFO:tasks.workunit.client.1.smithi192.stdout:3/71: symlink da/d15/l16 0 2024-02-16T22:56:08.401 INFO:tasks.workunit.client.1.smithi192.stdout:1/64: chown d3/l11 12529651 1 2024-02-16T22:56:08.403 INFO:tasks.workunit.client.1.smithi192.stdout:8/48: dread f2 [0,4194304] 0 2024-02-16T22:56:08.405 INFO:tasks.workunit.client.1.smithi192.stdout:7/123: dread d1/d2/f1b [0,4194304] 0 2024-02-16T22:56:08.406 INFO:tasks.workunit.client.1.smithi192.stdout:7/124: readlink d1/d2/de/l10 0 2024-02-16T22:56:08.407 INFO:tasks.workunit.client.1.smithi192.stdout:3/72: dwrite f9 [0,4194304] 0 2024-02-16T22:56:08.409 INFO:tasks.workunit.client.1.smithi192.stdout:6/103: write f9 [6644983,66216] 0 2024-02-16T22:56:08.416 INFO:tasks.workunit.client.1.smithi192.stdout:6/104: dwrite f6 [0,4194304] 0 2024-02-16T22:56:08.423 INFO:tasks.workunit.client.1.smithi192.stdout:0/58: getdents d1/d3 0 2024-02-16T22:56:08.425 INFO:tasks.workunit.client.1.smithi192.stdout:5/93: rename d6/d8/l9 to d6/d12/l1b 0 2024-02-16T22:56:08.427 INFO:tasks.workunit.client.1.smithi192.stdout:8/49: chown d5/d9/lc 17431651 1 2024-02-16T22:56:08.428 INFO:tasks.workunit.client.1.smithi192.stdout:7/125: creat d1/d2/de/f28 x:0 0 0 2024-02-16T22:56:08.428 INFO:tasks.workunit.client.1.smithi192.stdout:3/73: mknod da/d15/c17 0 2024-02-16T22:56:08.430 INFO:tasks.workunit.client.1.smithi192.stdout:8/50: dread f2 [4194304,4194304] 0 2024-02-16T22:56:08.431 INFO:tasks.workunit.client.1.smithi192.stdout:2/126: creat d22/f27 x:0 0 0 2024-02-16T22:56:08.475 INFO:tasks.workunit.client.1.smithi192.stdout:4/83: sync 2024-02-16T22:56:08.476 INFO:tasks.workunit.client.0.smithi064.stdout:5/8: sync 2024-02-16T22:56:08.476 INFO:tasks.workunit.client.0.smithi064.stdout:3/10: sync 2024-02-16T22:56:08.476 INFO:tasks.workunit.client.0.smithi064.stdout:3/11: fsync - no filename 2024-02-16T22:56:08.477 INFO:tasks.workunit.client.0.smithi064.stdout:3/12: write - no filename 2024-02-16T22:56:08.477 INFO:tasks.workunit.client.0.smithi064.stdout:3/13: stat l1 0 2024-02-16T22:56:08.477 INFO:tasks.workunit.client.0.smithi064.stdout:3/14: dread - no filename 2024-02-16T22:56:08.478 INFO:tasks.workunit.client.0.smithi064.stdout:3/15: stat l0 0 2024-02-16T22:56:08.478 INFO:tasks.workunit.client.0.smithi064.stdout:3/16: dwrite - no filename 2024-02-16T22:56:08.479 INFO:tasks.workunit.client.0.smithi064.stdout:5/9: creat f1 x:0 0 0 2024-02-16T22:56:08.479 INFO:tasks.workunit.client.0.smithi064.stdout:5/10: write f1 [385101,79616] 0 2024-02-16T22:56:08.480 INFO:tasks.workunit.client.0.smithi064.stdout:5/11: write f0 [289927,8592] 0 2024-02-16T22:56:08.482 INFO:tasks.workunit.client.0.smithi064.stdout:5/12: write f1 [794735,89323] 0 2024-02-16T22:56:08.482 INFO:tasks.workunit.client.0.smithi064.stdout:3/17: link l0 l3 0 2024-02-16T22:56:08.483 INFO:tasks.workunit.client.0.smithi064.stdout:3/18: mkdir d4 0 2024-02-16T22:56:08.484 INFO:tasks.workunit.client.0.smithi064.stdout:5/13: creat f2 x:0 0 0 2024-02-16T22:56:08.488 INFO:tasks.workunit.client.0.smithi064.stdout:5/14: unlink f1 0 2024-02-16T22:56:08.488 INFO:tasks.workunit.client.0.smithi064.stdout:3/19: creat d4/f5 x:0 0 0 2024-02-16T22:56:08.490 INFO:tasks.workunit.client.0.smithi064.stdout:5/15: creat f3 x:0 0 0 2024-02-16T22:56:08.490 INFO:tasks.workunit.client.0.smithi064.stdout:3/20: symlink d4/l6 0 2024-02-16T22:56:08.492 INFO:tasks.workunit.client.0.smithi064.stdout:5/16: rename f3 to f4 0 2024-02-16T22:56:08.494 INFO:tasks.workunit.client.1.smithi192.stdout:0/59: mknod d1/c11 0 2024-02-16T22:56:08.495 INFO:tasks.workunit.client.1.smithi192.stdout:1/65: fdatasync d3/fb 0 2024-02-16T22:56:08.499 INFO:tasks.workunit.client.1.smithi192.stdout:9/159: truncate d7/d15/f20 1232778 0 2024-02-16T22:56:08.504 INFO:tasks.workunit.client.0.smithi064.stdout:3/21: getdents d4 0 2024-02-16T22:56:08.505 INFO:tasks.workunit.client.0.smithi064.stdout:3/22: dread - d4/f5 zero size 2024-02-16T22:56:08.511 INFO:tasks.workunit.client.0.smithi064.stdout:3/23: dwrite d4/f5 [0,4194304] 0 2024-02-16T22:56:08.512 INFO:tasks.workunit.client.0.smithi064.stdout:3/24: chown d4 2446854 1 2024-02-16T22:56:08.517 INFO:tasks.workunit.client.0.smithi064.stdout:3/25: link l3 d4/l7 0 2024-02-16T22:56:08.518 INFO:tasks.workunit.client.0.smithi064.stdout:3/26: read d4/f5 [3415785,2481] 0 2024-02-16T22:56:08.525 INFO:tasks.workunit.client.0.smithi064.stdout:3/27: dwrite d4/f5 [0,4194304] 0 2024-02-16T22:56:08.527 INFO:tasks.workunit.client.0.smithi064.stdout:3/28: rmdir d4 39 2024-02-16T22:56:08.529 INFO:tasks.workunit.client.0.smithi064.stdout:3/29: write d4/f5 [3539819,104292] 0 2024-02-16T22:56:08.531 INFO:tasks.workunit.client.0.smithi064.stdout:3/30: creat d4/f8 x:0 0 0 2024-02-16T22:56:08.532 INFO:tasks.workunit.client.0.smithi064.stdout:3/31: chown l1 814 1 2024-02-16T22:56:08.538 INFO:tasks.workunit.client.0.smithi064.stdout:3/32: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:56:08.539 INFO:tasks.workunit.client.0.smithi064.stdout:3/33: stat d4/f5 0 2024-02-16T22:56:08.555 INFO:tasks.workunit.client.0.smithi064.stdout:3/34: fdatasync d4/f8 0 2024-02-16T22:56:08.559 INFO:tasks.workunit.client.0.smithi064.stdout:3/35: truncate d4/f5 3974067 0 2024-02-16T22:56:08.561 INFO:tasks.workunit.client.1.smithi192.stdout:5/94: creat d6/d12/d15/f1c x:0 0 0 2024-02-16T22:56:08.561 INFO:tasks.workunit.client.0.smithi064.stdout:3/36: rename l3 to d4/l9 0 2024-02-16T22:56:08.562 INFO:tasks.workunit.client.0.smithi064.stdout:3/37: write d4/f8 [522079,96538] 0 2024-02-16T22:56:08.566 INFO:tasks.workunit.client.1.smithi192.stdout:7/126: dwrite d1/d2/f1b [0,4194304] 0 2024-02-16T22:56:08.566 INFO:tasks.workunit.client.0.smithi064.stdout:3/38: fsync d4/f5 0 2024-02-16T22:56:08.567 INFO:tasks.workunit.client.1.smithi192.stdout:4/84: stat d1/d8/d16/c21 0 2024-02-16T22:56:08.568 INFO:tasks.workunit.client.1.smithi192.stdout:1/66: dwrite d3/fc [0,4194304] 0 2024-02-16T22:56:08.570 INFO:tasks.workunit.client.0.smithi064.stdout:3/39: chown l2 649663366 1 2024-02-16T22:56:08.571 INFO:tasks.workunit.client.1.smithi192.stdout:4/85: fdatasync d1/f4 0 2024-02-16T22:56:08.575 INFO:tasks.workunit.client.1.smithi192.stdout:8/51: dwrite f2 [0,4194304] 0 2024-02-16T22:56:08.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/160: dread d7/f12 [0,4194304] 0 2024-02-16T22:56:08.575 INFO:tasks.workunit.client.1.smithi192.stdout:4/86: write d1/f18 [3143100,4036] 0 2024-02-16T22:56:08.577 INFO:tasks.workunit.client.1.smithi192.stdout:8/52: chown d5/db/de 159256 1 2024-02-16T22:56:08.577 INFO:tasks.workunit.client.1.smithi192.stdout:0/60: dwrite d1/f5 [0,4194304] 0 2024-02-16T22:56:08.577 INFO:tasks.workunit.client.1.smithi192.stdout:7/127: rmdir d1/d2/de 39 2024-02-16T22:56:08.580 INFO:tasks.workunit.client.1.smithi192.stdout:4/87: write d1/d8/f1e [488815,113701] 0 2024-02-16T22:56:08.580 INFO:tasks.workunit.client.1.smithi192.stdout:1/67: fsync d3/fb 0 2024-02-16T22:56:08.583 INFO:tasks.workunit.client.1.smithi192.stdout:2/127: rmdir d22/d25 0 2024-02-16T22:56:08.584 INFO:tasks.workunit.client.1.smithi192.stdout:3/74: getdents da/d11 0 2024-02-16T22:56:08.585 INFO:tasks.workunit.client.1.smithi192.stdout:3/75: chown f8 7 1 2024-02-16T22:56:08.585 INFO:tasks.workunit.client.1.smithi192.stdout:4/88: dread d1/f5 [0,4194304] 0 2024-02-16T22:56:08.586 INFO:tasks.workunit.client.1.smithi192.stdout:3/76: write da/f12 [702504,75129] 0 2024-02-16T22:56:08.586 INFO:tasks.workunit.client.1.smithi192.stdout:3/77: read f9 [1366777,30170] 0 2024-02-16T22:56:08.588 INFO:tasks.workunit.client.1.smithi192.stdout:0/61: unlink d1/f7 0 2024-02-16T22:56:08.589 INFO:tasks.workunit.client.1.smithi192.stdout:8/53: mknod d5/c11 0 2024-02-16T22:56:08.591 INFO:tasks.workunit.client.0.smithi064.stdout:3/40: truncate d4/f8 1025265 0 2024-02-16T22:56:08.593 INFO:tasks.workunit.client.1.smithi192.stdout:4/89: dwrite d1/d8/fe [0,4194304] 0 2024-02-16T22:56:08.595 INFO:tasks.workunit.client.1.smithi192.stdout:1/68: mknod d3/de/c15 0 2024-02-16T22:56:08.597 INFO:tasks.workunit.client.1.smithi192.stdout:9/161: mkdir d7/d15/d36/d38 0 2024-02-16T22:56:08.598 INFO:tasks.workunit.client.0.smithi064.stdout:3/41: rename l2 to d4/la 0 2024-02-16T22:56:08.600 INFO:tasks.workunit.client.1.smithi192.stdout:5/95: dread d6/fd [0,4194304] 0 2024-02-16T22:56:08.600 INFO:tasks.workunit.client.0.smithi064.stdout:3/42: fsync d4/f5 0 2024-02-16T22:56:08.600 INFO:tasks.workunit.client.0.smithi064.stdout:3/43: fdatasync d4/f5 0 2024-02-16T22:56:08.600 INFO:tasks.workunit.client.0.smithi064.stdout:3/44: readlink d4/l6 0 2024-02-16T22:56:08.601 INFO:tasks.workunit.client.1.smithi192.stdout:2/128: mknod d22/d24/c28 0 2024-02-16T22:56:08.603 INFO:tasks.workunit.client.0.smithi064.stdout:3/45: mknod d4/cb 0 2024-02-16T22:56:08.605 INFO:tasks.workunit.client.0.smithi064.stdout:7/22: sync 2024-02-16T22:56:08.605 INFO:tasks.workunit.client.0.smithi064.stdout:2/26: sync 2024-02-16T22:56:08.605 INFO:tasks.workunit.client.0.smithi064.stdout:4/43: sync 2024-02-16T22:56:08.605 INFO:tasks.workunit.client.0.smithi064.stdout:8/53: sync 2024-02-16T22:56:08.605 INFO:tasks.workunit.client.0.smithi064.stdout:1/47: sync 2024-02-16T22:56:08.606 INFO:tasks.workunit.client.0.smithi064.stdout:6/81: sync 2024-02-16T22:56:08.606 INFO:tasks.workunit.client.0.smithi064.stdout:9/40: sync 2024-02-16T22:56:08.606 INFO:tasks.workunit.client.0.smithi064.stdout:0/14: sync 2024-02-16T22:56:08.607 INFO:tasks.workunit.client.0.smithi064.stdout:7/23: chown c2 9721 1 2024-02-16T22:56:08.607 INFO:tasks.workunit.client.1.smithi192.stdout:8/54: unlink d5/la 0 2024-02-16T22:56:08.607 INFO:tasks.workunit.client.1.smithi192.stdout:9/162: dwrite d7/df/f23 [0,4194304] 0 2024-02-16T22:56:08.608 INFO:tasks.workunit.client.1.smithi192.stdout:9/163: chown d7/d15/d36/d38 993230423 1 2024-02-16T22:56:08.608 INFO:tasks.workunit.client.1.smithi192.stdout:9/164: read d7/d19/f21 [590892,106988] 0 2024-02-16T22:56:08.609 INFO:tasks.workunit.client.0.smithi064.stdout:3/46: creat d4/fc x:0 0 0 2024-02-16T22:56:08.609 INFO:tasks.workunit.client.0.smithi064.stdout:2/27: fsync f0 0 2024-02-16T22:56:08.609 INFO:tasks.workunit.client.0.smithi064.stdout:7/24: write f0 [2074921,61211] 0 2024-02-16T22:56:08.611 INFO:tasks.workunit.client.1.smithi192.stdout:1/69: creat d3/f16 x:0 0 0 2024-02-16T22:56:08.616 INFO:tasks.workunit.client.1.smithi192.stdout:2/129: symlink d22/d24/l29 0 2024-02-16T22:56:08.617 INFO:tasks.workunit.client.0.smithi064.stdout:0/15: creat f3 x:0 0 0 2024-02-16T22:56:08.618 INFO:tasks.workunit.client.0.smithi064.stdout:1/48: dread d2/f3 [0,4194304] 0 2024-02-16T22:56:08.618 INFO:tasks.workunit.client.0.smithi064.stdout:6/82: rename d0/f8 to d0/d5/f11 0 2024-02-16T22:56:08.619 INFO:tasks.workunit.client.1.smithi192.stdout:4/90: dwrite d1/f19 [0,4194304] 0 2024-02-16T22:56:08.621 INFO:tasks.workunit.client.1.smithi192.stdout:3/78: link f8 da/df/f18 0 2024-02-16T22:56:08.622 INFO:tasks.workunit.client.0.smithi064.stdout:3/47: dwrite d4/fc [0,4194304] 0 2024-02-16T22:56:08.623 INFO:tasks.workunit.client.0.smithi064.stdout:2/28: dwrite f0 [0,4194304] 0 2024-02-16T22:56:08.623 INFO:tasks.workunit.client.0.smithi064.stdout:7/25: mkdir d5 0 2024-02-16T22:56:08.625 INFO:tasks.workunit.client.0.smithi064.stdout:1/49: read d2/f9 [263053,97631] 0 2024-02-16T22:56:08.625 INFO:tasks.workunit.client.0.smithi064.stdout:3/48: chown d4/f5 3237815 1 2024-02-16T22:56:08.625 INFO:tasks.workunit.client.1.smithi192.stdout:1/70: dwrite d3/fc [4194304,4194304] 0 2024-02-16T22:56:08.625 INFO:tasks.workunit.client.1.smithi192.stdout:4/91: write d1/d8/d16/f1f [334012,42300] 0 2024-02-16T22:56:08.626 INFO:tasks.workunit.client.1.smithi192.stdout:5/96: dwrite d6/fd [0,4194304] 0 2024-02-16T22:56:08.626 INFO:tasks.workunit.client.1.smithi192.stdout:0/62: fsync d1/d2/f4 0 2024-02-16T22:56:08.626 INFO:tasks.workunit.client.1.smithi192.stdout:0/63: write d1/d3/de/f10 [116384,59465] 0 2024-02-16T22:56:08.627 INFO:tasks.workunit.client.0.smithi064.stdout:1/50: write d2/fd [2557413,116333] 0 2024-02-16T22:56:08.627 INFO:tasks.workunit.client.1.smithi192.stdout:7/128: creat d1/d2/f29 x:0 0 0 2024-02-16T22:56:08.628 INFO:tasks.workunit.client.1.smithi192.stdout:0/64: dread d1/d3/fc [0,4194304] 0 2024-02-16T22:56:08.632 INFO:tasks.workunit.client.1.smithi192.stdout:9/165: readlink d7/d19/l2d 0 2024-02-16T22:56:08.632 INFO:tasks.workunit.client.1.smithi192.stdout:9/166: write d7/df/f16 [629765,79938] 0 2024-02-16T22:56:08.634 INFO:tasks.workunit.client.1.smithi192.stdout:9/167: chown d7/df/f16 459 1 2024-02-16T22:56:08.639 INFO:tasks.workunit.client.0.smithi064.stdout:7/26: dwrite f3 [0,4194304] 0 2024-02-16T22:56:08.643 INFO:tasks.workunit.client.0.smithi064.stdout:0/16: mknod c4 0 2024-02-16T22:56:08.643 INFO:tasks.workunit.client.0.smithi064.stdout:0/17: chown f3 81996242 1 2024-02-16T22:56:08.648 INFO:tasks.workunit.client.0.smithi064.stdout:6/83: mkdir d0/d12 0 2024-02-16T22:56:08.654 INFO:tasks.workunit.client.0.smithi064.stdout:7/27: dwrite f0 [0,4194304] 0 2024-02-16T22:56:08.655 INFO:tasks.workunit.client.0.smithi064.stdout:7/28: chown c2 10550666 1 2024-02-16T22:56:08.655 INFO:tasks.workunit.client.0.smithi064.stdout:2/29: mkdir d5 0 2024-02-16T22:56:08.656 INFO:tasks.workunit.client.1.smithi192.stdout:3/79: write da/df/f18 [1434275,50077] 0 2024-02-16T22:56:08.659 INFO:tasks.workunit.client.1.smithi192.stdout:1/71: readlink d3/de/l13 0 2024-02-16T22:56:08.660 INFO:tasks.workunit.client.0.smithi064.stdout:0/18: dwrite f3 [0,4194304] 0 2024-02-16T22:56:08.660 INFO:tasks.workunit.client.1.smithi192.stdout:1/72: chown d3/de/l13 99607 1 2024-02-16T22:56:08.662 INFO:tasks.workunit.client.0.smithi064.stdout:0/19: write f3 [3447406,108975] 0 2024-02-16T22:56:08.666 INFO:tasks.workunit.client.0.smithi064.stdout:9/41: write f1 [4624015,61280] 0 2024-02-16T22:56:08.666 INFO:tasks.workunit.client.0.smithi064.stdout:1/51: fdatasync d2/f3 0 2024-02-16T22:56:08.668 INFO:tasks.workunit.client.0.smithi064.stdout:9/42: chown c2 110 1 2024-02-16T22:56:08.671 INFO:tasks.workunit.client.0.smithi064.stdout:8/54: rmdir dc 0 2024-02-16T22:56:08.672 INFO:tasks.workunit.client.0.smithi064.stdout:2/30: dwrite f0 [0,4194304] 0 2024-02-16T22:56:08.676 INFO:tasks.workunit.client.0.smithi064.stdout:2/31: write f0 [3939327,64530] 0 2024-02-16T22:56:08.678 INFO:tasks.workunit.client.0.smithi064.stdout:0/20: dwrite f3 [0,4194304] 0 2024-02-16T22:56:08.679 INFO:tasks.workunit.client.1.smithi192.stdout:7/129: unlink d1/d1f/l26 0 2024-02-16T22:56:08.683 INFO:tasks.workunit.client.0.smithi064.stdout:6/84: mkdir d0/d6/d13 0 2024-02-16T22:56:08.684 INFO:tasks.workunit.client.1.smithi192.stdout:0/65: creat d1/d3/f12 x:0 0 0 2024-02-16T22:56:08.686 INFO:tasks.workunit.client.1.smithi192.stdout:0/66: write d1/fd [536089,116247] 0 2024-02-16T22:56:08.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/168: mkdir d7/d19/d37/d39 0 2024-02-16T22:56:08.694 INFO:tasks.workunit.client.1.smithi192.stdout:0/67: dwrite d1/d3/f12 [0,4194304] 0 2024-02-16T22:56:08.696 INFO:tasks.workunit.client.1.smithi192.stdout:9/169: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:08.696 INFO:tasks.workunit.client.1.smithi192.stdout:3/80: creat da/d11/f19 x:0 0 0 2024-02-16T22:56:08.698 INFO:tasks.workunit.client.0.smithi064.stdout:0/21: fsync f3 0 2024-02-16T22:56:08.699 INFO:tasks.workunit.client.0.smithi064.stdout:9/43: creat d3/f10 x:0 0 0 2024-02-16T22:56:08.699 INFO:tasks.workunit.client.1.smithi192.stdout:1/73: creat d3/de/f17 x:0 0 0 2024-02-16T22:56:08.700 INFO:tasks.workunit.client.1.smithi192.stdout:1/74: write d3/f16 [741836,76891] 0 2024-02-16T22:56:08.701 INFO:tasks.workunit.client.1.smithi192.stdout:9/170: dread d7/d15/f33 [0,4194304] 0 2024-02-16T22:56:08.701 INFO:tasks.workunit.client.1.smithi192.stdout:4/92: mknod d1/d8/c22 0 2024-02-16T22:56:08.702 INFO:tasks.workunit.client.1.smithi192.stdout:8/55: truncate d5/f8 3097594 0 2024-02-16T22:56:08.702 INFO:tasks.workunit.client.1.smithi192.stdout:9/171: write f2 [462271,87923] 0 2024-02-16T22:56:08.703 INFO:tasks.workunit.client.1.smithi192.stdout:4/93: write d1/d8/d16/f1a [139735,35323] 0 2024-02-16T22:56:08.705 INFO:tasks.workunit.client.1.smithi192.stdout:7/130: mkdir d1/d19/d2a 0 2024-02-16T22:56:08.706 INFO:tasks.workunit.client.1.smithi192.stdout:7/131: chown d1/f11 1 1 2024-02-16T22:56:08.708 INFO:tasks.workunit.client.1.smithi192.stdout:1/75: dwrite d3/de/f17 [0,4194304] 0 2024-02-16T22:56:08.710 INFO:tasks.workunit.client.1.smithi192.stdout:1/76: chown d3/de/ff 8559518 1 2024-02-16T22:56:08.710 INFO:tasks.workunit.client.0.smithi064.stdout:6/85: chown d0/d5/f7 3875261 1 2024-02-16T22:56:08.713 INFO:tasks.workunit.client.1.smithi192.stdout:6/105: sync 2024-02-16T22:56:08.714 INFO:tasks.workunit.client.1.smithi192.stdout:7/132: dread d1/d2/f23 [0,4194304] 0 2024-02-16T22:56:08.714 INFO:tasks.workunit.client.0.smithi064.stdout:3/49: read d4/f8 [551577,7483] 0 2024-02-16T22:56:08.717 INFO:tasks.workunit.client.0.smithi064.stdout:0/22: creat f5 x:0 0 0 2024-02-16T22:56:08.718 INFO:tasks.workunit.client.0.smithi064.stdout:0/23: write f3 [357541,74068] 0 2024-02-16T22:56:08.721 INFO:tasks.workunit.client.0.smithi064.stdout:9/44: mkdir d3/d11 0 2024-02-16T22:56:08.722 INFO:tasks.workunit.client.1.smithi192.stdout:9/172: mknod d7/d19/d28/c3a 0 2024-02-16T22:56:08.723 INFO:tasks.workunit.client.0.smithi064.stdout:0/24: dread f3 [0,4194304] 0 2024-02-16T22:56:08.723 INFO:tasks.workunit.client.0.smithi064.stdout:1/52: mkdir d2/df/d11 0 2024-02-16T22:56:08.727 INFO:tasks.workunit.client.1.smithi192.stdout:2/130: truncate f8 506791 0 2024-02-16T22:56:08.728 INFO:tasks.workunit.client.1.smithi192.stdout:2/131: write fa [1485086,16035] 0 2024-02-16T22:56:08.733 INFO:tasks.workunit.client.1.smithi192.stdout:3/81: rename da/d11/f19 to da/f1a 0 2024-02-16T22:56:08.734 INFO:tasks.workunit.client.0.smithi064.stdout:3/50: creat d4/fd x:0 0 0 2024-02-16T22:56:08.735 INFO:tasks.workunit.client.1.smithi192.stdout:6/106: rmdir da/df/d12/d15 39 2024-02-16T22:56:08.737 INFO:tasks.workunit.client.0.smithi064.stdout:9/45: creat d3/d9/f12 x:0 0 0 2024-02-16T22:56:08.737 INFO:tasks.workunit.client.0.smithi064.stdout:9/46: chown d3/d11 1 1 2024-02-16T22:56:08.738 INFO:tasks.workunit.client.0.smithi064.stdout:9/47: chown d3/d9/f12 163770147 1 2024-02-16T22:56:08.739 INFO:tasks.workunit.client.0.smithi064.stdout:9/48: write d3/d9/ff [543297,80057] 0 2024-02-16T22:56:08.740 INFO:tasks.workunit.client.0.smithi064.stdout:9/49: readlink d3/d9/lb 0 2024-02-16T22:56:08.741 INFO:tasks.workunit.client.0.smithi064.stdout:0/25: creat f6 x:0 0 0 2024-02-16T22:56:08.741 INFO:tasks.workunit.client.0.smithi064.stdout:0/26: read - f6 zero size 2024-02-16T22:56:08.742 INFO:tasks.workunit.client.0.smithi064.stdout:8/55: link c5 cd 0 2024-02-16T22:56:08.744 INFO:tasks.workunit.client.1.smithi192.stdout:9/173: unlink d7/df/f24 0 2024-02-16T22:56:08.745 INFO:tasks.workunit.client.0.smithi064.stdout:5/17: sync 2024-02-16T22:56:08.745 INFO:tasks.workunit.client.1.smithi192.stdout:9/174: dread - d7/d19/d37/f34 zero size 2024-02-16T22:56:08.746 INFO:tasks.workunit.client.0.smithi064.stdout:2/32: rmdir d5 0 2024-02-16T22:56:08.747 INFO:tasks.workunit.client.0.smithi064.stdout:5/18: write f0 [1028715,72599] 0 2024-02-16T22:56:08.748 INFO:tasks.workunit.client.0.smithi064.stdout:9/50: dwrite d3/f10 [0,4194304] 0 2024-02-16T22:56:08.752 INFO:tasks.workunit.client.0.smithi064.stdout:6/86: creat d0/d6/d10/f14 x:0 0 0 2024-02-16T22:56:08.752 INFO:tasks.workunit.client.0.smithi064.stdout:9/51: write d3/fc [1717216,28907] 0 2024-02-16T22:56:08.753 INFO:tasks.workunit.client.0.smithi064.stdout:6/87: readlink d0/d5/lb 0 2024-02-16T22:56:08.754 INFO:tasks.workunit.client.1.smithi192.stdout:8/56: truncate f2 181756 0 2024-02-16T22:56:08.755 INFO:tasks.workunit.client.0.smithi064.stdout:8/56: creat fe x:0 0 0 2024-02-16T22:56:08.758 INFO:tasks.workunit.client.0.smithi064.stdout:3/51: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:56:08.759 INFO:tasks.workunit.client.1.smithi192.stdout:3/82: write da/f14 [996525,114951] 0 2024-02-16T22:56:08.760 INFO:tasks.workunit.client.1.smithi192.stdout:3/83: write da/f12 [320121,100732] 0 2024-02-16T22:56:08.763 INFO:tasks.workunit.client.1.smithi192.stdout:4/94: rename d1/d8/d16 to d1/d15/d23 0 2024-02-16T22:56:08.764 INFO:tasks.workunit.client.1.smithi192.stdout:4/95: fdatasync d1/f18 0 2024-02-16T22:56:08.764 INFO:tasks.workunit.client.0.smithi064.stdout:5/19: fdatasync f4 0 2024-02-16T22:56:08.765 INFO:tasks.workunit.client.1.smithi192.stdout:7/133: mknod d1/d19/d2a/c2b 0 2024-02-16T22:56:08.770 INFO:tasks.workunit.client.0.smithi064.stdout:0/27: truncate f3 519919 0 2024-02-16T22:56:08.771 INFO:tasks.workunit.client.0.smithi064.stdout:9/52: rmdir d3 39 2024-02-16T22:56:08.772 INFO:tasks.workunit.client.0.smithi064.stdout:0/28: write f6 [452394,33181] 0 2024-02-16T22:56:08.772 INFO:tasks.workunit.client.1.smithi192.stdout:1/77: truncate d3/de/f17 1947626 0 2024-02-16T22:56:08.772 INFO:tasks.workunit.client.0.smithi064.stdout:0/29: chown l0 780486 1 2024-02-16T22:56:08.777 INFO:tasks.workunit.client.1.smithi192.stdout:0/68: link d1/d3/cf d1/d2/c13 0 2024-02-16T22:56:08.778 INFO:tasks.workunit.client.1.smithi192.stdout:0/69: read d1/d3/de/f10 [24992,121917] 0 2024-02-16T22:56:08.779 INFO:tasks.workunit.client.0.smithi064.stdout:8/57: mkdir df 0 2024-02-16T22:56:08.782 INFO:tasks.workunit.client.0.smithi064.stdout:8/58: dread f2 [0,4194304] 0 2024-02-16T22:56:08.783 INFO:tasks.workunit.client.0.smithi064.stdout:5/20: symlink l5 0 2024-02-16T22:56:08.783 INFO:tasks.workunit.client.0.smithi064.stdout:8/59: fsync f2 0 2024-02-16T22:56:08.784 INFO:tasks.workunit.client.0.smithi064.stdout:8/60: write fb [689871,28307] 0 2024-02-16T22:56:08.785 INFO:tasks.workunit.client.0.smithi064.stdout:8/61: write fe [717675,100952] 0 2024-02-16T22:56:08.786 INFO:tasks.workunit.client.0.smithi064.stdout:6/88: write d0/d5/f7 [6859647,36255] 0 2024-02-16T22:56:08.787 INFO:tasks.workunit.client.1.smithi192.stdout:3/84: mknod da/d11/c1b 0 2024-02-16T22:56:08.789 INFO:tasks.workunit.client.0.smithi064.stdout:8/62: dread fb [0,4194304] 0 2024-02-16T22:56:08.792 INFO:tasks.workunit.client.1.smithi192.stdout:4/96: unlink d1/f18 0 2024-02-16T22:56:08.793 INFO:tasks.workunit.client.1.smithi192.stdout:4/97: truncate d1/d8/fe 4485579 0 2024-02-16T22:56:08.794 INFO:tasks.workunit.client.1.smithi192.stdout:4/98: write d1/f4 [5189841,40355] 0 2024-02-16T22:56:08.797 INFO:tasks.workunit.client.1.smithi192.stdout:9/175: link d7/f31 d7/d19/d37/f3b 0 2024-02-16T22:56:08.799 INFO:tasks.workunit.client.0.smithi064.stdout:5/21: mkdir d6 0 2024-02-16T22:56:08.799 INFO:tasks.workunit.client.1.smithi192.stdout:4/99: dwrite d1/d15/d23/f1f [0,4194304] 0 2024-02-16T22:56:08.801 INFO:tasks.workunit.client.1.smithi192.stdout:0/70: rename d1/d2/c13 to d1/c14 0 2024-02-16T22:56:08.802 INFO:tasks.workunit.client.0.smithi064.stdout:9/53: symlink d3/l13 0 2024-02-16T22:56:08.804 INFO:tasks.workunit.client.0.smithi064.stdout:6/89: rename d0/d5/f7 to d0/d6/f15 0 2024-02-16T22:56:08.804 INFO:tasks.workunit.client.0.smithi064.stdout:9/54: read d3/d9/ff [367081,8715] 0 2024-02-16T22:56:08.805 INFO:tasks.workunit.client.0.smithi064.stdout:6/90: stat d0/d6/d10/f14 0 2024-02-16T22:56:08.806 INFO:tasks.workunit.client.0.smithi064.stdout:6/91: chown d0/d5/lb 58799 1 2024-02-16T22:56:08.808 INFO:tasks.workunit.client.1.smithi192.stdout:3/85: rmdir da 39 2024-02-16T22:56:08.812 INFO:tasks.workunit.client.1.smithi192.stdout:7/134: mkdir d1/d2/de/d2c 0 2024-02-16T22:56:08.818 INFO:tasks.workunit.client.1.smithi192.stdout:4/100: unlink d1/d15/d23/f1f 0 2024-02-16T22:56:08.821 INFO:tasks.workunit.client.0.smithi064.stdout:0/30: dread f3 [0,4194304] 0 2024-02-16T22:56:08.822 INFO:tasks.workunit.client.0.smithi064.stdout:0/31: readlink l2 0 2024-02-16T22:56:08.822 INFO:tasks.workunit.client.0.smithi064.stdout:8/63: symlink df/l10 0 2024-02-16T22:56:08.822 INFO:tasks.workunit.client.0.smithi064.stdout:8/64: stat c6 0 2024-02-16T22:56:08.823 INFO:tasks.workunit.client.0.smithi064.stdout:8/65: chown fb 1598 1 2024-02-16T22:56:08.823 INFO:tasks.workunit.client.1.smithi192.stdout:2/132: getdents d22 0 2024-02-16T22:56:08.823 INFO:tasks.workunit.client.0.smithi064.stdout:8/66: chown fe 252 1 2024-02-16T22:56:08.823 INFO:tasks.workunit.client.0.smithi064.stdout:6/92: write d0/d6/f15 [8223218,317] 0 2024-02-16T22:56:08.825 INFO:tasks.workunit.client.0.smithi064.stdout:4/44: sync 2024-02-16T22:56:08.825 INFO:tasks.workunit.client.0.smithi064.stdout:3/52: getdents d4 0 2024-02-16T22:56:08.826 INFO:tasks.workunit.client.1.smithi192.stdout:4/101: dwrite d1/d15/d23/f1a [0,4194304] 0 2024-02-16T22:56:08.826 INFO:tasks.workunit.client.0.smithi064.stdout:5/22: mknod d6/c7 0 2024-02-16T22:56:08.827 INFO:tasks.workunit.client.1.smithi192.stdout:9/176: dwrite d7/f31 [0,4194304] 0 2024-02-16T22:56:08.827 INFO:tasks.workunit.client.0.smithi064.stdout:5/23: fsync f0 0 2024-02-16T22:56:08.831 INFO:tasks.workunit.client.1.smithi192.stdout:0/71: link d1/f8 d1/f15 0 2024-02-16T22:56:08.833 INFO:tasks.workunit.client.0.smithi064.stdout:3/53: dwrite d4/fd [0,4194304] 0 2024-02-16T22:56:08.834 INFO:tasks.workunit.client.0.smithi064.stdout:8/67: mknod df/c11 0 2024-02-16T22:56:08.840 INFO:tasks.workunit.client.0.smithi064.stdout:3/54: dwrite d4/f5 [0,4194304] 0 2024-02-16T22:56:08.848 INFO:tasks.workunit.client.1.smithi192.stdout:5/97: sync 2024-02-16T22:56:08.852 INFO:tasks.workunit.client.0.smithi064.stdout:6/93: mknod d0/d6/d10/c16 0 2024-02-16T22:56:08.852 INFO:tasks.workunit.client.1.smithi192.stdout:8/57: write f2 [401042,6008] 0 2024-02-16T22:56:08.853 INFO:tasks.workunit.client.1.smithi192.stdout:2/133: creat d22/f2a x:0 0 0 2024-02-16T22:56:08.855 INFO:tasks.workunit.client.0.smithi064.stdout:9/55: truncate d3/f10 90155 0 2024-02-16T22:56:08.856 INFO:tasks.workunit.client.0.smithi064.stdout:5/24: mknod d6/c8 0 2024-02-16T22:56:08.857 INFO:tasks.workunit.client.0.smithi064.stdout:5/25: chown l5 0 1 2024-02-16T22:56:08.857 INFO:tasks.workunit.client.1.smithi192.stdout:8/58: dwrite f2 [0,4194304] 0 2024-02-16T22:56:08.858 INFO:tasks.workunit.client.1.smithi192.stdout:8/59: readlink d5/l7 0 2024-02-16T22:56:08.863 INFO:tasks.workunit.client.0.smithi064.stdout:8/68: creat df/f12 x:0 0 0 2024-02-16T22:56:08.864 INFO:tasks.workunit.client.0.smithi064.stdout:3/55: stat l0 0 2024-02-16T22:56:08.867 INFO:tasks.workunit.client.1.smithi192.stdout:4/102: dwrite d1/fd [0,4194304] 0 2024-02-16T22:56:08.868 INFO:tasks.workunit.client.0.smithi064.stdout:9/56: mkdir d3/d14 0 2024-02-16T22:56:08.871 INFO:tasks.workunit.client.1.smithi192.stdout:3/86: rename l2 to da/d11/l1c 0 2024-02-16T22:56:08.873 INFO:tasks.workunit.client.0.smithi064.stdout:5/26: rename d6/c8 to d6/c9 0 2024-02-16T22:56:08.874 INFO:tasks.workunit.client.1.smithi192.stdout:7/135: write d1/d2/de/f20 [1445864,101756] 0 2024-02-16T22:56:08.877 INFO:tasks.workunit.client.0.smithi064.stdout:8/69: creat df/f13 x:0 0 0 2024-02-16T22:56:08.886 INFO:tasks.workunit.client.1.smithi192.stdout:5/98: truncate d6/f10 1919251 0 2024-02-16T22:56:08.889 INFO:tasks.workunit.client.1.smithi192.stdout:7/136: rmdir d1 39 2024-02-16T22:56:08.894 INFO:tasks.workunit.client.1.smithi192.stdout:8/60: truncate f2 3044511 0 2024-02-16T22:56:08.895 INFO:tasks.workunit.client.1.smithi192.stdout:9/177: rename d7/l1e to d7/d19/d28/l3c 0 2024-02-16T22:56:08.896 INFO:tasks.workunit.client.1.smithi192.stdout:0/72: getdents d1/d3/de 0 2024-02-16T22:56:08.900 INFO:tasks.workunit.client.1.smithi192.stdout:8/61: rmdir d5/db 39 2024-02-16T22:56:08.901 INFO:tasks.workunit.client.1.smithi192.stdout:8/62: read f2 [1475106,71356] 0 2024-02-16T22:56:08.902 INFO:tasks.workunit.client.1.smithi192.stdout:5/99: dwrite f2 [0,4194304] 0 2024-02-16T22:56:08.903 INFO:tasks.workunit.client.1.smithi192.stdout:0/73: dread d1/d3/f12 [0,4194304] 0 2024-02-16T22:56:08.906 INFO:tasks.workunit.client.1.smithi192.stdout:3/87: rename f8 to da/d15/f1d 0 2024-02-16T22:56:08.907 INFO:tasks.workunit.client.1.smithi192.stdout:6/107: sync 2024-02-16T22:56:08.907 INFO:tasks.workunit.client.1.smithi192.stdout:1/78: sync 2024-02-16T22:56:08.909 INFO:tasks.workunit.client.1.smithi192.stdout:4/103: dwrite d1/d8/d11/f1d [0,4194304] 0 2024-02-16T22:56:08.911 INFO:tasks.workunit.client.1.smithi192.stdout:9/178: creat d7/d19/d37/f3d x:0 0 0 2024-02-16T22:56:08.912 INFO:tasks.workunit.client.0.smithi064.stdout:3/56: symlink d4/le 0 2024-02-16T22:56:08.912 INFO:tasks.workunit.client.0.smithi064.stdout:6/94: mkdir d0/d5/df/d17 0 2024-02-16T22:56:08.913 INFO:tasks.workunit.client.0.smithi064.stdout:5/27: dread f0 [0,4194304] 0 2024-02-16T22:56:08.914 INFO:tasks.workunit.client.0.smithi064.stdout:0/32: getdents . 0 2024-02-16T22:56:08.916 INFO:tasks.workunit.client.0.smithi064.stdout:5/28: read f0 [867705,47565] 0 2024-02-16T22:56:08.917 INFO:tasks.workunit.client.0.smithi064.stdout:5/29: readlink l5 0 2024-02-16T22:56:08.918 INFO:tasks.workunit.client.0.smithi064.stdout:0/33: dread f3 [0,4194304] 0 2024-02-16T22:56:08.919 INFO:tasks.workunit.client.0.smithi064.stdout:0/34: truncate f5 627424 0 2024-02-16T22:56:08.921 INFO:tasks.workunit.client.0.smithi064.stdout:0/35: read f5 [219407,127423] 0 2024-02-16T22:56:08.924 INFO:tasks.workunit.client.0.smithi064.stdout:3/57: dwrite d4/f8 [4194304,4194304] 0 2024-02-16T22:56:08.925 INFO:tasks.workunit.client.0.smithi064.stdout:0/36: dread f6 [0,4194304] 0 2024-02-16T22:56:08.926 INFO:tasks.workunit.client.0.smithi064.stdout:0/37: chown l0 540360342 1 2024-02-16T22:56:08.928 INFO:tasks.workunit.client.1.smithi192.stdout:3/88: chown da/d11/l1c 1 1 2024-02-16T22:56:08.929 INFO:tasks.workunit.client.1.smithi192.stdout:3/89: chown da/d11/c1b 1314 1 2024-02-16T22:56:08.929 INFO:tasks.workunit.client.1.smithi192.stdout:3/90: stat da 0 2024-02-16T22:56:08.932 INFO:tasks.workunit.client.0.smithi064.stdout:3/58: dwrite d4/fd [0,4194304] 0 2024-02-16T22:56:08.932 INFO:tasks.workunit.client.1.smithi192.stdout:1/79: mknod d3/de/c18 0 2024-02-16T22:56:08.938 INFO:tasks.workunit.client.1.smithi192.stdout:4/104: creat d1/d15/f24 x:0 0 0 2024-02-16T22:56:08.938 INFO:tasks.workunit.client.1.smithi192.stdout:4/105: stat d1/d15/c1b 0 2024-02-16T22:56:08.939 INFO:tasks.workunit.client.0.smithi064.stdout:3/59: dwrite d4/fd [4194304,4194304] 0 2024-02-16T22:56:08.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/137: mkdir d1/d2/d2d 0 2024-02-16T22:56:08.953 INFO:tasks.workunit.client.1.smithi192.stdout:5/100: fdatasync d6/f10 0 2024-02-16T22:56:08.953 INFO:tasks.workunit.client.1.smithi192.stdout:3/91: mknod da/d11/c1e 0 2024-02-16T22:56:08.953 INFO:tasks.workunit.client.1.smithi192.stdout:6/108: creat da/df/d12/d15/f21 x:0 0 0 2024-02-16T22:56:08.955 INFO:tasks.workunit.client.1.smithi192.stdout:3/92: write da/f12 [1099658,54912] 0 2024-02-16T22:56:08.956 INFO:tasks.workunit.client.1.smithi192.stdout:6/109: write da/df/f1c [3129847,128966] 0 2024-02-16T22:56:08.959 INFO:tasks.workunit.client.1.smithi192.stdout:7/138: dwrite d1/d2/f23 [0,4194304] 0 2024-02-16T22:56:08.959 INFO:tasks.workunit.client.1.smithi192.stdout:7/139: dread - d1/d2/f1e zero size 2024-02-16T22:56:08.959 INFO:tasks.workunit.client.1.smithi192.stdout:4/106: rename d1/c2 to d1/d8/c25 0 2024-02-16T22:56:08.960 INFO:tasks.workunit.client.1.smithi192.stdout:4/107: read d1/fd [4104293,24855] 0 2024-02-16T22:56:08.960 INFO:tasks.workunit.client.1.smithi192.stdout:0/74: write d1/f15 [1544019,44284] 0 2024-02-16T22:56:08.965 INFO:tasks.workunit.client.1.smithi192.stdout:1/80: rmdir d3/de 39 2024-02-16T22:56:08.967 INFO:tasks.workunit.client.1.smithi192.stdout:7/140: dwrite d1/d2/de/f28 [0,4194304] 0 2024-02-16T22:56:08.972 INFO:tasks.workunit.client.1.smithi192.stdout:6/110: symlink da/l22 0 2024-02-16T22:56:08.973 INFO:tasks.workunit.client.1.smithi192.stdout:8/63: truncate f2 1455527 0 2024-02-16T22:56:08.974 INFO:tasks.workunit.client.1.smithi192.stdout:8/64: rename d5/d9 to d5/d9/d12 22 2024-02-16T22:56:08.974 INFO:tasks.workunit.client.1.smithi192.stdout:0/75: mkdir d1/d3/d16 0 2024-02-16T22:56:08.975 INFO:tasks.workunit.client.1.smithi192.stdout:0/76: truncate d1/f8 4659946 0 2024-02-16T22:56:08.986 INFO:tasks.workunit.client.1.smithi192.stdout:6/111: symlink da/df/d12/d15/l23 0 2024-02-16T22:56:08.986 INFO:tasks.workunit.client.1.smithi192.stdout:6/112: fdatasync da/f1f 0 2024-02-16T22:56:08.986 INFO:tasks.workunit.client.1.smithi192.stdout:6/113: read - da/f1f zero size 2024-02-16T22:56:08.990 INFO:tasks.workunit.client.1.smithi192.stdout:0/77: mknod d1/d3/c17 0 2024-02-16T22:56:08.991 INFO:tasks.workunit.client.1.smithi192.stdout:0/78: read d1/f15 [4122147,95227] 0 2024-02-16T22:56:08.992 INFO:tasks.workunit.client.1.smithi192.stdout:0/79: chown d1/d3/d16 5301158 1 2024-02-16T22:56:08.999 INFO:tasks.workunit.client.1.smithi192.stdout:5/101: truncate d6/d12/f18 470654 0 2024-02-16T22:56:09.001 INFO:tasks.workunit.client.1.smithi192.stdout:1/81: creat d3/f19 x:0 0 0 2024-02-16T22:56:09.003 INFO:tasks.workunit.client.1.smithi192.stdout:2/134: sync 2024-02-16T22:56:09.004 INFO:tasks.workunit.client.1.smithi192.stdout:5/102: dwrite d6/fc [0,4194304] 0 2024-02-16T22:56:09.005 INFO:tasks.workunit.client.1.smithi192.stdout:7/141: link d1/d2/l8 d1/d19/l2e 0 2024-02-16T22:56:09.006 INFO:tasks.workunit.client.1.smithi192.stdout:7/142: truncate d1/d2/f25 195658 0 2024-02-16T22:56:09.007 INFO:tasks.workunit.client.1.smithi192.stdout:6/114: symlink da/df/d12/d15/d1b/l24 0 2024-02-16T22:56:09.008 INFO:tasks.workunit.client.1.smithi192.stdout:3/93: truncate f3 37338 0 2024-02-16T22:56:09.009 INFO:tasks.workunit.client.1.smithi192.stdout:8/65: symlink d5/db/dd/l13 0 2024-02-16T22:56:09.011 INFO:tasks.workunit.client.1.smithi192.stdout:7/143: dread d1/d2/de/f28 [0,4194304] 0 2024-02-16T22:56:09.019 INFO:tasks.workunit.client.1.smithi192.stdout:2/135: mkdir d22/d24/d2b 0 2024-02-16T22:56:09.020 INFO:tasks.workunit.client.1.smithi192.stdout:5/103: chown d6/l7 7 1 2024-02-16T22:56:09.021 INFO:tasks.workunit.client.1.smithi192.stdout:5/104: chown d6/f10 104 1 2024-02-16T22:56:09.021 INFO:tasks.workunit.client.1.smithi192.stdout:5/105: chown d6/f10 11 1 2024-02-16T22:56:09.025 INFO:tasks.workunit.client.1.smithi192.stdout:3/94: rmdir da/d15 39 2024-02-16T22:56:09.025 INFO:tasks.workunit.client.1.smithi192.stdout:3/95: chown da/d11 505954266 1 2024-02-16T22:56:09.028 INFO:tasks.workunit.client.1.smithi192.stdout:7/144: unlink d1/c14 0 2024-02-16T22:56:09.029 INFO:tasks.workunit.client.1.smithi192.stdout:0/80: write d1/d3/fc [81565,63787] 0 2024-02-16T22:56:09.030 INFO:tasks.workunit.client.1.smithi192.stdout:0/81: chown d1/d3/fc 50130 1 2024-02-16T22:56:09.030 INFO:tasks.workunit.client.1.smithi192.stdout:1/82: creat d3/de/f1a x:0 0 0 2024-02-16T22:56:09.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:08 smithi192 ceph-mon[36254]: pgmap v9: 65 pgs: 65 active+clean; 313 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 78 KiB/s rd, 16 MiB/s wr, 250 op/s 2024-02-16T22:56:09.037 INFO:tasks.workunit.client.1.smithi192.stdout:0/82: dwrite d1/d3/fc [0,4194304] 0 2024-02-16T22:56:09.043 INFO:tasks.workunit.client.1.smithi192.stdout:8/66: creat d5/db/de/df/f14 x:0 0 0 2024-02-16T22:56:09.051 INFO:tasks.workunit.client.1.smithi192.stdout:7/145: dwrite d1/d2/de/f28 [0,4194304] 0 2024-02-16T22:56:09.051 INFO:tasks.workunit.client.1.smithi192.stdout:2/136: mkdir d22/d24/d2b/d2c 0 2024-02-16T22:56:09.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/146: dread - d1/f12 zero size 2024-02-16T22:56:09.055 INFO:tasks.workunit.client.1.smithi192.stdout:0/83: rmdir d1 39 2024-02-16T22:56:09.056 INFO:tasks.workunit.client.1.smithi192.stdout:3/96: creat da/df/f1f x:0 0 0 2024-02-16T22:56:09.058 INFO:tasks.workunit.client.1.smithi192.stdout:5/106: write d6/f10 [111696,125736] 0 2024-02-16T22:56:09.063 INFO:tasks.workunit.client.1.smithi192.stdout:6/115: truncate f6 6317508 0 2024-02-16T22:56:09.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/137: readlink l1e 0 2024-02-16T22:56:09.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/138: fsync fa 0 2024-02-16T22:56:09.065 INFO:tasks.workunit.client.1.smithi192.stdout:7/147: mknod d1/d1f/c2f 0 2024-02-16T22:56:09.068 INFO:tasks.workunit.client.1.smithi192.stdout:7/148: dread d1/f11 [0,4194304] 0 2024-02-16T22:56:09.071 INFO:tasks.workunit.client.1.smithi192.stdout:5/107: mkdir d6/d12/d1d 0 2024-02-16T22:56:09.071 INFO:tasks.workunit.client.1.smithi192.stdout:5/108: fdatasync d6/fd 0 2024-02-16T22:56:09.072 INFO:tasks.workunit.client.1.smithi192.stdout:5/109: dread - d6/f11 zero size 2024-02-16T22:56:09.081 INFO:tasks.workunit.client.1.smithi192.stdout:0/84: rmdir d1/d3/de 39 2024-02-16T22:56:09.087 INFO:tasks.workunit.client.1.smithi192.stdout:8/67: creat d5/f15 x:0 0 0 2024-02-16T22:56:09.090 INFO:tasks.workunit.client.1.smithi192.stdout:1/83: getdents d3/de 0 2024-02-16T22:56:09.094 INFO:tasks.workunit.client.1.smithi192.stdout:2/139: creat d22/d24/d2b/d2c/f2d x:0 0 0 2024-02-16T22:56:09.096 INFO:tasks.workunit.client.1.smithi192.stdout:1/84: dwrite d3/f16 [0,4194304] 0 2024-02-16T22:56:09.102 INFO:tasks.workunit.client.1.smithi192.stdout:0/85: dread d1/f8 [0,4194304] 0 2024-02-16T22:56:09.102 INFO:tasks.workunit.client.1.smithi192.stdout:1/85: dread d3/fc [4194304,4194304] 0 2024-02-16T22:56:09.103 INFO:tasks.workunit.client.1.smithi192.stdout:0/86: truncate d1/fa 465075 0 2024-02-16T22:56:09.104 INFO:tasks.workunit.client.1.smithi192.stdout:0/87: stat d1/d3 0 2024-02-16T22:56:09.106 INFO:tasks.workunit.client.1.smithi192.stdout:6/116: write da/df/d12/f13 [4523419,57657] 0 2024-02-16T22:56:09.106 INFO:tasks.workunit.client.1.smithi192.stdout:6/117: write da/df/d12/f17 [3573629,37342] 0 2024-02-16T22:56:09.107 INFO:tasks.workunit.client.1.smithi192.stdout:6/118: fsync da/df/f11 0 2024-02-16T22:56:09.108 INFO:tasks.workunit.client.1.smithi192.stdout:6/119: chown da/l22 215886410 1 2024-02-16T22:56:09.108 INFO:tasks.workunit.client.1.smithi192.stdout:3/97: write f3 [45588,82318] 0 2024-02-16T22:56:09.109 INFO:tasks.workunit.client.1.smithi192.stdout:3/98: chown f4 177935568 1 2024-02-16T22:56:09.109 INFO:tasks.workunit.client.1.smithi192.stdout:6/120: chown da/df/f11 285286 1 2024-02-16T22:56:09.110 INFO:tasks.workunit.client.1.smithi192.stdout:6/121: rename da/df to da/df/d12/d15/d25 22 2024-02-16T22:56:09.110 INFO:tasks.workunit.client.1.smithi192.stdout:7/149: link d1/l1c d1/d1f/l30 0 2024-02-16T22:56:09.113 INFO:tasks.workunit.client.1.smithi192.stdout:3/99: dread da/df/f18 [0,4194304] 0 2024-02-16T22:56:09.113 INFO:tasks.workunit.client.1.smithi192.stdout:1/86: creat d3/de/f1b x:0 0 0 2024-02-16T22:56:09.114 INFO:tasks.workunit.client.1.smithi192.stdout:6/122: dread da/df/d12/f13 [0,4194304] 0 2024-02-16T22:56:09.115 INFO:tasks.workunit.client.1.smithi192.stdout:8/68: fsync f2 0 2024-02-16T22:56:09.128 INFO:tasks.workunit.client.1.smithi192.stdout:8/69: dread d5/f8 [0,4194304] 0 2024-02-16T22:56:09.128 INFO:tasks.workunit.client.1.smithi192.stdout:0/88: mkdir d1/d3/d18 0 2024-02-16T22:56:09.132 INFO:tasks.workunit.client.1.smithi192.stdout:7/150: rename d1/d1f/l30 to d1/d2/d2d/l31 0 2024-02-16T22:56:09.132 INFO:tasks.workunit.client.1.smithi192.stdout:1/87: write d3/de/f17 [684611,121996] 0 2024-02-16T22:56:09.135 INFO:tasks.workunit.client.1.smithi192.stdout:0/89: stat d1/d3/de 0 2024-02-16T22:56:09.142 INFO:tasks.workunit.client.1.smithi192.stdout:0/90: dwrite d1/fd [0,4194304] 0 2024-02-16T22:56:09.142 INFO:tasks.workunit.client.1.smithi192.stdout:6/123: creat da/df/d12/f26 x:0 0 0 2024-02-16T22:56:09.149 INFO:tasks.workunit.client.1.smithi192.stdout:8/70: write d5/f8 [2487159,101230] 0 2024-02-16T22:56:09.150 INFO:tasks.workunit.client.1.smithi192.stdout:3/100: rename da/d15 to da/d11/d20 0 2024-02-16T22:56:09.151 INFO:tasks.workunit.client.1.smithi192.stdout:9/179: sync 2024-02-16T22:56:09.152 INFO:tasks.workunit.client.1.smithi192.stdout:4/108: sync 2024-02-16T22:56:09.152 INFO:tasks.workunit.client.1.smithi192.stdout:4/109: chown d1/d15/d23 2333 1 2024-02-16T22:56:09.153 INFO:tasks.workunit.client.1.smithi192.stdout:9/180: write d7/d19/d37/f3b [818512,82160] 0 2024-02-16T22:56:09.156 INFO:tasks.workunit.client.1.smithi192.stdout:1/88: write d3/f4 [7517287,34908] 0 2024-02-16T22:56:09.157 INFO:tasks.workunit.client.1.smithi192.stdout:9/181: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:09.158 INFO:tasks.workunit.client.1.smithi192.stdout:9/182: fsync d7/f31 0 2024-02-16T22:56:09.165 INFO:tasks.workunit.client.1.smithi192.stdout:3/101: creat da/d11/f21 x:0 0 0 2024-02-16T22:56:09.165 INFO:tasks.workunit.client.1.smithi192.stdout:3/102: dread f3 [0,4194304] 0 2024-02-16T22:56:09.165 INFO:tasks.workunit.client.1.smithi192.stdout:7/151: link d1/d2/de/c13 d1/d1f/c32 0 2024-02-16T22:56:09.167 INFO:tasks.workunit.client.1.smithi192.stdout:4/110: mkdir d1/d26 0 2024-02-16T22:56:09.170 INFO:tasks.workunit.client.1.smithi192.stdout:6/124: mkdir da/d27 0 2024-02-16T22:56:09.171 INFO:tasks.workunit.client.0.smithi064.stdout:4/45: sync 2024-02-16T22:56:09.171 INFO:tasks.workunit.client.0.smithi064.stdout:7/29: sync 2024-02-16T22:56:09.171 INFO:tasks.workunit.client.0.smithi064.stdout:1/53: sync 2024-02-16T22:56:09.171 INFO:tasks.workunit.client.0.smithi064.stdout:2/33: sync 2024-02-16T22:56:09.172 INFO:tasks.workunit.client.0.smithi064.stdout:4/46: write f0 [2349022,117039] 0 2024-02-16T22:56:09.172 INFO:tasks.workunit.client.1.smithi192.stdout:0/91: dwrite d1/fd [4194304,4194304] 0 2024-02-16T22:56:09.174 INFO:tasks.workunit.client.1.smithi192.stdout:1/89: unlink c2 0 2024-02-16T22:56:09.175 INFO:tasks.workunit.client.0.smithi064.stdout:2/34: dread f0 [0,4194304] 0 2024-02-16T22:56:09.178 INFO:tasks.workunit.client.1.smithi192.stdout:8/71: mkdir d5/d16 0 2024-02-16T22:56:09.184 INFO:tasks.workunit.client.1.smithi192.stdout:7/152: mknod d1/d19/c33 0 2024-02-16T22:56:09.184 INFO:tasks.workunit.client.1.smithi192.stdout:7/153: dread - d1/f12 zero size 2024-02-16T22:56:09.185 INFO:tasks.workunit.client.1.smithi192.stdout:7/154: chown d1/d2/f1e 21388 1 2024-02-16T22:56:09.185 INFO:tasks.workunit.client.1.smithi192.stdout:4/111: symlink d1/l27 0 2024-02-16T22:56:09.191 INFO:tasks.workunit.client.1.smithi192.stdout:4/112: dwrite d1/d8/d11/f1d [0,4194304] 0 2024-02-16T22:56:09.198 INFO:tasks.workunit.client.1.smithi192.stdout:7/155: creat d1/d2/de/f34 x:0 0 0 2024-02-16T22:56:09.200 INFO:tasks.workunit.client.1.smithi192.stdout:6/125: rename da/df/d12/d15/c1a to da/df/c28 0 2024-02-16T22:56:09.201 INFO:tasks.workunit.client.1.smithi192.stdout:1/90: symlink d3/l1c 0 2024-02-16T22:56:09.204 INFO:tasks.workunit.client.1.smithi192.stdout:8/72: creat d5/d9/f17 x:0 0 0 2024-02-16T22:56:09.204 INFO:tasks.workunit.client.1.smithi192.stdout:8/73: read - d5/d9/f17 zero size 2024-02-16T22:56:09.205 INFO:tasks.workunit.client.1.smithi192.stdout:8/74: truncate d5/f15 133830 0 2024-02-16T22:56:09.207 INFO:tasks.workunit.client.1.smithi192.stdout:3/103: creat da/f22 x:0 0 0 2024-02-16T22:56:09.207 INFO:tasks.workunit.client.1.smithi192.stdout:3/104: chown da 5 1 2024-02-16T22:56:09.210 INFO:tasks.workunit.client.1.smithi192.stdout:4/113: rename d1/f1c to d1/d26/f28 0 2024-02-16T22:56:09.211 INFO:tasks.workunit.client.1.smithi192.stdout:6/126: fdatasync da/df/d12/f17 0 2024-02-16T22:56:09.212 INFO:tasks.workunit.client.1.smithi192.stdout:9/183: write d7/d15/f27 [952588,14490] 0 2024-02-16T22:56:09.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/91: mknod d3/de/c1d 0 2024-02-16T22:56:09.214 INFO:tasks.workunit.client.1.smithi192.stdout:7/156: mknod d1/c35 0 2024-02-16T22:56:09.214 INFO:tasks.workunit.client.1.smithi192.stdout:8/75: dwrite d5/d9/f17 [0,4194304] 0 2024-02-16T22:56:09.214 INFO:tasks.workunit.client.1.smithi192.stdout:1/92: stat d3/fc 0 2024-02-16T22:56:09.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/114: mkdir d1/d15/d29 0 2024-02-16T22:56:09.221 INFO:tasks.workunit.client.1.smithi192.stdout:0/92: dwrite d1/d3/de/f10 [0,4194304] 0 2024-02-16T22:56:09.226 INFO:tasks.workunit.client.1.smithi192.stdout:3/105: link da/f22 da/df/f23 0 2024-02-16T22:56:09.226 INFO:tasks.workunit.client.1.smithi192.stdout:4/115: write d1/d8/fe [700486,91764] 0 2024-02-16T22:56:09.227 INFO:tasks.workunit.client.1.smithi192.stdout:0/93: write d1/d3/de/f10 [1579008,50135] 0 2024-02-16T22:56:09.230 INFO:tasks.workunit.client.1.smithi192.stdout:6/127: dread da/df/d12/f13 [0,4194304] 0 2024-02-16T22:56:09.232 INFO:tasks.workunit.client.1.smithi192.stdout:0/94: write d1/f5 [4634984,54637] 0 2024-02-16T22:56:09.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/157: dread d1/d2/de/f20 [0,4194304] 0 2024-02-16T22:56:09.236 INFO:tasks.workunit.client.1.smithi192.stdout:9/184: dwrite d7/d19/d37/f3b [0,4194304] 0 2024-02-16T22:56:09.237 INFO:tasks.workunit.client.1.smithi192.stdout:1/93: dwrite d3/f16 [0,4194304] 0 2024-02-16T22:56:09.239 INFO:tasks.workunit.client.1.smithi192.stdout:0/95: write d1/fa [464507,21360] 0 2024-02-16T22:56:09.243 INFO:tasks.workunit.client.1.smithi192.stdout:1/94: dread d3/de/f17 [0,4194304] 0 2024-02-16T22:56:09.244 INFO:tasks.workunit.client.1.smithi192.stdout:4/116: dwrite d1/d8/d11/f1d [0,4194304] 0 2024-02-16T22:56:09.247 INFO:tasks.workunit.client.1.smithi192.stdout:4/117: chown d1/l27 403619493 1 2024-02-16T22:56:09.247 INFO:tasks.workunit.client.1.smithi192.stdout:5/110: sync 2024-02-16T22:56:09.248 INFO:tasks.workunit.client.1.smithi192.stdout:3/106: rename da/df/f18 to da/d11/d20/f24 0 2024-02-16T22:56:09.248 INFO:tasks.workunit.client.1.smithi192.stdout:5/111: fsync d6/fd 0 2024-02-16T22:56:09.251 INFO:tasks.workunit.client.1.smithi192.stdout:4/118: write d1/d8/f9 [2901710,62882] 0 2024-02-16T22:56:09.251 INFO:tasks.workunit.client.1.smithi192.stdout:7/158: symlink d1/d19/l36 0 2024-02-16T22:56:09.252 INFO:tasks.workunit.client.1.smithi192.stdout:7/159: dread - d1/d2/de/f24 zero size 2024-02-16T22:56:09.253 INFO:tasks.workunit.client.1.smithi192.stdout:7/160: dread - d1/d2/f1e zero size 2024-02-16T22:56:09.256 INFO:tasks.workunit.client.1.smithi192.stdout:9/185: creat d7/d19/d28/f3e x:0 0 0 2024-02-16T22:56:09.258 INFO:tasks.workunit.client.1.smithi192.stdout:0/96: creat d1/f19 x:0 0 0 2024-02-16T22:56:09.263 INFO:tasks.workunit.client.1.smithi192.stdout:3/107: fsync da/f1a 0 2024-02-16T22:56:09.264 INFO:tasks.workunit.client.1.smithi192.stdout:3/108: chown da/c10 1 1 2024-02-16T22:56:09.265 INFO:tasks.workunit.client.1.smithi192.stdout:6/128: rename da/df/d12/f17 to da/df/d18/f29 0 2024-02-16T22:56:09.266 INFO:tasks.workunit.client.1.smithi192.stdout:8/76: truncate d5/f8 182611 0 2024-02-16T22:56:09.267 INFO:tasks.workunit.client.1.smithi192.stdout:4/119: write d1/d26/f28 [1472658,103117] 0 2024-02-16T22:56:09.273 INFO:tasks.workunit.client.1.smithi192.stdout:4/120: dwrite d1/d8/f9 [4194304,4194304] 0 2024-02-16T22:56:09.275 INFO:tasks.workunit.client.1.smithi192.stdout:1/95: mknod d3/c1e 0 2024-02-16T22:56:09.280 INFO:tasks.workunit.client.1.smithi192.stdout:3/109: write f4 [37148,118870] 0 2024-02-16T22:56:09.281 INFO:tasks.workunit.client.1.smithi192.stdout:3/110: write da/fe [667385,29611] 0 2024-02-16T22:56:09.282 INFO:tasks.workunit.client.1.smithi192.stdout:1/96: dwrite d3/de/f1a [0,4194304] 0 2024-02-16T22:56:09.283 INFO:tasks.workunit.client.1.smithi192.stdout:1/97: write d3/f4 [2528408,69963] 0 2024-02-16T22:56:09.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/186: fsync f0 0 2024-02-16T22:56:09.290 INFO:tasks.workunit.client.1.smithi192.stdout:3/111: dwrite da/f14 [0,4194304] 0 2024-02-16T22:56:09.291 INFO:tasks.workunit.client.1.smithi192.stdout:3/112: write da/fe [211249,24878] 0 2024-02-16T22:56:09.295 INFO:tasks.workunit.client.1.smithi192.stdout:3/113: dread f4 [0,4194304] 0 2024-02-16T22:56:09.296 INFO:tasks.workunit.client.1.smithi192.stdout:4/121: dread d1/d8/fc [0,4194304] 0 2024-02-16T22:56:09.304 INFO:tasks.workunit.client.1.smithi192.stdout:0/97: link d1/f15 d1/d3/d16/f1a 0 2024-02-16T22:56:09.308 INFO:tasks.workunit.client.1.smithi192.stdout:6/129: link da/df/d12/d15/d1b/l24 da/df/l2a 0 2024-02-16T22:56:09.309 INFO:tasks.workunit.client.1.smithi192.stdout:6/130: stat da/df/d12/d15/f21 0 2024-02-16T22:56:09.309 INFO:tasks.workunit.client.1.smithi192.stdout:6/131: dread - da/f1f zero size 2024-02-16T22:56:09.310 INFO:tasks.workunit.client.1.smithi192.stdout:3/114: symlink da/df/l25 0 2024-02-16T22:56:09.312 INFO:tasks.workunit.client.1.smithi192.stdout:4/122: fdatasync d1/fd 0 2024-02-16T22:56:09.314 INFO:tasks.workunit.client.1.smithi192.stdout:7/161: getdents d1/d2 0 2024-02-16T22:56:09.316 INFO:tasks.workunit.client.1.smithi192.stdout:9/187: mkdir d7/d15/d36/d38/d3f 0 2024-02-16T22:56:09.321 INFO:tasks.workunit.client.1.smithi192.stdout:6/132: rmdir da/df/d12 39 2024-02-16T22:56:09.323 INFO:tasks.workunit.client.1.smithi192.stdout:3/115: rmdir da/d11/d20 39 2024-02-16T22:56:09.328 INFO:tasks.workunit.client.1.smithi192.stdout:7/162: mkdir d1/d2/d37 0 2024-02-16T22:56:09.329 INFO:tasks.workunit.client.1.smithi192.stdout:6/133: dwrite da/df/f1c [0,4194304] 0 2024-02-16T22:56:09.330 INFO:tasks.workunit.client.1.smithi192.stdout:7/163: chown d1/f12 265499870 1 2024-02-16T22:56:09.331 INFO:tasks.workunit.client.1.smithi192.stdout:1/98: dwrite d3/fc [8388608,4194304] 0 2024-02-16T22:56:09.332 INFO:tasks.workunit.client.1.smithi192.stdout:1/99: chown d3/de/c14 1 1 2024-02-16T22:56:09.339 INFO:tasks.workunit.client.1.smithi192.stdout:6/134: chown da/df/c28 231526386 1 2024-02-16T22:56:09.340 INFO:tasks.workunit.client.1.smithi192.stdout:2/140: sync 2024-02-16T22:56:09.350 INFO:tasks.workunit.client.1.smithi192.stdout:0/98: getdents d1/d3/d16 0 2024-02-16T22:56:09.352 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:08 smithi064 ceph-mon[29226]: pgmap v9: 65 pgs: 65 active+clean; 313 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 78 KiB/s rd, 16 MiB/s wr, 250 op/s 2024-02-16T22:56:09.352 INFO:tasks.workunit.client.1.smithi192.stdout:4/123: dwrite d1/d8/fc [0,4194304] 0 2024-02-16T22:56:09.352 INFO:tasks.workunit.client.0.smithi064.stdout:9/57: mkdir d3/d11/d15 0 2024-02-16T22:56:09.353 INFO:tasks.workunit.client.0.smithi064.stdout:9/58: chown d3/l6 119173232 1 2024-02-16T22:56:09.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/116: creat da/f26 x:0 0 0 2024-02-16T22:56:09.354 INFO:tasks.workunit.client.0.smithi064.stdout:8/70: rmdir df 39 2024-02-16T22:56:09.355 INFO:tasks.workunit.client.0.smithi064.stdout:8/71: chown f1 8564 1 2024-02-16T22:56:09.356 INFO:tasks.workunit.client.1.smithi192.stdout:9/188: dwrite f1 [0,4194304] 0 2024-02-16T22:56:09.357 INFO:tasks.workunit.client.1.smithi192.stdout:6/135: mknod da/df/d12/d15/d1b/c2b 0 2024-02-16T22:56:09.359 INFO:tasks.workunit.client.1.smithi192.stdout:6/136: dread - da/f1f zero size 2024-02-16T22:56:09.360 INFO:tasks.workunit.client.0.smithi064.stdout:5/30: rmdir d6 39 2024-02-16T22:56:09.360 INFO:tasks.workunit.client.1.smithi192.stdout:7/164: fsync d1/d2/de/f28 0 2024-02-16T22:56:09.364 INFO:tasks.workunit.client.1.smithi192.stdout:1/100: creat d3/f1f x:0 0 0 2024-02-16T22:56:09.373 INFO:tasks.workunit.client.0.smithi064.stdout:3/60: mknod d4/cf 0 2024-02-16T22:56:09.373 INFO:tasks.workunit.client.0.smithi064.stdout:3/61: rename d4 to d4/d10 22 2024-02-16T22:56:09.374 INFO:tasks.workunit.client.1.smithi192.stdout:0/99: truncate d1/d3/d16/f1a 5338664 0 2024-02-16T22:56:09.375 INFO:tasks.workunit.client.1.smithi192.stdout:1/101: write d3/de/ff [3348473,83307] 0 2024-02-16T22:56:09.377 INFO:tasks.workunit.client.1.smithi192.stdout:1/102: truncate d3/f16 4839643 0 2024-02-16T22:56:09.377 INFO:tasks.workunit.client.0.smithi064.stdout:4/47: mkdir dc/de 0 2024-02-16T22:56:09.378 INFO:tasks.workunit.client.0.smithi064.stdout:1/54: sync 2024-02-16T22:56:09.379 INFO:tasks.workunit.client.0.smithi064.stdout:4/48: chown c4 0 1 2024-02-16T22:56:09.379 INFO:tasks.workunit.client.0.smithi064.stdout:1/55: readlink d2/lc 0 2024-02-16T22:56:09.379 INFO:tasks.workunit.client.0.smithi064.stdout:3/62: dwrite d4/f8 [4194304,4194304] 0 2024-02-16T22:56:09.380 INFO:tasks.workunit.client.0.smithi064.stdout:2/35: symlink l6 0 2024-02-16T22:56:09.382 INFO:tasks.workunit.client.1.smithi192.stdout:7/165: getdents d1/d2/de/d2c 0 2024-02-16T22:56:09.383 INFO:tasks.workunit.client.1.smithi192.stdout:7/166: readlink d1/d19/l36 0 2024-02-16T22:56:09.384 INFO:tasks.workunit.client.1.smithi192.stdout:0/100: symlink d1/d2/l1b 0 2024-02-16T22:56:09.385 INFO:tasks.workunit.client.1.smithi192.stdout:7/167: read d1/d2/de/f20 [990517,97548] 0 2024-02-16T22:56:09.385 INFO:tasks.workunit.client.0.smithi064.stdout:6/95: mkdir d0/d5/df/d17/d18 0 2024-02-16T22:56:09.386 INFO:tasks.workunit.client.0.smithi064.stdout:1/56: dread d2/f3 [0,4194304] 0 2024-02-16T22:56:09.386 INFO:tasks.workunit.client.1.smithi192.stdout:3/117: creat da/f27 x:0 0 0 2024-02-16T22:56:09.388 INFO:tasks.workunit.client.1.smithi192.stdout:9/189: dwrite d7/fe [0,4194304] 0 2024-02-16T22:56:09.389 INFO:tasks.workunit.client.0.smithi064.stdout:3/63: dwrite d4/fd [8388608,4194304] 0 2024-02-16T22:56:09.389 INFO:tasks.workunit.client.0.smithi064.stdout:1/57: write d2/fd [1802951,93500] 0 2024-02-16T22:56:09.390 INFO:tasks.workunit.client.0.smithi064.stdout:1/58: rename d2 to d2/df/d11/d12 22 2024-02-16T22:56:09.392 INFO:tasks.workunit.client.0.smithi064.stdout:8/72: write f1 [2998597,51218] 0 2024-02-16T22:56:09.397 INFO:tasks.workunit.client.1.smithi192.stdout:6/137: dwrite f9 [0,4194304] 0 2024-02-16T22:56:09.398 INFO:tasks.workunit.client.0.smithi064.stdout:7/30: getdents d5 0 2024-02-16T22:56:09.401 INFO:tasks.workunit.client.1.smithi192.stdout:9/190: mknod d7/d15/d36/c40 0 2024-02-16T22:56:09.401 INFO:tasks.workunit.client.1.smithi192.stdout:9/191: chown d7/d19/d28/f3e 17718540 1 2024-02-16T22:56:09.402 INFO:tasks.workunit.client.0.smithi064.stdout:2/36: dread f0 [4194304,4194304] 0 2024-02-16T22:56:09.402 INFO:tasks.workunit.client.0.smithi064.stdout:2/37: rmdir - no directory 2024-02-16T22:56:09.403 INFO:tasks.workunit.client.1.smithi192.stdout:7/168: creat d1/d2/de/d2c/f38 x:0 0 0 2024-02-16T22:56:09.404 INFO:tasks.workunit.client.0.smithi064.stdout:9/59: creat d3/d11/d15/f16 x:0 0 0 2024-02-16T22:56:09.406 INFO:tasks.workunit.client.1.smithi192.stdout:1/103: rename d3/ld to d3/l20 0 2024-02-16T22:56:09.407 INFO:tasks.workunit.client.0.smithi064.stdout:5/31: creat d6/fa x:0 0 0 2024-02-16T22:56:09.407 INFO:tasks.workunit.client.0.smithi064.stdout:5/32: readlink l5 0 2024-02-16T22:56:09.408 INFO:tasks.workunit.client.0.smithi064.stdout:5/33: stat d6/c7 0 2024-02-16T22:56:09.408 INFO:tasks.workunit.client.1.smithi192.stdout:3/118: mknod da/d11/d20/c28 0 2024-02-16T22:56:09.411 INFO:tasks.workunit.client.0.smithi064.stdout:6/96: rename d0/d6/d13 to d0/d6/d19 0 2024-02-16T22:56:09.412 INFO:tasks.workunit.client.0.smithi064.stdout:6/97: write d0/d6/f15 [8012464,116966] 0 2024-02-16T22:56:09.412 INFO:tasks.workunit.client.0.smithi064.stdout:6/98: readlink d0/l9 0 2024-02-16T22:56:09.413 INFO:tasks.workunit.client.0.smithi064.stdout:6/99: chown d0/d5/f11 435002390 1 2024-02-16T22:56:09.413 INFO:tasks.workunit.client.0.smithi064.stdout:6/100: stat d0/d6/f15 0 2024-02-16T22:56:09.415 INFO:tasks.workunit.client.1.smithi192.stdout:8/77: sync 2024-02-16T22:56:09.415 INFO:tasks.workunit.client.1.smithi192.stdout:5/112: sync 2024-02-16T22:56:09.418 INFO:tasks.workunit.client.1.smithi192.stdout:8/78: dread d5/f15 [0,4194304] 0 2024-02-16T22:56:09.419 INFO:tasks.workunit.client.1.smithi192.stdout:9/192: dwrite d7/d19/d28/f2c [0,4194304] 0 2024-02-16T22:56:09.419 INFO:tasks.workunit.client.0.smithi064.stdout:0/38: link c4 c7 0 2024-02-16T22:56:09.420 INFO:tasks.workunit.client.0.smithi064.stdout:0/39: chown f5 8370 1 2024-02-16T22:56:09.421 INFO:tasks.workunit.client.1.smithi192.stdout:0/101: write d1/d3/d16/f1a [1195247,130658] 0 2024-02-16T22:56:09.422 INFO:tasks.workunit.client.0.smithi064.stdout:6/101: dwrite d0/d6/d10/f14 [0,4194304] 0 2024-02-16T22:56:09.422 INFO:tasks.workunit.client.1.smithi192.stdout:6/138: creat da/d27/f2c x:0 0 0 2024-02-16T22:56:09.423 INFO:tasks.workunit.client.1.smithi192.stdout:6/139: truncate da/df/d12/f13 6660595 0 2024-02-16T22:56:09.424 INFO:tasks.workunit.client.1.smithi192.stdout:6/140: write da/df/d12/f26 [847656,95035] 0 2024-02-16T22:56:09.429 INFO:tasks.workunit.client.0.smithi064.stdout:4/49: link fb dc/ff 0 2024-02-16T22:56:09.433 INFO:tasks.workunit.client.1.smithi192.stdout:7/169: mknod d1/d2/c39 0 2024-02-16T22:56:09.435 INFO:tasks.workunit.client.1.smithi192.stdout:5/113: creat d6/d12/f1e x:0 0 0 2024-02-16T22:56:09.435 INFO:tasks.workunit.client.0.smithi064.stdout:9/60: mknod d3/d11/c17 0 2024-02-16T22:56:09.438 INFO:tasks.workunit.client.0.smithi064.stdout:5/34: creat d6/fb x:0 0 0 2024-02-16T22:56:09.441 INFO:tasks.workunit.client.1.smithi192.stdout:0/102: fdatasync d1/d3/d16/f1a 0 2024-02-16T22:56:09.443 INFO:tasks.workunit.client.1.smithi192.stdout:3/119: symlink da/d11/l29 0 2024-02-16T22:56:09.445 INFO:tasks.workunit.client.0.smithi064.stdout:1/59: dwrite d2/f9 [0,4194304] 0 2024-02-16T22:56:09.445 INFO:tasks.workunit.client.1.smithi192.stdout:9/193: truncate d7/d15/f20 509641 0 2024-02-16T22:56:09.445 INFO:tasks.workunit.client.1.smithi192.stdout:9/194: chown d7/d19/d37/d39 1599 1 2024-02-16T22:56:09.448 INFO:tasks.workunit.client.1.smithi192.stdout:6/141: unlink da/df/d18/l20 0 2024-02-16T22:56:09.449 INFO:tasks.workunit.client.1.smithi192.stdout:6/142: write da/df/f11 [806388,1545] 0 2024-02-16T22:56:09.449 INFO:tasks.workunit.client.0.smithi064.stdout:5/35: dwrite f4 [0,4194304] 0 2024-02-16T22:56:09.450 INFO:tasks.workunit.client.1.smithi192.stdout:6/143: write da/f1f [384142,4926] 0 2024-02-16T22:56:09.450 INFO:tasks.workunit.client.0.smithi064.stdout:5/36: write d6/fb [271907,36170] 0 2024-02-16T22:56:09.451 INFO:tasks.workunit.client.0.smithi064.stdout:8/73: unlink df/f13 0 2024-02-16T22:56:09.451 INFO:tasks.workunit.client.1.smithi192.stdout:6/144: read - da/df/d12/d15/f21 zero size 2024-02-16T22:56:09.452 INFO:tasks.workunit.client.0.smithi064.stdout:8/74: write df/f12 [977911,56126] 0 2024-02-16T22:56:09.452 INFO:tasks.workunit.client.1.smithi192.stdout:6/145: write da/df/d12/d15/d1b/f1e [7085251,127841] 0 2024-02-16T22:56:09.454 INFO:tasks.workunit.client.1.smithi192.stdout:6/146: truncate da/df/f11 1630163 0 2024-02-16T22:56:09.455 INFO:tasks.workunit.client.1.smithi192.stdout:6/147: dread - da/df/d12/d15/f21 zero size 2024-02-16T22:56:09.457 INFO:tasks.workunit.client.0.smithi064.stdout:0/40: fdatasync f5 0 2024-02-16T22:56:09.459 INFO:tasks.workunit.client.0.smithi064.stdout:7/31: unlink l4 0 2024-02-16T22:56:09.460 INFO:tasks.workunit.client.0.smithi064.stdout:7/32: read f0 [1202567,17031] 0 2024-02-16T22:56:09.460 INFO:tasks.workunit.client.0.smithi064.stdout:3/64: truncate d4/fd 6025498 0 2024-02-16T22:56:09.462 INFO:tasks.workunit.client.1.smithi192.stdout:1/104: rename d3/de/f17 to d3/f21 0 2024-02-16T22:56:09.462 INFO:tasks.workunit.client.1.smithi192.stdout:2/141: sync 2024-02-16T22:56:09.464 INFO:tasks.workunit.client.0.smithi064.stdout:6/102: rename d0/l9 to d0/d5/df/d17/l1a 0 2024-02-16T22:56:09.464 INFO:tasks.workunit.client.0.smithi064.stdout:6/103: stat d0/d6/d10/f14 0 2024-02-16T22:56:09.465 INFO:tasks.workunit.client.0.smithi064.stdout:6/104: stat d0/d5/df/d17/d18 0 2024-02-16T22:56:09.467 INFO:tasks.workunit.client.0.smithi064.stdout:4/50: mkdir dc/d10 0 2024-02-16T22:56:09.468 INFO:tasks.workunit.client.1.smithi192.stdout:1/105: dwrite d3/fb [0,4194304] 0 2024-02-16T22:56:09.469 INFO:tasks.workunit.client.1.smithi192.stdout:3/120: rmdir da/d11/d20 39 2024-02-16T22:56:09.470 INFO:tasks.workunit.client.0.smithi064.stdout:9/61: chown d3/f10 1613946 1 2024-02-16T22:56:09.477 INFO:tasks.workunit.client.1.smithi192.stdout:8/79: write d5/f8 [1222947,69184] 0 2024-02-16T22:56:09.481 INFO:tasks.workunit.client.1.smithi192.stdout:2/142: write f10 [451399,12776] 0 2024-02-16T22:56:09.482 INFO:tasks.workunit.client.1.smithi192.stdout:9/195: rename d7/d19/d28/c3a to d7/d15/d36/d38/c41 0 2024-02-16T22:56:09.483 INFO:tasks.workunit.client.0.smithi064.stdout:8/75: symlink df/l14 0 2024-02-16T22:56:09.487 INFO:tasks.workunit.client.1.smithi192.stdout:2/143: dwrite fa [0,4194304] 0 2024-02-16T22:56:09.487 INFO:tasks.workunit.client.0.smithi064.stdout:6/105: mkdir d0/d1b 0 2024-02-16T22:56:09.492 INFO:tasks.workunit.client.1.smithi192.stdout:5/114: link d6/d8/la d6/d12/d1d/l1f 0 2024-02-16T22:56:09.495 INFO:tasks.workunit.client.1.smithi192.stdout:3/121: rename da/d11/c1e to da/df/c2a 0 2024-02-16T22:56:09.496 INFO:tasks.workunit.client.1.smithi192.stdout:3/122: truncate da/f26 838876 0 2024-02-16T22:56:09.497 INFO:tasks.workunit.client.0.smithi064.stdout:5/37: link d6/fa d6/fc 0 2024-02-16T22:56:09.498 INFO:tasks.workunit.client.0.smithi064.stdout:9/62: write d3/f10 [38853,68738] 0 2024-02-16T22:56:09.501 INFO:tasks.workunit.client.0.smithi064.stdout:1/60: truncate d2/fd 741864 0 2024-02-16T22:56:09.502 INFO:tasks.workunit.client.1.smithi192.stdout:2/144: creat d22/d24/d2b/d2c/f2e x:0 0 0 2024-02-16T22:56:09.503 INFO:tasks.workunit.client.0.smithi064.stdout:7/33: mkdir d5/d6 0 2024-02-16T22:56:09.503 INFO:tasks.workunit.client.1.smithi192.stdout:2/145: chown d22/f2a 13 1 2024-02-16T22:56:09.503 INFO:tasks.workunit.client.1.smithi192.stdout:8/80: creat d5/d16/f18 x:0 0 0 2024-02-16T22:56:09.504 INFO:tasks.workunit.client.1.smithi192.stdout:8/81: rename d5/db/de/df to d5/db/de/df/d19 22 2024-02-16T22:56:09.506 INFO:tasks.workunit.client.0.smithi064.stdout:6/106: write d0/d5/f11 [193626,98762] 0 2024-02-16T22:56:09.508 INFO:tasks.workunit.client.0.smithi064.stdout:4/51: mknod dc/de/c11 0 2024-02-16T22:56:09.509 INFO:tasks.workunit.client.0.smithi064.stdout:4/52: chown f5 0 1 2024-02-16T22:56:09.510 INFO:tasks.workunit.client.0.smithi064.stdout:2/38: getdents . 0 2024-02-16T22:56:09.511 INFO:tasks.workunit.client.1.smithi192.stdout:0/103: getdents d1/d3 0 2024-02-16T22:56:09.511 INFO:tasks.workunit.client.0.smithi064.stdout:6/107: dread d0/d5/f11 [0,4194304] 0 2024-02-16T22:56:09.513 INFO:tasks.workunit.client.1.smithi192.stdout:1/106: rename d3/f4 to d3/f22 0 2024-02-16T22:56:09.514 INFO:tasks.workunit.client.1.smithi192.stdout:0/104: dread d1/d3/f12 [0,4194304] 0 2024-02-16T22:56:09.514 INFO:tasks.workunit.client.1.smithi192.stdout:1/107: fsync d3/fc 0 2024-02-16T22:56:09.514 INFO:tasks.workunit.client.0.smithi064.stdout:3/65: truncate d4/fd 5001294 0 2024-02-16T22:56:09.515 INFO:tasks.workunit.client.0.smithi064.stdout:5/38: symlink d6/ld 0 2024-02-16T22:56:09.515 INFO:tasks.workunit.client.0.smithi064.stdout:4/53: dwrite f5 [0,4194304] 0 2024-02-16T22:56:09.515 INFO:tasks.workunit.client.0.smithi064.stdout:5/39: fsync f2 0 2024-02-16T22:56:09.516 INFO:tasks.workunit.client.1.smithi192.stdout:7/170: getdents d1/d2/de/d2c 0 2024-02-16T22:56:09.516 INFO:tasks.workunit.client.0.smithi064.stdout:5/40: chown f4 3519767 1 2024-02-16T22:56:09.517 INFO:tasks.workunit.client.0.smithi064.stdout:5/41: dread - d6/fc zero size 2024-02-16T22:56:09.518 INFO:tasks.workunit.client.0.smithi064.stdout:9/63: creat d3/f18 x:0 0 0 2024-02-16T22:56:09.521 INFO:tasks.workunit.client.0.smithi064.stdout:4/54: dwrite f0 [0,4194304] 0 2024-02-16T22:56:09.522 INFO:tasks.workunit.client.1.smithi192.stdout:5/115: mkdir d6/d20 0 2024-02-16T22:56:09.522 INFO:tasks.workunit.client.0.smithi064.stdout:1/61: creat d2/df/f13 x:0 0 0 2024-02-16T22:56:09.524 INFO:tasks.workunit.client.1.smithi192.stdout:3/123: mkdir da/d11/d2b 0 2024-02-16T22:56:09.525 INFO:tasks.workunit.client.1.smithi192.stdout:3/124: readlink l7 0 2024-02-16T22:56:09.525 INFO:tasks.workunit.client.1.smithi192.stdout:3/125: chown f3 190 1 2024-02-16T22:56:09.525 INFO:tasks.workunit.client.1.smithi192.stdout:3/126: stat da/df/f23 0 2024-02-16T22:56:09.529 INFO:tasks.workunit.client.1.smithi192.stdout:3/127: dread da/f12 [0,4194304] 0 2024-02-16T22:56:09.537 INFO:tasks.workunit.client.1.smithi192.stdout:5/116: chown c4 1 1 2024-02-16T22:56:09.537 INFO:tasks.workunit.client.1.smithi192.stdout:9/196: getdents d7/df 0 2024-02-16T22:56:09.539 INFO:tasks.workunit.client.1.smithi192.stdout:9/197: chown d7/df/f23 1 1 2024-02-16T22:56:09.539 INFO:tasks.workunit.client.1.smithi192.stdout:8/82: dwrite f2 [0,4194304] 0 2024-02-16T22:56:09.539 INFO:tasks.workunit.client.1.smithi192.stdout:8/83: fsync d5/f8 0 2024-02-16T22:56:09.541 INFO:tasks.workunit.client.1.smithi192.stdout:3/128: truncate da/f1a 801413 0 2024-02-16T22:56:09.546 INFO:tasks.workunit.client.1.smithi192.stdout:9/198: dwrite d7/d19/d37/f3d [0,4194304] 0 2024-02-16T22:56:09.546 INFO:tasks.workunit.client.1.smithi192.stdout:0/105: rename d1/d2/cb to d1/d3/c1c 0 2024-02-16T22:56:09.551 INFO:tasks.workunit.client.1.smithi192.stdout:4/124: sync 2024-02-16T22:56:09.553 INFO:tasks.workunit.client.1.smithi192.stdout:8/84: dread d5/f15 [0,4194304] 0 2024-02-16T22:56:09.553 INFO:tasks.workunit.client.1.smithi192.stdout:8/85: readlink l1 0 2024-02-16T22:56:09.558 INFO:tasks.workunit.client.1.smithi192.stdout:9/199: symlink d7/d15/l42 0 2024-02-16T22:56:09.558 INFO:tasks.workunit.client.1.smithi192.stdout:2/146: link c1a d22/d24/c2f 0 2024-02-16T22:56:09.560 INFO:tasks.workunit.client.1.smithi192.stdout:2/147: write d22/f27 [848327,63805] 0 2024-02-16T22:56:09.563 INFO:tasks.workunit.client.1.smithi192.stdout:1/108: dwrite d3/f21 [0,4194304] 0 2024-02-16T22:56:09.563 INFO:tasks.workunit.client.1.smithi192.stdout:8/86: fdatasync d5/f15 0 2024-02-16T22:56:09.564 INFO:tasks.workunit.client.1.smithi192.stdout:7/171: rename d1/d2/f23 to d1/d19/f3a 0 2024-02-16T22:56:09.565 INFO:tasks.workunit.client.1.smithi192.stdout:9/200: mkdir d7/d19/d28/d43 0 2024-02-16T22:56:09.565 INFO:tasks.workunit.client.1.smithi192.stdout:7/172: fsync d1/f12 0 2024-02-16T22:56:09.567 INFO:tasks.workunit.client.1.smithi192.stdout:2/148: readlink l1d 0 2024-02-16T22:56:09.574 INFO:tasks.workunit.client.1.smithi192.stdout:7/173: chown d1/d2/l8 1900 1 2024-02-16T22:56:09.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/201: dwrite f2 [4194304,4194304] 0 2024-02-16T22:56:09.575 INFO:tasks.workunit.client.1.smithi192.stdout:5/117: getdents d6/d12 0 2024-02-16T22:56:09.575 INFO:tasks.workunit.client.1.smithi192.stdout:2/149: creat d22/d24/d2b/d2c/f30 x:0 0 0 2024-02-16T22:56:09.576 INFO:tasks.workunit.client.1.smithi192.stdout:0/106: symlink d1/d3/d18/l1d 0 2024-02-16T22:56:09.576 INFO:tasks.workunit.client.1.smithi192.stdout:7/174: dread - d1/d2/de/f24 zero size 2024-02-16T22:56:09.577 INFO:tasks.workunit.client.1.smithi192.stdout:3/129: rename da/d11/l1c to da/l2c 0 2024-02-16T22:56:09.577 INFO:tasks.workunit.client.1.smithi192.stdout:4/125: truncate d1/d8/fc 2664579 0 2024-02-16T22:56:09.577 INFO:tasks.workunit.client.1.smithi192.stdout:1/109: rename d3 to d3/de/d23 22 2024-02-16T22:56:09.578 INFO:tasks.workunit.client.1.smithi192.stdout:7/175: read - d1/d2/de/f24 zero size 2024-02-16T22:56:09.578 INFO:tasks.workunit.client.1.smithi192.stdout:2/150: rmdir d22/d24 39 2024-02-16T22:56:09.583 INFO:tasks.workunit.client.1.smithi192.stdout:3/130: dwrite da/f26 [0,4194304] 0 2024-02-16T22:56:09.583 INFO:tasks.workunit.client.1.smithi192.stdout:8/87: rename d5/db/dd/l13 to d5/db/de/df/l1a 0 2024-02-16T22:56:09.584 INFO:tasks.workunit.client.1.smithi192.stdout:3/131: fdatasync da/f27 0 2024-02-16T22:56:09.584 INFO:tasks.workunit.client.1.smithi192.stdout:8/88: write d5/f8 [1950915,73694] 0 2024-02-16T22:56:09.585 INFO:tasks.workunit.client.1.smithi192.stdout:1/110: symlink d3/de/l24 0 2024-02-16T22:56:09.585 INFO:tasks.workunit.client.1.smithi192.stdout:4/126: creat d1/d8/d11/f2a x:0 0 0 2024-02-16T22:56:09.586 INFO:tasks.workunit.client.1.smithi192.stdout:1/111: read - d3/f1f zero size 2024-02-16T22:56:09.586 INFO:tasks.workunit.client.1.smithi192.stdout:8/89: chown d5/db/de 3 1 2024-02-16T22:56:09.587 INFO:tasks.workunit.client.1.smithi192.stdout:9/202: creat d7/d19/d37/d39/f44 x:0 0 0 2024-02-16T22:56:09.588 INFO:tasks.workunit.client.1.smithi192.stdout:0/107: mknod d1/d3/de/c1e 0 2024-02-16T22:56:09.588 INFO:tasks.workunit.client.1.smithi192.stdout:1/112: fdatasync d3/de/f1a 0 2024-02-16T22:56:09.589 INFO:tasks.workunit.client.1.smithi192.stdout:0/108: chown d1/d3 128842 1 2024-02-16T22:56:09.590 INFO:tasks.workunit.client.1.smithi192.stdout:8/90: fdatasync d5/f8 0 2024-02-16T22:56:09.591 INFO:tasks.workunit.client.1.smithi192.stdout:5/118: getdents d6 0 2024-02-16T22:56:09.594 INFO:tasks.workunit.client.1.smithi192.stdout:7/176: creat d1/d2/f3b x:0 0 0 2024-02-16T22:56:09.595 INFO:tasks.workunit.client.1.smithi192.stdout:0/109: creat d1/f1f x:0 0 0 2024-02-16T22:56:09.598 INFO:tasks.workunit.client.1.smithi192.stdout:8/91: dread d5/d9/f17 [0,4194304] 0 2024-02-16T22:56:09.598 INFO:tasks.workunit.client.1.smithi192.stdout:1/113: dwrite d3/f19 [0,4194304] 0 2024-02-16T22:56:09.598 INFO:tasks.workunit.client.1.smithi192.stdout:5/119: symlink d6/d8/l21 0 2024-02-16T22:56:09.599 INFO:tasks.workunit.client.1.smithi192.stdout:8/92: fdatasync f2 0 2024-02-16T22:56:09.601 INFO:tasks.workunit.client.1.smithi192.stdout:9/203: unlink d7/d15/d36/d38/c41 0 2024-02-16T22:56:09.602 INFO:tasks.workunit.client.1.smithi192.stdout:9/204: write d7/fe [4472666,54169] 0 2024-02-16T22:56:09.603 INFO:tasks.workunit.client.1.smithi192.stdout:7/177: creat d1/d19/d2a/f3c x:0 0 0 2024-02-16T22:56:09.608 INFO:tasks.workunit.client.1.smithi192.stdout:4/127: read d1/d8/fc [854249,100362] 0 2024-02-16T22:56:09.610 INFO:tasks.workunit.client.1.smithi192.stdout:1/114: symlink d3/de/l25 0 2024-02-16T22:56:09.611 INFO:tasks.workunit.client.1.smithi192.stdout:5/120: mknod d6/d8/c22 0 2024-02-16T22:56:09.612 INFO:tasks.workunit.client.1.smithi192.stdout:5/121: readlink d6/d12/d1d/l1f 0 2024-02-16T22:56:09.614 INFO:tasks.workunit.client.1.smithi192.stdout:9/205: rename d7/d19/f21 to d7/d19/f45 0 2024-02-16T22:56:09.618 INFO:tasks.workunit.client.1.smithi192.stdout:4/128: rmdir d1/d8 39 2024-02-16T22:56:09.619 INFO:tasks.workunit.client.1.smithi192.stdout:1/115: fdatasync d3/fb 0 2024-02-16T22:56:09.619 INFO:tasks.workunit.client.1.smithi192.stdout:2/151: getdents d22/d24/d2b 0 2024-02-16T22:56:09.622 INFO:tasks.workunit.client.1.smithi192.stdout:0/110: write d1/d2/f4 [2342952,20538] 0 2024-02-16T22:56:09.623 INFO:tasks.workunit.client.1.smithi192.stdout:7/178: unlink c0 0 2024-02-16T22:56:09.624 INFO:tasks.workunit.client.1.smithi192.stdout:3/132: link da/l2c da/df/l2d 0 2024-02-16T22:56:09.624 INFO:tasks.workunit.client.1.smithi192.stdout:7/179: fdatasync d1/d2/f3b 0 2024-02-16T22:56:09.624 INFO:tasks.workunit.client.1.smithi192.stdout:1/116: dwrite d3/f21 [0,4194304] 0 2024-02-16T22:56:09.627 INFO:tasks.workunit.client.1.smithi192.stdout:6/148: sync 2024-02-16T22:56:09.629 INFO:tasks.workunit.client.1.smithi192.stdout:5/122: mknod d6/d12/d1d/c23 0 2024-02-16T22:56:09.630 INFO:tasks.workunit.client.1.smithi192.stdout:5/123: truncate d6/fd 2285757 0 2024-02-16T22:56:09.632 INFO:tasks.workunit.client.1.smithi192.stdout:3/133: creat da/df/f2e x:0 0 0 2024-02-16T22:56:09.633 INFO:tasks.workunit.client.1.smithi192.stdout:4/129: creat d1/d8/d11/f2b x:0 0 0 2024-02-16T22:56:09.634 INFO:tasks.workunit.client.1.smithi192.stdout:3/134: write da/d11/f21 [266909,42403] 0 2024-02-16T22:56:09.634 INFO:tasks.workunit.client.1.smithi192.stdout:4/130: write d1/d8/d11/f2a [141120,16846] 0 2024-02-16T22:56:09.652 INFO:tasks.workunit.client.1.smithi192.stdout:5/124: dread d6/f10 [0,4194304] 0 2024-02-16T22:56:09.652 INFO:tasks.workunit.client.0.smithi064.stdout:8/76: truncate fe 173376 0 2024-02-16T22:56:09.653 INFO:tasks.workunit.client.0.smithi064.stdout:5/42: rename d6/fb to d6/fe 0 2024-02-16T22:56:09.653 INFO:tasks.workunit.client.0.smithi064.stdout:1/62: write d2/df/f13 [966675,117138] 0 2024-02-16T22:56:09.653 INFO:tasks.workunit.client.1.smithi192.stdout:8/93: getdents d5/d16 0 2024-02-16T22:56:09.654 INFO:tasks.workunit.client.1.smithi192.stdout:8/94: chown d5/db 177569913 1 2024-02-16T22:56:09.654 INFO:tasks.workunit.client.1.smithi192.stdout:8/95: stat f2 0 2024-02-16T22:56:09.654 INFO:tasks.workunit.client.0.smithi064.stdout:1/63: write d2/f9 [651978,36195] 0 2024-02-16T22:56:09.655 INFO:tasks.workunit.client.1.smithi192.stdout:8/96: stat d5/d9/f17 0 2024-02-16T22:56:09.655 INFO:tasks.workunit.client.1.smithi192.stdout:5/125: dread f2 [0,4194304] 0 2024-02-16T22:56:09.655 INFO:tasks.workunit.client.0.smithi064.stdout:7/34: creat d5/d6/f7 x:0 0 0 2024-02-16T22:56:09.656 INFO:tasks.workunit.client.1.smithi192.stdout:6/149: creat da/df/d12/d15/f2d x:0 0 0 2024-02-16T22:56:09.659 INFO:tasks.workunit.client.1.smithi192.stdout:6/150: write da/df/f1c [4363484,35060] 0 2024-02-16T22:56:09.661 INFO:tasks.workunit.client.1.smithi192.stdout:0/111: rename d1/d3/c9 to d1/c20 0 2024-02-16T22:56:09.661 INFO:tasks.workunit.client.1.smithi192.stdout:6/151: write da/df/d12/f26 [103582,46209] 0 2024-02-16T22:56:09.662 INFO:tasks.workunit.client.1.smithi192.stdout:9/206: dwrite f0 [0,4194304] 0 2024-02-16T22:56:09.663 INFO:tasks.workunit.client.0.smithi064.stdout:9/64: mknod d3/d14/c19 0 2024-02-16T22:56:09.663 INFO:tasks.workunit.client.1.smithi192.stdout:9/207: read d7/d15/f33 [3216807,114766] 0 2024-02-16T22:56:09.665 INFO:tasks.workunit.client.0.smithi064.stdout:4/55: creat dc/d10/f12 x:0 0 0 2024-02-16T22:56:09.667 INFO:tasks.workunit.client.1.smithi192.stdout:2/152: creat d22/f31 x:0 0 0 2024-02-16T22:56:09.667 INFO:tasks.workunit.client.0.smithi064.stdout:2/39: truncate f0 683688 0 2024-02-16T22:56:09.668 INFO:tasks.workunit.client.1.smithi192.stdout:2/153: read - d22/d24/d2b/d2c/f30 zero size 2024-02-16T22:56:09.668 INFO:tasks.workunit.client.0.smithi064.stdout:8/77: creat df/f15 x:0 0 0 2024-02-16T22:56:09.669 INFO:tasks.workunit.client.0.smithi064.stdout:6/108: truncate d0/d6/d10/f14 2117312 0 2024-02-16T22:56:09.669 INFO:tasks.workunit.client.1.smithi192.stdout:6/152: dwrite da/df/d12/f13 [4194304,4194304] 0 2024-02-16T22:56:09.670 INFO:tasks.workunit.client.0.smithi064.stdout:6/109: chown d0/d6/d10 99 1 2024-02-16T22:56:09.671 INFO:tasks.workunit.client.1.smithi192.stdout:5/126: unlink d6/c14 0 2024-02-16T22:56:09.673 INFO:tasks.workunit.client.0.smithi064.stdout:5/43: creat d6/ff x:0 0 0 2024-02-16T22:56:09.675 INFO:tasks.workunit.client.1.smithi192.stdout:6/153: dwrite da/df/f1c [0,4194304] 0 2024-02-16T22:56:09.676 INFO:tasks.workunit.client.1.smithi192.stdout:6/154: readlink da/l22 0 2024-02-16T22:56:09.676 INFO:tasks.workunit.client.0.smithi064.stdout:1/64: mkdir d2/df/d14 0 2024-02-16T22:56:09.679 INFO:tasks.workunit.client.1.smithi192.stdout:7/180: creat d1/d2/f3d x:0 0 0 2024-02-16T22:56:09.681 INFO:tasks.workunit.client.0.smithi064.stdout:9/65: rename d3/ca to d3/c1a 0 2024-02-16T22:56:09.682 INFO:tasks.workunit.client.0.smithi064.stdout:9/66: dread - d3/d9/f12 zero size 2024-02-16T22:56:09.683 INFO:tasks.workunit.client.0.smithi064.stdout:4/56: mknod dc/de/c13 0 2024-02-16T22:56:09.684 INFO:tasks.workunit.client.1.smithi192.stdout:6/155: dwrite da/df/f1d [0,4194304] 0 2024-02-16T22:56:09.684 INFO:tasks.workunit.client.1.smithi192.stdout:6/156: readlink da/l22 0 2024-02-16T22:56:09.688 INFO:tasks.workunit.client.0.smithi064.stdout:2/40: unlink c2 0 2024-02-16T22:56:09.695 INFO:tasks.workunit.client.0.smithi064.stdout:6/110: creat d0/d6/d10/f1c x:0 0 0 2024-02-16T22:56:09.697 INFO:tasks.workunit.client.0.smithi064.stdout:5/44: creat d6/f10 x:0 0 0 2024-02-16T22:56:09.699 INFO:tasks.workunit.client.0.smithi064.stdout:1/65: symlink d2/df/l15 0 2024-02-16T22:56:09.703 INFO:tasks.workunit.client.1.smithi192.stdout:2/154: dwrite f4 [0,4194304] 0 2024-02-16T22:56:09.704 INFO:tasks.workunit.client.0.smithi064.stdout:9/67: mknod d3/c1b 0 2024-02-16T22:56:09.705 INFO:tasks.workunit.client.0.smithi064.stdout:1/66: dwrite d2/f3 [4194304,4194304] 0 2024-02-16T22:56:09.707 INFO:tasks.workunit.client.0.smithi064.stdout:4/57: creat dc/d10/f14 x:0 0 0 2024-02-16T22:56:09.715 INFO:tasks.workunit.client.1.smithi192.stdout:7/181: dwrite d1/d2/f29 [0,4194304] 0 2024-02-16T22:56:09.715 INFO:tasks.workunit.client.0.smithi064.stdout:4/58: dwrite f0 [0,4194304] 0 2024-02-16T22:56:09.716 INFO:tasks.workunit.client.1.smithi192.stdout:7/182: chown d1/d2/f3b 2259297 1 2024-02-16T22:56:09.716 INFO:tasks.workunit.client.1.smithi192.stdout:3/135: link da/df/f2e da/d11/f2f 0 2024-02-16T22:56:09.718 INFO:tasks.workunit.client.1.smithi192.stdout:3/136: read da/d11/f21 [242491,8144] 0 2024-02-16T22:56:09.718 INFO:tasks.workunit.client.0.smithi064.stdout:5/45: rename l5 to d6/l11 0 2024-02-16T22:56:09.720 INFO:tasks.workunit.client.1.smithi192.stdout:6/157: mknod da/df/d18/c2e 0 2024-02-16T22:56:09.722 INFO:tasks.workunit.client.0.smithi064.stdout:9/68: mknod d3/d11/c1c 0 2024-02-16T22:56:09.724 INFO:tasks.workunit.client.1.smithi192.stdout:9/208: mknod d7/d15/d36/d38/d3f/c46 0 2024-02-16T22:56:09.724 INFO:tasks.workunit.client.1.smithi192.stdout:9/209: dread - d7/d15/f32 zero size 2024-02-16T22:56:09.725 INFO:tasks.workunit.client.1.smithi192.stdout:9/210: fdatasync d7/d19/d37/f34 0 2024-02-16T22:56:09.731 INFO:tasks.workunit.client.0.smithi064.stdout:7/35: truncate f0 1000828 0 2024-02-16T22:56:09.731 INFO:tasks.workunit.client.0.smithi064.stdout:8/78: link df/c11 df/c16 0 2024-02-16T22:56:09.732 INFO:tasks.workunit.client.0.smithi064.stdout:7/36: write f3 [2071664,94171] 0 2024-02-16T22:56:09.733 INFO:tasks.workunit.client.1.smithi192.stdout:8/97: link d5/f15 d5/db/de/df/f1b 0 2024-02-16T22:56:09.737 INFO:tasks.workunit.client.0.smithi064.stdout:4/59: unlink c4 0 2024-02-16T22:56:09.737 INFO:tasks.workunit.client.0.smithi064.stdout:4/60: chown dc/d10/f12 0 1 2024-02-16T22:56:09.738 INFO:tasks.workunit.client.0.smithi064.stdout:5/46: symlink d6/l12 0 2024-02-16T22:56:09.739 INFO:tasks.workunit.client.0.smithi064.stdout:5/47: chown d6/l12 27257 1 2024-02-16T22:56:09.742 INFO:tasks.workunit.client.0.smithi064.stdout:9/69: mknod d3/c1d 0 2024-02-16T22:56:09.742 INFO:tasks.workunit.client.0.smithi064.stdout:9/70: read - d3/d9/f12 zero size 2024-02-16T22:56:09.743 INFO:tasks.workunit.client.1.smithi192.stdout:6/158: creat da/d27/f2f x:0 0 0 2024-02-16T22:56:09.743 INFO:tasks.workunit.client.0.smithi064.stdout:9/71: write d3/fc [511457,4329] 0 2024-02-16T22:56:09.744 INFO:tasks.workunit.client.0.smithi064.stdout:1/67: creat d2/df/d14/f16 x:0 0 0 2024-02-16T22:56:09.745 INFO:tasks.workunit.client.0.smithi064.stdout:9/72: write d3/d11/d15/f16 [304537,126613] 0 2024-02-16T22:56:09.746 INFO:tasks.workunit.client.0.smithi064.stdout:1/68: read d2/df/f13 [165716,110891] 0 2024-02-16T22:56:09.746 INFO:tasks.workunit.client.0.smithi064.stdout:9/73: dread - d3/d9/f12 zero size 2024-02-16T22:56:09.746 INFO:tasks.workunit.client.1.smithi192.stdout:9/211: creat d7/d15/d36/d38/f47 x:0 0 0 2024-02-16T22:56:09.747 INFO:tasks.workunit.client.1.smithi192.stdout:9/212: chown f0 43031 1 2024-02-16T22:56:09.747 INFO:tasks.workunit.client.1.smithi192.stdout:6/159: dread da/df/d12/d15/d1b/f1e [4194304,4194304] 0 2024-02-16T22:56:09.750 INFO:tasks.workunit.client.0.smithi064.stdout:8/79: mknod df/c17 0 2024-02-16T22:56:09.751 INFO:tasks.workunit.client.1.smithi192.stdout:9/213: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:09.752 INFO:tasks.workunit.client.1.smithi192.stdout:2/155: truncate f1 915580 0 2024-02-16T22:56:09.752 INFO:tasks.workunit.client.0.smithi064.stdout:9/74: dwrite d3/d9/f12 [0,4194304] 0 2024-02-16T22:56:09.757 INFO:tasks.workunit.client.1.smithi192.stdout:7/183: symlink d1/l3e 0 2024-02-16T22:56:09.757 INFO:tasks.workunit.client.1.smithi192.stdout:7/184: read d1/d2/de/f20 [403643,34250] 0 2024-02-16T22:56:09.758 INFO:tasks.workunit.client.0.smithi064.stdout:4/61: creat dc/d10/f15 x:0 0 0 2024-02-16T22:56:09.759 INFO:tasks.workunit.client.0.smithi064.stdout:4/62: fdatasync dc/d10/f15 0 2024-02-16T22:56:09.759 INFO:tasks.workunit.client.0.smithi064.stdout:4/63: stat f3 0 2024-02-16T22:56:09.760 INFO:tasks.workunit.client.0.smithi064.stdout:4/64: write f1 [1012989,104581] 0 2024-02-16T22:56:09.761 INFO:tasks.workunit.client.0.smithi064.stdout:5/48: creat d6/f13 x:0 0 0 2024-02-16T22:56:09.761 INFO:tasks.workunit.client.0.smithi064.stdout:4/65: readlink l9 0 2024-02-16T22:56:09.762 INFO:tasks.workunit.client.1.smithi192.stdout:3/137: creat da/d11/d20/f30 x:0 0 0 2024-02-16T22:56:09.763 INFO:tasks.workunit.client.1.smithi192.stdout:3/138: write da/f14 [2138319,124451] 0 2024-02-16T22:56:09.765 INFO:tasks.workunit.client.1.smithi192.stdout:5/127: write d6/f10 [1950125,86892] 0 2024-02-16T22:56:09.767 INFO:tasks.workunit.client.1.smithi192.stdout:6/160: rename l5 to da/df/d12/d15/l30 0 2024-02-16T22:56:09.768 INFO:tasks.workunit.client.1.smithi192.stdout:6/161: write da/df/d12/d15/f21 [129588,67471] 0 2024-02-16T22:56:09.771 INFO:tasks.workunit.client.1.smithi192.stdout:9/214: symlink d7/d15/d36/l48 0 2024-02-16T22:56:09.776 INFO:tasks.workunit.client.0.smithi064.stdout:9/75: rmdir d3/d9 39 2024-02-16T22:56:09.776 INFO:tasks.workunit.client.1.smithi192.stdout:9/215: dwrite d7/d19/d37/d39/f44 [0,4194304] 0 2024-02-16T22:56:09.777 INFO:tasks.workunit.client.1.smithi192.stdout:9/216: read - d7/d15/d36/d38/f47 zero size 2024-02-16T22:56:09.778 INFO:tasks.workunit.client.0.smithi064.stdout:6/111: dwrite d0/d6/d10/f14 [0,4194304] 0 2024-02-16T22:56:09.778 INFO:tasks.workunit.client.1.smithi192.stdout:9/217: write d7/d15/f27 [1877588,102431] 0 2024-02-16T22:56:09.780 INFO:tasks.workunit.client.0.smithi064.stdout:6/112: write d0/d6/d10/f14 [1454689,37339] 0 2024-02-16T22:56:09.780 INFO:tasks.workunit.client.0.smithi064.stdout:4/66: unlink f0 0 2024-02-16T22:56:09.782 INFO:tasks.workunit.client.0.smithi064.stdout:6/113: write d0/d6/d10/f14 [514525,31475] 0 2024-02-16T22:56:09.784 INFO:tasks.workunit.client.0.smithi064.stdout:4/67: truncate dc/d10/f15 981926 0 2024-02-16T22:56:09.785 INFO:tasks.workunit.client.0.smithi064.stdout:8/80: write fb [688652,38670] 0 2024-02-16T22:56:09.789 INFO:tasks.workunit.client.0.smithi064.stdout:1/69: unlink d2/l5 0 2024-02-16T22:56:09.791 INFO:tasks.workunit.client.1.smithi192.stdout:5/128: write d6/d12/f1e [510440,66264] 0 2024-02-16T22:56:09.792 INFO:tasks.workunit.client.1.smithi192.stdout:3/139: write f9 [1989621,87325] 0 2024-02-16T22:56:09.793 INFO:tasks.workunit.client.1.smithi192.stdout:6/162: creat da/df/d12/d15/f31 x:0 0 0 2024-02-16T22:56:09.794 INFO:tasks.workunit.client.1.smithi192.stdout:6/163: write da/df/f1c [1102379,78702] 0 2024-02-16T22:56:09.794 INFO:tasks.workunit.client.1.smithi192.stdout:2/156: symlink d22/l32 0 2024-02-16T22:56:09.798 INFO:tasks.workunit.client.0.smithi064.stdout:6/114: mkdir d0/d1d 0 2024-02-16T22:56:09.798 INFO:tasks.workunit.client.0.smithi064.stdout:6/115: chown d0/d6/d19 191923 1 2024-02-16T22:56:09.798 INFO:tasks.workunit.client.1.smithi192.stdout:7/185: truncate d1/d2/de/f28 3511611 0 2024-02-16T22:56:09.799 INFO:tasks.workunit.client.0.smithi064.stdout:0/41: sync 2024-02-16T22:56:09.800 INFO:tasks.workunit.client.0.smithi064.stdout:8/81: symlink df/l18 0 2024-02-16T22:56:09.800 INFO:tasks.workunit.client.0.smithi064.stdout:8/82: stat l7 0 2024-02-16T22:56:09.801 INFO:tasks.workunit.client.0.smithi064.stdout:9/76: mkdir d3/d9/d1e 0 2024-02-16T22:56:09.802 INFO:tasks.workunit.client.0.smithi064.stdout:6/116: dread d0/d6/d10/f14 [0,4194304] 0 2024-02-16T22:56:09.803 INFO:tasks.workunit.client.0.smithi064.stdout:9/77: read - d3/f18 zero size 2024-02-16T22:56:09.803 INFO:tasks.workunit.client.0.smithi064.stdout:9/78: chown d3/c1b 51255 1 2024-02-16T22:56:09.805 INFO:tasks.workunit.client.1.smithi192.stdout:3/140: unlink da/df/l25 0 2024-02-16T22:56:09.807 INFO:tasks.workunit.client.1.smithi192.stdout:9/218: dwrite d7/f12 [4194304,4194304] 0 2024-02-16T22:56:09.808 INFO:tasks.workunit.client.0.smithi064.stdout:0/42: dread f6 [0,4194304] 0 2024-02-16T22:56:09.808 INFO:tasks.workunit.client.1.smithi192.stdout:5/129: dread d6/fc [0,4194304] 0 2024-02-16T22:56:09.810 INFO:tasks.workunit.client.0.smithi064.stdout:8/83: chown df/c16 2 1 2024-02-16T22:56:09.812 INFO:tasks.workunit.client.0.smithi064.stdout:6/117: dwrite d0/d6/d10/f1c [0,4194304] 0 2024-02-16T22:56:09.814 INFO:tasks.workunit.client.0.smithi064.stdout:9/79: dwrite d3/d9/f12 [0,4194304] 0 2024-02-16T22:56:09.814 INFO:tasks.workunit.client.1.smithi192.stdout:2/157: read f1b [273697,78739] 0 2024-02-16T22:56:09.814 INFO:tasks.workunit.client.1.smithi192.stdout:8/98: getdents d5/db/de 0 2024-02-16T22:56:09.815 INFO:tasks.workunit.client.0.smithi064.stdout:9/80: dread - d3/f18 zero size 2024-02-16T22:56:09.816 INFO:tasks.workunit.client.1.smithi192.stdout:9/219: dwrite d7/df/f23 [0,4194304] 0 2024-02-16T22:56:09.816 INFO:tasks.workunit.client.1.smithi192.stdout:7/186: truncate d1/f11 2763870 0 2024-02-16T22:56:09.817 INFO:tasks.workunit.client.1.smithi192.stdout:5/130: dread f2 [0,4194304] 0 2024-02-16T22:56:09.817 INFO:tasks.workunit.client.0.smithi064.stdout:9/81: write d3/fc [489119,96792] 0 2024-02-16T22:56:09.817 INFO:tasks.workunit.client.0.smithi064.stdout:9/82: readlink d3/l13 0 2024-02-16T22:56:09.821 INFO:tasks.workunit.client.0.smithi064.stdout:0/43: unlink c4 0 2024-02-16T22:56:09.822 INFO:tasks.workunit.client.0.smithi064.stdout:4/68: getdents dc/de 0 2024-02-16T22:56:09.823 INFO:tasks.workunit.client.0.smithi064.stdout:4/69: chown dc/ff 280089 1 2024-02-16T22:56:09.825 INFO:tasks.workunit.client.1.smithi192.stdout:6/164: mknod da/df/c32 0 2024-02-16T22:56:09.826 INFO:tasks.workunit.client.0.smithi064.stdout:9/83: creat d3/d11/f1f x:0 0 0 2024-02-16T22:56:09.827 INFO:tasks.workunit.client.0.smithi064.stdout:9/84: chown d3/l8 689 1 2024-02-16T22:56:09.828 INFO:tasks.workunit.client.1.smithi192.stdout:8/99: creat d5/db/dd/f1c x:0 0 0 2024-02-16T22:56:09.830 INFO:tasks.workunit.client.1.smithi192.stdout:9/220: creat d7/d15/d36/d38/f49 x:0 0 0 2024-02-16T22:56:09.832 INFO:tasks.workunit.client.1.smithi192.stdout:9/221: write d7/fe [2639708,130108] 0 2024-02-16T22:56:09.832 INFO:tasks.workunit.client.1.smithi192.stdout:5/131: readlink d6/d8/l21 0 2024-02-16T22:56:09.833 INFO:tasks.workunit.client.1.smithi192.stdout:6/165: dwrite da/d27/f2f [0,4194304] 0 2024-02-16T22:56:09.833 INFO:tasks.workunit.client.1.smithi192.stdout:9/222: read - d7/d19/d28/f3e zero size 2024-02-16T22:56:09.835 INFO:tasks.workunit.client.0.smithi064.stdout:4/70: mkdir dc/dd/d16 0 2024-02-16T22:56:09.835 INFO:tasks.workunit.client.1.smithi192.stdout:7/187: symlink d1/d2/de/l3f 0 2024-02-16T22:56:09.836 INFO:tasks.workunit.client.0.smithi064.stdout:4/71: dread - dc/d10/f12 zero size 2024-02-16T22:56:09.836 INFO:tasks.workunit.client.1.smithi192.stdout:3/141: symlink da/d11/d2b/l31 0 2024-02-16T22:56:09.837 INFO:tasks.workunit.client.0.smithi064.stdout:4/72: dread - dc/ff zero size 2024-02-16T22:56:09.837 INFO:tasks.workunit.client.0.smithi064.stdout:0/44: link l0 l8 0 2024-02-16T22:56:09.838 INFO:tasks.workunit.client.1.smithi192.stdout:3/142: truncate da/df/f1f 525958 0 2024-02-16T22:56:09.838 INFO:tasks.workunit.client.1.smithi192.stdout:6/166: dread f9 [0,4194304] 0 2024-02-16T22:56:09.839 INFO:tasks.workunit.client.0.smithi064.stdout:9/85: mknod d3/c20 0 2024-02-16T22:56:09.839 INFO:tasks.workunit.client.1.smithi192.stdout:2/158: mknod d22/c33 0 2024-02-16T22:56:09.841 INFO:tasks.workunit.client.1.smithi192.stdout:2/159: fsync d22/d24/d2b/d2c/f2d 0 2024-02-16T22:56:09.843 INFO:tasks.workunit.client.0.smithi064.stdout:4/73: dwrite f3 [0,4194304] 0 2024-02-16T22:56:09.844 INFO:tasks.workunit.client.1.smithi192.stdout:6/167: dread da/df/f11 [0,4194304] 0 2024-02-16T22:56:09.845 INFO:tasks.workunit.client.0.smithi064.stdout:0/45: fsync f5 0 2024-02-16T22:56:09.850 INFO:tasks.workunit.client.0.smithi064.stdout:4/74: creat dc/f17 x:0 0 0 2024-02-16T22:56:09.851 INFO:tasks.workunit.client.1.smithi192.stdout:6/168: dwrite da/df/d12/d15/f31 [0,4194304] 0 2024-02-16T22:56:09.852 INFO:tasks.workunit.client.0.smithi064.stdout:4/75: fsync dc/d10/f12 0 2024-02-16T22:56:09.852 INFO:tasks.workunit.client.0.smithi064.stdout:4/76: chown fb 5 1 2024-02-16T22:56:09.852 INFO:tasks.workunit.client.0.smithi064.stdout:4/77: write dc/d10/f15 [1317954,86826] 0 2024-02-16T22:56:09.853 INFO:tasks.workunit.client.1.smithi192.stdout:9/223: unlink d7/d19/d37/f3b 0 2024-02-16T22:56:09.855 INFO:tasks.workunit.client.1.smithi192.stdout:7/188: write d1/d2/f1e [964968,68664] 0 2024-02-16T22:56:09.855 INFO:tasks.workunit.client.1.smithi192.stdout:3/143: mkdir da/d11/d2b/d32 0 2024-02-16T22:56:09.857 INFO:tasks.workunit.client.0.smithi064.stdout:9/86: link d3/fc d3/d9/f21 0 2024-02-16T22:56:09.858 INFO:tasks.workunit.client.1.smithi192.stdout:6/169: dread da/df/d12/d15/f31 [0,4194304] 0 2024-02-16T22:56:09.863 INFO:tasks.workunit.client.0.smithi064.stdout:4/78: mkdir dc/d10/d18 0 2024-02-16T22:56:09.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/160: creat d22/d24/f34 x:0 0 0 2024-02-16T22:56:09.864 INFO:tasks.workunit.client.1.smithi192.stdout:2/161: readlink d22/d24/l29 0 2024-02-16T22:56:09.864 INFO:tasks.workunit.client.1.smithi192.stdout:2/162: stat d22/d24/d2b/d2c/f2e 0 2024-02-16T22:56:09.865 INFO:tasks.workunit.client.0.smithi064.stdout:0/46: unlink l0 0 2024-02-16T22:56:09.865 INFO:tasks.workunit.client.1.smithi192.stdout:2/163: truncate f10 767001 0 2024-02-16T22:56:09.870 INFO:tasks.workunit.client.1.smithi192.stdout:4/131: sync 2024-02-16T22:56:09.870 INFO:tasks.workunit.client.1.smithi192.stdout:1/117: sync 2024-02-16T22:56:09.870 INFO:tasks.workunit.client.1.smithi192.stdout:0/112: sync 2024-02-16T22:56:09.871 INFO:tasks.workunit.client.1.smithi192.stdout:1/118: fdatasync d3/de/f1b 0 2024-02-16T22:56:09.871 INFO:tasks.workunit.client.1.smithi192.stdout:5/132: unlink d6/f11 0 2024-02-16T22:56:09.874 INFO:tasks.workunit.client.0.smithi064.stdout:4/79: rename dc/d10/f15 to dc/f19 0 2024-02-16T22:56:09.875 INFO:tasks.workunit.client.1.smithi192.stdout:1/119: dread d3/de/f1a [0,4194304] 0 2024-02-16T22:56:09.878 INFO:tasks.workunit.client.0.smithi064.stdout:9/87: symlink d3/d9/d1e/l22 0 2024-02-16T22:56:09.880 INFO:tasks.workunit.client.1.smithi192.stdout:9/224: rename d7/d19/c29 to d7/d19/d37/c4a 0 2024-02-16T22:56:09.881 INFO:tasks.workunit.client.1.smithi192.stdout:6/170: dread da/fb [0,4194304] 0 2024-02-16T22:56:09.881 INFO:tasks.workunit.client.1.smithi192.stdout:9/225: chown d7/df/f23 48063 1 2024-02-16T22:56:09.881 INFO:tasks.workunit.client.1.smithi192.stdout:3/144: symlink da/df/l33 0 2024-02-16T22:56:09.881 INFO:tasks.workunit.client.1.smithi192.stdout:0/113: dread d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:09.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/145: chown f4 14 1 2024-02-16T22:56:09.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/146: dread - da/df/f23 zero size 2024-02-16T22:56:09.883 INFO:tasks.workunit.client.0.smithi064.stdout:9/88: mkdir d3/d11/d15/d23 0 2024-02-16T22:56:09.884 INFO:tasks.workunit.client.0.smithi064.stdout:4/80: mknod dc/d10/d18/c1a 0 2024-02-16T22:56:09.885 INFO:tasks.workunit.client.0.smithi064.stdout:4/81: write f5 [2212040,72135] 0 2024-02-16T22:56:09.886 INFO:tasks.workunit.client.0.smithi064.stdout:4/82: dread - dc/d10/f14 zero size 2024-02-16T22:56:09.886 INFO:tasks.workunit.client.0.smithi064.stdout:9/89: mknod d3/d9/c24 0 2024-02-16T22:56:09.888 INFO:tasks.workunit.client.1.smithi192.stdout:2/164: symlink d22/d24/d2b/l35 0 2024-02-16T22:56:09.889 INFO:tasks.workunit.client.0.smithi064.stdout:9/90: read d3/d11/d15/f16 [116658,38157] 0 2024-02-16T22:56:09.892 INFO:tasks.workunit.client.0.smithi064.stdout:4/83: dwrite f3 [0,4194304] 0 2024-02-16T22:56:09.903 INFO:tasks.workunit.client.1.smithi192.stdout:8/100: dwrite d5/db/de/df/f1b [0,4194304] 0 2024-02-16T22:56:09.904 INFO:tasks.workunit.client.0.smithi064.stdout:4/84: unlink f1 0 2024-02-16T22:56:09.907 INFO:tasks.workunit.client.0.smithi064.stdout:9/91: creat d3/d11/d15/d23/f25 x:0 0 0 2024-02-16T22:56:09.915 INFO:tasks.workunit.client.1.smithi192.stdout:6/171: creat da/df/d12/d15/f33 x:0 0 0 2024-02-16T22:56:09.916 INFO:tasks.workunit.client.1.smithi192.stdout:9/226: dwrite d7/d15/f33 [0,4194304] 0 2024-02-16T22:56:09.927 INFO:tasks.workunit.client.0.smithi064.stdout:4/85: symlink dc/d10/l1b 0 2024-02-16T22:56:09.994 INFO:tasks.workunit.client.1.smithi192.stdout:3/147: mknod da/d11/d20/c34 0 2024-02-16T22:56:09.994 INFO:tasks.workunit.client.1.smithi192.stdout:3/148: dread - da/d11/f2f zero size 2024-02-16T22:56:09.997 INFO:tasks.workunit.client.1.smithi192.stdout:2/165: symlink d22/d24/d2b/l36 0 2024-02-16T22:56:09.998 INFO:tasks.workunit.client.1.smithi192.stdout:3/149: dread da/f26 [0,4194304] 0 2024-02-16T22:56:10.002 INFO:tasks.workunit.client.1.smithi192.stdout:3/150: dread da/fe [0,4194304] 0 2024-02-16T22:56:10.002 INFO:tasks.workunit.client.1.smithi192.stdout:7/189: truncate d1/d2/de/f20 1149861 0 2024-02-16T22:56:10.003 INFO:tasks.workunit.client.1.smithi192.stdout:4/132: mknod d1/d15/d29/c2c 0 2024-02-16T22:56:10.005 INFO:tasks.workunit.client.0.smithi064.stdout:9/92: dwrite d3/fc [0,4194304] 0 2024-02-16T22:56:10.006 INFO:tasks.workunit.client.1.smithi192.stdout:5/133: mknod d6/d8/c24 0 2024-02-16T22:56:10.006 INFO:tasks.workunit.client.1.smithi192.stdout:5/134: chown d6/d8/la 10893 1 2024-02-16T22:56:10.008 INFO:tasks.workunit.client.1.smithi192.stdout:8/101: rename l1 to d5/db/de/l1d 0 2024-02-16T22:56:10.022 INFO:tasks.workunit.client.1.smithi192.stdout:6/172: sync 2024-02-16T22:56:10.024 INFO:tasks.workunit.client.1.smithi192.stdout:7/190: dwrite d1/d19/f3a [0,4194304] 0 2024-02-16T22:56:10.026 INFO:tasks.workunit.client.1.smithi192.stdout:9/227: rename d7/d15/f32 to d7/d15/d36/d38/f4b 0 2024-02-16T22:56:10.026 INFO:tasks.workunit.client.1.smithi192.stdout:7/191: chown d1/d1f 113938 1 2024-02-16T22:56:10.028 INFO:tasks.workunit.client.1.smithi192.stdout:1/120: creat d3/f26 x:0 0 0 2024-02-16T22:56:10.028 INFO:tasks.workunit.client.1.smithi192.stdout:9/228: stat d7/d19/d28/f3e 0 2024-02-16T22:56:10.029 INFO:tasks.workunit.client.1.smithi192.stdout:0/114: creat d1/d3/de/f21 x:0 0 0 2024-02-16T22:56:10.031 INFO:tasks.workunit.client.1.smithi192.stdout:5/135: dwrite d6/fd [0,4194304] 0 2024-02-16T22:56:10.031 INFO:tasks.workunit.client.1.smithi192.stdout:9/229: read d7/d15/f27 [785367,42778] 0 2024-02-16T22:56:10.032 INFO:tasks.workunit.client.1.smithi192.stdout:2/166: write fe [5177689,45423] 0 2024-02-16T22:56:10.036 INFO:tasks.workunit.client.1.smithi192.stdout:4/133: link d1/d15/f24 d1/d8/d11/f2d 0 2024-02-16T22:56:10.044 INFO:tasks.workunit.client.1.smithi192.stdout:7/192: fdatasync d1/d2/f6 0 2024-02-16T22:56:10.044 INFO:tasks.workunit.client.1.smithi192.stdout:1/121: mkdir d3/de/d27 0 2024-02-16T22:56:10.046 INFO:tasks.workunit.client.1.smithi192.stdout:3/151: dwrite da/f22 [0,4194304] 0 2024-02-16T22:56:10.047 INFO:tasks.workunit.client.1.smithi192.stdout:5/136: write d6/d12/f1e [408554,24530] 0 2024-02-16T22:56:10.051 INFO:tasks.workunit.client.1.smithi192.stdout:8/102: creat d5/f1e x:0 0 0 2024-02-16T22:56:10.054 INFO:tasks.workunit.client.1.smithi192.stdout:8/103: dread d5/d9/f17 [0,4194304] 0 2024-02-16T22:56:10.061 INFO:tasks.workunit.client.1.smithi192.stdout:0/115: fdatasync d1/d3/f12 0 2024-02-16T22:56:10.069 INFO:tasks.workunit.client.1.smithi192.stdout:4/134: symlink d1/l2e 0 2024-02-16T22:56:10.072 INFO:tasks.workunit.client.1.smithi192.stdout:6/173: rename da/f1f to da/f34 0 2024-02-16T22:56:10.073 INFO:tasks.workunit.client.1.smithi192.stdout:8/104: rmdir d5/db/de 39 2024-02-16T22:56:10.073 INFO:tasks.workunit.client.1.smithi192.stdout:8/105: chown d5/db 3515 1 2024-02-16T22:56:10.076 INFO:tasks.workunit.client.1.smithi192.stdout:0/116: write d1/d3/fc [3310723,41879] 0 2024-02-16T22:56:10.077 INFO:tasks.workunit.client.1.smithi192.stdout:1/122: mknod d3/de/d27/c28 0 2024-02-16T22:56:10.078 INFO:tasks.workunit.client.1.smithi192.stdout:0/117: fsync d1/d3/fc 0 2024-02-16T22:56:10.079 INFO:tasks.workunit.client.1.smithi192.stdout:3/152: mkdir da/d11/d35 0 2024-02-16T22:56:10.082 INFO:tasks.workunit.client.1.smithi192.stdout:4/135: write d1/d8/fc [2180428,66037] 0 2024-02-16T22:56:10.085 INFO:tasks.workunit.client.1.smithi192.stdout:6/174: rmdir da/df/d12/d15/d1b 39 2024-02-16T22:56:10.093 INFO:tasks.workunit.client.1.smithi192.stdout:0/118: mkdir d1/d3/de/d22 0 2024-02-16T22:56:10.094 INFO:tasks.workunit.client.1.smithi192.stdout:3/153: rename da/df/l33 to da/d11/d35/l36 0 2024-02-16T22:56:10.096 INFO:tasks.workunit.client.1.smithi192.stdout:2/167: getdents d22/d24/d2b/d2c 0 2024-02-16T22:56:10.099 INFO:tasks.workunit.client.1.smithi192.stdout:4/136: rename d1/d15/f24 to d1/d15/f2f 0 2024-02-16T22:56:10.099 INFO:tasks.workunit.client.0.smithi064.stdout:3/66: sync 2024-02-16T22:56:10.101 INFO:tasks.workunit.client.1.smithi192.stdout:7/193: dwrite d1/f11 [0,4194304] 0 2024-02-16T22:56:10.102 INFO:tasks.workunit.client.1.smithi192.stdout:8/106: mknod d5/db/de/df/c1f 0 2024-02-16T22:56:10.102 INFO:tasks.workunit.client.1.smithi192.stdout:6/175: dread da/df/f11 [0,4194304] 0 2024-02-16T22:56:10.103 INFO:tasks.workunit.client.1.smithi192.stdout:7/194: read d1/d2/f1e [34708,5817] 0 2024-02-16T22:56:10.104 INFO:tasks.workunit.client.1.smithi192.stdout:7/195: write d1/f11 [3907184,50766] 0 2024-02-16T22:56:10.108 INFO:tasks.workunit.client.1.smithi192.stdout:8/107: dwrite f2 [0,4194304] 0 2024-02-16T22:56:10.109 INFO:tasks.workunit.client.1.smithi192.stdout:3/154: dread da/f1a [0,4194304] 0 2024-02-16T22:56:10.110 INFO:tasks.workunit.client.1.smithi192.stdout:3/155: chown f3 194 1 2024-02-16T22:56:10.110 INFO:tasks.workunit.client.1.smithi192.stdout:2/168: stat c20 0 2024-02-16T22:56:10.113 INFO:tasks.workunit.client.1.smithi192.stdout:5/137: getdents d6 0 2024-02-16T22:56:10.113 INFO:tasks.workunit.client.1.smithi192.stdout:3/156: dread da/f1a [0,4194304] 0 2024-02-16T22:56:10.114 INFO:tasks.workunit.client.1.smithi192.stdout:3/157: chown l7 31 1 2024-02-16T22:56:10.116 INFO:tasks.workunit.client.1.smithi192.stdout:8/108: dwrite d5/db/de/df/f14 [0,4194304] 0 2024-02-16T22:56:10.117 INFO:tasks.workunit.client.1.smithi192.stdout:8/109: stat d5/db/de/df/f14 0 2024-02-16T22:56:10.118 INFO:tasks.workunit.client.1.smithi192.stdout:8/110: write f2 [4229028,94703] 0 2024-02-16T22:56:10.120 INFO:tasks.workunit.client.1.smithi192.stdout:6/176: readlink da/df/l2a 0 2024-02-16T22:56:10.124 INFO:tasks.workunit.client.1.smithi192.stdout:6/177: dread da/df/f11 [0,4194304] 0 2024-02-16T22:56:10.125 INFO:tasks.workunit.client.1.smithi192.stdout:6/178: chown da/df/d12/d15 640409263 1 2024-02-16T22:56:10.126 INFO:tasks.workunit.client.1.smithi192.stdout:8/111: dwrite d5/f1e [0,4194304] 0 2024-02-16T22:56:10.129 INFO:tasks.workunit.client.1.smithi192.stdout:0/119: write d1/d2/f4 [3559483,52947] 0 2024-02-16T22:56:10.133 INFO:tasks.workunit.client.1.smithi192.stdout:2/169: rmdir d22/d24 39 2024-02-16T22:56:10.140 INFO:tasks.workunit.client.1.smithi192.stdout:3/158: fsync da/d11/f2f 0 2024-02-16T22:56:10.145 INFO:tasks.workunit.client.1.smithi192.stdout:3/159: dwrite da/df/f1f [0,4194304] 0 2024-02-16T22:56:10.146 INFO:tasks.workunit.client.1.smithi192.stdout:1/123: getdents d3 0 2024-02-16T22:56:10.149 INFO:tasks.workunit.client.1.smithi192.stdout:6/179: write f9 [5109937,31295] 0 2024-02-16T22:56:10.150 INFO:tasks.workunit.client.1.smithi192.stdout:8/112: mknod d5/d16/c20 0 2024-02-16T22:56:10.152 INFO:tasks.workunit.client.1.smithi192.stdout:7/196: creat d1/d2/d2d/f40 x:0 0 0 2024-02-16T22:56:10.153 INFO:tasks.workunit.client.1.smithi192.stdout:0/120: symlink d1/l23 0 2024-02-16T22:56:10.156 INFO:tasks.workunit.client.1.smithi192.stdout:6/180: dwrite da/df/d12/f26 [0,4194304] 0 2024-02-16T22:56:10.157 INFO:tasks.workunit.client.1.smithi192.stdout:4/137: creat d1/f30 x:0 0 0 2024-02-16T22:56:10.158 INFO:tasks.workunit.client.1.smithi192.stdout:4/138: dread - d1/f30 zero size 2024-02-16T22:56:10.163 INFO:tasks.workunit.client.0.smithi064.stdout:7/37: sync 2024-02-16T22:56:10.163 INFO:tasks.workunit.client.0.smithi064.stdout:2/41: sync 2024-02-16T22:56:10.163 INFO:tasks.workunit.client.0.smithi064.stdout:7/38: readlink - no filename 2024-02-16T22:56:10.163 INFO:tasks.workunit.client.0.smithi064.stdout:5/49: sync 2024-02-16T22:56:10.164 INFO:tasks.workunit.client.1.smithi192.stdout:9/230: sync 2024-02-16T22:56:10.164 INFO:tasks.workunit.client.1.smithi192.stdout:3/160: mknod da/d11/d2b/c37 0 2024-02-16T22:56:10.167 INFO:tasks.workunit.client.1.smithi192.stdout:1/124: rename d3/c1e to d3/c29 0 2024-02-16T22:56:10.171 INFO:tasks.workunit.client.1.smithi192.stdout:8/113: mkdir d5/db/de/d21 0 2024-02-16T22:56:10.171 INFO:tasks.workunit.client.1.smithi192.stdout:8/114: readlink d5/l10 0 2024-02-16T22:56:10.173 INFO:tasks.workunit.client.1.smithi192.stdout:5/138: truncate d6/fd 63949 0 2024-02-16T22:56:10.175 INFO:tasks.workunit.client.1.smithi192.stdout:0/121: creat d1/f24 x:0 0 0 2024-02-16T22:56:10.177 INFO:tasks.workunit.client.1.smithi192.stdout:2/170: creat d22/d24/d2b/d2c/f37 x:0 0 0 2024-02-16T22:56:10.177 INFO:tasks.workunit.client.1.smithi192.stdout:2/171: readlink l1d 0 2024-02-16T22:56:10.178 INFO:tasks.workunit.client.1.smithi192.stdout:2/172: truncate d22/f31 702530 0 2024-02-16T22:56:10.178 INFO:tasks.workunit.client.1.smithi192.stdout:2/173: readlink d22/l32 0 2024-02-16T22:56:10.179 INFO:tasks.workunit.client.1.smithi192.stdout:6/181: creat da/d27/f35 x:0 0 0 2024-02-16T22:56:10.179 INFO:tasks.workunit.client.1.smithi192.stdout:6/182: write da/df/f1c [5242664,9740] 0 2024-02-16T22:56:10.180 INFO:tasks.workunit.client.1.smithi192.stdout:6/183: write da/d27/f2f [3166707,94298] 0 2024-02-16T22:56:10.181 INFO:tasks.workunit.client.1.smithi192.stdout:9/231: fdatasync d7/f8 0 2024-02-16T22:56:10.183 INFO:tasks.workunit.client.1.smithi192.stdout:3/161: fsync da/d11/f21 0 2024-02-16T22:56:10.184 INFO:tasks.workunit.client.1.smithi192.stdout:1/125: fdatasync d3/f26 0 2024-02-16T22:56:10.189 INFO:tasks.workunit.client.1.smithi192.stdout:5/139: symlink d6/d12/d1d/l25 0 2024-02-16T22:56:10.190 INFO:tasks.workunit.client.1.smithi192.stdout:5/140: stat d6/lf 0 2024-02-16T22:56:10.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/174: creat d22/d24/d2b/f38 x:0 0 0 2024-02-16T22:56:10.191 INFO:tasks.workunit.client.1.smithi192.stdout:2/175: fdatasync d22/f31 0 2024-02-16T22:56:10.196 INFO:tasks.workunit.client.1.smithi192.stdout:6/184: unlink da/df/d12/f26 0 2024-02-16T22:56:10.206 INFO:tasks.workunit.client.1.smithi192.stdout:0/122: mknod d1/d3/de/d22/c25 0 2024-02-16T22:56:10.206 INFO:tasks.workunit.client.1.smithi192.stdout:1/126: dwrite d3/f19 [4194304,4194304] 0 2024-02-16T22:56:10.206 INFO:tasks.workunit.client.1.smithi192.stdout:0/123: chown d1 7328170 1 2024-02-16T22:56:10.221 INFO:tasks.workunit.client.1.smithi192.stdout:7/197: link d1/l1c d1/d19/d2a/l41 0 2024-02-16T22:56:10.221 INFO:tasks.workunit.client.1.smithi192.stdout:0/124: chown d1/d3/de/f21 5 1 2024-02-16T22:56:10.222 INFO:tasks.workunit.client.1.smithi192.stdout:1/127: write d3/f16 [5590639,32954] 0 2024-02-16T22:56:10.224 INFO:tasks.workunit.client.1.smithi192.stdout:6/185: mknod da/df/c36 0 2024-02-16T22:56:10.226 INFO:tasks.workunit.client.1.smithi192.stdout:9/232: link d7/d15/d36/l48 d7/d19/d28/l4c 0 2024-02-16T22:56:10.226 INFO:tasks.workunit.client.1.smithi192.stdout:9/233: chown d7/d19/l2d 72 1 2024-02-16T22:56:10.227 INFO:tasks.workunit.client.1.smithi192.stdout:0/125: dwrite d1/d3/fc [4194304,4194304] 0 2024-02-16T22:56:10.227 INFO:tasks.workunit.client.1.smithi192.stdout:8/115: getdents d5/db/dd 0 2024-02-16T22:56:10.228 INFO:tasks.workunit.client.1.smithi192.stdout:9/234: fsync d7/df/f23 0 2024-02-16T22:56:10.229 INFO:tasks.workunit.client.1.smithi192.stdout:3/162: rename da/fe to da/d11/f38 0 2024-02-16T22:56:10.230 INFO:tasks.workunit.client.1.smithi192.stdout:3/163: write f9 [3922784,30789] 0 2024-02-16T22:56:10.241 INFO:tasks.workunit.client.1.smithi192.stdout:0/126: stat d1/d3/d18/l1d 0 2024-02-16T22:56:10.246 INFO:tasks.workunit.client.1.smithi192.stdout:5/141: rename d6/d8/c13 to d6/d12/c26 0 2024-02-16T22:56:10.247 INFO:tasks.workunit.client.1.smithi192.stdout:3/164: mkdir da/d11/d20/d39 0 2024-02-16T22:56:10.249 INFO:tasks.workunit.client.1.smithi192.stdout:7/198: symlink d1/d2/d37/l42 0 2024-02-16T22:56:10.250 INFO:tasks.workunit.client.1.smithi192.stdout:1/128: mknod d3/c2a 0 2024-02-16T22:56:10.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/186: creat da/df/d12/d15/d1b/f37 x:0 0 0 2024-02-16T22:56:10.254 INFO:tasks.workunit.client.1.smithi192.stdout:8/116: symlink d5/l22 0 2024-02-16T22:56:10.254 INFO:tasks.workunit.client.1.smithi192.stdout:8/117: fdatasync d5/d16/f18 0 2024-02-16T22:56:10.255 INFO:tasks.workunit.client.1.smithi192.stdout:0/127: creat d1/d3/de/d22/f26 x:0 0 0 2024-02-16T22:56:10.260 INFO:tasks.workunit.client.1.smithi192.stdout:8/118: dwrite d5/db/de/df/f1b [4194304,4194304] 0 2024-02-16T22:56:10.261 INFO:tasks.workunit.client.1.smithi192.stdout:7/199: mknod d1/d19/c43 0 2024-02-16T22:56:10.262 INFO:tasks.workunit.client.1.smithi192.stdout:1/129: fdatasync d3/f21 0 2024-02-16T22:56:10.271 INFO:tasks.workunit.client.1.smithi192.stdout:5/142: truncate d6/fd 252924 0 2024-02-16T22:56:10.274 INFO:tasks.workunit.client.1.smithi192.stdout:7/200: mknod d1/d2/d2d/c44 0 2024-02-16T22:56:10.275 INFO:tasks.workunit.client.1.smithi192.stdout:7/201: write d1/d19/f3a [4142446,118038] 0 2024-02-16T22:56:10.277 INFO:tasks.workunit.client.1.smithi192.stdout:4/139: sync 2024-02-16T22:56:10.278 INFO:tasks.workunit.client.1.smithi192.stdout:9/235: write d7/f31 [3945383,1775] 0 2024-02-16T22:56:10.285 INFO:tasks.workunit.client.1.smithi192.stdout:0/128: creat d1/d3/d16/f27 x:0 0 0 2024-02-16T22:56:10.286 INFO:tasks.workunit.client.1.smithi192.stdout:0/129: dread d1/f8 [4194304,4194304] 0 2024-02-16T22:56:10.288 INFO:tasks.workunit.client.1.smithi192.stdout:0/130: write d1/f1f [639174,126386] 0 2024-02-16T22:56:10.288 INFO:tasks.workunit.client.1.smithi192.stdout:0/131: chown d1/l23 62 1 2024-02-16T22:56:10.289 INFO:tasks.workunit.client.1.smithi192.stdout:0/132: write d1/d3/d16/f1a [1721388,43825] 0 2024-02-16T22:56:10.293 INFO:tasks.workunit.client.1.smithi192.stdout:3/165: write da/d11/f38 [403808,119373] 0 2024-02-16T22:56:10.293 INFO:tasks.workunit.client.1.smithi192.stdout:8/119: truncate d5/f8 1697453 0 2024-02-16T22:56:10.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/120: read d5/db/de/df/f14 [2313915,58354] 0 2024-02-16T22:56:10.298 INFO:tasks.workunit.client.1.smithi192.stdout:4/140: creat d1/d26/f31 x:0 0 0 2024-02-16T22:56:10.302 INFO:tasks.workunit.client.1.smithi192.stdout:9/236: unlink d7/f31 0 2024-02-16T22:56:10.304 INFO:tasks.workunit.client.1.smithi192.stdout:1/130: rmdir d3/de 39 2024-02-16T22:56:10.305 INFO:tasks.workunit.client.1.smithi192.stdout:0/133: dread d1/fd [4194304,4194304] 0 2024-02-16T22:56:10.306 INFO:tasks.workunit.client.1.smithi192.stdout:9/237: dread d7/df/f16 [0,4194304] 0 2024-02-16T22:56:10.308 INFO:tasks.workunit.client.1.smithi192.stdout:6/187: creat da/df/d12/f38 x:0 0 0 2024-02-16T22:56:10.308 INFO:tasks.workunit.client.1.smithi192.stdout:6/188: chown da/df/d12 35587 1 2024-02-16T22:56:10.313 INFO:tasks.workunit.client.1.smithi192.stdout:5/143: dwrite d6/d12/f18 [0,4194304] 0 2024-02-16T22:56:10.314 INFO:tasks.workunit.client.1.smithi192.stdout:6/189: dwrite da/df/d12/d15/d1b/f1e [4194304,4194304] 0 2024-02-16T22:56:10.314 INFO:tasks.workunit.client.1.smithi192.stdout:5/144: chown d6/d8/la 5922 1 2024-02-16T22:56:10.316 INFO:tasks.workunit.client.1.smithi192.stdout:5/145: write d6/d12/f1e [516505,14532] 0 2024-02-16T22:56:10.316 INFO:tasks.workunit.client.1.smithi192.stdout:3/166: creat da/df/f3a x:0 0 0 2024-02-16T22:56:10.319 INFO:tasks.workunit.client.1.smithi192.stdout:7/202: write d1/d2/f1e [1361552,103565] 0 2024-02-16T22:56:10.319 INFO:tasks.workunit.client.1.smithi192.stdout:8/121: fdatasync d5/f8 0 2024-02-16T22:56:10.321 INFO:tasks.workunit.client.1.smithi192.stdout:8/122: read - d5/db/dd/f1c zero size 2024-02-16T22:56:10.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/141: readlink d1/lf 0 2024-02-16T22:56:10.325 INFO:tasks.workunit.client.1.smithi192.stdout:5/146: dwrite d6/d12/f18 [0,4194304] 0 2024-02-16T22:56:10.325 INFO:tasks.workunit.client.1.smithi192.stdout:5/147: chown d6/l7 955174345 1 2024-02-16T22:56:10.327 INFO:tasks.workunit.client.1.smithi192.stdout:5/148: dread - d6/d12/d15/f1c zero size 2024-02-16T22:56:10.330 INFO:tasks.workunit.client.1.smithi192.stdout:0/134: chown d1/d3/c1c 2075069693 1 2024-02-16T22:56:10.333 INFO:tasks.workunit.client.0.smithi064.stdout:8/84: sync 2024-02-16T22:56:10.334 INFO:tasks.workunit.client.0.smithi064.stdout:6/118: sync 2024-02-16T22:56:10.334 INFO:tasks.workunit.client.0.smithi064.stdout:1/70: sync 2024-02-16T22:56:10.337 INFO:tasks.workunit.client.0.smithi064.stdout:6/119: dread d0/d6/d10/f1c [0,4194304] 0 2024-02-16T22:56:10.339 INFO:tasks.workunit.client.1.smithi192.stdout:6/190: rmdir da/df/d12 39 2024-02-16T22:56:10.345 INFO:tasks.workunit.client.1.smithi192.stdout:7/203: mknod d1/d2/d37/c45 0 2024-02-16T22:56:10.349 INFO:tasks.workunit.client.1.smithi192.stdout:1/131: unlink d3/de/l24 0 2024-02-16T22:56:10.351 INFO:tasks.workunit.client.0.smithi064.stdout:4/86: creat dc/de/f1c x:0 0 0 2024-02-16T22:56:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:0/135: creat d1/d3/de/d22/f28 x:0 0 0 2024-02-16T22:56:10.353 INFO:tasks.workunit.client.1.smithi192.stdout:6/191: readlink da/ld 0 2024-02-16T22:56:10.358 INFO:tasks.workunit.client.0.smithi064.stdout:9/93: creat d3/d14/f26 x:0 0 0 2024-02-16T22:56:10.360 INFO:tasks.workunit.client.1.smithi192.stdout:2/176: sync 2024-02-16T22:56:10.360 INFO:tasks.workunit.client.1.smithi192.stdout:8/123: symlink d5/db/l23 0 2024-02-16T22:56:10.361 INFO:tasks.workunit.client.1.smithi192.stdout:8/124: chown d5/l7 73472 1 2024-02-16T22:56:10.364 INFO:tasks.workunit.client.1.smithi192.stdout:8/125: dread d5/db/de/df/f14 [0,4194304] 0 2024-02-16T22:56:10.364 INFO:tasks.workunit.client.0.smithi064.stdout:7/39: mkdir d5/d6/d8 0 2024-02-16T22:56:10.365 INFO:tasks.workunit.client.1.smithi192.stdout:8/126: read d5/f8 [1524466,74084] 0 2024-02-16T22:56:10.365 INFO:tasks.workunit.client.1.smithi192.stdout:5/149: mknod d6/d20/c27 0 2024-02-16T22:56:10.368 INFO:tasks.workunit.client.0.smithi064.stdout:8/85: creat df/f19 x:0 0 0 2024-02-16T22:56:10.368 INFO:tasks.workunit.client.0.smithi064.stdout:8/86: fdatasync f3 0 2024-02-16T22:56:10.369 INFO:tasks.workunit.client.1.smithi192.stdout:6/192: fdatasync da/df/f11 0 2024-02-16T22:56:10.372 INFO:tasks.workunit.client.0.smithi064.stdout:6/120: mknod d0/d6/da/c1e 0 2024-02-16T22:56:10.372 INFO:tasks.workunit.client.1.smithi192.stdout:2/177: mkdir d22/d24/d39 0 2024-02-16T22:56:10.372 INFO:tasks.workunit.client.1.smithi192.stdout:3/167: dwrite da/d11/f21 [0,4194304] 0 2024-02-16T22:56:10.373 INFO:tasks.workunit.client.1.smithi192.stdout:7/204: dread d1/d2/de/f28 [0,4194304] 0 2024-02-16T22:56:10.373 INFO:tasks.workunit.client.0.smithi064.stdout:4/87: mkdir dc/d1d 0 2024-02-16T22:56:10.376 INFO:tasks.workunit.client.0.smithi064.stdout:9/94: creat d3/d14/f27 x:0 0 0 2024-02-16T22:56:10.376 INFO:tasks.workunit.client.1.smithi192.stdout:8/127: dwrite d5/db/de/df/f1b [4194304,4194304] 0 2024-02-16T22:56:10.378 INFO:tasks.workunit.client.1.smithi192.stdout:5/150: creat d6/d12/d15/f28 x:0 0 0 2024-02-16T22:56:10.380 INFO:tasks.workunit.client.0.smithi064.stdout:3/67: truncate d4/fd 2925646 0 2024-02-16T22:56:10.380 INFO:tasks.workunit.client.1.smithi192.stdout:4/142: dwrite d1/fd [0,4194304] 0 2024-02-16T22:56:10.381 INFO:tasks.workunit.client.1.smithi192.stdout:4/143: stat d1/lf 0 2024-02-16T22:56:10.382 INFO:tasks.workunit.client.0.smithi064.stdout:2/42: fsync f0 0 2024-02-16T22:56:10.385 INFO:tasks.workunit.client.0.smithi064.stdout:7/40: creat d5/d6/f9 x:0 0 0 2024-02-16T22:56:10.386 INFO:tasks.workunit.client.0.smithi064.stdout:7/41: chown f3 11787046 1 2024-02-16T22:56:10.390 INFO:tasks.workunit.client.1.smithi192.stdout:1/132: truncate d3/fc 898116 0 2024-02-16T22:56:10.390 INFO:tasks.workunit.client.1.smithi192.stdout:1/133: fdatasync d3/de/ff 0 2024-02-16T22:56:10.393 INFO:tasks.workunit.client.1.smithi192.stdout:2/178: mknod d22/d24/c3a 0 2024-02-16T22:56:10.396 INFO:tasks.workunit.client.0.smithi064.stdout:8/87: creat df/f1a x:0 0 0 2024-02-16T22:56:10.398 INFO:tasks.workunit.client.1.smithi192.stdout:0/136: rename d1/d3/d16 to d1/d3/d29 0 2024-02-16T22:56:10.401 INFO:tasks.workunit.client.0.smithi064.stdout:6/121: read d0/d5/f11 [4174924,25168] 0 2024-02-16T22:56:10.403 INFO:tasks.workunit.client.1.smithi192.stdout:8/128: rmdir d5/db 39 2024-02-16T22:56:10.405 INFO:tasks.workunit.client.0.smithi064.stdout:0/47: getdents . 0 2024-02-16T22:56:10.406 INFO:tasks.workunit.client.1.smithi192.stdout:4/144: mknod d1/d26/c32 0 2024-02-16T22:56:10.407 INFO:tasks.workunit.client.0.smithi064.stdout:9/95: creat d3/d11/f28 x:0 0 0 2024-02-16T22:56:10.408 INFO:tasks.workunit.client.0.smithi064.stdout:9/96: write d3/d14/f27 [12178,74645] 0 2024-02-16T22:56:10.409 INFO:tasks.workunit.client.0.smithi064.stdout:3/68: creat d4/f11 x:0 0 0 2024-02-16T22:56:10.411 INFO:tasks.workunit.client.0.smithi064.stdout:2/43: mkdir d7 0 2024-02-16T22:56:10.412 INFO:tasks.workunit.client.1.smithi192.stdout:0/137: unlink d1/c14 0 2024-02-16T22:56:10.412 INFO:tasks.workunit.client.0.smithi064.stdout:9/97: dread d3/d9/f21 [0,4194304] 0 2024-02-16T22:56:10.413 INFO:tasks.workunit.client.0.smithi064.stdout:7/42: unlink d5/d6/f7 0 2024-02-16T22:56:10.414 INFO:tasks.workunit.client.1.smithi192.stdout:2/179: dread f1b [0,4194304] 0 2024-02-16T22:56:10.414 INFO:tasks.workunit.client.0.smithi064.stdout:1/71: creat d2/f17 x:0 0 0 2024-02-16T22:56:10.415 INFO:tasks.workunit.client.1.smithi192.stdout:2/180: dread - d22/d24/d2b/d2c/f2d zero size 2024-02-16T22:56:10.416 INFO:tasks.workunit.client.0.smithi064.stdout:6/122: fsync d0/d6/d10/f1c 0 2024-02-16T22:56:10.420 INFO:tasks.workunit.client.1.smithi192.stdout:2/181: dwrite f4 [4194304,4194304] 0 2024-02-16T22:56:10.420 INFO:tasks.workunit.client.1.smithi192.stdout:9/238: sync 2024-02-16T22:56:10.434 INFO:tasks.workunit.client.0.smithi064.stdout:5/50: sync 2024-02-16T22:56:10.434 INFO:tasks.workunit.client.1.smithi192.stdout:4/145: symlink d1/d8/d11/l33 0 2024-02-16T22:56:10.435 INFO:tasks.workunit.client.0.smithi064.stdout:5/51: chown d6/l12 829892 1 2024-02-16T22:56:10.435 INFO:tasks.workunit.client.1.smithi192.stdout:4/146: write d1/d8/d11/f2b [284951,51864] 0 2024-02-16T22:56:10.435 INFO:tasks.workunit.client.0.smithi064.stdout:0/48: write f3 [1409081,48571] 0 2024-02-16T22:56:10.436 INFO:tasks.workunit.client.1.smithi192.stdout:1/134: symlink d3/l2b 0 2024-02-16T22:56:10.442 INFO:tasks.workunit.client.1.smithi192.stdout:3/168: link l7 da/d11/d2b/d32/l3b 0 2024-02-16T22:56:10.445 INFO:tasks.workunit.client.1.smithi192.stdout:7/205: rename d1/d2/de/d2c/f38 to d1/d2/f46 0 2024-02-16T22:56:10.445 INFO:tasks.workunit.client.1.smithi192.stdout:7/206: readlink d1/d2/de/l16 0 2024-02-16T22:56:10.445 INFO:tasks.workunit.client.1.smithi192.stdout:7/207: read - d1/d2/f3d zero size 2024-02-16T22:56:10.446 INFO:tasks.workunit.client.1.smithi192.stdout:7/208: write d1/d2/de/f24 [113318,98171] 0 2024-02-16T22:56:10.447 INFO:tasks.workunit.client.1.smithi192.stdout:8/129: mknod d5/db/de/df/c24 0 2024-02-16T22:56:10.447 INFO:tasks.workunit.client.1.smithi192.stdout:7/209: write d1/d19/f1d [2750676,18544] 0 2024-02-16T22:56:10.448 INFO:tasks.workunit.client.0.smithi064.stdout:1/72: dread d2/fd [0,4194304] 0 2024-02-16T22:56:10.449 INFO:tasks.workunit.client.0.smithi064.stdout:2/44: truncate f0 445094 0 2024-02-16T22:56:10.449 INFO:tasks.workunit.client.0.smithi064.stdout:8/88: dwrite f2 [0,4194304] 0 2024-02-16T22:56:10.452 INFO:tasks.workunit.client.1.smithi192.stdout:4/147: mkdir d1/d26/d34 0 2024-02-16T22:56:10.461 INFO:tasks.workunit.client.1.smithi192.stdout:3/169: rmdir da 39 2024-02-16T22:56:10.462 INFO:tasks.workunit.client.1.smithi192.stdout:9/239: dwrite d7/f8 [0,4194304] 0 2024-02-16T22:56:10.468 INFO:tasks.workunit.client.1.smithi192.stdout:1/135: mkdir d3/d2c 0 2024-02-16T22:56:10.472 INFO:tasks.workunit.client.1.smithi192.stdout:2/182: rename c1f to d22/d24/d2b/c3b 0 2024-02-16T22:56:10.473 INFO:tasks.workunit.client.1.smithi192.stdout:0/138: getdents d1/d3/d18 0 2024-02-16T22:56:10.474 INFO:tasks.workunit.client.1.smithi192.stdout:1/136: chown d3/l11 378 1 2024-02-16T22:56:10.475 INFO:tasks.workunit.client.1.smithi192.stdout:9/240: link d7/d15/d36/d38/f47 d7/d19/d28/d43/f4d 0 2024-02-16T22:56:10.476 INFO:tasks.workunit.client.1.smithi192.stdout:9/241: write d7/d15/f20 [3892652,78006] 0 2024-02-16T22:56:10.477 INFO:tasks.workunit.client.1.smithi192.stdout:8/130: creat d5/f25 x:0 0 0 2024-02-16T22:56:10.478 INFO:tasks.workunit.client.1.smithi192.stdout:4/148: link d1/d26/f28 d1/f35 0 2024-02-16T22:56:10.480 INFO:tasks.workunit.client.1.smithi192.stdout:4/149: read d1/d26/f28 [2088729,56216] 0 2024-02-16T22:56:10.483 INFO:tasks.workunit.client.1.smithi192.stdout:7/210: rename d1/d19/d2a/l41 to d1/d2/d37/l47 0 2024-02-16T22:56:10.484 INFO:tasks.workunit.client.1.smithi192.stdout:9/242: creat d7/d19/d37/d39/f4e x:0 0 0 2024-02-16T22:56:10.485 INFO:tasks.workunit.client.1.smithi192.stdout:3/170: creat da/d11/f3c x:0 0 0 2024-02-16T22:56:10.485 INFO:tasks.workunit.client.1.smithi192.stdout:7/211: chown d1/d2/d2d 28889836 1 2024-02-16T22:56:10.487 INFO:tasks.workunit.client.1.smithi192.stdout:1/137: dread d3/fc [0,4194304] 0 2024-02-16T22:56:10.491 INFO:tasks.workunit.client.1.smithi192.stdout:8/131: getdents d5/d9 0 2024-02-16T22:56:10.493 INFO:tasks.workunit.client.0.smithi064.stdout:2/45: chown l4 15284 1 2024-02-16T22:56:10.493 INFO:tasks.workunit.client.1.smithi192.stdout:2/183: dread f19 [0,4194304] 0 2024-02-16T22:56:10.493 INFO:tasks.workunit.client.1.smithi192.stdout:3/171: write da/f12 [1563427,42575] 0 2024-02-16T22:56:10.494 INFO:tasks.workunit.client.0.smithi064.stdout:5/52: symlink d6/l14 0 2024-02-16T22:56:10.494 INFO:tasks.workunit.client.0.smithi064.stdout:0/49: rename f5 to f9 0 2024-02-16T22:56:10.494 INFO:tasks.workunit.client.0.smithi064.stdout:5/53: readlink d6/l14 0 2024-02-16T22:56:10.495 INFO:tasks.workunit.client.0.smithi064.stdout:9/98: unlink d3/c1a 0 2024-02-16T22:56:10.495 INFO:tasks.workunit.client.1.smithi192.stdout:7/212: chown d1/d2/d2d/l31 61627 1 2024-02-16T22:56:10.495 INFO:tasks.workunit.client.1.smithi192.stdout:6/193: sync 2024-02-16T22:56:10.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/151: sync 2024-02-16T22:56:10.497 INFO:tasks.workunit.client.0.smithi064.stdout:4/88: getdents dc/d10/d18 0 2024-02-16T22:56:10.499 INFO:tasks.workunit.client.1.smithi192.stdout:4/150: dwrite d1/d8/d11/f2d [0,4194304] 0 2024-02-16T22:56:10.499 INFO:tasks.workunit.client.1.smithi192.stdout:0/139: dread d1/fd [0,4194304] 0 2024-02-16T22:56:10.500 INFO:tasks.workunit.client.0.smithi064.stdout:9/99: creat d3/d14/f29 x:0 0 0 2024-02-16T22:56:10.500 INFO:tasks.workunit.client.0.smithi064.stdout:5/54: dread d6/fe [0,4194304] 0 2024-02-16T22:56:10.501 INFO:tasks.workunit.client.1.smithi192.stdout:0/140: fsync d1/d3/f12 0 2024-02-16T22:56:10.501 INFO:tasks.workunit.client.1.smithi192.stdout:1/138: unlink d3/f1f 0 2024-02-16T22:56:10.501 INFO:tasks.workunit.client.1.smithi192.stdout:5/152: dread d6/d12/f1e [0,4194304] 0 2024-02-16T22:56:10.502 INFO:tasks.workunit.client.0.smithi064.stdout:0/50: mkdir da 0 2024-02-16T22:56:10.503 INFO:tasks.workunit.client.0.smithi064.stdout:2/46: getdents d7 0 2024-02-16T22:56:10.505 INFO:tasks.workunit.client.0.smithi064.stdout:1/73: dwrite d2/f3 [4194304,4194304] 0 2024-02-16T22:56:10.505 INFO:tasks.workunit.client.0.smithi064.stdout:5/55: write d6/f13 [275799,122460] 0 2024-02-16T22:56:10.506 INFO:tasks.workunit.client.1.smithi192.stdout:7/213: readlink d1/d2/lc 0 2024-02-16T22:56:10.506 INFO:tasks.workunit.client.1.smithi192.stdout:6/194: write da/f34 [28940,100848] 0 2024-02-16T22:56:10.507 INFO:tasks.workunit.client.1.smithi192.stdout:1/139: dwrite d3/de/ff [0,4194304] 0 2024-02-16T22:56:10.509 INFO:tasks.workunit.client.0.smithi064.stdout:8/89: dwrite f9 [0,4194304] 0 2024-02-16T22:56:10.509 INFO:tasks.workunit.client.0.smithi064.stdout:5/56: dread - d6/fc zero size 2024-02-16T22:56:10.511 INFO:tasks.workunit.client.0.smithi064.stdout:3/69: dwrite d4/fd [0,4194304] 0 2024-02-16T22:56:10.511 INFO:tasks.workunit.client.0.smithi064.stdout:9/100: truncate d3/d11/f28 375608 0 2024-02-16T22:56:10.513 INFO:tasks.workunit.client.0.smithi064.stdout:8/90: truncate df/f1a 953469 0 2024-02-16T22:56:10.513 INFO:tasks.workunit.client.1.smithi192.stdout:6/195: dread f6 [0,4194304] 0 2024-02-16T22:56:10.514 INFO:tasks.workunit.client.1.smithi192.stdout:8/132: unlink d5/db/de/df/l1a 0 2024-02-16T22:56:10.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/133: dread - d5/f25 zero size 2024-02-16T22:56:10.515 INFO:tasks.workunit.client.0.smithi064.stdout:8/91: dread fb [0,4194304] 0 2024-02-16T22:56:10.516 INFO:tasks.workunit.client.1.smithi192.stdout:8/134: write d5/d16/f18 [179651,43075] 0 2024-02-16T22:56:10.516 INFO:tasks.workunit.client.0.smithi064.stdout:8/92: chown df/l10 76398 1 2024-02-16T22:56:10.516 INFO:tasks.workunit.client.1.smithi192.stdout:8/135: truncate d5/f1e 5110145 0 2024-02-16T22:56:10.520 INFO:tasks.workunit.client.0.smithi064.stdout:0/51: creat da/fb x:0 0 0 2024-02-16T22:56:10.526 INFO:tasks.workunit.client.0.smithi064.stdout:1/74: mkdir d2/df/d14/d18 0 2024-02-16T22:56:10.529 INFO:tasks.workunit.client.1.smithi192.stdout:4/151: creat d1/d26/f36 x:0 0 0 2024-02-16T22:56:10.530 INFO:tasks.workunit.client.0.smithi064.stdout:4/89: dwrite dc/f19 [0,4194304] 0 2024-02-16T22:56:10.531 INFO:tasks.workunit.client.0.smithi064.stdout:4/90: dread - dc/f17 zero size 2024-02-16T22:56:10.531 INFO:tasks.workunit.client.0.smithi064.stdout:9/101: mkdir d3/d14/d2a 0 2024-02-16T22:56:10.532 INFO:tasks.workunit.client.1.smithi192.stdout:0/141: creat d1/f2a x:0 0 0 2024-02-16T22:56:10.533 INFO:tasks.workunit.client.0.smithi064.stdout:3/70: creat d4/f12 x:0 0 0 2024-02-16T22:56:10.533 INFO:tasks.workunit.client.0.smithi064.stdout:9/102: write d3/d11/d15/d23/f25 [782545,80527] 0 2024-02-16T22:56:10.534 INFO:tasks.workunit.client.1.smithi192.stdout:9/243: getdents d7/d15/d36/d38 0 2024-02-16T22:56:10.534 INFO:tasks.workunit.client.1.smithi192.stdout:0/142: truncate d1/d3/fc 9403105 0 2024-02-16T22:56:10.534 INFO:tasks.workunit.client.1.smithi192.stdout:5/153: creat d6/f29 x:0 0 0 2024-02-16T22:56:10.535 INFO:tasks.workunit.client.1.smithi192.stdout:0/143: fsync d1/d3/fc 0 2024-02-16T22:56:10.540 INFO:tasks.workunit.client.0.smithi064.stdout:9/103: dwrite d3/d14/f27 [0,4194304] 0 2024-02-16T22:56:10.540 INFO:tasks.workunit.client.1.smithi192.stdout:2/184: creat d22/f3c x:0 0 0 2024-02-16T22:56:10.541 INFO:tasks.workunit.client.0.smithi064.stdout:3/71: mkdir d4/d13 0 2024-02-16T22:56:10.542 INFO:tasks.workunit.client.0.smithi064.stdout:9/104: chown d3/d11/d15 61 1 2024-02-16T22:56:10.542 INFO:tasks.workunit.client.0.smithi064.stdout:3/72: read d4/fc [1518301,36089] 0 2024-02-16T22:56:10.542 INFO:tasks.workunit.client.0.smithi064.stdout:9/105: dread - d3/f18 zero size 2024-02-16T22:56:10.544 INFO:tasks.workunit.client.1.smithi192.stdout:7/214: dwrite d1/d2/f46 [0,4194304] 0 2024-02-16T22:56:10.545 INFO:tasks.workunit.client.0.smithi064.stdout:1/75: creat d2/df/d14/d18/f19 x:0 0 0 2024-02-16T22:56:10.546 INFO:tasks.workunit.client.1.smithi192.stdout:4/152: creat d1/d26/f37 x:0 0 0 2024-02-16T22:56:10.547 INFO:tasks.workunit.client.1.smithi192.stdout:9/244: mknod d7/d15/d36/c4f 0 2024-02-16T22:56:10.548 INFO:tasks.workunit.client.0.smithi064.stdout:3/73: mkdir d4/d14 0 2024-02-16T22:56:10.551 INFO:tasks.workunit.client.0.smithi064.stdout:8/93: fsync f9 0 2024-02-16T22:56:10.551 INFO:tasks.workunit.client.0.smithi064.stdout:8/94: fsync fa 0 2024-02-16T22:56:10.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/144: rmdir d1/d2 39 2024-02-16T22:56:10.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/145: dread - d1/d3/de/d22/f28 zero size 2024-02-16T22:56:10.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/146: dread - d1/f19 zero size 2024-02-16T22:56:10.557 INFO:tasks.workunit.client.0.smithi064.stdout:3/74: creat d4/f15 x:0 0 0 2024-02-16T22:56:10.557 INFO:tasks.workunit.client.0.smithi064.stdout:0/52: fsync da/fb 0 2024-02-16T22:56:10.558 INFO:tasks.workunit.client.1.smithi192.stdout:8/136: mkdir d5/d26 0 2024-02-16T22:56:10.559 INFO:tasks.workunit.client.0.smithi064.stdout:1/76: mkdir d2/df/d11/d1a 0 2024-02-16T22:56:10.559 INFO:tasks.workunit.client.0.smithi064.stdout:1/77: chown d2/df 0 1 2024-02-16T22:56:10.560 INFO:tasks.workunit.client.0.smithi064.stdout:2/47: write f0 [737022,100836] 0 2024-02-16T22:56:10.565 INFO:tasks.workunit.client.0.smithi064.stdout:9/106: rename d3/d14 to d3/d11/d2b 0 2024-02-16T22:56:10.566 INFO:tasks.workunit.client.0.smithi064.stdout:1/78: symlink d2/df/d14/l1b 0 2024-02-16T22:56:10.567 INFO:tasks.workunit.client.0.smithi064.stdout:1/79: readlink l1 0 2024-02-16T22:56:10.568 INFO:tasks.workunit.client.0.smithi064.stdout:1/80: fdatasync d2/f17 0 2024-02-16T22:56:10.568 INFO:tasks.workunit.client.0.smithi064.stdout:5/57: dwrite d6/fc [0,4194304] 0 2024-02-16T22:56:10.570 INFO:tasks.workunit.client.1.smithi192.stdout:1/140: creat d3/f2d x:0 0 0 2024-02-16T22:56:10.573 INFO:tasks.workunit.client.0.smithi064.stdout:3/75: creat d4/d13/f16 x:0 0 0 2024-02-16T22:56:10.574 INFO:tasks.workunit.client.1.smithi192.stdout:3/172: link da/df/f2e da/f3d 0 2024-02-16T22:56:10.574 INFO:tasks.workunit.client.1.smithi192.stdout:7/215: dread d1/d19/f1d [0,4194304] 0 2024-02-16T22:56:10.577 INFO:tasks.workunit.client.0.smithi064.stdout:9/107: unlink d3/d11/d15/f16 0 2024-02-16T22:56:10.580 INFO:tasks.workunit.client.0.smithi064.stdout:1/81: creat d2/df/f1c x:0 0 0 2024-02-16T22:56:10.581 INFO:tasks.workunit.client.1.smithi192.stdout:2/185: mkdir d22/d24/d39/d3d 0 2024-02-16T22:56:10.587 INFO:tasks.workunit.client.0.smithi064.stdout:9/108: rmdir d3/d9 39 2024-02-16T22:56:10.590 INFO:tasks.workunit.client.0.smithi064.stdout:0/53: write f6 [485401,40700] 0 2024-02-16T22:56:10.593 INFO:tasks.workunit.client.1.smithi192.stdout:0/147: mknod d1/d3/c2b 0 2024-02-16T22:56:10.597 INFO:tasks.workunit.client.0.smithi064.stdout:3/76: rename d4/f15 to d4/f17 0 2024-02-16T22:56:10.597 INFO:tasks.workunit.client.0.smithi064.stdout:2/48: truncate f0 136128 0 2024-02-16T22:56:10.598 INFO:tasks.workunit.client.1.smithi192.stdout:4/153: dwrite d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:10.598 INFO:tasks.workunit.client.1.smithi192.stdout:6/196: rename da/df/d12/f38 to da/df/d12/f39 0 2024-02-16T22:56:10.598 INFO:tasks.workunit.client.1.smithi192.stdout:3/173: write f3 [810895,35114] 0 2024-02-16T22:56:10.599 INFO:tasks.workunit.client.0.smithi064.stdout:5/58: truncate d6/fc 701814 0 2024-02-16T22:56:10.601 INFO:tasks.workunit.client.0.smithi064.stdout:1/82: mkdir d2/df/d11/d1a/d1d 0 2024-02-16T22:56:10.603 INFO:tasks.workunit.client.1.smithi192.stdout:8/137: truncate f2 451498 0 2024-02-16T22:56:10.604 INFO:tasks.workunit.client.1.smithi192.stdout:5/154: getdents d6 0 2024-02-16T22:56:10.604 INFO:tasks.workunit.client.0.smithi064.stdout:0/54: mkdir da/dc 0 2024-02-16T22:56:10.607 INFO:tasks.workunit.client.0.smithi064.stdout:3/77: creat d4/f18 x:0 0 0 2024-02-16T22:56:10.607 INFO:tasks.workunit.client.0.smithi064.stdout:1/83: dwrite d2/f9 [0,4194304] 0 2024-02-16T22:56:10.608 INFO:tasks.workunit.client.0.smithi064.stdout:1/84: chown d2/df/d11 1486375587 1 2024-02-16T22:56:10.608 INFO:tasks.workunit.client.1.smithi192.stdout:3/174: dwrite da/d11/d20/f30 [0,4194304] 0 2024-02-16T22:56:10.609 INFO:tasks.workunit.client.0.smithi064.stdout:1/85: dread - d2/df/d14/f16 zero size 2024-02-16T22:56:10.614 INFO:tasks.workunit.client.1.smithi192.stdout:8/138: dwrite d5/d16/f18 [0,4194304] 0 2024-02-16T22:56:10.616 INFO:tasks.workunit.client.0.smithi064.stdout:9/109: mknod d3/d9/c2c 0 2024-02-16T22:56:10.617 INFO:tasks.workunit.client.1.smithi192.stdout:7/216: truncate d1/d2/f6 423584 0 2024-02-16T22:56:10.618 INFO:tasks.workunit.client.0.smithi064.stdout:9/110: write d3/d11/d2b/f26 [19325,67849] 0 2024-02-16T22:56:10.620 INFO:tasks.workunit.client.1.smithi192.stdout:7/217: write d1/d2/de/f34 [510203,127752] 0 2024-02-16T22:56:10.620 INFO:tasks.workunit.client.1.smithi192.stdout:2/186: dwrite f19 [0,4194304] 0 2024-02-16T22:56:10.623 INFO:tasks.workunit.client.1.smithi192.stdout:9/245: sync 2024-02-16T22:56:10.625 INFO:tasks.workunit.client.1.smithi192.stdout:9/246: fsync d7/d15/f33 0 2024-02-16T22:56:10.626 INFO:tasks.workunit.client.1.smithi192.stdout:9/247: fdatasync f2 0 2024-02-16T22:56:10.629 INFO:tasks.workunit.client.1.smithi192.stdout:9/248: dread - d7/d15/d36/d38/f47 zero size 2024-02-16T22:56:10.629 INFO:tasks.workunit.client.0.smithi064.stdout:3/78: mknod d4/d13/c19 0 2024-02-16T22:56:10.630 INFO:tasks.workunit.client.1.smithi192.stdout:0/148: dread d1/d2/f4 [0,4194304] 0 2024-02-16T22:56:10.635 INFO:tasks.workunit.client.0.smithi064.stdout:1/86: rename d2/fd to d2/df/f1e 0 2024-02-16T22:56:10.637 INFO:tasks.workunit.client.0.smithi064.stdout:9/111: symlink d3/d11/d15/d23/l2d 0 2024-02-16T22:56:10.638 INFO:tasks.workunit.client.0.smithi064.stdout:9/112: write d3/d11/d2b/f27 [303328,3790] 0 2024-02-16T22:56:10.639 INFO:tasks.workunit.client.1.smithi192.stdout:1/141: truncate d3/fb 7341231 0 2024-02-16T22:56:10.639 INFO:tasks.workunit.client.0.smithi064.stdout:5/59: truncate f0 324837 0 2024-02-16T22:56:10.640 INFO:tasks.workunit.client.1.smithi192.stdout:5/155: symlink d6/d12/l2a 0 2024-02-16T22:56:10.641 INFO:tasks.workunit.client.1.smithi192.stdout:5/156: read - d6/d12/d15/f1c zero size 2024-02-16T22:56:10.641 INFO:tasks.workunit.client.0.smithi064.stdout:0/55: symlink da/dc/ld 0 2024-02-16T22:56:10.641 INFO:tasks.workunit.client.1.smithi192.stdout:5/157: dread - d6/f29 zero size 2024-02-16T22:56:10.642 INFO:tasks.workunit.client.0.smithi064.stdout:0/56: write da/fb [843898,41327] 0 2024-02-16T22:56:10.643 INFO:tasks.workunit.client.0.smithi064.stdout:3/79: mknod d4/d13/c1a 0 2024-02-16T22:56:10.643 INFO:tasks.workunit.client.0.smithi064.stdout:5/60: dread d6/f13 [0,4194304] 0 2024-02-16T22:56:10.647 INFO:tasks.workunit.client.1.smithi192.stdout:8/139: unlink d5/db/de/df/f14 0 2024-02-16T22:56:10.649 INFO:tasks.workunit.client.0.smithi064.stdout:1/87: creat d2/df/f1f x:0 0 0 2024-02-16T22:56:10.649 INFO:tasks.workunit.client.0.smithi064.stdout:1/88: chown d2/ce 1187 1 2024-02-16T22:56:10.650 INFO:tasks.workunit.client.1.smithi192.stdout:2/187: mkdir d22/d24/d2b/d3e 0 2024-02-16T22:56:10.653 INFO:tasks.workunit.client.1.smithi192.stdout:7/218: symlink d1/d2/d37/l48 0 2024-02-16T22:56:10.656 INFO:tasks.workunit.client.0.smithi064.stdout:0/57: dread f9 [0,4194304] 0 2024-02-16T22:56:10.657 INFO:tasks.workunit.client.0.smithi064.stdout:5/61: fdatasync d6/fa 0 2024-02-16T22:56:10.658 INFO:tasks.workunit.client.1.smithi192.stdout:4/154: mknod d1/d8/c38 0 2024-02-16T22:56:10.659 INFO:tasks.workunit.client.1.smithi192.stdout:4/155: write d1/f5 [2344860,38793] 0 2024-02-16T22:56:10.659 INFO:tasks.workunit.client.0.smithi064.stdout:0/58: dread da/fb [0,4194304] 0 2024-02-16T22:56:10.660 INFO:tasks.workunit.client.0.smithi064.stdout:3/80: mkdir d4/d13/d1b 0 2024-02-16T22:56:10.660 INFO:tasks.workunit.client.0.smithi064.stdout:0/59: chown f9 369855707 1 2024-02-16T22:56:10.660 INFO:tasks.workunit.client.0.smithi064.stdout:3/81: readlink d4/l6 0 2024-02-16T22:56:10.661 INFO:tasks.workunit.client.0.smithi064.stdout:3/82: write d4/f12 [201033,8198] 0 2024-02-16T22:56:10.664 INFO:tasks.workunit.client.0.smithi064.stdout:1/89: symlink d2/df/d14/l20 0 2024-02-16T22:56:10.664 INFO:tasks.workunit.client.0.smithi064.stdout:1/90: fdatasync d2/df/f1c 0 2024-02-16T22:56:10.666 INFO:tasks.workunit.client.1.smithi192.stdout:6/197: unlink da/df/d12/d15/l30 0 2024-02-16T22:56:10.667 INFO:tasks.workunit.client.1.smithi192.stdout:6/198: chown da/d27/f35 3539 1 2024-02-16T22:56:10.667 INFO:tasks.workunit.client.0.smithi064.stdout:9/113: mkdir d3/d11/d2b/d2a/d2e 0 2024-02-16T22:56:10.668 INFO:tasks.workunit.client.1.smithi192.stdout:0/149: symlink d1/d3/d29/l2c 0 2024-02-16T22:56:10.671 INFO:tasks.workunit.client.0.smithi064.stdout:0/60: fsync f9 0 2024-02-16T22:56:10.673 INFO:tasks.workunit.client.0.smithi064.stdout:2/49: link f0 d7/f8 0 2024-02-16T22:56:10.675 INFO:tasks.workunit.client.1.smithi192.stdout:8/140: creat d5/d16/f27 x:0 0 0 2024-02-16T22:56:10.676 INFO:tasks.workunit.client.1.smithi192.stdout:8/141: truncate d5/d16/f18 4491321 0 2024-02-16T22:56:10.676 INFO:tasks.workunit.client.0.smithi064.stdout:1/91: symlink d2/df/d14/d18/l21 0 2024-02-16T22:56:10.677 INFO:tasks.workunit.client.0.smithi064.stdout:1/92: read - d2/df/d14/d18/f19 zero size 2024-02-16T22:56:10.678 INFO:tasks.workunit.client.1.smithi192.stdout:2/188: mkdir d22/d24/d2b/d2c/d3f 0 2024-02-16T22:56:10.682 INFO:tasks.workunit.client.0.smithi064.stdout:3/83: creat d4/d13/d1b/f1c x:0 0 0 2024-02-16T22:56:10.683 INFO:tasks.workunit.client.1.smithi192.stdout:9/249: symlink d7/d19/d28/d43/l50 0 2024-02-16T22:56:10.687 INFO:tasks.workunit.client.1.smithi192.stdout:6/199: dread f9 [4194304,4194304] 0 2024-02-16T22:56:10.687 INFO:tasks.workunit.client.1.smithi192.stdout:0/150: rename d1/d3/d18/l1d to d1/d2/l2d 0 2024-02-16T22:56:10.689 INFO:tasks.workunit.client.1.smithi192.stdout:6/200: truncate da/df/d12/d15/f33 1008751 0 2024-02-16T22:56:10.692 INFO:tasks.workunit.client.1.smithi192.stdout:0/151: dread d1/d3/fc [4194304,4194304] 0 2024-02-16T22:56:10.693 INFO:tasks.workunit.client.1.smithi192.stdout:2/189: truncate f1b 2109818 0 2024-02-16T22:56:10.693 INFO:tasks.workunit.client.1.smithi192.stdout:7/219: dread d1/d2/f1b [0,4194304] 0 2024-02-16T22:56:10.694 INFO:tasks.workunit.client.1.smithi192.stdout:0/152: chown d1/d3/fc 11363 1 2024-02-16T22:56:10.694 INFO:tasks.workunit.client.1.smithi192.stdout:5/158: write d6/f10 [314847,85018] 0 2024-02-16T22:56:10.695 INFO:tasks.workunit.client.1.smithi192.stdout:4/156: mknod d1/c39 0 2024-02-16T22:56:10.695 INFO:tasks.workunit.client.1.smithi192.stdout:1/142: link d3/de/l25 d3/de/d27/l2e 0 2024-02-16T22:56:10.696 INFO:tasks.workunit.client.1.smithi192.stdout:3/175: getdents da/d11/d2b 0 2024-02-16T22:56:10.697 INFO:tasks.workunit.client.1.smithi192.stdout:3/176: write da/d11/f38 [1535461,86911] 0 2024-02-16T22:56:10.698 INFO:tasks.workunit.client.1.smithi192.stdout:0/153: symlink d1/d2/l2e 0 2024-02-16T22:56:10.699 INFO:tasks.workunit.client.1.smithi192.stdout:2/190: dread f21 [0,4194304] 0 2024-02-16T22:56:10.700 INFO:tasks.workunit.client.1.smithi192.stdout:5/159: fdatasync f2 0 2024-02-16T22:56:10.700 INFO:tasks.workunit.client.1.smithi192.stdout:7/220: truncate d1/d2/de/f28 3958834 0 2024-02-16T22:56:10.701 INFO:tasks.workunit.client.1.smithi192.stdout:9/250: getdents d7/df 0 2024-02-16T22:56:10.702 INFO:tasks.workunit.client.1.smithi192.stdout:6/201: link da/df/f1c da/d27/f3a 0 2024-02-16T22:56:10.705 INFO:tasks.workunit.client.1.smithi192.stdout:3/177: dwrite da/df/f1f [0,4194304] 0 2024-02-16T22:56:10.709 INFO:tasks.workunit.client.1.smithi192.stdout:6/202: fsync f9 0 2024-02-16T22:56:10.710 INFO:tasks.workunit.client.1.smithi192.stdout:8/142: getdents d5/db 0 2024-02-16T22:56:10.711 INFO:tasks.workunit.client.1.smithi192.stdout:2/191: unlink f1b 0 2024-02-16T22:56:10.711 INFO:tasks.workunit.client.1.smithi192.stdout:8/143: chown d5/f1e 7689249 1 2024-02-16T22:56:10.713 INFO:tasks.workunit.client.1.smithi192.stdout:3/178: rename c6 to da/d11/d20/c3e 0 2024-02-16T22:56:10.713 INFO:tasks.workunit.client.1.smithi192.stdout:2/192: write d22/d24/d2b/d2c/f2e [20904,122057] 0 2024-02-16T22:56:10.714 INFO:tasks.workunit.client.1.smithi192.stdout:4/157: link d1/d8/d11/f2d d1/d8/f3a 0 2024-02-16T22:56:10.715 INFO:tasks.workunit.client.1.smithi192.stdout:1/143: write d3/f22 [6831201,112554] 0 2024-02-16T22:56:10.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/158: chown d1/d26/f28 287529902 1 2024-02-16T22:56:10.717 INFO:tasks.workunit.client.1.smithi192.stdout:1/144: chown d3/f22 24756 1 2024-02-16T22:56:10.718 INFO:tasks.workunit.client.0.smithi064.stdout:2/50: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:10.720 INFO:tasks.workunit.client.1.smithi192.stdout:6/203: write da/df/d12/f13 [6385900,85006] 0 2024-02-16T22:56:10.720 INFO:tasks.workunit.client.0.smithi064.stdout:2/51: dread f0 [0,4194304] 0 2024-02-16T22:56:10.720 INFO:tasks.workunit.client.1.smithi192.stdout:4/159: write d1/d26/f37 [185547,122217] 0 2024-02-16T22:56:10.721 INFO:tasks.workunit.client.0.smithi064.stdout:9/114: link d3/ce d3/d11/d2b/c2f 0 2024-02-16T22:56:10.721 INFO:tasks.workunit.client.0.smithi064.stdout:2/52: chown l1 204372 1 2024-02-16T22:56:10.721 INFO:tasks.workunit.client.1.smithi192.stdout:0/154: link d1/d3/de/f10 d1/d3/f2f 0 2024-02-16T22:56:10.722 INFO:tasks.workunit.client.1.smithi192.stdout:8/144: dwrite d5/f1e [4194304,4194304] 0 2024-02-16T22:56:10.722 INFO:tasks.workunit.client.0.smithi064.stdout:0/61: dwrite da/fb [0,4194304] 0 2024-02-16T22:56:10.723 INFO:tasks.workunit.client.0.smithi064.stdout:0/62: stat da/dc/ld 0 2024-02-16T22:56:10.725 INFO:tasks.workunit.client.0.smithi064.stdout:5/62: getdents d6 0 2024-02-16T22:56:10.727 INFO:tasks.workunit.client.1.smithi192.stdout:4/160: truncate d1/d8/f1e 748877 0 2024-02-16T22:56:10.727 INFO:tasks.workunit.client.1.smithi192.stdout:5/160: dwrite d6/d12/f1e [0,4194304] 0 2024-02-16T22:56:10.731 INFO:tasks.workunit.client.0.smithi064.stdout:3/84: mkdir d4/d14/d1d 0 2024-02-16T22:56:10.732 INFO:tasks.workunit.client.1.smithi192.stdout:9/251: link d7/d15/c26 d7/d19/d37/d39/c51 0 2024-02-16T22:56:10.733 INFO:tasks.workunit.client.1.smithi192.stdout:9/252: fsync d7/d15/d36/d38/f49 0 2024-02-16T22:56:10.736 INFO:tasks.workunit.client.0.smithi064.stdout:3/85: dread d4/fd [0,4194304] 0 2024-02-16T22:56:10.737 INFO:tasks.workunit.client.0.smithi064.stdout:3/86: write d4/f11 [225341,44281] 0 2024-02-16T22:56:10.738 INFO:tasks.workunit.client.0.smithi064.stdout:9/115: creat d3/d11/d2b/d2a/f30 x:0 0 0 2024-02-16T22:56:10.739 INFO:tasks.workunit.client.0.smithi064.stdout:3/87: dread - d4/d13/f16 zero size 2024-02-16T22:56:10.739 INFO:tasks.workunit.client.0.smithi064.stdout:3/88: dread - d4/d13/f16 zero size 2024-02-16T22:56:10.741 INFO:tasks.workunit.client.0.smithi064.stdout:3/89: write d4/f18 [819194,11304] 0 2024-02-16T22:56:10.741 INFO:tasks.workunit.client.0.smithi064.stdout:0/63: mkdir da/dc/de 0 2024-02-16T22:56:10.741 INFO:tasks.workunit.client.0.smithi064.stdout:2/53: write f0 [613600,104959] 0 2024-02-16T22:56:10.742 INFO:tasks.workunit.client.0.smithi064.stdout:3/90: fsync d4/f8 0 2024-02-16T22:56:10.743 INFO:tasks.workunit.client.0.smithi064.stdout:5/63: mknod d6/c15 0 2024-02-16T22:56:10.745 INFO:tasks.workunit.client.0.smithi064.stdout:5/64: dread d6/fa [0,4194304] 0 2024-02-16T22:56:10.745 INFO:tasks.workunit.client.0.smithi064.stdout:5/65: chown d6/c9 1 1 2024-02-16T22:56:10.746 INFO:tasks.workunit.client.0.smithi064.stdout:0/64: dread da/fb [0,4194304] 0 2024-02-16T22:56:10.748 INFO:tasks.workunit.client.1.smithi192.stdout:4/161: unlink d1/l2e 0 2024-02-16T22:56:10.749 INFO:tasks.workunit.client.1.smithi192.stdout:8/145: dwrite d5/d9/f17 [0,4194304] 0 2024-02-16T22:56:10.749 INFO:tasks.workunit.client.1.smithi192.stdout:5/161: fdatasync d6/fc 0 2024-02-16T22:56:10.752 INFO:tasks.workunit.client.1.smithi192.stdout:2/193: creat d22/d24/d39/d3d/f40 x:0 0 0 2024-02-16T22:56:10.752 INFO:tasks.workunit.client.0.smithi064.stdout:3/91: creat d4/f1e x:0 0 0 2024-02-16T22:56:10.754 INFO:tasks.workunit.client.0.smithi064.stdout:5/66: mknod d6/c16 0 2024-02-16T22:56:10.755 INFO:tasks.workunit.client.1.smithi192.stdout:1/145: mknod d3/c2f 0 2024-02-16T22:56:10.756 INFO:tasks.workunit.client.0.smithi064.stdout:2/54: dread f0 [0,4194304] 0 2024-02-16T22:56:10.756 INFO:tasks.workunit.client.0.smithi064.stdout:9/116: mkdir d3/d11/d2b/d2a/d2e/d31 0 2024-02-16T22:56:10.758 INFO:tasks.workunit.client.1.smithi192.stdout:1/146: write d3/f2d [370193,50742] 0 2024-02-16T22:56:10.758 INFO:tasks.workunit.client.1.smithi192.stdout:3/179: creat da/d11/f3f x:0 0 0 2024-02-16T22:56:10.758 INFO:tasks.workunit.client.1.smithi192.stdout:5/162: dread - d6/d12/d15/f28 zero size 2024-02-16T22:56:10.760 INFO:tasks.workunit.client.1.smithi192.stdout:4/162: dwrite d1/f19 [0,4194304] 0 2024-02-16T22:56:10.760 INFO:tasks.workunit.client.1.smithi192.stdout:0/155: write d1/fd [8028687,22340] 0 2024-02-16T22:56:10.762 INFO:tasks.workunit.client.1.smithi192.stdout:5/163: write d6/f29 [660838,25251] 0 2024-02-16T22:56:10.763 INFO:tasks.workunit.client.1.smithi192.stdout:8/146: dread d5/d16/f18 [0,4194304] 0 2024-02-16T22:56:10.764 INFO:tasks.workunit.client.0.smithi064.stdout:3/92: mkdir d4/d1f 0 2024-02-16T22:56:10.765 INFO:tasks.workunit.client.1.smithi192.stdout:5/164: chown d6/d8/la 38 1 2024-02-16T22:56:10.765 INFO:tasks.workunit.client.0.smithi064.stdout:3/93: write d4/f1e [133174,109561] 0 2024-02-16T22:56:10.766 INFO:tasks.workunit.client.0.smithi064.stdout:3/94: fdatasync d4/f5 0 2024-02-16T22:56:10.769 INFO:tasks.workunit.client.0.smithi064.stdout:5/67: creat d6/f17 x:0 0 0 2024-02-16T22:56:10.769 INFO:tasks.workunit.client.1.smithi192.stdout:6/204: dwrite da/df/f1c [0,4194304] 0 2024-02-16T22:56:10.772 INFO:tasks.workunit.client.1.smithi192.stdout:0/156: dwrite d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:10.772 INFO:tasks.workunit.client.1.smithi192.stdout:6/205: write da/df/d12/d15/f21 [396994,36619] 0 2024-02-16T22:56:10.773 INFO:tasks.workunit.client.1.smithi192.stdout:0/157: fdatasync d1/d3/de/d22/f26 0 2024-02-16T22:56:10.774 INFO:tasks.workunit.client.1.smithi192.stdout:0/158: chown d1/d3/de 1060680218 1 2024-02-16T22:56:10.775 INFO:tasks.workunit.client.0.smithi064.stdout:9/117: rename d3/c1b to d3/d11/d2b/d2a/c32 0 2024-02-16T22:56:10.775 INFO:tasks.workunit.client.0.smithi064.stdout:3/95: dwrite d4/f11 [0,4194304] 0 2024-02-16T22:56:10.776 INFO:tasks.workunit.client.1.smithi192.stdout:9/253: write d7/df/f16 [553887,9889] 0 2024-02-16T22:56:10.780 INFO:tasks.workunit.client.0.smithi064.stdout:9/118: dread d3/d9/f12 [0,4194304] 0 2024-02-16T22:56:10.785 INFO:tasks.workunit.client.1.smithi192.stdout:4/163: dread d1/d8/fe [0,4194304] 0 2024-02-16T22:56:10.787 INFO:tasks.workunit.client.1.smithi192.stdout:5/165: fdatasync f2 0 2024-02-16T22:56:10.788 INFO:tasks.workunit.client.0.smithi064.stdout:0/65: write f9 [470181,108396] 0 2024-02-16T22:56:10.788 INFO:tasks.workunit.client.0.smithi064.stdout:2/55: creat d7/f9 x:0 0 0 2024-02-16T22:56:10.790 INFO:tasks.workunit.client.1.smithi192.stdout:7/221: sync 2024-02-16T22:56:10.790 INFO:tasks.workunit.client.0.smithi064.stdout:2/56: dread f0 [0,4194304] 0 2024-02-16T22:56:10.791 INFO:tasks.workunit.client.1.smithi192.stdout:8/147: rename d5/l7 to d5/db/l28 0 2024-02-16T22:56:10.791 INFO:tasks.workunit.client.0.smithi064.stdout:9/119: dread d3/d11/d15/d23/f25 [0,4194304] 0 2024-02-16T22:56:10.792 INFO:tasks.workunit.client.0.smithi064.stdout:5/68: mkdir d6/d18 0 2024-02-16T22:56:10.794 INFO:tasks.workunit.client.1.smithi192.stdout:3/180: dread f4 [0,4194304] 0 2024-02-16T22:56:10.798 INFO:tasks.workunit.client.0.smithi064.stdout:9/120: dwrite f1 [0,4194304] 0 2024-02-16T22:56:10.799 INFO:tasks.workunit.client.0.smithi064.stdout:2/57: mknod d7/ca 0 2024-02-16T22:56:10.803 INFO:tasks.workunit.client.1.smithi192.stdout:9/254: creat d7/d19/d28/f52 x:0 0 0 2024-02-16T22:56:10.808 INFO:tasks.workunit.client.1.smithi192.stdout:1/147: dread d3/fb [0,4194304] 0 2024-02-16T22:56:10.808 INFO:tasks.workunit.client.1.smithi192.stdout:5/166: creat d6/f2b x:0 0 0 2024-02-16T22:56:10.808 INFO:tasks.workunit.client.0.smithi064.stdout:3/96: creat d4/d1f/f20 x:0 0 0 2024-02-16T22:56:10.810 INFO:tasks.workunit.client.1.smithi192.stdout:5/167: write d6/f29 [1676382,28510] 0 2024-02-16T22:56:10.810 INFO:tasks.workunit.client.1.smithi192.stdout:7/222: mkdir d1/d2/de/d2c/d49 0 2024-02-16T22:56:10.810 INFO:tasks.workunit.client.1.smithi192.stdout:5/168: fsync d6/d12/f18 0 2024-02-16T22:56:10.811 INFO:tasks.workunit.client.0.smithi064.stdout:0/66: fsync f9 0 2024-02-16T22:56:10.811 INFO:tasks.workunit.client.0.smithi064.stdout:0/67: read da/fb [1762088,99712] 0 2024-02-16T22:56:10.813 INFO:tasks.workunit.client.0.smithi064.stdout:2/58: mknod d7/cb 0 2024-02-16T22:56:10.814 INFO:tasks.workunit.client.0.smithi064.stdout:2/59: truncate d7/f9 80638 0 2024-02-16T22:56:10.815 INFO:tasks.workunit.client.1.smithi192.stdout:2/194: creat d22/f41 x:0 0 0 2024-02-16T22:56:10.816 INFO:tasks.workunit.client.0.smithi064.stdout:9/121: dread d3/d9/ff [0,4194304] 0 2024-02-16T22:56:10.816 INFO:tasks.workunit.client.0.smithi064.stdout:3/97: mknod d4/c21 0 2024-02-16T22:56:10.816 INFO:tasks.workunit.client.0.smithi064.stdout:0/68: dread da/fb [0,4194304] 0 2024-02-16T22:56:10.816 INFO:tasks.workunit.client.1.smithi192.stdout:0/159: dread d1/d3/f2f [0,4194304] 0 2024-02-16T22:56:10.819 INFO:tasks.workunit.client.0.smithi064.stdout:5/69: rmdir d6 39 2024-02-16T22:56:10.820 INFO:tasks.workunit.client.1.smithi192.stdout:9/255: creat d7/d19/d28/f53 x:0 0 0 2024-02-16T22:56:10.821 INFO:tasks.workunit.client.1.smithi192.stdout:9/256: write d7/d15/d36/d38/f49 [632111,31000] 0 2024-02-16T22:56:10.822 INFO:tasks.workunit.client.0.smithi064.stdout:9/122: dwrite d3/f18 [0,4194304] 0 2024-02-16T22:56:10.822 INFO:tasks.workunit.client.0.smithi064.stdout:3/98: creat d4/d14/f22 x:0 0 0 2024-02-16T22:56:10.824 INFO:tasks.workunit.client.0.smithi064.stdout:9/123: write d3/d11/d2b/f26 [614442,40891] 0 2024-02-16T22:56:10.826 INFO:tasks.workunit.client.1.smithi192.stdout:1/148: truncate d3/fc 800670 0 2024-02-16T22:56:10.826 INFO:tasks.workunit.client.1.smithi192.stdout:1/149: read d3/de/ff [1774670,118478] 0 2024-02-16T22:56:10.826 INFO:tasks.workunit.client.1.smithi192.stdout:1/150: stat d3/de/f1a 0 2024-02-16T22:56:10.827 INFO:tasks.workunit.client.0.smithi064.stdout:2/60: link d7/f9 d7/fc 0 2024-02-16T22:56:10.829 INFO:tasks.workunit.client.1.smithi192.stdout:4/164: write d1/d8/f3a [3563665,39960] 0 2024-02-16T22:56:10.832 INFO:tasks.workunit.client.1.smithi192.stdout:7/223: unlink d1/d2/d37/l48 0 2024-02-16T22:56:10.834 INFO:tasks.workunit.client.1.smithi192.stdout:5/169: symlink d6/d12/l2c 0 2024-02-16T22:56:10.835 INFO:tasks.workunit.client.1.smithi192.stdout:5/170: fsync d6/d12/f18 0 2024-02-16T22:56:10.836 INFO:tasks.workunit.client.1.smithi192.stdout:4/165: dwrite d1/d15/f2f [0,4194304] 0 2024-02-16T22:56:10.836 INFO:tasks.workunit.client.0.smithi064.stdout:0/69: mknod da/dc/de/cf 0 2024-02-16T22:56:10.837 INFO:tasks.workunit.client.1.smithi192.stdout:2/195: chown f8 31741 1 2024-02-16T22:56:10.839 INFO:tasks.workunit.client.0.smithi064.stdout:2/61: write d7/f8 [1421291,96802] 0 2024-02-16T22:56:10.840 INFO:tasks.workunit.client.0.smithi064.stdout:2/62: chown l6 70687 1 2024-02-16T22:56:10.842 INFO:tasks.workunit.client.1.smithi192.stdout:5/171: dwrite d6/d12/f1e [0,4194304] 0 2024-02-16T22:56:10.843 INFO:tasks.workunit.client.0.smithi064.stdout:2/63: read f0 [846097,10414] 0 2024-02-16T22:56:10.845 INFO:tasks.workunit.client.1.smithi192.stdout:9/257: mkdir d7/d15/d36/d54 0 2024-02-16T22:56:10.846 INFO:tasks.workunit.client.1.smithi192.stdout:9/258: write d7/d19/d37/d39/f4e [245807,61557] 0 2024-02-16T22:56:10.847 INFO:tasks.workunit.client.0.smithi064.stdout:0/70: write da/fb [3148154,79866] 0 2024-02-16T22:56:10.850 INFO:tasks.workunit.client.1.smithi192.stdout:1/151: creat d3/de/d27/f30 x:0 0 0 2024-02-16T22:56:10.852 INFO:tasks.workunit.client.0.smithi064.stdout:5/70: mkdir d6/d18/d19 0 2024-02-16T22:56:10.853 INFO:tasks.workunit.client.1.smithi192.stdout:9/259: dwrite d7/d19/d37/f3d [4194304,4194304] 0 2024-02-16T22:56:10.854 INFO:tasks.workunit.client.1.smithi192.stdout:6/206: rename da/df/d12/d15/d1b/l24 to da/df/d12/l3b 0 2024-02-16T22:56:10.861 INFO:tasks.workunit.client.1.smithi192.stdout:4/166: creat d1/d15/d23/f3b x:0 0 0 2024-02-16T22:56:10.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/167: readlink d1/d8/d11/l33 0 2024-02-16T22:56:10.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/196: mkdir d22/d24/d39/d42 0 2024-02-16T22:56:10.869 INFO:tasks.workunit.client.1.smithi192.stdout:3/181: dwrite da/f3d [0,4194304] 0 2024-02-16T22:56:10.869 INFO:tasks.workunit.client.1.smithi192.stdout:5/172: mkdir d6/d12/d2d 0 2024-02-16T22:56:10.870 INFO:tasks.workunit.client.1.smithi192.stdout:5/173: fsync d6/f2b 0 2024-02-16T22:56:10.876 INFO:tasks.workunit.client.1.smithi192.stdout:2/197: creat d22/d24/d2b/d2c/f43 x:0 0 0 2024-02-16T22:56:10.880 INFO:tasks.workunit.client.1.smithi192.stdout:5/174: creat d6/d12/d15/f2e x:0 0 0 2024-02-16T22:56:10.880 INFO:tasks.workunit.client.1.smithi192.stdout:3/182: dwrite da/df/f1f [0,4194304] 0 2024-02-16T22:56:10.880 INFO:tasks.workunit.client.0.smithi064.stdout:7/43: sync 2024-02-16T22:56:10.883 INFO:tasks.workunit.client.1.smithi192.stdout:1/152: dwrite d3/f22 [4194304,4194304] 0 2024-02-16T22:56:10.883 INFO:tasks.workunit.client.1.smithi192.stdout:6/207: mknod da/df/d12/c3c 0 2024-02-16T22:56:10.884 INFO:tasks.workunit.client.1.smithi192.stdout:7/224: rename d1/d1f/c32 to d1/d2/de/d2c/d49/c4a 0 2024-02-16T22:56:10.887 INFO:tasks.workunit.client.1.smithi192.stdout:0/160: getdents d1/d3/de 0 2024-02-16T22:56:10.887 INFO:tasks.workunit.client.1.smithi192.stdout:2/198: symlink d22/d24/d2b/l44 0 2024-02-16T22:56:10.889 INFO:tasks.workunit.client.1.smithi192.stdout:5/175: mknod d6/d12/c2f 0 2024-02-16T22:56:10.890 INFO:tasks.workunit.client.1.smithi192.stdout:8/148: truncate d5/db/de/df/f1b 6858990 0 2024-02-16T22:56:10.892 INFO:tasks.workunit.client.1.smithi192.stdout:4/168: dread d1/d8/d11/f1d [0,4194304] 0 2024-02-16T22:56:10.893 INFO:tasks.workunit.client.1.smithi192.stdout:5/176: write d6/d12/d15/f2e [772447,106345] 0 2024-02-16T22:56:10.896 INFO:tasks.workunit.client.1.smithi192.stdout:9/260: truncate d7/f12 3446958 0 2024-02-16T22:56:10.896 INFO:tasks.workunit.client.1.smithi192.stdout:4/169: stat d1/d8/d11 0 2024-02-16T22:56:10.897 INFO:tasks.workunit.client.1.smithi192.stdout:8/149: mknod d5/db/de/c29 0 2024-02-16T22:56:10.898 INFO:tasks.workunit.client.1.smithi192.stdout:6/208: mknod da/df/d12/c3d 0 2024-02-16T22:56:10.898 INFO:tasks.workunit.client.1.smithi192.stdout:5/177: write d6/f2b [371399,128512] 0 2024-02-16T22:56:10.899 INFO:tasks.workunit.client.1.smithi192.stdout:7/225: creat d1/f4b x:0 0 0 2024-02-16T22:56:10.900 INFO:tasks.workunit.client.1.smithi192.stdout:3/183: rename da/c10 to da/d11/c40 0 2024-02-16T22:56:10.900 INFO:tasks.workunit.client.1.smithi192.stdout:9/261: read d7/df/f16 [302537,19691] 0 2024-02-16T22:56:10.901 INFO:tasks.workunit.client.1.smithi192.stdout:1/153: truncate d3/fc 251740 0 2024-02-16T22:56:10.901 INFO:tasks.workunit.client.1.smithi192.stdout:4/170: mkdir d1/d15/d29/d3c 0 2024-02-16T22:56:10.903 INFO:tasks.workunit.client.1.smithi192.stdout:7/226: mkdir d1/d19/d4c 0 2024-02-16T22:56:10.904 INFO:tasks.workunit.client.1.smithi192.stdout:2/199: dwrite f19 [4194304,4194304] 0 2024-02-16T22:56:10.905 INFO:tasks.workunit.client.1.smithi192.stdout:3/184: creat da/d11/d20/f41 x:0 0 0 2024-02-16T22:56:10.909 INFO:tasks.workunit.client.1.smithi192.stdout:0/161: dwrite d1/fd [8388608,4194304] 0 2024-02-16T22:56:10.909 INFO:tasks.workunit.client.1.smithi192.stdout:1/154: symlink d3/de/l31 0 2024-02-16T22:56:10.917 INFO:tasks.workunit.client.1.smithi192.stdout:7/227: rmdir d1/d1f 39 2024-02-16T22:56:10.917 INFO:tasks.workunit.client.1.smithi192.stdout:1/155: dread - d3/de/d27/f30 zero size 2024-02-16T22:56:10.917 INFO:tasks.workunit.client.1.smithi192.stdout:6/209: creat da/df/d12/d15/f3e x:0 0 0 2024-02-16T22:56:10.919 INFO:tasks.workunit.client.1.smithi192.stdout:8/150: dwrite d5/f8 [0,4194304] 0 2024-02-16T22:56:10.920 INFO:tasks.workunit.client.1.smithi192.stdout:2/200: dread d22/d24/d2b/d2c/f2e [0,4194304] 0 2024-02-16T22:56:10.921 INFO:tasks.workunit.client.1.smithi192.stdout:7/228: fdatasync d1/d2/f3d 0 2024-02-16T22:56:10.921 INFO:tasks.workunit.client.1.smithi192.stdout:0/162: creat d1/d3/de/d22/f30 x:0 0 0 2024-02-16T22:56:10.921 INFO:tasks.workunit.client.1.smithi192.stdout:1/156: fdatasync d3/fb 0 2024-02-16T22:56:10.929 INFO:tasks.workunit.client.1.smithi192.stdout:5/178: getdents d6/d12/d1d 0 2024-02-16T22:56:10.932 INFO:tasks.workunit.client.1.smithi192.stdout:4/171: dwrite d1/d8/f9 [4194304,4194304] 0 2024-02-16T22:56:10.932 INFO:tasks.workunit.client.1.smithi192.stdout:3/185: dwrite da/f27 [0,4194304] 0 2024-02-16T22:56:10.932 INFO:tasks.workunit.client.0.smithi064.stdout:0/71: fdatasync da/fb 0 2024-02-16T22:56:10.934 INFO:tasks.workunit.client.0.smithi064.stdout:3/99: getdents d4/d13/d1b 0 2024-02-16T22:56:10.935 INFO:tasks.workunit.client.1.smithi192.stdout:5/179: readlink d6/d12/d1d/l1f 0 2024-02-16T22:56:10.939 INFO:tasks.workunit.client.0.smithi064.stdout:5/71: mkdir d6/d18/d19/d1a 0 2024-02-16T22:56:10.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/229: chown d1/d2/f25 541 1 2024-02-16T22:56:10.940 INFO:tasks.workunit.client.1.smithi192.stdout:6/210: dread f9 [0,4194304] 0 2024-02-16T22:56:10.943 INFO:tasks.workunit.client.0.smithi064.stdout:9/124: dwrite d3/fc [4194304,4194304] 0 2024-02-16T22:56:10.943 INFO:tasks.workunit.client.1.smithi192.stdout:2/201: truncate f2 718300 0 2024-02-16T22:56:10.944 INFO:tasks.workunit.client.0.smithi064.stdout:0/72: creat da/f10 x:0 0 0 2024-02-16T22:56:10.944 INFO:tasks.workunit.client.1.smithi192.stdout:1/157: dwrite d3/f21 [0,4194304] 0 2024-02-16T22:56:10.945 INFO:tasks.workunit.client.1.smithi192.stdout:0/163: dwrite d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:10.947 INFO:tasks.workunit.client.0.smithi064.stdout:7/44: creat d5/d6/fa x:0 0 0 2024-02-16T22:56:10.948 INFO:tasks.workunit.client.1.smithi192.stdout:3/186: mkdir da/df/d42 0 2024-02-16T22:56:10.950 INFO:tasks.workunit.client.0.smithi064.stdout:2/64: dwrite d7/fc [0,4194304] 0 2024-02-16T22:56:10.950 INFO:tasks.workunit.client.0.smithi064.stdout:0/73: truncate da/f10 322492 0 2024-02-16T22:56:10.954 INFO:tasks.workunit.client.0.smithi064.stdout:3/100: dwrite d4/d13/f16 [0,4194304] 0 2024-02-16T22:56:10.955 INFO:tasks.workunit.client.0.smithi064.stdout:3/101: dread - d4/d1f/f20 zero size 2024-02-16T22:56:10.961 INFO:tasks.workunit.client.1.smithi192.stdout:7/230: creat d1/d2/de/f4d x:0 0 0 2024-02-16T22:56:10.962 INFO:tasks.workunit.client.0.smithi064.stdout:5/72: rename d6/f10 to d6/f1b 0 2024-02-16T22:56:10.964 INFO:tasks.workunit.client.1.smithi192.stdout:3/187: dwrite da/f22 [0,4194304] 0 2024-02-16T22:56:10.964 INFO:tasks.workunit.client.1.smithi192.stdout:9/262: dwrite d7/d15/d36/d38/f47 [0,4194304] 0 2024-02-16T22:56:10.965 INFO:tasks.workunit.client.1.smithi192.stdout:9/263: fsync d7/df/f23 0 2024-02-16T22:56:10.969 INFO:tasks.workunit.client.1.smithi192.stdout:6/211: dread da/fb [0,4194304] 0 2024-02-16T22:56:10.972 INFO:tasks.workunit.client.1.smithi192.stdout:4/172: link d1/d8/fe d1/d15/d23/f3d 0 2024-02-16T22:56:10.973 INFO:tasks.workunit.client.1.smithi192.stdout:4/173: fsync d1/f35 0 2024-02-16T22:56:10.975 INFO:tasks.workunit.client.0.smithi064.stdout:5/73: mkdir d6/d1c 0 2024-02-16T22:56:10.983 INFO:tasks.workunit.client.1.smithi192.stdout:7/231: write d1/d19/f1d [3600924,92399] 0 2024-02-16T22:56:10.984 INFO:tasks.workunit.client.1.smithi192.stdout:7/232: stat d1/d2/l8 0 2024-02-16T22:56:10.985 INFO:tasks.workunit.client.1.smithi192.stdout:3/188: fsync da/d11/d20/f24 0 2024-02-16T22:56:10.992 INFO:tasks.workunit.client.1.smithi192.stdout:5/180: truncate d6/d12/f18 3417048 0 2024-02-16T22:56:10.993 INFO:tasks.workunit.client.0.smithi064.stdout:0/74: link da/dc/ld da/dc/l11 0 2024-02-16T22:56:10.994 INFO:tasks.workunit.client.0.smithi064.stdout:3/102: link l0 d4/l23 0 2024-02-16T22:56:10.995 INFO:tasks.workunit.client.1.smithi192.stdout:5/181: dread d6/f10 [0,4194304] 0 2024-02-16T22:56:10.997 INFO:tasks.workunit.client.0.smithi064.stdout:0/75: mkdir da/dc/de/d12 0 2024-02-16T22:56:10.998 INFO:tasks.workunit.client.0.smithi064.stdout:2/65: write f0 [2302064,116650] 0 2024-02-16T22:56:10.998 INFO:tasks.workunit.client.0.smithi064.stdout:0/76: write da/fb [3904599,85367] 0 2024-02-16T22:56:10.999 INFO:tasks.workunit.client.1.smithi192.stdout:6/212: dread da/df/d12/d15/f31 [0,4194304] 0 2024-02-16T22:56:10.999 INFO:tasks.workunit.client.0.smithi064.stdout:0/77: write f9 [1486783,29451] 0 2024-02-16T22:56:11.000 INFO:tasks.workunit.client.0.smithi064.stdout:3/103: creat d4/d1f/f24 x:0 0 0 2024-02-16T22:56:11.002 INFO:tasks.workunit.client.1.smithi192.stdout:4/174: mknod d1/d15/c3e 0 2024-02-16T22:56:11.002 INFO:tasks.workunit.client.1.smithi192.stdout:5/182: dwrite d6/d12/d15/f1c [0,4194304] 0 2024-02-16T22:56:11.003 INFO:tasks.workunit.client.0.smithi064.stdout:2/66: dread d7/f9 [0,4194304] 0 2024-02-16T22:56:11.004 INFO:tasks.workunit.client.0.smithi064.stdout:3/104: dread d4/f11 [0,4194304] 0 2024-02-16T22:56:11.005 INFO:tasks.workunit.client.1.smithi192.stdout:8/151: getdents d5/db/de/df 0 2024-02-16T22:56:11.006 INFO:tasks.workunit.client.0.smithi064.stdout:7/45: write f0 [1883150,58567] 0 2024-02-16T22:56:11.012 INFO:tasks.workunit.client.1.smithi192.stdout:5/183: dwrite d6/fd [0,4194304] 0 2024-02-16T22:56:11.012 INFO:tasks.workunit.client.1.smithi192.stdout:4/175: dwrite d1/d26/f36 [0,4194304] 0 2024-02-16T22:56:11.012 INFO:tasks.workunit.client.0.smithi064.stdout:0/78: creat da/dc/f13 x:0 0 0 2024-02-16T22:56:11.014 INFO:tasks.workunit.client.1.smithi192.stdout:9/264: write d7/d15/f27 [463268,25830] 0 2024-02-16T22:56:11.015 INFO:tasks.workunit.client.1.smithi192.stdout:7/233: mknod d1/d2/d2d/c4e 0 2024-02-16T22:56:11.016 INFO:tasks.workunit.client.0.smithi064.stdout:5/74: getdents d6/d18/d19 0 2024-02-16T22:56:11.016 INFO:tasks.workunit.client.1.smithi192.stdout:9/265: write d7/f8 [4433599,65053] 0 2024-02-16T22:56:11.022 INFO:tasks.workunit.client.1.smithi192.stdout:3/189: mkdir da/d11/d2b/d43 0 2024-02-16T22:56:11.027 INFO:tasks.workunit.client.1.smithi192.stdout:2/202: write f8 [662872,55771] 0 2024-02-16T22:56:11.028 INFO:tasks.workunit.client.1.smithi192.stdout:1/158: rmdir d3/d2c 0 2024-02-16T22:56:11.028 INFO:tasks.workunit.client.1.smithi192.stdout:2/203: chown d22/d24/d2b/d2c/f2d 0 1 2024-02-16T22:56:11.028 INFO:tasks.workunit.client.1.smithi192.stdout:1/159: fdatasync d3/f26 0 2024-02-16T22:56:11.029 INFO:tasks.workunit.client.0.smithi064.stdout:5/75: creat d6/f1d x:0 0 0 2024-02-16T22:56:11.032 INFO:tasks.workunit.client.1.smithi192.stdout:6/213: mkdir da/df/d18/d3f 0 2024-02-16T22:56:11.033 INFO:tasks.workunit.client.0.smithi064.stdout:3/105: rename l0 to d4/d13/l25 0 2024-02-16T22:56:11.034 INFO:tasks.workunit.client.0.smithi064.stdout:3/106: chown d4/f5 78 1 2024-02-16T22:56:11.036 INFO:tasks.workunit.client.0.smithi064.stdout:0/79: fdatasync f9 0 2024-02-16T22:56:11.037 INFO:tasks.workunit.client.1.smithi192.stdout:8/152: unlink d5/f8 0 2024-02-16T22:56:11.037 INFO:tasks.workunit.client.0.smithi064.stdout:7/46: getdents d5/d6/d8 0 2024-02-16T22:56:11.037 INFO:tasks.workunit.client.0.smithi064.stdout:7/47: fdatasync f3 0 2024-02-16T22:56:11.038 INFO:tasks.workunit.client.1.smithi192.stdout:8/153: write d5/d16/f27 [826321,21429] 0 2024-02-16T22:56:11.040 INFO:tasks.workunit.client.0.smithi064.stdout:5/76: mknod d6/d18/d19/c1e 0 2024-02-16T22:56:11.042 INFO:tasks.workunit.client.1.smithi192.stdout:4/176: mknod d1/d15/d29/c3f 0 2024-02-16T22:56:11.043 INFO:tasks.workunit.client.0.smithi064.stdout:3/107: creat d4/d13/f26 x:0 0 0 2024-02-16T22:56:11.044 INFO:tasks.workunit.client.1.smithi192.stdout:8/154: dwrite d5/f1e [4194304,4194304] 0 2024-02-16T22:56:11.045 INFO:tasks.workunit.client.0.smithi064.stdout:7/48: dwrite d5/d6/f9 [0,4194304] 0 2024-02-16T22:56:11.051 INFO:tasks.workunit.client.1.smithi192.stdout:9/266: creat d7/d19/d37/f55 x:0 0 0 2024-02-16T22:56:11.052 INFO:tasks.workunit.client.0.smithi064.stdout:6/123: sync 2024-02-16T22:56:11.053 INFO:tasks.workunit.client.0.smithi064.stdout:3/108: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:56:11.054 INFO:tasks.workunit.client.0.smithi064.stdout:6/124: write d0/d6/d10/f1c [5122272,98796] 0 2024-02-16T22:56:11.054 INFO:tasks.workunit.client.0.smithi064.stdout:3/109: fsync d4/d1f/f24 0 2024-02-16T22:56:11.055 INFO:tasks.workunit.client.0.smithi064.stdout:2/67: getdents d7 0 2024-02-16T22:56:11.061 INFO:tasks.workunit.client.0.smithi064.stdout:5/77: creat d6/d18/d19/f1f x:0 0 0 2024-02-16T22:56:11.061 INFO:tasks.workunit.client.1.smithi192.stdout:6/214: dread da/df/f1d [0,4194304] 0 2024-02-16T22:56:11.061 INFO:tasks.workunit.client.1.smithi192.stdout:3/190: mknod da/df/c44 0 2024-02-16T22:56:11.063 INFO:tasks.workunit.client.0.smithi064.stdout:5/78: readlink d6/l11 0 2024-02-16T22:56:11.065 INFO:tasks.workunit.client.0.smithi064.stdout:0/80: rename l8 to da/dc/de/l14 0 2024-02-16T22:56:11.067 INFO:tasks.workunit.client.1.smithi192.stdout:2/204: rmdir d22 39 2024-02-16T22:56:11.067 INFO:tasks.workunit.client.0.smithi064.stdout:3/110: dwrite d4/d1f/f20 [0,4194304] 0 2024-02-16T22:56:11.069 INFO:tasks.workunit.client.0.smithi064.stdout:2/68: dwrite f0 [0,4194304] 0 2024-02-16T22:56:11.070 INFO:tasks.workunit.client.1.smithi192.stdout:1/160: creat d3/de/f32 x:0 0 0 2024-02-16T22:56:11.070 INFO:tasks.workunit.client.1.smithi192.stdout:1/161: fdatasync d3/f22 0 2024-02-16T22:56:11.077 INFO:tasks.workunit.client.1.smithi192.stdout:5/184: getdents d6/d12/d2d 0 2024-02-16T22:56:11.079 INFO:tasks.workunit.client.1.smithi192.stdout:5/185: truncate d6/d12/d15/f2e 1196334 0 2024-02-16T22:56:11.080 INFO:tasks.workunit.client.0.smithi064.stdout:6/125: dwrite d0/d6/d10/f14 [0,4194304] 0 2024-02-16T22:56:11.080 INFO:tasks.workunit.client.1.smithi192.stdout:7/234: fdatasync d1/d2/de/f20 0 2024-02-16T22:56:11.080 INFO:tasks.workunit.client.0.smithi064.stdout:2/69: mknod d7/cd 0 2024-02-16T22:56:11.082 INFO:tasks.workunit.client.1.smithi192.stdout:9/267: rename d7/l17 to d7/d19/d37/d39/l56 0 2024-02-16T22:56:11.083 INFO:tasks.workunit.client.1.smithi192.stdout:9/268: dread - d7/d19/d37/f34 zero size 2024-02-16T22:56:11.084 INFO:tasks.workunit.client.0.smithi064.stdout:3/111: mkdir d4/d13/d1b/d27 0 2024-02-16T22:56:11.085 INFO:tasks.workunit.client.0.smithi064.stdout:3/112: read - d4/d14/f22 zero size 2024-02-16T22:56:11.086 INFO:tasks.workunit.client.0.smithi064.stdout:3/113: dread - d4/d1f/f24 zero size 2024-02-16T22:56:11.087 INFO:tasks.workunit.client.0.smithi064.stdout:7/49: rmdir d5/d6/d8 0 2024-02-16T22:56:11.088 INFO:tasks.workunit.client.1.smithi192.stdout:3/191: symlink da/d11/d35/l45 0 2024-02-16T22:56:11.089 INFO:tasks.workunit.client.0.smithi064.stdout:5/79: creat d6/d1c/f20 x:0 0 0 2024-02-16T22:56:11.090 INFO:tasks.workunit.client.0.smithi064.stdout:5/80: fdatasync f4 0 2024-02-16T22:56:11.091 INFO:tasks.workunit.client.0.smithi064.stdout:0/81: mknod da/dc/de/d12/c15 0 2024-02-16T22:56:11.093 INFO:tasks.workunit.client.0.smithi064.stdout:7/50: dwrite f0 [0,4194304] 0 2024-02-16T22:56:11.099 INFO:tasks.workunit.client.0.smithi064.stdout:3/114: creat d4/d13/f28 x:0 0 0 2024-02-16T22:56:11.099 INFO:tasks.workunit.client.0.smithi064.stdout:2/70: fsync d7/f8 0 2024-02-16T22:56:11.101 INFO:tasks.workunit.client.0.smithi064.stdout:3/115: dread d4/f1e [0,4194304] 0 2024-02-16T22:56:11.103 INFO:tasks.workunit.client.0.smithi064.stdout:2/71: dread d7/f9 [0,4194304] 0 2024-02-16T22:56:11.103 INFO:tasks.workunit.client.0.smithi064.stdout:2/72: chown d7/f8 51367584 1 2024-02-16T22:56:11.104 INFO:tasks.workunit.client.0.smithi064.stdout:2/73: chown d7/ca 14302 1 2024-02-16T22:56:11.104 INFO:tasks.workunit.client.1.smithi192.stdout:5/186: mknod d6/d12/d1d/c30 0 2024-02-16T22:56:11.105 INFO:tasks.workunit.client.0.smithi064.stdout:5/81: mknod d6/d1c/c21 0 2024-02-16T22:56:11.105 INFO:tasks.workunit.client.0.smithi064.stdout:5/82: readlink d6/l14 0 2024-02-16T22:56:11.106 INFO:tasks.workunit.client.0.smithi064.stdout:5/83: dread - d6/d18/d19/f1f zero size 2024-02-16T22:56:11.107 INFO:tasks.workunit.client.0.smithi064.stdout:0/82: creat da/dc/de/f16 x:0 0 0 2024-02-16T22:56:11.108 INFO:tasks.workunit.client.0.smithi064.stdout:0/83: write da/dc/de/f16 [463956,123003] 0 2024-02-16T22:56:11.110 INFO:tasks.workunit.client.0.smithi064.stdout:7/51: rename f3 to d5/d6/fb 0 2024-02-16T22:56:11.112 INFO:tasks.workunit.client.1.smithi192.stdout:9/269: creat d7/d19/d28/d43/f57 x:0 0 0 2024-02-16T22:56:11.118 INFO:tasks.workunit.client.0.smithi064.stdout:7/52: dwrite d5/d6/f9 [4194304,4194304] 0 2024-02-16T22:56:11.119 INFO:tasks.workunit.client.1.smithi192.stdout:2/205: creat d22/d24/d2b/f45 x:0 0 0 2024-02-16T22:56:11.119 INFO:tasks.workunit.client.0.smithi064.stdout:7/53: write d5/d6/fa [171129,66798] 0 2024-02-16T22:56:11.120 INFO:tasks.workunit.client.0.smithi064.stdout:2/74: mkdir d7/de 0 2024-02-16T22:56:11.121 INFO:tasks.workunit.client.1.smithi192.stdout:4/177: truncate d1/d8/d11/f2a 154401 0 2024-02-16T22:56:11.122 INFO:tasks.workunit.client.1.smithi192.stdout:1/162: symlink d3/l33 0 2024-02-16T22:56:11.123 INFO:tasks.workunit.client.1.smithi192.stdout:4/178: fdatasync d1/d26/f36 0 2024-02-16T22:56:11.124 INFO:tasks.workunit.client.1.smithi192.stdout:5/187: symlink d6/d8/l31 0 2024-02-16T22:56:11.125 INFO:tasks.workunit.client.0.smithi064.stdout:0/84: chown c1 1746044 1 2024-02-16T22:56:11.126 INFO:tasks.workunit.client.1.smithi192.stdout:1/163: write d3/f21 [3265862,87949] 0 2024-02-16T22:56:11.127 INFO:tasks.workunit.client.1.smithi192.stdout:5/188: stat d6/d12/d15/f2e 0 2024-02-16T22:56:11.127 INFO:tasks.workunit.client.1.smithi192.stdout:7/235: creat d1/d1f/f4f x:0 0 0 2024-02-16T22:56:11.128 INFO:tasks.workunit.client.1.smithi192.stdout:2/206: dwrite d22/d24/d2b/d2c/f30 [0,4194304] 0 2024-02-16T22:56:11.131 INFO:tasks.workunit.client.1.smithi192.stdout:3/192: symlink da/d11/d20/d39/l46 0 2024-02-16T22:56:11.131 INFO:tasks.workunit.client.0.smithi064.stdout:5/84: dwrite d6/f1b [0,4194304] 0 2024-02-16T22:56:11.133 INFO:tasks.workunit.client.0.smithi064.stdout:5/85: dread - d6/ff zero size 2024-02-16T22:56:11.133 INFO:tasks.workunit.client.1.smithi192.stdout:6/215: truncate da/df/d18/f29 68134 0 2024-02-16T22:56:11.134 INFO:tasks.workunit.client.0.smithi064.stdout:5/86: write d6/d18/d19/f1f [777651,37683] 0 2024-02-16T22:56:11.136 INFO:tasks.workunit.client.0.smithi064.stdout:5/87: chown d6/f13 1 1 2024-02-16T22:56:11.137 INFO:tasks.workunit.client.1.smithi192.stdout:7/236: dwrite d1/d2/de/f24 [0,4194304] 0 2024-02-16T22:56:11.138 INFO:tasks.workunit.client.1.smithi192.stdout:7/237: chown d1/d2/lc 29296 1 2024-02-16T22:56:11.138 INFO:tasks.workunit.client.0.smithi064.stdout:7/54: write d5/d6/fb [859652,109442] 0 2024-02-16T22:56:11.140 INFO:tasks.workunit.client.0.smithi064.stdout:4/91: sync 2024-02-16T22:56:11.140 INFO:tasks.workunit.client.1.smithi192.stdout:4/179: mknod d1/d26/c40 0 2024-02-16T22:56:11.140 INFO:tasks.workunit.client.1.smithi192.stdout:1/164: mkdir d3/de/d27/d34 0 2024-02-16T22:56:11.141 INFO:tasks.workunit.client.0.smithi064.stdout:8/95: sync 2024-02-16T22:56:11.141 INFO:tasks.workunit.client.0.smithi064.stdout:3/116: dwrite d4/fc [0,4194304] 0 2024-02-16T22:56:11.141 INFO:tasks.workunit.client.0.smithi064.stdout:4/92: fsync dc/d10/f12 0 2024-02-16T22:56:11.141 INFO:tasks.workunit.client.0.smithi064.stdout:6/126: getdents d0/d5 0 2024-02-16T22:56:11.142 INFO:tasks.workunit.client.1.smithi192.stdout:5/189: write d6/f10 [4549891,39388] 0 2024-02-16T22:56:11.142 INFO:tasks.workunit.client.0.smithi064.stdout:3/117: truncate d4/f17 360483 0 2024-02-16T22:56:11.143 INFO:tasks.workunit.client.0.smithi064.stdout:2/75: symlink d7/de/lf 0 2024-02-16T22:56:11.143 INFO:tasks.workunit.client.0.smithi064.stdout:4/93: chown dc/d10/f14 11 1 2024-02-16T22:56:11.143 INFO:tasks.workunit.client.1.smithi192.stdout:5/190: chown d6/d12/d1d/l25 859 1 2024-02-16T22:56:11.144 INFO:tasks.workunit.client.0.smithi064.stdout:3/118: chown d4/cf 0 1 2024-02-16T22:56:11.144 INFO:tasks.workunit.client.0.smithi064.stdout:4/94: fsync f5 0 2024-02-16T22:56:11.145 INFO:tasks.workunit.client.0.smithi064.stdout:4/95: stat c7 0 2024-02-16T22:56:11.147 INFO:tasks.workunit.client.0.smithi064.stdout:4/96: chown dc 4003100 1 2024-02-16T22:56:11.147 INFO:tasks.workunit.client.0.smithi064.stdout:5/88: symlink d6/d18/d19/l22 0 2024-02-16T22:56:11.147 INFO:tasks.workunit.client.0.smithi064.stdout:7/55: mknod d5/cc 0 2024-02-16T22:56:11.147 INFO:tasks.workunit.client.1.smithi192.stdout:2/207: mknod d22/d24/d39/d3d/c46 0 2024-02-16T22:56:11.149 INFO:tasks.workunit.client.1.smithi192.stdout:2/208: dread - d22/d24/d2b/d2c/f37 zero size 2024-02-16T22:56:11.153 INFO:tasks.workunit.client.0.smithi064.stdout:2/76: write d7/fc [3889069,49576] 0 2024-02-16T22:56:11.154 INFO:tasks.workunit.client.1.smithi192.stdout:7/238: stat d1/d2/de/d2c/d49/c4a 0 2024-02-16T22:56:11.155 INFO:tasks.workunit.client.0.smithi064.stdout:3/119: write d4/f11 [2881829,4708] 0 2024-02-16T22:56:11.158 INFO:tasks.workunit.client.1.smithi192.stdout:4/180: symlink d1/d15/l41 0 2024-02-16T22:56:11.158 INFO:tasks.workunit.client.1.smithi192.stdout:4/181: read - d1/d26/f31 zero size 2024-02-16T22:56:11.158 INFO:tasks.workunit.client.1.smithi192.stdout:3/193: creat da/df/d42/f47 x:0 0 0 2024-02-16T22:56:11.159 INFO:tasks.workunit.client.0.smithi064.stdout:7/56: mknod d5/d6/cd 0 2024-02-16T22:56:11.159 INFO:tasks.workunit.client.0.smithi064.stdout:7/57: chown d5 1 1 2024-02-16T22:56:11.160 INFO:tasks.workunit.client.0.smithi064.stdout:8/96: truncate fe 897982 0 2024-02-16T22:56:11.161 INFO:tasks.workunit.client.1.smithi192.stdout:6/216: mkdir da/d40 0 2024-02-16T22:56:11.163 INFO:tasks.workunit.client.0.smithi064.stdout:3/120: chown d4/l9 25708 1 2024-02-16T22:56:11.164 INFO:tasks.workunit.client.1.smithi192.stdout:6/217: write da/df/d12/d15/d1b/f37 [66434,119587] 0 2024-02-16T22:56:11.164 INFO:tasks.workunit.client.1.smithi192.stdout:4/182: dwrite d1/d26/f36 [0,4194304] 0 2024-02-16T22:56:11.167 INFO:tasks.workunit.client.0.smithi064.stdout:5/89: fdatasync d6/f1b 0 2024-02-16T22:56:11.167 INFO:tasks.workunit.client.0.smithi064.stdout:3/121: fdatasync d4/f11 0 2024-02-16T22:56:11.167 INFO:tasks.workunit.client.1.smithi192.stdout:1/165: rename d3/fc to d3/de/f35 0 2024-02-16T22:56:11.167 INFO:tasks.workunit.client.1.smithi192.stdout:1/166: chown d3/de/d27/c28 3775064 1 2024-02-16T22:56:11.169 INFO:tasks.workunit.client.0.smithi064.stdout:5/90: write f2 [533160,40267] 0 2024-02-16T22:56:11.169 INFO:tasks.workunit.client.1.smithi192.stdout:3/194: symlink da/df/d42/l48 0 2024-02-16T22:56:11.170 INFO:tasks.workunit.client.0.smithi064.stdout:5/91: write d6/f1d [4135,24156] 0 2024-02-16T22:56:11.171 INFO:tasks.workunit.client.0.smithi064.stdout:7/58: symlink d5/d6/le 0 2024-02-16T22:56:11.172 INFO:tasks.workunit.client.1.smithi192.stdout:5/191: dread f2 [0,4194304] 0 2024-02-16T22:56:11.173 INFO:tasks.workunit.client.0.smithi064.stdout:5/92: dread d6/fa [0,4194304] 0 2024-02-16T22:56:11.174 INFO:tasks.workunit.client.0.smithi064.stdout:5/93: fdatasync d6/f1b 0 2024-02-16T22:56:11.174 INFO:tasks.workunit.client.1.smithi192.stdout:2/209: mkdir d22/d47 0 2024-02-16T22:56:11.174 INFO:tasks.workunit.client.1.smithi192.stdout:5/192: write d6/d12/d15/f28 [621201,19557] 0 2024-02-16T22:56:11.176 INFO:tasks.workunit.client.1.smithi192.stdout:7/239: symlink d1/d2/l50 0 2024-02-16T22:56:11.179 INFO:tasks.workunit.client.1.smithi192.stdout:4/183: mknod d1/d15/c42 0 2024-02-16T22:56:11.182 INFO:tasks.workunit.client.1.smithi192.stdout:9/270: truncate f1 2101271 0 2024-02-16T22:56:11.182 INFO:tasks.workunit.client.0.smithi064.stdout:3/122: symlink d4/d13/l29 0 2024-02-16T22:56:11.184 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:10 smithi192 ceph-mon[36254]: pgmap v10: 65 pgs: 65 active+clean; 313 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 71 KiB/s rd, 15 MiB/s wr, 229 op/s 2024-02-16T22:56:11.184 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:10 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:56:11.184 INFO:tasks.workunit.client.1.smithi192.stdout:3/195: unlink c5 0 2024-02-16T22:56:11.184 INFO:tasks.workunit.client.1.smithi192.stdout:3/196: fsync f3 0 2024-02-16T22:56:11.186 INFO:tasks.workunit.client.1.smithi192.stdout:3/197: fdatasync da/df/f23 0 2024-02-16T22:56:11.186 INFO:tasks.workunit.client.1.smithi192.stdout:5/193: creat d6/d12/f32 x:0 0 0 2024-02-16T22:56:11.187 INFO:tasks.workunit.client.0.smithi064.stdout:4/97: write fb [493397,66257] 0 2024-02-16T22:56:11.188 INFO:tasks.workunit.client.1.smithi192.stdout:9/271: dread d7/d15/d36/d38/f49 [0,4194304] 0 2024-02-16T22:56:11.190 INFO:tasks.workunit.client.1.smithi192.stdout:4/184: rmdir d1/d8/d11 39 2024-02-16T22:56:11.191 INFO:tasks.workunit.client.1.smithi192.stdout:4/185: stat d1/d15/d23/f1a 0 2024-02-16T22:56:11.192 INFO:tasks.workunit.client.0.smithi064.stdout:3/123: unlink d4/fc 0 2024-02-16T22:56:11.194 INFO:tasks.workunit.client.1.smithi192.stdout:6/218: rename da/c10 to da/d27/c41 0 2024-02-16T22:56:11.195 INFO:tasks.workunit.client.1.smithi192.stdout:6/219: write da/df/f1c [2140263,91408] 0 2024-02-16T22:56:11.196 INFO:tasks.workunit.client.1.smithi192.stdout:0/164: sync 2024-02-16T22:56:11.197 INFO:tasks.workunit.client.1.smithi192.stdout:1/167: rmdir d3 39 2024-02-16T22:56:11.197 INFO:tasks.workunit.client.1.smithi192.stdout:0/165: chown d1/f5 1765 1 2024-02-16T22:56:11.197 INFO:tasks.workunit.client.1.smithi192.stdout:0/166: truncate d1/f2a 650258 0 2024-02-16T22:56:11.199 INFO:tasks.workunit.client.0.smithi064.stdout:5/94: link d6/f1b d6/f23 0 2024-02-16T22:56:11.202 INFO:tasks.workunit.client.0.smithi064.stdout:3/124: truncate d4/fd 5035782 0 2024-02-16T22:56:11.203 INFO:tasks.workunit.client.0.smithi064.stdout:2/77: truncate d7/fc 443954 0 2024-02-16T22:56:11.206 INFO:tasks.workunit.client.0.smithi064.stdout:8/97: getdents df 0 2024-02-16T22:56:11.208 INFO:tasks.workunit.client.0.smithi064.stdout:4/98: getdents dc/dd/d16 0 2024-02-16T22:56:11.209 INFO:tasks.workunit.client.0.smithi064.stdout:4/99: chown dc/d1d 255000 1 2024-02-16T22:56:11.209 INFO:tasks.workunit.client.0.smithi064.stdout:7/59: write f0 [4854965,74992] 0 2024-02-16T22:56:11.209 INFO:tasks.workunit.client.0.smithi064.stdout:7/60: chown c2 1373 1 2024-02-16T22:56:11.210 INFO:tasks.workunit.client.1.smithi192.stdout:5/194: creat d6/f33 x:0 0 0 2024-02-16T22:56:11.211 INFO:tasks.workunit.client.0.smithi064.stdout:4/100: write dc/d10/f12 [568815,36426] 0 2024-02-16T22:56:11.211 INFO:tasks.workunit.client.0.smithi064.stdout:7/61: write d5/d6/fa [713803,1735] 0 2024-02-16T22:56:11.211 INFO:tasks.workunit.client.1.smithi192.stdout:5/195: write d6/fc [4664128,86402] 0 2024-02-16T22:56:11.215 INFO:tasks.workunit.client.1.smithi192.stdout:5/196: write d6/d12/f1e [255737,105270] 0 2024-02-16T22:56:11.215 INFO:tasks.workunit.client.1.smithi192.stdout:7/240: mknod d1/d19/d4c/c51 0 2024-02-16T22:56:11.215 INFO:tasks.workunit.client.1.smithi192.stdout:7/241: chown d1/d19/d4c/c51 12703042 1 2024-02-16T22:56:11.216 INFO:tasks.workunit.client.1.smithi192.stdout:9/272: creat d7/d15/f58 x:0 0 0 2024-02-16T22:56:11.217 INFO:tasks.workunit.client.0.smithi064.stdout:2/78: rename f0 to d7/f10 0 2024-02-16T22:56:11.219 INFO:tasks.workunit.client.1.smithi192.stdout:4/186: rename d1/d15/f2f to d1/d15/d29/d3c/f43 0 2024-02-16T22:56:11.221 INFO:tasks.workunit.client.0.smithi064.stdout:8/98: creat df/f1b x:0 0 0 2024-02-16T22:56:11.221 INFO:tasks.workunit.client.0.smithi064.stdout:2/79: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:11.221 INFO:tasks.workunit.client.0.smithi064.stdout:2/80: chown d7/f8 5853 1 2024-02-16T22:56:11.223 INFO:tasks.workunit.client.1.smithi192.stdout:6/220: rmdir da 39 2024-02-16T22:56:11.224 INFO:tasks.workunit.client.1.smithi192.stdout:2/210: truncate f19 6504260 0 2024-02-16T22:56:11.225 INFO:tasks.workunit.client.1.smithi192.stdout:2/211: chown f8 0 1 2024-02-16T22:56:11.225 INFO:tasks.workunit.client.0.smithi064.stdout:4/101: unlink fb 0 2024-02-16T22:56:11.226 INFO:tasks.workunit.client.1.smithi192.stdout:2/212: truncate d22/d24/d2b/d2c/f43 538421 0 2024-02-16T22:56:11.227 INFO:tasks.workunit.client.1.smithi192.stdout:1/168: stat d3/de/l25 0 2024-02-16T22:56:11.231 INFO:tasks.workunit.client.1.smithi192.stdout:1/169: dread d3/f21 [0,4194304] 0 2024-02-16T22:56:11.231 INFO:tasks.workunit.client.1.smithi192.stdout:3/198: write da/d11/d20/f24 [1294823,71869] 0 2024-02-16T22:56:11.231 INFO:tasks.workunit.client.1.smithi192.stdout:0/167: chown d1/d3/c17 122641 1 2024-02-16T22:56:11.232 INFO:tasks.workunit.client.0.smithi064.stdout:4/102: dwrite dc/d10/f12 [0,4194304] 0 2024-02-16T22:56:11.232 INFO:tasks.workunit.client.1.smithi192.stdout:3/199: readlink da/d11/d2b/l31 0 2024-02-16T22:56:11.235 INFO:tasks.workunit.client.0.smithi064.stdout:8/99: mknod df/c1c 0 2024-02-16T22:56:11.240 INFO:tasks.workunit.client.1.smithi192.stdout:7/242: unlink d1/c35 0 2024-02-16T22:56:11.243 INFO:tasks.workunit.client.0.smithi064.stdout:5/95: write f0 [791346,51474] 0 2024-02-16T22:56:11.249 INFO:tasks.workunit.client.0.smithi064.stdout:4/103: mknod dc/d10/c1e 0 2024-02-16T22:56:11.249 INFO:tasks.workunit.client.1.smithi192.stdout:2/213: rename d22 to d22/d48 22 2024-02-16T22:56:11.251 INFO:tasks.workunit.client.0.smithi064.stdout:2/81: rename d7/fc to d7/de/f11 0 2024-02-16T22:56:11.252 INFO:tasks.workunit.client.1.smithi192.stdout:8/155: sync 2024-02-16T22:56:11.255 INFO:tasks.workunit.client.0.smithi064.stdout:7/62: link d5/cc d5/d6/cf 0 2024-02-16T22:56:11.256 INFO:tasks.workunit.client.0.smithi064.stdout:7/63: write d5/d6/fb [1475511,97546] 0 2024-02-16T22:56:11.256 INFO:tasks.workunit.client.1.smithi192.stdout:2/214: dwrite f10 [0,4194304] 0 2024-02-16T22:56:11.257 INFO:tasks.workunit.client.1.smithi192.stdout:0/168: creat d1/d3/de/d22/f31 x:0 0 0 2024-02-16T22:56:11.262 INFO:tasks.workunit.client.0.smithi064.stdout:2/82: write d7/f8 [2257318,48630] 0 2024-02-16T22:56:11.264 INFO:tasks.workunit.client.1.smithi192.stdout:4/187: unlink d1/d8/d11/f1d 0 2024-02-16T22:56:11.264 INFO:tasks.workunit.client.1.smithi192.stdout:4/188: stat d1/d15/d23/c21 0 2024-02-16T22:56:11.265 INFO:tasks.workunit.client.1.smithi192.stdout:4/189: stat d1/d8/d11/f2d 0 2024-02-16T22:56:11.265 INFO:tasks.workunit.client.1.smithi192.stdout:4/190: fdatasync d1/f19 0 2024-02-16T22:56:11.266 INFO:tasks.workunit.client.0.smithi064.stdout:3/125: dwrite d4/fd [0,4194304] 0 2024-02-16T22:56:11.267 INFO:tasks.workunit.client.0.smithi064.stdout:5/96: rename f2 to d6/f24 0 2024-02-16T22:56:11.268 INFO:tasks.workunit.client.0.smithi064.stdout:5/97: fdatasync d6/f1d 0 2024-02-16T22:56:11.269 INFO:tasks.workunit.client.0.smithi064.stdout:8/100: write fb [694602,106882] 0 2024-02-16T22:56:11.269 INFO:tasks.workunit.client.1.smithi192.stdout:6/221: dread da/df/d12/f13 [0,4194304] 0 2024-02-16T22:56:11.270 INFO:tasks.workunit.client.0.smithi064.stdout:8/101: chown df/f15 23845507 1 2024-02-16T22:56:11.270 INFO:tasks.workunit.client.0.smithi064.stdout:5/98: truncate d6/f1d 33270 0 2024-02-16T22:56:11.271 INFO:tasks.workunit.client.0.smithi064.stdout:8/102: truncate fb 1536354 0 2024-02-16T22:56:11.272 INFO:tasks.workunit.client.0.smithi064.stdout:8/103: write df/f1a [1920019,92076] 0 2024-02-16T22:56:11.272 INFO:tasks.workunit.client.0.smithi064.stdout:7/64: rmdir d5 39 2024-02-16T22:56:11.273 INFO:tasks.workunit.client.1.smithi192.stdout:9/273: dread d7/f12 [0,4194304] 0 2024-02-16T22:56:11.273 INFO:tasks.workunit.client.1.smithi192.stdout:4/191: dwrite d1/fd [4194304,4194304] 0 2024-02-16T22:56:11.275 INFO:tasks.workunit.client.1.smithi192.stdout:9/274: write d7/d19/d28/d43/f57 [776331,43940] 0 2024-02-16T22:56:11.279 INFO:tasks.workunit.client.0.smithi064.stdout:8/104: dwrite df/f1a [0,4194304] 0 2024-02-16T22:56:11.280 INFO:tasks.workunit.client.0.smithi064.stdout:8/105: chown fa 2738 1 2024-02-16T22:56:11.283 INFO:tasks.workunit.client.0.smithi064.stdout:1/93: sync 2024-02-16T22:56:11.283 INFO:tasks.workunit.client.0.smithi064.stdout:9/125: sync 2024-02-16T22:56:11.283 INFO:tasks.workunit.client.0.smithi064.stdout:3/126: creat d4/d14/f2a x:0 0 0 2024-02-16T22:56:11.284 INFO:tasks.workunit.client.0.smithi064.stdout:3/127: fsync d4/d13/f28 0 2024-02-16T22:56:11.285 INFO:tasks.workunit.client.1.smithi192.stdout:1/170: truncate d3/fb 1605989 0 2024-02-16T22:56:11.285 INFO:tasks.workunit.client.1.smithi192.stdout:5/197: dwrite d6/d12/f18 [0,4194304] 0 2024-02-16T22:56:11.285 INFO:tasks.workunit.client.0.smithi064.stdout:9/126: chown d3/d11/f28 1 1 2024-02-16T22:56:11.286 INFO:tasks.workunit.client.1.smithi192.stdout:1/171: dread - d3/de/f1b zero size 2024-02-16T22:56:11.286 INFO:tasks.workunit.client.1.smithi192.stdout:0/169: symlink d1/d2/l32 0 2024-02-16T22:56:11.287 INFO:tasks.workunit.client.0.smithi064.stdout:5/99: rmdir d6/d18 39 2024-02-16T22:56:11.288 INFO:tasks.workunit.client.1.smithi192.stdout:5/198: write d6/d12/f32 [701819,84562] 0 2024-02-16T22:56:11.289 INFO:tasks.workunit.client.0.smithi064.stdout:3/128: dread d4/f12 [0,4194304] 0 2024-02-16T22:56:11.289 INFO:tasks.workunit.client.0.smithi064.stdout:3/129: fdatasync d4/f11 0 2024-02-16T22:56:11.290 INFO:tasks.workunit.client.0.smithi064.stdout:3/130: fdatasync d4/f8 0 2024-02-16T22:56:11.290 INFO:tasks.workunit.client.1.smithi192.stdout:3/200: link f4 da/df/f49 0 2024-02-16T22:56:11.293 INFO:tasks.workunit.client.0.smithi064.stdout:4/104: chown dc/d10/d18 89 1 2024-02-16T22:56:11.294 INFO:tasks.workunit.client.0.smithi064.stdout:4/105: chown ca 8 1 2024-02-16T22:56:11.297 INFO:tasks.workunit.client.0.smithi064.stdout:8/106: mknod df/c1d 0 2024-02-16T22:56:11.298 INFO:tasks.workunit.client.1.smithi192.stdout:5/199: dwrite d6/fc [4194304,4194304] 0 2024-02-16T22:56:11.299 INFO:tasks.workunit.client.1.smithi192.stdout:6/222: fsync da/df/f11 0 2024-02-16T22:56:11.299 INFO:tasks.workunit.client.1.smithi192.stdout:5/200: dread - d6/f33 zero size 2024-02-16T22:56:11.300 INFO:tasks.workunit.client.1.smithi192.stdout:6/223: chown da/ld 394688 1 2024-02-16T22:56:11.305 INFO:tasks.workunit.client.0.smithi064.stdout:2/83: write d7/de/f11 [1363232,79808] 0 2024-02-16T22:56:11.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/215: getdents d22/d24/d2b/d2c/d3f 0 2024-02-16T22:56:11.306 INFO:tasks.workunit.client.0.smithi064.stdout:3/131: mkdir d4/d14/d2b 0 2024-02-16T22:56:11.307 INFO:tasks.workunit.client.0.smithi064.stdout:7/65: creat d5/f10 x:0 0 0 2024-02-16T22:56:11.308 INFO:tasks.workunit.client.1.smithi192.stdout:2/216: write d22/f27 [1779981,40565] 0 2024-02-16T22:56:11.308 INFO:tasks.workunit.client.1.smithi192.stdout:5/201: dwrite d6/d12/d15/f28 [0,4194304] 0 2024-02-16T22:56:11.308 INFO:tasks.workunit.client.1.smithi192.stdout:3/201: rmdir da/df 39 2024-02-16T22:56:11.311 INFO:tasks.workunit.client.1.smithi192.stdout:4/192: creat d1/d26/d34/f44 x:0 0 0 2024-02-16T22:56:11.312 INFO:tasks.workunit.client.0.smithi064.stdout:8/107: mknod df/c1e 0 2024-02-16T22:56:11.313 INFO:tasks.workunit.client.0.smithi064.stdout:8/108: readlink df/l10 0 2024-02-16T22:56:11.314 INFO:tasks.workunit.client.1.smithi192.stdout:4/193: dread - d1/d26/d34/f44 zero size 2024-02-16T22:56:11.315 INFO:tasks.workunit.client.1.smithi192.stdout:4/194: fsync d1/f30 0 2024-02-16T22:56:11.316 INFO:tasks.workunit.client.0.smithi064.stdout:7/66: dwrite d5/f10 [0,4194304] 0 2024-02-16T22:56:11.317 INFO:tasks.workunit.client.0.smithi064.stdout:5/100: mkdir d6/d25 0 2024-02-16T22:56:11.319 INFO:tasks.workunit.client.1.smithi192.stdout:8/156: dread d5/f15 [0,4194304] 0 2024-02-16T22:56:11.319 INFO:tasks.workunit.client.1.smithi192.stdout:6/224: rename da/df/c36 to da/df/d18/d3f/c42 0 2024-02-16T22:56:11.319 INFO:tasks.workunit.client.1.smithi192.stdout:5/202: dwrite d6/d12/f32 [0,4194304] 0 2024-02-16T22:56:11.325 INFO:tasks.workunit.client.0.smithi064.stdout:3/132: write d4/f12 [1133181,99033] 0 2024-02-16T22:56:11.328 INFO:tasks.workunit.client.1.smithi192.stdout:5/203: dwrite d6/d12/d15/f28 [0,4194304] 0 2024-02-16T22:56:11.330 INFO:tasks.workunit.client.1.smithi192.stdout:3/202: chown da/d11/c40 1 1 2024-02-16T22:56:11.331 INFO:tasks.workunit.client.0.smithi064.stdout:7/67: dwrite d5/d6/fb [4194304,4194304] 0 2024-02-16T22:56:11.334 INFO:tasks.workunit.client.0.smithi064.stdout:1/94: link d2/ce d2/df/d14/c22 0 2024-02-16T22:56:11.334 INFO:tasks.workunit.client.1.smithi192.stdout:2/217: creat d22/d24/f49 x:0 0 0 2024-02-16T22:56:11.334 INFO:tasks.workunit.client.0.smithi064.stdout:1/95: dread - d2/df/d14/f16 zero size 2024-02-16T22:56:11.335 INFO:tasks.workunit.client.1.smithi192.stdout:7/243: fsync d1/d2/f25 0 2024-02-16T22:56:11.341 INFO:tasks.workunit.client.1.smithi192.stdout:5/204: dwrite d6/d12/f18 [4194304,4194304] 0 2024-02-16T22:56:11.343 INFO:tasks.workunit.client.0.smithi064.stdout:1/96: dwrite d2/df/d14/d18/f19 [0,4194304] 0 2024-02-16T22:56:11.344 INFO:tasks.workunit.client.1.smithi192.stdout:4/195: unlink d1/d15/d23/f3d 0 2024-02-16T22:56:11.344 INFO:tasks.workunit.client.0.smithi064.stdout:4/106: truncate dc/ff 1330420 0 2024-02-16T22:56:11.347 INFO:tasks.workunit.client.0.smithi064.stdout:7/68: symlink d5/l11 0 2024-02-16T22:56:11.347 INFO:tasks.workunit.client.0.smithi064.stdout:7/69: chown c2 3446 1 2024-02-16T22:56:11.349 INFO:tasks.workunit.client.1.smithi192.stdout:6/225: unlink da/df/f11 0 2024-02-16T22:56:11.351 INFO:tasks.workunit.client.0.smithi064.stdout:5/101: rename d6/f1d to d6/d18/d19/f26 0 2024-02-16T22:56:11.357 INFO:tasks.workunit.client.1.smithi192.stdout:0/170: truncate d1/d3/de/f10 3606287 0 2024-02-16T22:56:11.359 INFO:tasks.workunit.client.1.smithi192.stdout:0/171: write d1/d3/d29/f27 [116511,51783] 0 2024-02-16T22:56:11.359 INFO:tasks.workunit.client.0.smithi064.stdout:3/133: creat d4/d13/d1b/d27/f2c x:0 0 0 2024-02-16T22:56:11.360 INFO:tasks.workunit.client.1.smithi192.stdout:7/244: mknod d1/d1f/c52 0 2024-02-16T22:56:11.362 INFO:tasks.workunit.client.0.smithi064.stdout:1/97: mkdir d2/df/d23 0 2024-02-16T22:56:11.362 INFO:tasks.workunit.client.1.smithi192.stdout:9/275: getdents d7/d19/d37/d39 0 2024-02-16T22:56:11.364 INFO:tasks.workunit.client.1.smithi192.stdout:5/205: creat d6/d12/d1d/f34 x:0 0 0 2024-02-16T22:56:11.364 INFO:tasks.workunit.client.0.smithi064.stdout:4/107: write dc/f19 [2407462,103484] 0 2024-02-16T22:56:11.365 INFO:tasks.workunit.client.1.smithi192.stdout:5/206: chown d6/d12/d1d/l1f 3019258 1 2024-02-16T22:56:11.365 INFO:tasks.workunit.client.0.smithi064.stdout:7/70: mknod d5/c12 0 2024-02-16T22:56:11.366 INFO:tasks.workunit.client.1.smithi192.stdout:5/207: write d6/d12/d15/f1c [4856850,129844] 0 2024-02-16T22:56:11.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:10 smithi064 ceph-mon[29226]: pgmap v10: 65 pgs: 65 active+clean; 313 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 71 KiB/s rd, 15 MiB/s wr, 229 op/s 2024-02-16T22:56:11.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:10 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:56:11.369 INFO:tasks.workunit.client.0.smithi064.stdout:7/71: write d5/d6/f9 [4523205,51823] 0 2024-02-16T22:56:11.370 INFO:tasks.workunit.client.0.smithi064.stdout:5/102: unlink d6/fe 0 2024-02-16T22:56:11.371 INFO:tasks.workunit.client.1.smithi192.stdout:8/157: symlink d5/db/dd/l2a 0 2024-02-16T22:56:11.372 INFO:tasks.workunit.client.0.smithi064.stdout:3/134: dwrite d4/d1f/f20 [0,4194304] 0 2024-02-16T22:56:11.372 INFO:tasks.workunit.client.0.smithi064.stdout:2/84: getdents d7/de 0 2024-02-16T22:56:11.373 INFO:tasks.workunit.client.1.smithi192.stdout:8/158: chown d5/d9/f17 986591398 1 2024-02-16T22:56:11.373 INFO:tasks.workunit.client.1.smithi192.stdout:6/226: rmdir da 39 2024-02-16T22:56:11.374 INFO:tasks.workunit.client.0.smithi064.stdout:5/103: chown d6/d18/d19 0 1 2024-02-16T22:56:11.374 INFO:tasks.workunit.client.1.smithi192.stdout:5/208: dwrite d6/f10 [0,4194304] 0 2024-02-16T22:56:11.374 INFO:tasks.workunit.client.1.smithi192.stdout:8/159: chown d5/db/dd/l2a 180453 1 2024-02-16T22:56:11.374 INFO:tasks.workunit.client.1.smithi192.stdout:5/209: stat d6/d8/c22 0 2024-02-16T22:56:11.375 INFO:tasks.workunit.client.0.smithi064.stdout:1/98: dwrite d2/df/f1c [0,4194304] 0 2024-02-16T22:56:11.378 INFO:tasks.workunit.client.0.smithi064.stdout:1/99: chown d2/df/d14/l1b 14396645 1 2024-02-16T22:56:11.380 INFO:tasks.workunit.client.1.smithi192.stdout:9/276: creat d7/d15/d36/d38/d3f/f59 x:0 0 0 2024-02-16T22:56:11.383 INFO:tasks.workunit.client.0.smithi064.stdout:7/72: rename d5/c12 to d5/d6/c13 0 2024-02-16T22:56:11.388 INFO:tasks.workunit.client.0.smithi064.stdout:2/85: dwrite d7/de/f11 [0,4194304] 0 2024-02-16T22:56:11.391 INFO:tasks.workunit.client.0.smithi064.stdout:3/135: symlink d4/d13/d1b/d27/l2d 0 2024-02-16T22:56:11.393 INFO:tasks.workunit.client.1.smithi192.stdout:2/218: creat d22/f4a x:0 0 0 2024-02-16T22:56:11.393 INFO:tasks.workunit.client.0.smithi064.stdout:5/104: truncate d6/f13 1235780 0 2024-02-16T22:56:11.395 INFO:tasks.workunit.client.0.smithi064.stdout:7/73: dwrite d5/f10 [0,4194304] 0 2024-02-16T22:56:11.395 INFO:tasks.workunit.client.1.smithi192.stdout:5/210: symlink d6/d20/l35 0 2024-02-16T22:56:11.396 INFO:tasks.workunit.client.0.smithi064.stdout:7/74: chown c2 6437589 1 2024-02-16T22:56:11.399 INFO:tasks.workunit.client.0.smithi064.stdout:2/86: dwrite d7/f9 [0,4194304] 0 2024-02-16T22:56:11.400 INFO:tasks.workunit.client.0.smithi064.stdout:2/87: chown d7/cd 181613374 1 2024-02-16T22:56:11.404 INFO:tasks.workunit.client.0.smithi064.stdout:2/88: dread d7/f10 [0,4194304] 0 2024-02-16T22:56:11.405 INFO:tasks.workunit.client.0.smithi064.stdout:2/89: chown l3 11 1 2024-02-16T22:56:11.405 INFO:tasks.workunit.client.1.smithi192.stdout:8/160: creat d5/db/de/df/f2b x:0 0 0 2024-02-16T22:56:11.408 INFO:tasks.workunit.client.1.smithi192.stdout:7/245: mknod d1/d2/c53 0 2024-02-16T22:56:11.412 INFO:tasks.workunit.client.1.smithi192.stdout:5/211: creat d6/d12/d1d/f36 x:0 0 0 2024-02-16T22:56:11.414 INFO:tasks.workunit.client.0.smithi064.stdout:7/75: symlink d5/l14 0 2024-02-16T22:56:11.415 INFO:tasks.workunit.client.0.smithi064.stdout:7/76: write f0 [5025505,50343] 0 2024-02-16T22:56:11.416 INFO:tasks.workunit.client.0.smithi064.stdout:1/100: creat d2/df/d11/d1a/d1d/f24 x:0 0 0 2024-02-16T22:56:11.417 INFO:tasks.workunit.client.1.smithi192.stdout:8/161: rename d5/db/de/c29 to d5/db/de/df/c2c 0 2024-02-16T22:56:11.419 INFO:tasks.workunit.client.1.smithi192.stdout:7/246: mkdir d1/d2/de/d2c/d49/d54 0 2024-02-16T22:56:11.420 INFO:tasks.workunit.client.1.smithi192.stdout:7/247: readlink d1/d2/l8 0 2024-02-16T22:56:11.420 INFO:tasks.workunit.client.0.smithi064.stdout:2/90: creat d7/f12 x:0 0 0 2024-02-16T22:56:11.422 INFO:tasks.workunit.client.1.smithi192.stdout:2/219: symlink d22/l4b 0 2024-02-16T22:56:11.424 INFO:tasks.workunit.client.0.smithi064.stdout:0/85: sync 2024-02-16T22:56:11.424 INFO:tasks.workunit.client.0.smithi064.stdout:6/127: sync 2024-02-16T22:56:11.424 INFO:tasks.workunit.client.1.smithi192.stdout:5/212: unlink d6/d12/d15/f28 0 2024-02-16T22:56:11.426 INFO:tasks.workunit.client.1.smithi192.stdout:8/162: creat d5/db/de/df/f2d x:0 0 0 2024-02-16T22:56:11.427 INFO:tasks.workunit.client.0.smithi064.stdout:4/108: getdents dc 0 2024-02-16T22:56:11.431 INFO:tasks.workunit.client.1.smithi192.stdout:4/196: dwrite d1/d8/d11/f2a [0,4194304] 0 2024-02-16T22:56:11.431 INFO:tasks.workunit.client.1.smithi192.stdout:9/277: dread d7/d15/f33 [0,4194304] 0 2024-02-16T22:56:11.432 INFO:tasks.workunit.client.1.smithi192.stdout:2/220: creat d22/d24/d39/f4c x:0 0 0 2024-02-16T22:56:11.433 INFO:tasks.workunit.client.0.smithi064.stdout:4/109: dwrite f5 [4194304,4194304] 0 2024-02-16T22:56:11.434 INFO:tasks.workunit.client.0.smithi064.stdout:4/110: write dc/d10/f14 [342387,2686] 0 2024-02-16T22:56:11.434 INFO:tasks.workunit.client.1.smithi192.stdout:9/278: write d7/d19/d28/f52 [314176,95795] 0 2024-02-16T22:56:11.436 INFO:tasks.workunit.client.1.smithi192.stdout:2/221: write d22/d24/d2b/d2c/f30 [3427943,14004] 0 2024-02-16T22:56:11.438 INFO:tasks.workunit.client.1.smithi192.stdout:2/222: fdatasync d22/f31 0 2024-02-16T22:56:11.439 INFO:tasks.workunit.client.1.smithi192.stdout:8/163: dwrite d5/db/de/df/f2d [0,4194304] 0 2024-02-16T22:56:11.441 INFO:tasks.workunit.client.1.smithi192.stdout:7/248: dread d1/d2/f6 [0,4194304] 0 2024-02-16T22:56:11.445 INFO:tasks.workunit.client.1.smithi192.stdout:5/213: mknod d6/d12/d2d/c37 0 2024-02-16T22:56:11.446 INFO:tasks.workunit.client.1.smithi192.stdout:9/279: mkdir d7/d19/d37/d5a 0 2024-02-16T22:56:11.447 INFO:tasks.workunit.client.1.smithi192.stdout:5/214: stat d6/d12/d1d/l25 0 2024-02-16T22:56:11.448 INFO:tasks.workunit.client.1.smithi192.stdout:4/197: link d1/f4 d1/d15/f45 0 2024-02-16T22:56:11.449 INFO:tasks.workunit.client.1.smithi192.stdout:0/172: dwrite d1/d3/de/f10 [0,4194304] 0 2024-02-16T22:56:11.449 INFO:tasks.workunit.client.1.smithi192.stdout:8/164: creat d5/db/dd/f2e x:0 0 0 2024-02-16T22:56:11.451 INFO:tasks.workunit.client.1.smithi192.stdout:7/249: mkdir d1/d2/d2d/d55 0 2024-02-16T22:56:11.452 INFO:tasks.workunit.client.1.smithi192.stdout:9/280: readlink d7/l1c 0 2024-02-16T22:56:11.452 INFO:tasks.workunit.client.1.smithi192.stdout:0/173: fdatasync d1/d3/de/d22/f30 0 2024-02-16T22:56:11.454 INFO:tasks.workunit.client.1.smithi192.stdout:4/198: mknod d1/d15/d29/c46 0 2024-02-16T22:56:11.455 INFO:tasks.workunit.client.1.smithi192.stdout:8/165: rmdir d5/d9 39 2024-02-16T22:56:11.455 INFO:tasks.workunit.client.1.smithi192.stdout:0/174: write d1/d3/d29/f1a [2857951,126367] 0 2024-02-16T22:56:11.458 INFO:tasks.workunit.client.0.smithi064.stdout:4/111: write dc/d10/f12 [626452,16144] 0 2024-02-16T22:56:11.458 INFO:tasks.workunit.client.1.smithi192.stdout:7/250: mknod d1/d2/de/d2c/d49/c56 0 2024-02-16T22:56:11.459 INFO:tasks.workunit.client.1.smithi192.stdout:5/215: dwrite d6/d12/d1d/f36 [0,4194304] 0 2024-02-16T22:56:11.460 INFO:tasks.workunit.client.0.smithi064.stdout:3/136: truncate d4/fd 1865245 0 2024-02-16T22:56:11.467 INFO:tasks.workunit.client.1.smithi192.stdout:8/166: chown d5/d9/f17 46358662 1 2024-02-16T22:56:11.467 INFO:tasks.workunit.client.1.smithi192.stdout:7/251: dwrite d1/d2/f3b [0,4194304] 0 2024-02-16T22:56:11.467 INFO:tasks.workunit.client.0.smithi064.stdout:2/91: dwrite d7/f10 [0,4194304] 0 2024-02-16T22:56:11.468 INFO:tasks.workunit.client.1.smithi192.stdout:8/167: write d5/db/dd/f1c [632176,11018] 0 2024-02-16T22:56:11.468 INFO:tasks.workunit.client.0.smithi064.stdout:6/128: creat d0/d12/f1f x:0 0 0 2024-02-16T22:56:11.469 INFO:tasks.workunit.client.0.smithi064.stdout:0/86: dwrite f3 [0,4194304] 0 2024-02-16T22:56:11.471 INFO:tasks.workunit.client.0.smithi064.stdout:6/129: write d0/d6/d10/f1c [2205315,118815] 0 2024-02-16T22:56:11.472 INFO:tasks.workunit.client.1.smithi192.stdout:2/223: getdents d22 0 2024-02-16T22:56:11.472 INFO:tasks.workunit.client.0.smithi064.stdout:6/130: dread - d0/d12/f1f zero size 2024-02-16T22:56:11.475 INFO:tasks.workunit.client.1.smithi192.stdout:4/199: symlink d1/d26/d34/l47 0 2024-02-16T22:56:11.476 INFO:tasks.workunit.client.1.smithi192.stdout:4/200: write d1/f19 [1489854,24179] 0 2024-02-16T22:56:11.476 INFO:tasks.workunit.client.1.smithi192.stdout:2/224: read d22/f27 [1624079,130556] 0 2024-02-16T22:56:11.477 INFO:tasks.workunit.client.1.smithi192.stdout:2/225: chown d22/d24/d2b/f45 1872 1 2024-02-16T22:56:11.483 INFO:tasks.workunit.client.1.smithi192.stdout:8/168: fsync d5/db/de/df/f2d 0 2024-02-16T22:56:11.484 INFO:tasks.workunit.client.1.smithi192.stdout:8/169: readlink d5/db/dd/l2a 0 2024-02-16T22:56:11.486 INFO:tasks.workunit.client.1.smithi192.stdout:5/216: rename l0 to d6/d12/d15/l38 0 2024-02-16T22:56:11.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/217: readlink d6/d8/l31 0 2024-02-16T22:56:11.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/218: chown d6/d8 178560696 1 2024-02-16T22:56:11.488 INFO:tasks.workunit.client.1.smithi192.stdout:8/170: dread d5/f15 [0,4194304] 0 2024-02-16T22:56:11.489 INFO:tasks.workunit.client.0.smithi064.stdout:3/137: symlink d4/l2e 0 2024-02-16T22:56:11.490 INFO:tasks.workunit.client.0.smithi064.stdout:3/138: write d4/d1f/f20 [3977004,72286] 0 2024-02-16T22:56:11.491 INFO:tasks.workunit.client.0.smithi064.stdout:3/139: chown d4/d13/f16 13 1 2024-02-16T22:56:11.491 INFO:tasks.workunit.client.0.smithi064.stdout:2/92: creat d7/f13 x:0 0 0 2024-02-16T22:56:11.494 INFO:tasks.workunit.client.0.smithi064.stdout:0/87: rmdir da/dc 39 2024-02-16T22:56:11.498 INFO:tasks.workunit.client.1.smithi192.stdout:4/201: mkdir d1/d15/d48 0 2024-02-16T22:56:11.499 INFO:tasks.workunit.client.1.smithi192.stdout:4/202: chown d1/d15/d29/d3c 16031 1 2024-02-16T22:56:11.499 INFO:tasks.workunit.client.0.smithi064.stdout:4/112: truncate dc/ff 1815621 0 2024-02-16T22:56:11.500 INFO:tasks.workunit.client.0.smithi064.stdout:4/113: stat dc/d10/l1b 0 2024-02-16T22:56:11.501 INFO:tasks.workunit.client.0.smithi064.stdout:7/77: getdents d5/d6 0 2024-02-16T22:56:11.502 INFO:tasks.workunit.client.0.smithi064.stdout:3/140: mknod d4/d14/c2f 0 2024-02-16T22:56:11.503 INFO:tasks.workunit.client.0.smithi064.stdout:3/141: chown d4/f11 0 1 2024-02-16T22:56:11.503 INFO:tasks.workunit.client.0.smithi064.stdout:3/142: chown d4/d13 109 1 2024-02-16T22:56:11.503 INFO:tasks.workunit.client.0.smithi064.stdout:3/143: dread - d4/d13/f28 zero size 2024-02-16T22:56:11.503 INFO:tasks.workunit.client.0.smithi064.stdout:2/93: mkdir d7/d14 0 2024-02-16T22:56:11.507 INFO:tasks.workunit.client.1.smithi192.stdout:7/252: symlink d1/d2/l57 0 2024-02-16T22:56:11.511 INFO:tasks.workunit.client.0.smithi064.stdout:2/94: dwrite d7/f10 [4194304,4194304] 0 2024-02-16T22:56:11.511 INFO:tasks.workunit.client.1.smithi192.stdout:4/203: mknod d1/d15/d29/c49 0 2024-02-16T22:56:11.511 INFO:tasks.workunit.client.0.smithi064.stdout:2/95: chown l6 247496245 1 2024-02-16T22:56:11.512 INFO:tasks.workunit.client.0.smithi064.stdout:2/96: readlink l6 0 2024-02-16T22:56:11.512 INFO:tasks.workunit.client.1.smithi192.stdout:4/204: write d1/d26/f28 [4261978,105382] 0 2024-02-16T22:56:11.513 INFO:tasks.workunit.client.1.smithi192.stdout:2/226: symlink d22/d24/d39/d42/l4d 0 2024-02-16T22:56:11.516 INFO:tasks.workunit.client.0.smithi064.stdout:3/144: dread d4/f1e [0,4194304] 0 2024-02-16T22:56:11.517 INFO:tasks.workunit.client.0.smithi064.stdout:3/145: read d4/f11 [3529124,16363] 0 2024-02-16T22:56:11.517 INFO:tasks.workunit.client.0.smithi064.stdout:0/88: creat da/dc/de/f17 x:0 0 0 2024-02-16T22:56:11.519 INFO:tasks.workunit.client.1.smithi192.stdout:9/281: truncate d7/d15/d36/d38/f47 1380912 0 2024-02-16T22:56:11.520 INFO:tasks.workunit.client.0.smithi064.stdout:3/146: dread d4/d1f/f20 [0,4194304] 0 2024-02-16T22:56:11.520 INFO:tasks.workunit.client.1.smithi192.stdout:2/227: dwrite fa [0,4194304] 0 2024-02-16T22:56:11.523 INFO:tasks.workunit.client.0.smithi064.stdout:7/78: getdents d5/d6 0 2024-02-16T22:56:11.528 INFO:tasks.workunit.client.1.smithi192.stdout:2/228: dwrite d22/d24/d39/d3d/f40 [0,4194304] 0 2024-02-16T22:56:11.529 INFO:tasks.workunit.client.1.smithi192.stdout:2/229: fsync d22/d24/d2b/d2c/f2d 0 2024-02-16T22:56:11.529 INFO:tasks.workunit.client.0.smithi064.stdout:3/147: dwrite d4/d14/f22 [0,4194304] 0 2024-02-16T22:56:11.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/219: truncate d6/d12/d1d/f36 1657006 0 2024-02-16T22:56:11.534 INFO:tasks.workunit.client.1.smithi192.stdout:9/282: mkdir d7/d19/d5b 0 2024-02-16T22:56:11.539 INFO:tasks.workunit.client.1.smithi192.stdout:5/220: dwrite d6/fd [0,4194304] 0 2024-02-16T22:56:11.539 INFO:tasks.workunit.client.1.smithi192.stdout:8/171: truncate d5/f1e 4834162 0 2024-02-16T22:56:11.541 INFO:tasks.workunit.client.0.smithi064.stdout:6/131: link d0/d5/df/d17/l1a d0/d6/d19/l20 0 2024-02-16T22:56:11.542 INFO:tasks.workunit.client.1.smithi192.stdout:7/253: truncate d1/d2/de/f28 934134 0 2024-02-16T22:56:11.543 INFO:tasks.workunit.client.1.smithi192.stdout:7/254: write d1/d1f/f4f [301045,60833] 0 2024-02-16T22:56:11.548 INFO:tasks.workunit.client.1.smithi192.stdout:8/172: dwrite d5/db/dd/f2e [0,4194304] 0 2024-02-16T22:56:11.570 INFO:tasks.workunit.client.1.smithi192.stdout:1/172: sync 2024-02-16T22:56:11.626 INFO:tasks.workunit.client.1.smithi192.stdout:0/175: sync 2024-02-16T22:56:11.626 INFO:tasks.workunit.client.1.smithi192.stdout:6/227: sync 2024-02-16T22:56:11.626 INFO:tasks.workunit.client.1.smithi192.stdout:3/203: sync 2024-02-16T22:56:11.630 INFO:tasks.workunit.client.1.smithi192.stdout:0/176: dwrite d1/fd [4194304,4194304] 0 2024-02-16T22:56:11.636 INFO:tasks.workunit.client.1.smithi192.stdout:0/177: dwrite d1/f19 [0,4194304] 0 2024-02-16T22:56:11.639 INFO:tasks.workunit.client.1.smithi192.stdout:9/283: rmdir d7/d15/d36/d38/d3f 39 2024-02-16T22:56:11.642 INFO:tasks.workunit.client.1.smithi192.stdout:5/221: creat d6/d12/d1d/f39 x:0 0 0 2024-02-16T22:56:11.642 INFO:tasks.workunit.client.1.smithi192.stdout:5/222: chown d6/d12/c1a 35 1 2024-02-16T22:56:11.646 INFO:tasks.workunit.client.0.smithi064.stdout:3/148: dread d4/fd [0,4194304] 0 2024-02-16T22:56:11.646 INFO:tasks.workunit.client.1.smithi192.stdout:2/230: creat d22/d24/d2b/d2c/d3f/f4e x:0 0 0 2024-02-16T22:56:11.647 INFO:tasks.workunit.client.1.smithi192.stdout:2/231: dread - d22/d24/d39/f4c zero size 2024-02-16T22:56:11.648 INFO:tasks.workunit.client.1.smithi192.stdout:4/205: truncate d1/d15/d29/d3c/f43 2413577 0 2024-02-16T22:56:11.649 INFO:tasks.workunit.client.1.smithi192.stdout:7/255: fsync d1/d2/f4 0 2024-02-16T22:56:11.649 INFO:tasks.workunit.client.1.smithi192.stdout:7/256: fsync d1/d2/f3d 0 2024-02-16T22:56:11.651 INFO:tasks.workunit.client.0.smithi064.stdout:2/97: truncate d7/de/f11 2341060 0 2024-02-16T22:56:11.652 INFO:tasks.workunit.client.0.smithi064.stdout:6/132: unlink d0/d6/da/c1e 0 2024-02-16T22:56:11.654 INFO:tasks.workunit.client.1.smithi192.stdout:8/173: rename d5/f15 to d5/db/de/d21/f2f 0 2024-02-16T22:56:11.656 INFO:tasks.workunit.client.1.smithi192.stdout:1/173: unlink d3/c2f 0 2024-02-16T22:56:11.660 INFO:tasks.workunit.client.0.smithi064.stdout:0/89: link f9 da/f18 0 2024-02-16T22:56:11.662 INFO:tasks.workunit.client.1.smithi192.stdout:3/204: creat da/d11/d20/f4a x:0 0 0 2024-02-16T22:56:11.663 INFO:tasks.workunit.client.0.smithi064.stdout:2/98: creat d7/de/f15 x:0 0 0 2024-02-16T22:56:11.666 INFO:tasks.workunit.client.1.smithi192.stdout:0/178: symlink d1/d3/d29/l33 0 2024-02-16T22:56:11.666 INFO:tasks.workunit.client.0.smithi064.stdout:7/79: truncate f0 3192623 0 2024-02-16T22:56:11.667 INFO:tasks.workunit.client.1.smithi192.stdout:1/174: dread d3/f16 [0,4194304] 0 2024-02-16T22:56:11.668 INFO:tasks.workunit.client.1.smithi192.stdout:1/175: write d3/de/f1a [3860177,73662] 0 2024-02-16T22:56:11.671 INFO:tasks.workunit.client.0.smithi064.stdout:7/80: dread d5/d6/f9 [4194304,4194304] 0 2024-02-16T22:56:11.671 INFO:tasks.workunit.client.0.smithi064.stdout:0/90: readlink da/dc/ld 0 2024-02-16T22:56:11.674 INFO:tasks.workunit.client.1.smithi192.stdout:5/223: stat c4 0 2024-02-16T22:56:11.674 INFO:tasks.workunit.client.0.smithi064.stdout:6/133: mkdir d0/d1d/d21 0 2024-02-16T22:56:11.675 INFO:tasks.workunit.client.0.smithi064.stdout:6/134: write d0/d12/f1f [591929,128007] 0 2024-02-16T22:56:11.677 INFO:tasks.workunit.client.0.smithi064.stdout:3/149: link d4/f1e d4/d14/f30 0 2024-02-16T22:56:11.678 INFO:tasks.workunit.client.1.smithi192.stdout:5/224: dwrite f2 [0,4194304] 0 2024-02-16T22:56:11.679 INFO:tasks.workunit.client.0.smithi064.stdout:3/150: write d4/d13/d1b/f1c [355806,88533] 0 2024-02-16T22:56:11.686 INFO:tasks.workunit.client.1.smithi192.stdout:8/174: symlink d5/d16/l30 0 2024-02-16T22:56:11.690 INFO:tasks.workunit.client.0.smithi064.stdout:6/135: creat d0/f22 x:0 0 0 2024-02-16T22:56:11.691 INFO:tasks.workunit.client.0.smithi064.stdout:6/136: chown d0/d5/df/d17/l1a 21545 1 2024-02-16T22:56:11.691 INFO:tasks.workunit.client.0.smithi064.stdout:6/137: chown d0/d6/f15 69453008 1 2024-02-16T22:56:11.692 INFO:tasks.workunit.client.0.smithi064.stdout:3/151: symlink d4/d1f/l31 0 2024-02-16T22:56:11.693 INFO:tasks.workunit.client.0.smithi064.stdout:3/152: chown l1 245 1 2024-02-16T22:56:11.693 INFO:tasks.workunit.client.0.smithi064.stdout:3/153: chown d4/la 11 1 2024-02-16T22:56:11.696 INFO:tasks.workunit.client.1.smithi192.stdout:3/205: unlink da/d11/f21 0 2024-02-16T22:56:11.697 INFO:tasks.workunit.client.0.smithi064.stdout:0/91: dwrite f9 [0,4194304] 0 2024-02-16T22:56:11.699 INFO:tasks.workunit.client.1.smithi192.stdout:0/179: mkdir d1/d2/d34 0 2024-02-16T22:56:11.702 INFO:tasks.workunit.client.1.smithi192.stdout:1/176: read d3/de/f35 [239154,17531] 0 2024-02-16T22:56:11.702 INFO:tasks.workunit.client.1.smithi192.stdout:1/177: read - d3/de/d27/f30 zero size 2024-02-16T22:56:11.704 INFO:tasks.workunit.client.1.smithi192.stdout:1/178: chown d3/de/d27/l2e 796 1 2024-02-16T22:56:11.704 INFO:tasks.workunit.client.0.smithi064.stdout:0/92: dwrite f9 [0,4194304] 0 2024-02-16T22:56:11.705 INFO:tasks.workunit.client.0.smithi064.stdout:0/93: stat da/dc/de/f16 0 2024-02-16T22:56:11.705 INFO:tasks.workunit.client.1.smithi192.stdout:1/179: write d3/de/d27/f30 [394539,80020] 0 2024-02-16T22:56:11.709 INFO:tasks.workunit.client.0.smithi064.stdout:3/154: creat d4/d1f/f32 x:0 0 0 2024-02-16T22:56:11.711 INFO:tasks.workunit.client.0.smithi064.stdout:3/155: fdatasync d4/d1f/f24 0 2024-02-16T22:56:11.712 INFO:tasks.workunit.client.0.smithi064.stdout:3/156: stat d4/d14 0 2024-02-16T22:56:11.713 INFO:tasks.workunit.client.1.smithi192.stdout:4/206: unlink d1/d8/d11/f2d 0 2024-02-16T22:56:11.714 INFO:tasks.workunit.client.0.smithi064.stdout:2/99: unlink d7/f9 0 2024-02-16T22:56:11.714 INFO:tasks.workunit.client.0.smithi064.stdout:0/94: dwrite da/f10 [0,4194304] 0 2024-02-16T22:56:11.714 INFO:tasks.workunit.client.0.smithi064.stdout:3/157: truncate d4/f17 708362 0 2024-02-16T22:56:11.715 INFO:tasks.workunit.client.1.smithi192.stdout:5/225: creat d6/d12/d1d/f3a x:0 0 0 2024-02-16T22:56:11.716 INFO:tasks.workunit.client.0.smithi064.stdout:3/158: read d4/f8 [1239207,128539] 0 2024-02-16T22:56:11.716 INFO:tasks.workunit.client.0.smithi064.stdout:2/100: write d7/f13 [921835,124913] 0 2024-02-16T22:56:11.717 INFO:tasks.workunit.client.1.smithi192.stdout:9/284: dwrite f1 [0,4194304] 0 2024-02-16T22:56:11.718 INFO:tasks.workunit.client.0.smithi064.stdout:2/101: chown d7/de/lf 2674 1 2024-02-16T22:56:11.720 INFO:tasks.workunit.client.0.smithi064.stdout:2/102: dread - d7/de/f15 zero size 2024-02-16T22:56:11.723 INFO:tasks.workunit.client.0.smithi064.stdout:0/95: dread da/fb [0,4194304] 0 2024-02-16T22:56:11.726 INFO:tasks.workunit.client.0.smithi064.stdout:7/81: dread f0 [0,4194304] 0 2024-02-16T22:56:11.727 INFO:tasks.workunit.client.0.smithi064.stdout:3/159: dwrite d4/f5 [0,4194304] 0 2024-02-16T22:56:11.732 INFO:tasks.workunit.client.0.smithi064.stdout:0/96: creat da/dc/de/d12/f19 x:0 0 0 2024-02-16T22:56:11.733 INFO:tasks.workunit.client.0.smithi064.stdout:0/97: write da/dc/de/f16 [1094039,22741] 0 2024-02-16T22:56:11.735 INFO:tasks.workunit.client.0.smithi064.stdout:3/160: symlink d4/d1f/l33 0 2024-02-16T22:56:11.739 INFO:tasks.workunit.client.0.smithi064.stdout:2/103: mknod d7/de/c16 0 2024-02-16T22:56:11.741 INFO:tasks.workunit.client.0.smithi064.stdout:6/138: getdents d0/d6/da 0 2024-02-16T22:56:11.743 INFO:tasks.workunit.client.0.smithi064.stdout:0/98: mknod da/dc/de/c1a 0 2024-02-16T22:56:11.747 INFO:tasks.workunit.client.0.smithi064.stdout:0/99: dread da/f10 [0,4194304] 0 2024-02-16T22:56:11.747 INFO:tasks.workunit.client.0.smithi064.stdout:6/139: unlink d0/d5/df/d17/l1a 0 2024-02-16T22:56:11.747 INFO:tasks.workunit.client.0.smithi064.stdout:0/100: fdatasync f9 0 2024-02-16T22:56:11.749 INFO:tasks.workunit.client.0.smithi064.stdout:3/161: truncate d4/f1e 38362 0 2024-02-16T22:56:11.749 INFO:tasks.workunit.client.0.smithi064.stdout:3/162: chown d4/d14/f22 59128 1 2024-02-16T22:56:11.750 INFO:tasks.workunit.client.0.smithi064.stdout:9/127: sync 2024-02-16T22:56:11.751 INFO:tasks.workunit.client.0.smithi064.stdout:9/128: truncate d3/d11/f1f 518246 0 2024-02-16T22:56:11.755 INFO:tasks.workunit.client.0.smithi064.stdout:3/163: mkdir d4/d1f/d34 0 2024-02-16T22:56:11.756 INFO:tasks.workunit.client.0.smithi064.stdout:7/82: truncate d5/d6/fb 1242686 0 2024-02-16T22:56:11.757 INFO:tasks.workunit.client.0.smithi064.stdout:7/83: read d5/f10 [1181201,39693] 0 2024-02-16T22:56:11.761 INFO:tasks.workunit.client.0.smithi064.stdout:3/164: mkdir d4/d13/d35 0 2024-02-16T22:56:11.764 INFO:tasks.workunit.client.0.smithi064.stdout:0/101: link f6 da/f1b 0 2024-02-16T22:56:11.765 INFO:tasks.workunit.client.0.smithi064.stdout:6/140: dwrite d0/d5/f11 [0,4194304] 0 2024-02-16T22:56:11.765 INFO:tasks.workunit.client.0.smithi064.stdout:0/102: write f3 [1136731,126595] 0 2024-02-16T22:56:11.766 INFO:tasks.workunit.client.0.smithi064.stdout:6/141: stat d0/d5/df 0 2024-02-16T22:56:11.766 INFO:tasks.workunit.client.0.smithi064.stdout:9/129: symlink d3/l33 0 2024-02-16T22:56:11.766 INFO:tasks.workunit.client.0.smithi064.stdout:0/103: dread - da/dc/f13 zero size 2024-02-16T22:56:11.767 INFO:tasks.workunit.client.0.smithi064.stdout:9/130: fdatasync f1 0 2024-02-16T22:56:11.767 INFO:tasks.workunit.client.0.smithi064.stdout:8/109: sync 2024-02-16T22:56:11.767 INFO:tasks.workunit.client.0.smithi064.stdout:4/114: sync 2024-02-16T22:56:11.767 INFO:tasks.workunit.client.0.smithi064.stdout:1/101: sync 2024-02-16T22:56:11.767 INFO:tasks.workunit.client.0.smithi064.stdout:5/105: sync 2024-02-16T22:56:11.769 INFO:tasks.workunit.client.0.smithi064.stdout:7/84: mknod d5/d6/c15 0 2024-02-16T22:56:11.771 INFO:tasks.workunit.client.0.smithi064.stdout:7/85: chown d5/d6/f9 45924 1 2024-02-16T22:56:11.772 INFO:tasks.workunit.client.0.smithi064.stdout:2/104: truncate d7/de/f15 443743 0 2024-02-16T22:56:11.772 INFO:tasks.workunit.client.0.smithi064.stdout:2/105: readlink l1 0 2024-02-16T22:56:11.773 INFO:tasks.workunit.client.0.smithi064.stdout:2/106: write d7/de/f15 [1193661,112307] 0 2024-02-16T22:56:11.774 INFO:tasks.workunit.client.0.smithi064.stdout:3/165: creat d4/d13/d1b/d27/f36 x:0 0 0 2024-02-16T22:56:11.777 INFO:tasks.workunit.client.0.smithi064.stdout:4/115: creat dc/d10/d18/f1f x:0 0 0 2024-02-16T22:56:11.778 INFO:tasks.workunit.client.0.smithi064.stdout:9/131: fdatasync d3/d11/d15/d23/f25 0 2024-02-16T22:56:11.779 INFO:tasks.workunit.client.0.smithi064.stdout:9/132: truncate d3/d11/d2b/f29 807582 0 2024-02-16T22:56:11.780 INFO:tasks.workunit.client.0.smithi064.stdout:1/102: rename d2/df/d23 to d2/df/d11/d25 0 2024-02-16T22:56:11.780 INFO:tasks.workunit.client.0.smithi064.stdout:1/103: chown d2 1 1 2024-02-16T22:56:11.781 INFO:tasks.workunit.client.0.smithi064.stdout:9/133: truncate d3/d11/d2b/d2a/f30 864503 0 2024-02-16T22:56:11.797 INFO:tasks.workunit.client.1.smithi192.stdout:3/206: rmdir da/d11 39 2024-02-16T22:56:11.797 INFO:tasks.workunit.client.1.smithi192.stdout:5/226: symlink d6/d12/l3b 0 2024-02-16T22:56:11.798 INFO:tasks.workunit.client.1.smithi192.stdout:4/207: dwrite d1/d8/f9 [0,4194304] 0 2024-02-16T22:56:11.798 INFO:tasks.workunit.client.1.smithi192.stdout:6/228: truncate da/d27/f2f 2602101 0 2024-02-16T22:56:11.799 INFO:tasks.workunit.client.1.smithi192.stdout:9/285: write d7/df/f16 [365496,45985] 0 2024-02-16T22:56:11.801 INFO:tasks.workunit.client.1.smithi192.stdout:8/175: symlink d5/d26/l31 0 2024-02-16T22:56:11.801 INFO:tasks.workunit.client.1.smithi192.stdout:2/232: getdents d22/d24/d39/d42 0 2024-02-16T22:56:11.802 INFO:tasks.workunit.client.1.smithi192.stdout:4/208: write d1/d15/d23/f3b [153822,55039] 0 2024-02-16T22:56:11.803 INFO:tasks.workunit.client.1.smithi192.stdout:7/257: getdents d1/d2 0 2024-02-16T22:56:11.804 INFO:tasks.workunit.client.1.smithi192.stdout:5/227: mkdir d6/d12/d1d/d3c 0 2024-02-16T22:56:11.805 INFO:tasks.workunit.client.1.smithi192.stdout:3/207: chown da/d11/d20/f4a 29 1 2024-02-16T22:56:11.805 INFO:tasks.workunit.client.1.smithi192.stdout:9/286: mkdir d7/d19/d37/d5c 0 2024-02-16T22:56:11.806 INFO:tasks.workunit.client.1.smithi192.stdout:2/233: write d22/f4a [4585,40257] 0 2024-02-16T22:56:11.806 INFO:tasks.workunit.client.1.smithi192.stdout:8/176: write d5/db/de/df/f2b [1045180,28231] 0 2024-02-16T22:56:11.807 INFO:tasks.workunit.client.1.smithi192.stdout:4/209: write d1/f35 [2452300,84069] 0 2024-02-16T22:56:11.807 INFO:tasks.workunit.client.1.smithi192.stdout:1/180: dread d3/f21 [0,4194304] 0 2024-02-16T22:56:11.807 INFO:tasks.workunit.client.1.smithi192.stdout:7/258: creat d1/d2/f58 x:0 0 0 2024-02-16T22:56:11.809 INFO:tasks.workunit.client.1.smithi192.stdout:5/228: mknod d6/c3d 0 2024-02-16T22:56:11.809 INFO:tasks.workunit.client.1.smithi192.stdout:2/234: chown d22/d24/c26 4 1 2024-02-16T22:56:11.811 INFO:tasks.workunit.client.1.smithi192.stdout:2/235: write fe [2865112,105001] 0 2024-02-16T22:56:11.813 INFO:tasks.workunit.client.1.smithi192.stdout:2/236: readlink d22/d24/d2b/l44 0 2024-02-16T22:56:11.813 INFO:tasks.workunit.client.1.smithi192.stdout:4/210: read d1/d8/fe [82493,100848] 0 2024-02-16T22:56:11.816 INFO:tasks.workunit.client.1.smithi192.stdout:4/211: write d1/d26/f31 [689921,69714] 0 2024-02-16T22:56:11.817 INFO:tasks.workunit.client.1.smithi192.stdout:7/259: dwrite d1/f11 [4194304,4194304] 0 2024-02-16T22:56:11.818 INFO:tasks.workunit.client.1.smithi192.stdout:8/177: truncate f2 598614 0 2024-02-16T22:56:11.819 INFO:tasks.workunit.client.1.smithi192.stdout:8/178: fsync d5/f25 0 2024-02-16T22:56:11.820 INFO:tasks.workunit.client.1.smithi192.stdout:8/179: readlink d5/d16/l30 0 2024-02-16T22:56:11.820 INFO:tasks.workunit.client.1.smithi192.stdout:1/181: mkdir d3/de/d27/d34/d36 0 2024-02-16T22:56:11.823 INFO:tasks.workunit.client.1.smithi192.stdout:2/237: write d22/d24/d2b/d2c/f2e [337528,42447] 0 2024-02-16T22:56:11.824 INFO:tasks.workunit.client.1.smithi192.stdout:2/238: fdatasync d22/d24/d2b/d2c/f2d 0 2024-02-16T22:56:11.824 INFO:tasks.workunit.client.1.smithi192.stdout:5/229: link d6/d12/d1d/f3a d6/d8/f3e 0 2024-02-16T22:56:11.824 INFO:tasks.workunit.client.1.smithi192.stdout:2/239: write f8 [1392627,72637] 0 2024-02-16T22:56:11.826 INFO:tasks.workunit.client.1.smithi192.stdout:5/230: write d6/d12/d15/f2e [762889,14861] 0 2024-02-16T22:56:11.826 INFO:tasks.workunit.client.1.smithi192.stdout:5/231: chown d6/f29 111 1 2024-02-16T22:56:11.827 INFO:tasks.workunit.client.1.smithi192.stdout:5/232: dread - d6/f33 zero size 2024-02-16T22:56:11.828 INFO:tasks.workunit.client.1.smithi192.stdout:4/212: creat d1/d15/d23/f4a x:0 0 0 2024-02-16T22:56:11.831 INFO:tasks.workunit.client.1.smithi192.stdout:5/233: chown d6/d12/d1d/f34 12549 1 2024-02-16T22:56:11.831 INFO:tasks.workunit.client.1.smithi192.stdout:8/180: stat d5/db/l28 0 2024-02-16T22:56:11.832 INFO:tasks.workunit.client.1.smithi192.stdout:8/181: chown d5/db 2938 1 2024-02-16T22:56:11.832 INFO:tasks.workunit.client.1.smithi192.stdout:1/182: mknod d3/de/d27/c37 0 2024-02-16T22:56:11.832 INFO:tasks.workunit.client.1.smithi192.stdout:3/208: dwrite da/d11/d20/f24 [0,4194304] 0 2024-02-16T22:56:11.833 INFO:tasks.workunit.client.1.smithi192.stdout:7/260: mknod d1/d2/d37/c59 0 2024-02-16T22:56:11.834 INFO:tasks.workunit.client.1.smithi192.stdout:7/261: write d1/d19/f1d [4087199,76916] 0 2024-02-16T22:56:11.834 INFO:tasks.workunit.client.0.smithi064.stdout:6/142: creat d0/d1b/f23 x:0 0 0 2024-02-16T22:56:11.835 INFO:tasks.workunit.client.1.smithi192.stdout:7/262: write d1/f12 [291177,43878] 0 2024-02-16T22:56:11.835 INFO:tasks.workunit.client.1.smithi192.stdout:4/213: mknod d1/d15/d29/d3c/c4b 0 2024-02-16T22:56:11.836 INFO:tasks.workunit.client.0.smithi064.stdout:5/106: mknod d6/c27 0 2024-02-16T22:56:11.836 INFO:tasks.workunit.client.0.smithi064.stdout:2/107: rename d7/de/c16 to d7/d14/c17 0 2024-02-16T22:56:11.837 INFO:tasks.workunit.client.0.smithi064.stdout:1/104: creat d2/df/d14/d18/f26 x:0 0 0 2024-02-16T22:56:11.840 INFO:tasks.workunit.client.0.smithi064.stdout:8/110: truncate df/f12 603663 0 2024-02-16T22:56:11.840 INFO:tasks.workunit.client.0.smithi064.stdout:3/166: chown d4/f1e 4341503 1 2024-02-16T22:56:11.843 INFO:tasks.workunit.client.0.smithi064.stdout:9/134: rmdir d3/d11/d2b/d2a 39 2024-02-16T22:56:11.847 INFO:tasks.workunit.client.0.smithi064.stdout:6/143: mknod d0/c24 0 2024-02-16T22:56:11.848 INFO:tasks.workunit.client.0.smithi064.stdout:6/144: write d0/d6/f15 [4265881,61416] 0 2024-02-16T22:56:11.851 INFO:tasks.workunit.client.1.smithi192.stdout:2/240: mkdir d22/d24/d2b/d3e/d4f 0 2024-02-16T22:56:11.852 INFO:tasks.workunit.client.0.smithi064.stdout:6/145: dread d0/d5/f11 [0,4194304] 0 2024-02-16T22:56:11.857 INFO:tasks.workunit.client.1.smithi192.stdout:4/214: rmdir d1 39 2024-02-16T22:56:11.859 INFO:tasks.workunit.client.0.smithi064.stdout:6/146: dwrite d0/d6/d10/f14 [0,4194304] 0 2024-02-16T22:56:11.861 INFO:tasks.workunit.client.0.smithi064.stdout:8/111: mknod df/c1f 0 2024-02-16T22:56:11.863 INFO:tasks.workunit.client.0.smithi064.stdout:8/112: truncate df/f1a 4757959 0 2024-02-16T22:56:11.863 INFO:tasks.workunit.client.1.smithi192.stdout:9/287: dread d7/d15/f27 [0,4194304] 0 2024-02-16T22:56:11.866 INFO:tasks.workunit.client.1.smithi192.stdout:5/234: truncate f2 4615239 0 2024-02-16T22:56:11.868 INFO:tasks.workunit.client.1.smithi192.stdout:8/182: rename d5/db/de/df/c2c to d5/d16/c32 0 2024-02-16T22:56:11.869 INFO:tasks.workunit.client.1.smithi192.stdout:8/183: chown d5/db/dd/l2a 14683 1 2024-02-16T22:56:11.873 INFO:tasks.workunit.client.1.smithi192.stdout:7/263: mknod d1/d2/de/d2c/d49/c5a 0 2024-02-16T22:56:11.875 INFO:tasks.workunit.client.1.smithi192.stdout:7/264: write d1/f11 [8994724,8097] 0 2024-02-16T22:56:11.876 INFO:tasks.workunit.client.0.smithi064.stdout:3/167: dwrite d4/fd [0,4194304] 0 2024-02-16T22:56:11.876 INFO:tasks.workunit.client.0.smithi064.stdout:5/107: mknod d6/d25/c28 0 2024-02-16T22:56:11.877 INFO:tasks.workunit.client.1.smithi192.stdout:1/183: dwrite d3/f22 [0,4194304] 0 2024-02-16T22:56:11.877 INFO:tasks.workunit.client.1.smithi192.stdout:9/288: creat d7/d15/f5d x:0 0 0 2024-02-16T22:56:11.878 INFO:tasks.workunit.client.1.smithi192.stdout:5/235: symlink d6/d12/d2d/l3f 0 2024-02-16T22:56:11.879 INFO:tasks.workunit.client.1.smithi192.stdout:8/184: dwrite d5/d16/f27 [0,4194304] 0 2024-02-16T22:56:11.881 INFO:tasks.workunit.client.0.smithi064.stdout:1/105: link d2/df/d11/d1a/d1d/f24 d2/df/d11/f27 0 2024-02-16T22:56:11.881 INFO:tasks.workunit.client.0.smithi064.stdout:1/106: chown l1 4583 1 2024-02-16T22:56:11.882 INFO:tasks.workunit.client.0.smithi064.stdout:6/147: creat d0/d6/da/f25 x:0 0 0 2024-02-16T22:56:11.884 INFO:tasks.workunit.client.1.smithi192.stdout:5/236: dread d6/d12/f18 [4194304,4194304] 0 2024-02-16T22:56:11.885 INFO:tasks.workunit.client.0.smithi064.stdout:8/113: mknod df/c20 0 2024-02-16T22:56:11.886 INFO:tasks.workunit.client.1.smithi192.stdout:8/185: dread d5/d16/f27 [0,4194304] 0 2024-02-16T22:56:11.886 INFO:tasks.workunit.client.0.smithi064.stdout:8/114: readlink df/l10 0 2024-02-16T22:56:11.887 INFO:tasks.workunit.client.1.smithi192.stdout:8/186: chown d5/db/de/df/f2b 10399 1 2024-02-16T22:56:11.891 INFO:tasks.workunit.client.1.smithi192.stdout:7/265: read d1/d2/f1b [2594099,28767] 0 2024-02-16T22:56:11.894 INFO:tasks.workunit.client.1.smithi192.stdout:9/289: mkdir d7/d19/d5e 0 2024-02-16T22:56:11.895 INFO:tasks.workunit.client.1.smithi192.stdout:9/290: read d7/d15/f33 [232459,48267] 0 2024-02-16T22:56:11.896 INFO:tasks.workunit.client.0.smithi064.stdout:7/86: dread d5/d6/fb [0,4194304] 0 2024-02-16T22:56:11.897 INFO:tasks.workunit.client.0.smithi064.stdout:7/87: chown d5/d6/fa 649 1 2024-02-16T22:56:11.898 INFO:tasks.workunit.client.0.smithi064.stdout:1/107: creat d2/df/d14/d18/f28 x:0 0 0 2024-02-16T22:56:11.900 INFO:tasks.workunit.client.0.smithi064.stdout:6/148: creat d0/d6/f26 x:0 0 0 2024-02-16T22:56:11.902 INFO:tasks.workunit.client.0.smithi064.stdout:8/115: symlink df/l21 0 2024-02-16T22:56:11.902 INFO:tasks.workunit.client.0.smithi064.stdout:8/116: write df/f1b [446255,17678] 0 2024-02-16T22:56:11.903 INFO:tasks.workunit.client.0.smithi064.stdout:8/117: chown df/c20 4721 1 2024-02-16T22:56:11.904 INFO:tasks.workunit.client.0.smithi064.stdout:9/135: mknod d3/d11/d2b/d2a/d2e/d31/c34 0 2024-02-16T22:56:11.906 INFO:tasks.workunit.client.0.smithi064.stdout:8/118: dread fb [0,4194304] 0 2024-02-16T22:56:11.909 INFO:tasks.workunit.client.1.smithi192.stdout:0/180: sync 2024-02-16T22:56:11.910 INFO:tasks.workunit.client.1.smithi192.stdout:0/181: chown d1/d3/d29/f1a 403463 1 2024-02-16T22:56:11.911 INFO:tasks.workunit.client.0.smithi064.stdout:9/136: dwrite d3/d11/d2b/f26 [0,4194304] 0 2024-02-16T22:56:11.912 INFO:tasks.workunit.client.0.smithi064.stdout:7/88: fsync d5/d6/f9 0 2024-02-16T22:56:11.912 INFO:tasks.workunit.client.0.smithi064.stdout:9/137: dread d3/f10 [0,4194304] 0 2024-02-16T22:56:11.913 INFO:tasks.workunit.client.0.smithi064.stdout:9/138: readlink d3/l13 0 2024-02-16T22:56:11.913 INFO:tasks.workunit.client.0.smithi064.stdout:9/139: fsync d3/f18 0 2024-02-16T22:56:11.915 INFO:tasks.workunit.client.0.smithi064.stdout:9/140: write f1 [5714260,45462] 0 2024-02-16T22:56:11.915 INFO:tasks.workunit.client.0.smithi064.stdout:7/89: dread d5/d6/fb [0,4194304] 0 2024-02-16T22:56:11.916 INFO:tasks.workunit.client.0.smithi064.stdout:9/141: read d3/d11/d2b/f27 [2344373,50562] 0 2024-02-16T22:56:11.917 INFO:tasks.workunit.client.0.smithi064.stdout:1/108: creat d2/df/d11/f29 x:0 0 0 2024-02-16T22:56:11.920 INFO:tasks.workunit.client.1.smithi192.stdout:3/209: rename da/f27 to da/d11/f4b 0 2024-02-16T22:56:11.920 INFO:tasks.workunit.client.0.smithi064.stdout:7/90: dread f0 [0,4194304] 0 2024-02-16T22:56:11.922 INFO:tasks.workunit.client.0.smithi064.stdout:6/149: creat d0/d5/f27 x:0 0 0 2024-02-16T22:56:11.923 INFO:tasks.workunit.client.1.smithi192.stdout:9/291: symlink d7/d15/l5f 0 2024-02-16T22:56:11.924 INFO:tasks.workunit.client.0.smithi064.stdout:9/142: dwrite d3/d11/f28 [0,4194304] 0 2024-02-16T22:56:11.925 INFO:tasks.workunit.client.1.smithi192.stdout:1/184: write d3/f16 [5012138,81461] 0 2024-02-16T22:56:11.926 INFO:tasks.workunit.client.1.smithi192.stdout:1/185: stat d3/f22 0 2024-02-16T22:56:11.930 INFO:tasks.workunit.client.0.smithi064.stdout:8/119: mkdir df/d22 0 2024-02-16T22:56:11.932 INFO:tasks.workunit.client.1.smithi192.stdout:2/241: getdents d22/d24/d2b/d2c 0 2024-02-16T22:56:11.932 INFO:tasks.workunit.client.0.smithi064.stdout:5/108: creat d6/f29 x:0 0 0 2024-02-16T22:56:11.934 INFO:tasks.workunit.client.1.smithi192.stdout:5/237: creat d6/d12/d1d/d3c/f40 x:0 0 0 2024-02-16T22:56:11.935 INFO:tasks.workunit.client.0.smithi064.stdout:2/108: link d7/cd d7/de/c18 0 2024-02-16T22:56:11.935 INFO:tasks.workunit.client.1.smithi192.stdout:5/238: truncate d6/d12/d1d/f39 938027 0 2024-02-16T22:56:11.937 INFO:tasks.workunit.client.1.smithi192.stdout:4/215: write d1/d15/d29/d3c/f43 [3035235,123329] 0 2024-02-16T22:56:11.941 INFO:tasks.workunit.client.0.smithi064.stdout:5/109: dwrite f0 [0,4194304] 0 2024-02-16T22:56:11.942 INFO:tasks.workunit.client.0.smithi064.stdout:5/110: truncate d6/f1b 4695950 0 2024-02-16T22:56:11.951 INFO:tasks.workunit.client.1.smithi192.stdout:7/266: rename d1/d2/de/d2c/d49/c5a to d1/d2/d2d/c5b 0 2024-02-16T22:56:11.954 INFO:tasks.workunit.client.0.smithi064.stdout:7/91: symlink d5/d6/l16 0 2024-02-16T22:56:11.955 INFO:tasks.workunit.client.1.smithi192.stdout:3/210: dwrite da/df/f49 [0,4194304] 0 2024-02-16T22:56:11.955 INFO:tasks.workunit.client.1.smithi192.stdout:2/242: chown c11 33964 1 2024-02-16T22:56:11.956 INFO:tasks.workunit.client.0.smithi064.stdout:6/150: symlink d0/d6/da/l28 0 2024-02-16T22:56:11.958 INFO:tasks.workunit.client.1.smithi192.stdout:1/186: dread d3/f19 [0,4194304] 0 2024-02-16T22:56:11.959 INFO:tasks.workunit.client.1.smithi192.stdout:1/187: chown d3/f21 10 1 2024-02-16T22:56:11.960 INFO:tasks.workunit.client.1.smithi192.stdout:1/188: chown d3/f22 2 1 2024-02-16T22:56:11.960 INFO:tasks.workunit.client.0.smithi064.stdout:9/143: symlink d3/d11/d15/d23/l35 0 2024-02-16T22:56:11.961 INFO:tasks.workunit.client.1.smithi192.stdout:3/211: dread da/d11/f4b [0,4194304] 0 2024-02-16T22:56:11.962 INFO:tasks.workunit.client.0.smithi064.stdout:9/144: read d3/d11/f1f [285710,34585] 0 2024-02-16T22:56:11.963 INFO:tasks.workunit.client.1.smithi192.stdout:5/239: dwrite d6/d12/d1d/f36 [0,4194304] 0 2024-02-16T22:56:11.967 INFO:tasks.workunit.client.0.smithi064.stdout:8/120: dread fe [0,4194304] 0 2024-02-16T22:56:11.968 INFO:tasks.workunit.client.0.smithi064.stdout:8/121: truncate fa 153589 0 2024-02-16T22:56:11.970 INFO:tasks.workunit.client.1.smithi192.stdout:6/229: sync 2024-02-16T22:56:11.970 INFO:tasks.workunit.client.1.smithi192.stdout:6/230: stat da/df/d12/f13 0 2024-02-16T22:56:11.975 INFO:tasks.workunit.client.0.smithi064.stdout:5/111: rmdir d6 39 2024-02-16T22:56:11.977 INFO:tasks.workunit.client.1.smithi192.stdout:4/216: mkdir d1/d15/d48/d4c 0 2024-02-16T22:56:11.979 INFO:tasks.workunit.client.1.smithi192.stdout:1/189: creat d3/de/d27/d34/f38 x:0 0 0 2024-02-16T22:56:11.980 INFO:tasks.workunit.client.1.smithi192.stdout:8/187: getdents d5/db/de/d21 0 2024-02-16T22:56:11.982 INFO:tasks.workunit.client.0.smithi064.stdout:6/151: unlink d0/d6/da/l28 0 2024-02-16T22:56:11.984 INFO:tasks.workunit.client.1.smithi192.stdout:5/240: symlink d6/d12/l41 0 2024-02-16T22:56:11.987 INFO:tasks.workunit.client.0.smithi064.stdout:9/145: creat d3/d9/f36 x:0 0 0 2024-02-16T22:56:11.989 INFO:tasks.workunit.client.1.smithi192.stdout:7/267: rename d1/d2/l57 to d1/d19/l5c 0 2024-02-16T22:56:11.991 INFO:tasks.workunit.client.0.smithi064.stdout:9/146: dread d3/d11/f28 [0,4194304] 0 2024-02-16T22:56:11.994 INFO:tasks.workunit.client.0.smithi064.stdout:8/122: rename f2 to df/f23 0 2024-02-16T22:56:11.996 INFO:tasks.workunit.client.1.smithi192.stdout:2/243: dwrite f21 [0,4194304] 0 2024-02-16T22:56:11.998 INFO:tasks.workunit.client.1.smithi192.stdout:8/188: creat d5/d9/f33 x:0 0 0 2024-02-16T22:56:12.001 INFO:tasks.workunit.client.0.smithi064.stdout:6/152: mknod d0/d5/c29 0 2024-02-16T22:56:12.003 INFO:tasks.workunit.client.1.smithi192.stdout:5/241: creat d6/d12/d2d/f42 x:0 0 0 2024-02-16T22:56:12.005 INFO:tasks.workunit.client.1.smithi192.stdout:0/182: getdents d1 0 2024-02-16T22:56:12.008 INFO:tasks.workunit.client.1.smithi192.stdout:0/183: dread d1/d3/de/f10 [0,4194304] 0 2024-02-16T22:56:12.009 INFO:tasks.workunit.client.0.smithi064.stdout:9/147: symlink d3/d9/d1e/l37 0 2024-02-16T22:56:12.010 INFO:tasks.workunit.client.1.smithi192.stdout:3/212: truncate da/df/f49 3857682 0 2024-02-16T22:56:12.011 INFO:tasks.workunit.client.1.smithi192.stdout:9/292: link d7/d15/d36/d38/d3f/c46 d7/d19/d37/d5c/c60 0 2024-02-16T22:56:12.011 INFO:tasks.workunit.client.1.smithi192.stdout:9/293: chown d7/d19 49 1 2024-02-16T22:56:12.011 INFO:tasks.workunit.client.1.smithi192.stdout:9/294: write d7/d15/f20 [2920975,7317] 0 2024-02-16T22:56:12.012 INFO:tasks.workunit.client.1.smithi192.stdout:9/295: chown d7/d19/d28/f52 31158 1 2024-02-16T22:56:12.013 INFO:tasks.workunit.client.0.smithi064.stdout:2/109: truncate d7/de/f11 3387805 0 2024-02-16T22:56:12.017 INFO:tasks.workunit.client.0.smithi064.stdout:8/123: fsync f8 0 2024-02-16T22:56:12.017 INFO:tasks.workunit.client.0.smithi064.stdout:2/110: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:12.018 INFO:tasks.workunit.client.0.smithi064.stdout:2/111: readlink l3 0 2024-02-16T22:56:12.019 INFO:tasks.workunit.client.0.smithi064.stdout:5/112: rmdir d6/d18 39 2024-02-16T22:56:12.021 INFO:tasks.workunit.client.1.smithi192.stdout:1/190: link d3/de/d27/f30 d3/de/d27/f39 0 2024-02-16T22:56:12.021 INFO:tasks.workunit.client.1.smithi192.stdout:4/217: truncate d1/f19 1942689 0 2024-02-16T22:56:12.022 INFO:tasks.workunit.client.0.smithi064.stdout:1/109: getdents d2/df/d11 0 2024-02-16T22:56:12.022 INFO:tasks.workunit.client.1.smithi192.stdout:4/218: write d1/d26/f36 [2903789,116184] 0 2024-02-16T22:56:12.023 INFO:tasks.workunit.client.1.smithi192.stdout:4/219: readlink d1/d26/d34/l47 0 2024-02-16T22:56:12.023 INFO:tasks.workunit.client.1.smithi192.stdout:4/220: chown d1/d8/fe 4702250 1 2024-02-16T22:56:12.024 INFO:tasks.workunit.client.0.smithi064.stdout:7/92: link d5/d6/l16 d5/d6/l17 0 2024-02-16T22:56:12.024 INFO:tasks.workunit.client.0.smithi064.stdout:1/110: truncate d2/df/d14/f16 289463 0 2024-02-16T22:56:12.027 INFO:tasks.workunit.client.1.smithi192.stdout:5/242: creat d6/d12/d2d/f43 x:0 0 0 2024-02-16T22:56:12.028 INFO:tasks.workunit.client.0.smithi064.stdout:7/93: dread d5/d6/fa [0,4194304] 0 2024-02-16T22:56:12.032 INFO:tasks.workunit.client.1.smithi192.stdout:7/268: dread d1/d2/f25 [0,4194304] 0 2024-02-16T22:56:12.036 INFO:tasks.workunit.client.1.smithi192.stdout:8/189: dwrite d5/d16/f18 [0,4194304] 0 2024-02-16T22:56:12.036 INFO:tasks.workunit.client.0.smithi064.stdout:0/104: sync 2024-02-16T22:56:12.039 INFO:tasks.workunit.client.0.smithi064.stdout:0/105: dread - da/dc/de/f17 zero size 2024-02-16T22:56:12.039 INFO:tasks.workunit.client.0.smithi064.stdout:0/106: chown da/dc/de/d12 993761785 1 2024-02-16T22:56:12.039 INFO:tasks.workunit.client.0.smithi064.stdout:8/124: creat df/f24 x:0 0 0 2024-02-16T22:56:12.040 INFO:tasks.workunit.client.0.smithi064.stdout:8/125: fdatasync f9 0 2024-02-16T22:56:12.040 INFO:tasks.workunit.client.0.smithi064.stdout:8/126: chown df/c17 2848 1 2024-02-16T22:56:12.040 INFO:tasks.workunit.client.0.smithi064.stdout:0/107: write da/dc/de/f16 [673192,42807] 0 2024-02-16T22:56:12.041 INFO:tasks.workunit.client.0.smithi064.stdout:2/112: unlink l6 0 2024-02-16T22:56:12.046 INFO:tasks.workunit.client.1.smithi192.stdout:4/221: mkdir d1/d15/d29/d3c/d4d 0 2024-02-16T22:56:12.047 INFO:tasks.workunit.client.0.smithi064.stdout:1/111: mkdir d2/df/d14/d2a 0 2024-02-16T22:56:12.047 INFO:tasks.workunit.client.0.smithi064.stdout:6/153: getdents d0/d5/df/d17/d18 0 2024-02-16T22:56:12.048 INFO:tasks.workunit.client.0.smithi064.stdout:7/94: creat d5/d6/f18 x:0 0 0 2024-02-16T22:56:12.049 INFO:tasks.workunit.client.1.smithi192.stdout:5/243: creat d6/d12/f44 x:0 0 0 2024-02-16T22:56:12.051 INFO:tasks.workunit.client.0.smithi064.stdout:9/148: mkdir d3/d11/d2b/d38 0 2024-02-16T22:56:12.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/269: mknod d1/d19/c5d 0 2024-02-16T22:56:12.052 INFO:tasks.workunit.client.0.smithi064.stdout:9/149: chown d3/d11/d2b/f27 770863322 1 2024-02-16T22:56:12.053 INFO:tasks.workunit.client.0.smithi064.stdout:1/112: dwrite d2/f3 [4194304,4194304] 0 2024-02-16T22:56:12.055 INFO:tasks.workunit.client.1.smithi192.stdout:0/184: rename d1/d3/f2f to d1/f35 0 2024-02-16T22:56:12.058 INFO:tasks.workunit.client.0.smithi064.stdout:0/108: rmdir da/dc 39 2024-02-16T22:56:12.063 INFO:tasks.workunit.client.0.smithi064.stdout:5/113: symlink d6/l2a 0 2024-02-16T22:56:12.068 INFO:tasks.workunit.client.1.smithi192.stdout:5/244: symlink d6/d12/d15/l45 0 2024-02-16T22:56:12.069 INFO:tasks.workunit.client.0.smithi064.stdout:9/150: creat d3/d11/d2b/d2a/d2e/f39 x:0 0 0 2024-02-16T22:56:12.074 INFO:tasks.workunit.client.1.smithi192.stdout:3/213: creat da/f4c x:0 0 0 2024-02-16T22:56:12.074 INFO:tasks.workunit.client.1.smithi192.stdout:3/214: readlink da/d11/d2b/l31 0 2024-02-16T22:56:12.076 INFO:tasks.workunit.client.0.smithi064.stdout:2/113: link d7/f8 d7/f19 0 2024-02-16T22:56:12.077 INFO:tasks.workunit.client.1.smithi192.stdout:8/190: truncate d5/db/de/df/f1b 2995378 0 2024-02-16T22:56:12.079 INFO:tasks.workunit.client.1.smithi192.stdout:1/191: link d3/c2a d3/de/d27/c3a 0 2024-02-16T22:56:12.080 INFO:tasks.workunit.client.1.smithi192.stdout:1/192: chown d3/de/c1d 76171 1 2024-02-16T22:56:12.081 INFO:tasks.workunit.client.0.smithi064.stdout:5/114: symlink d6/d1c/l2b 0 2024-02-16T22:56:12.081 INFO:tasks.workunit.client.1.smithi192.stdout:9/296: truncate d7/f8 80516 0 2024-02-16T22:56:12.082 INFO:tasks.workunit.client.1.smithi192.stdout:2/244: getdents d22 0 2024-02-16T22:56:12.083 INFO:tasks.workunit.client.0.smithi064.stdout:6/154: chown d0/d6/d19/l20 12161544 1 2024-02-16T22:56:12.084 INFO:tasks.workunit.client.0.smithi064.stdout:6/155: dread - d0/d5/f27 zero size 2024-02-16T22:56:12.085 INFO:tasks.workunit.client.0.smithi064.stdout:6/156: stat d0/d12 0 2024-02-16T22:56:12.085 INFO:tasks.workunit.client.0.smithi064.stdout:6/157: write d0/d1b/f23 [374585,126628] 0 2024-02-16T22:56:12.087 INFO:tasks.workunit.client.0.smithi064.stdout:7/95: write d5/d6/fb [2193419,126653] 0 2024-02-16T22:56:12.090 INFO:tasks.workunit.client.0.smithi064.stdout:1/113: write d2/df/d11/d1a/d1d/f24 [568889,53063] 0 2024-02-16T22:56:12.091 INFO:tasks.workunit.client.1.smithi192.stdout:5/245: dread d6/fc [0,4194304] 0 2024-02-16T22:56:12.092 INFO:tasks.workunit.client.0.smithi064.stdout:8/127: link l0 df/d22/l25 0 2024-02-16T22:56:12.093 INFO:tasks.workunit.client.1.smithi192.stdout:3/215: creat da/df/d42/f4d x:0 0 0 2024-02-16T22:56:12.094 INFO:tasks.workunit.client.0.smithi064.stdout:2/114: fdatasync d7/f19 0 2024-02-16T22:56:12.095 INFO:tasks.workunit.client.1.smithi192.stdout:5/246: dread d6/d12/d15/f1c [0,4194304] 0 2024-02-16T22:56:12.096 INFO:tasks.workunit.client.1.smithi192.stdout:8/191: mkdir d5/d16/d34 0 2024-02-16T22:56:12.097 INFO:tasks.workunit.client.0.smithi064.stdout:7/96: dwrite d5/d6/f18 [0,4194304] 0 2024-02-16T22:56:12.099 INFO:tasks.workunit.client.0.smithi064.stdout:9/151: dread d3/d9/f12 [0,4194304] 0 2024-02-16T22:56:12.102 INFO:tasks.workunit.client.0.smithi064.stdout:9/152: write d3/d9/f36 [1014264,75542] 0 2024-02-16T22:56:12.102 INFO:tasks.workunit.client.0.smithi064.stdout:4/116: sync 2024-02-16T22:56:12.102 INFO:tasks.workunit.client.0.smithi064.stdout:3/168: sync 2024-02-16T22:56:12.104 INFO:tasks.workunit.client.0.smithi064.stdout:0/109: truncate f9 1864997 0 2024-02-16T22:56:12.105 INFO:tasks.workunit.client.1.smithi192.stdout:4/222: truncate d1/d8/f9 4785489 0 2024-02-16T22:56:12.106 INFO:tasks.workunit.client.0.smithi064.stdout:1/114: dwrite d2/df/d14/d18/f26 [0,4194304] 0 2024-02-16T22:56:12.106 INFO:tasks.workunit.client.1.smithi192.stdout:4/223: write d1/d26/f28 [1748689,33984] 0 2024-02-16T22:56:12.107 INFO:tasks.workunit.client.1.smithi192.stdout:1/193: fdatasync d3/de/d27/f30 0 2024-02-16T22:56:12.107 INFO:tasks.workunit.client.0.smithi064.stdout:7/97: dread f0 [0,4194304] 0 2024-02-16T22:56:12.110 INFO:tasks.workunit.client.1.smithi192.stdout:9/297: unlink d7/d19/d37/d39/l56 0 2024-02-16T22:56:12.115 INFO:tasks.workunit.client.0.smithi064.stdout:8/128: rename df/c1c to df/d22/c26 0 2024-02-16T22:56:12.118 INFO:tasks.workunit.client.1.smithi192.stdout:3/216: mkdir da/df/d42/d4e 0 2024-02-16T22:56:12.120 INFO:tasks.workunit.client.0.smithi064.stdout:2/115: creat d7/f1a x:0 0 0 2024-02-16T22:56:12.121 INFO:tasks.workunit.client.1.smithi192.stdout:5/247: mkdir d6/d12/d15/d46 0 2024-02-16T22:56:12.124 INFO:tasks.workunit.client.1.smithi192.stdout:8/192: symlink d5/d16/l35 0 2024-02-16T22:56:12.126 INFO:tasks.workunit.client.0.smithi064.stdout:9/153: rmdir d3 39 2024-02-16T22:56:12.128 INFO:tasks.workunit.client.0.smithi064.stdout:3/169: mknod d4/c37 0 2024-02-16T22:56:12.129 INFO:tasks.workunit.client.1.smithi192.stdout:8/193: dwrite d5/db/de/df/f2d [0,4194304] 0 2024-02-16T22:56:12.134 INFO:tasks.workunit.client.0.smithi064.stdout:1/115: symlink d2/df/d11/d1a/d1d/l2b 0 2024-02-16T22:56:12.136 INFO:tasks.workunit.client.1.smithi192.stdout:2/245: dread f19 [0,4194304] 0 2024-02-16T22:56:12.141 INFO:tasks.workunit.client.1.smithi192.stdout:9/298: creat d7/d19/d28/f61 x:0 0 0 2024-02-16T22:56:12.142 INFO:tasks.workunit.client.0.smithi064.stdout:6/158: getdents d0/d1d/d21 0 2024-02-16T22:56:12.145 INFO:tasks.workunit.client.0.smithi064.stdout:6/159: dread d0/d1b/f23 [0,4194304] 0 2024-02-16T22:56:12.145 INFO:tasks.workunit.client.0.smithi064.stdout:5/115: dwrite d6/d18/d19/f26 [0,4194304] 0 2024-02-16T22:56:12.146 INFO:tasks.workunit.client.1.smithi192.stdout:7/270: rename d1/d2/f3d to d1/f5e 0 2024-02-16T22:56:12.147 INFO:tasks.workunit.client.1.smithi192.stdout:7/271: write d1/f11 [1572673,130278] 0 2024-02-16T22:56:12.147 INFO:tasks.workunit.client.0.smithi064.stdout:6/160: write d0/f22 [804140,113934] 0 2024-02-16T22:56:12.147 INFO:tasks.workunit.client.0.smithi064.stdout:5/116: chown d6/fa 160 1 2024-02-16T22:56:12.148 INFO:tasks.workunit.client.0.smithi064.stdout:5/117: chown f4 486 1 2024-02-16T22:56:12.148 INFO:tasks.workunit.client.1.smithi192.stdout:7/272: write d1/d19/f1d [2626230,12165] 0 2024-02-16T22:56:12.151 INFO:tasks.workunit.client.0.smithi064.stdout:4/117: rename l8 to dc/dd/d16/l20 0 2024-02-16T22:56:12.152 INFO:tasks.workunit.client.0.smithi064.stdout:3/170: rename d4/d1f to d4/d1f/d38 22 2024-02-16T22:56:12.153 INFO:tasks.workunit.client.1.smithi192.stdout:3/217: mknod da/d11/d35/c4f 0 2024-02-16T22:56:12.155 INFO:tasks.workunit.client.0.smithi064.stdout:2/116: symlink d7/d14/l1b 0 2024-02-16T22:56:12.157 INFO:tasks.workunit.client.0.smithi064.stdout:7/98: dwrite d5/f10 [0,4194304] 0 2024-02-16T22:56:12.158 INFO:tasks.workunit.client.1.smithi192.stdout:8/194: dread d5/db/de/d21/f2f [0,4194304] 0 2024-02-16T22:56:12.159 INFO:tasks.workunit.client.0.smithi064.stdout:3/171: dread d4/d13/f16 [0,4194304] 0 2024-02-16T22:56:12.159 INFO:tasks.workunit.client.0.smithi064.stdout:8/129: write f8 [1599181,4252] 0 2024-02-16T22:56:12.162 INFO:tasks.workunit.client.0.smithi064.stdout:0/110: mknod da/dc/c1c 0 2024-02-16T22:56:12.162 INFO:tasks.workunit.client.0.smithi064.stdout:0/111: stat c1 0 2024-02-16T22:56:12.163 INFO:tasks.workunit.client.0.smithi064.stdout:1/116: mkdir d2/df/d14/d2c 0 2024-02-16T22:56:12.164 INFO:tasks.workunit.client.0.smithi064.stdout:7/99: dread d5/f10 [0,4194304] 0 2024-02-16T22:56:12.164 INFO:tasks.workunit.client.0.smithi064.stdout:0/112: write da/dc/de/f16 [443291,130075] 0 2024-02-16T22:56:12.165 INFO:tasks.workunit.client.0.smithi064.stdout:7/100: fdatasync d5/d6/fb 0 2024-02-16T22:56:12.169 INFO:tasks.workunit.client.0.smithi064.stdout:1/117: dread d2/df/d11/d1a/d1d/f24 [0,4194304] 0 2024-02-16T22:56:12.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/194: link d3/de/d27/f30 d3/de/d27/f3b 0 2024-02-16T22:56:12.171 INFO:tasks.workunit.client.0.smithi064.stdout:5/118: mknod d6/d18/c2c 0 2024-02-16T22:56:12.172 INFO:tasks.workunit.client.0.smithi064.stdout:5/119: fdatasync d6/d18/d19/f26 0 2024-02-16T22:56:12.175 INFO:tasks.workunit.client.0.smithi064.stdout:4/118: mknod dc/d10/d18/c21 0 2024-02-16T22:56:12.175 INFO:tasks.workunit.client.0.smithi064.stdout:1/118: dwrite d2/df/d14/d18/f26 [0,4194304] 0 2024-02-16T22:56:12.176 INFO:tasks.workunit.client.0.smithi064.stdout:4/119: write dc/f17 [274827,113496] 0 2024-02-16T22:56:12.179 INFO:tasks.workunit.client.1.smithi192.stdout:0/185: rename d1/f35 to d1/f36 0 2024-02-16T22:56:12.183 INFO:tasks.workunit.client.1.smithi192.stdout:8/195: write d5/d16/f27 [1394353,94180] 0 2024-02-16T22:56:12.184 INFO:tasks.workunit.client.1.smithi192.stdout:4/224: rmdir d1/d15/d29/d3c/d4d 0 2024-02-16T22:56:12.185 INFO:tasks.workunit.client.0.smithi064.stdout:8/130: rmdir df 39 2024-02-16T22:56:12.188 INFO:tasks.workunit.client.0.smithi064.stdout:3/172: rename d4/d14/d1d to d4/d14/d39 0 2024-02-16T22:56:12.189 INFO:tasks.workunit.client.1.smithi192.stdout:1/195: mknod d3/de/c3c 0 2024-02-16T22:56:12.190 INFO:tasks.workunit.client.0.smithi064.stdout:2/117: write d7/f8 [2226415,65529] 0 2024-02-16T22:56:12.190 INFO:tasks.workunit.client.1.smithi192.stdout:1/196: write d3/f2d [869823,56787] 0 2024-02-16T22:56:12.208 INFO:tasks.workunit.client.0.smithi064.stdout:0/113: mkdir da/dc/de/d12/d1d 0 2024-02-16T22:56:12.209 INFO:tasks.workunit.client.1.smithi192.stdout:9/299: truncate d7/d19/d28/f2c 1482830 0 2024-02-16T22:56:12.210 INFO:tasks.workunit.client.0.smithi064.stdout:7/101: write f0 [1398550,45383] 0 2024-02-16T22:56:12.210 INFO:tasks.workunit.client.0.smithi064.stdout:7/102: readlink d5/l14 0 2024-02-16T22:56:12.211 INFO:tasks.workunit.client.0.smithi064.stdout:7/103: write d5/d6/f18 [2186003,42012] 0 2024-02-16T22:56:12.213 INFO:tasks.workunit.client.1.smithi192.stdout:8/196: rename c4 to d5/db/de/df/c36 0 2024-02-16T22:56:12.214 INFO:tasks.workunit.client.1.smithi192.stdout:8/197: truncate d5/f25 959870 0 2024-02-16T22:56:12.215 INFO:tasks.workunit.client.0.smithi064.stdout:7/104: dread f0 [0,4194304] 0 2024-02-16T22:56:12.215 INFO:tasks.workunit.client.1.smithi192.stdout:4/225: creat d1/d8/d11/f4e x:0 0 0 2024-02-16T22:56:12.216 INFO:tasks.workunit.client.1.smithi192.stdout:4/226: write d1/d8/fc [2224862,94323] 0 2024-02-16T22:56:12.219 INFO:tasks.workunit.client.1.smithi192.stdout:6/231: sync 2024-02-16T22:56:12.222 INFO:tasks.workunit.client.1.smithi192.stdout:7/273: symlink d1/d2/d2d/d55/l5f 0 2024-02-16T22:56:12.223 INFO:tasks.workunit.client.1.smithi192.stdout:3/218: dwrite da/d11/d20/f1d [0,4194304] 0 2024-02-16T22:56:12.224 INFO:tasks.workunit.client.1.smithi192.stdout:9/300: creat d7/df/f62 x:0 0 0 2024-02-16T22:56:12.225 INFO:tasks.workunit.client.0.smithi064.stdout:3/173: symlink d4/d13/d1b/d27/l3a 0 2024-02-16T22:56:12.226 INFO:tasks.workunit.client.1.smithi192.stdout:6/232: dwrite da/df/d12/d15/f3e [0,4194304] 0 2024-02-16T22:56:12.229 INFO:tasks.workunit.client.0.smithi064.stdout:3/174: dread d4/f5 [0,4194304] 0 2024-02-16T22:56:12.231 INFO:tasks.workunit.client.0.smithi064.stdout:3/175: chown d4/d13/d35 0 1 2024-02-16T22:56:12.232 INFO:tasks.workunit.client.1.smithi192.stdout:4/227: creat d1/d15/f4f x:0 0 0 2024-02-16T22:56:12.232 INFO:tasks.workunit.client.0.smithi064.stdout:2/118: dread d7/de/f11 [0,4194304] 0 2024-02-16T22:56:12.233 INFO:tasks.workunit.client.0.smithi064.stdout:2/119: fdatasync d7/de/f15 0 2024-02-16T22:56:12.233 INFO:tasks.workunit.client.1.smithi192.stdout:2/246: link c1a d22/d24/c50 0 2024-02-16T22:56:12.240 INFO:tasks.workunit.client.0.smithi064.stdout:7/105: creat d5/d6/f19 x:0 0 0 2024-02-16T22:56:12.240 INFO:tasks.workunit.client.0.smithi064.stdout:2/120: dwrite d7/f13 [0,4194304] 0 2024-02-16T22:56:12.243 INFO:tasks.workunit.client.0.smithi064.stdout:2/121: chown d7/d14/l1b 3323133 1 2024-02-16T22:56:12.244 INFO:tasks.workunit.client.0.smithi064.stdout:6/161: rename d0/d6/f26 to d0/d6/d19/f2a 0 2024-02-16T22:56:12.248 INFO:tasks.workunit.client.0.smithi064.stdout:3/176: unlink d4/la 0 2024-02-16T22:56:12.249 INFO:tasks.workunit.client.0.smithi064.stdout:3/177: chown d4/d14/d2b 735982 1 2024-02-16T22:56:12.252 INFO:tasks.workunit.client.0.smithi064.stdout:0/114: dwrite da/f10 [4194304,4194304] 0 2024-02-16T22:56:12.254 INFO:tasks.workunit.client.0.smithi064.stdout:0/115: chown da/dc/c1c 45368593 1 2024-02-16T22:56:12.258 INFO:tasks.workunit.client.0.smithi064.stdout:6/162: creat d0/d5/f2b x:0 0 0 2024-02-16T22:56:12.263 INFO:tasks.workunit.client.0.smithi064.stdout:0/116: chown c7 78900560 1 2024-02-16T22:56:12.265 INFO:tasks.workunit.client.0.smithi064.stdout:9/154: rename d3/c4 to d3/d11/d15/d23/c3a 0 2024-02-16T22:56:12.265 INFO:tasks.workunit.client.0.smithi064.stdout:0/117: truncate f6 887231 0 2024-02-16T22:56:12.265 INFO:tasks.workunit.client.1.smithi192.stdout:1/197: symlink d3/l3d 0 2024-02-16T22:56:12.265 INFO:tasks.workunit.client.1.smithi192.stdout:0/186: mkdir d1/d3/de/d22/d37 0 2024-02-16T22:56:12.266 INFO:tasks.workunit.client.1.smithi192.stdout:2/247: chown f21 58 1 2024-02-16T22:56:12.266 INFO:tasks.workunit.client.1.smithi192.stdout:8/198: write d5/db/de/d21/f2f [3648407,69791] 0 2024-02-16T22:56:12.267 INFO:tasks.workunit.client.0.smithi064.stdout:6/163: symlink d0/d12/l2c 0 2024-02-16T22:56:12.268 INFO:tasks.workunit.client.1.smithi192.stdout:8/199: chown d5/l10 20 1 2024-02-16T22:56:12.269 INFO:tasks.workunit.client.1.smithi192.stdout:9/301: dread d7/d19/d28/d43/f4d [0,4194304] 0 2024-02-16T22:56:12.269 INFO:tasks.workunit.client.1.smithi192.stdout:8/200: write d5/d9/f17 [3224951,128761] 0 2024-02-16T22:56:12.269 INFO:tasks.workunit.client.1.smithi192.stdout:8/201: chown d5/d16/f18 39387349 1 2024-02-16T22:56:12.270 INFO:tasks.workunit.client.0.smithi064.stdout:6/164: read d0/f22 [878698,12887] 0 2024-02-16T22:56:12.270 INFO:tasks.workunit.client.0.smithi064.stdout:8/131: getdents df 0 2024-02-16T22:56:12.270 INFO:tasks.workunit.client.0.smithi064.stdout:3/178: dwrite d4/f5 [0,4194304] 0 2024-02-16T22:56:12.271 INFO:tasks.workunit.client.0.smithi064.stdout:7/106: dwrite d5/d6/f9 [4194304,4194304] 0 2024-02-16T22:56:12.273 INFO:tasks.workunit.client.0.smithi064.stdout:3/179: dread d4/f1e [0,4194304] 0 2024-02-16T22:56:12.273 INFO:tasks.workunit.client.0.smithi064.stdout:5/120: rename d6/l2a to d6/d1c/l2d 0 2024-02-16T22:56:12.273 INFO:tasks.workunit.client.0.smithi064.stdout:6/165: write d0/d6/f15 [5623270,115375] 0 2024-02-16T22:56:12.274 INFO:tasks.workunit.client.0.smithi064.stdout:0/118: dread f9 [0,4194304] 0 2024-02-16T22:56:12.274 INFO:tasks.workunit.client.0.smithi064.stdout:3/180: chown d4/f1e 6894 1 2024-02-16T22:56:12.275 INFO:tasks.workunit.client.0.smithi064.stdout:5/121: chown d6/c15 4 1 2024-02-16T22:56:12.275 INFO:tasks.workunit.client.1.smithi192.stdout:1/198: mkdir d3/de/d3e 0 2024-02-16T22:56:12.276 INFO:tasks.workunit.client.0.smithi064.stdout:9/155: mknod d3/d11/d2b/d2a/d2e/c3b 0 2024-02-16T22:56:12.276 INFO:tasks.workunit.client.1.smithi192.stdout:8/202: dwrite d5/db/de/df/f1b [0,4194304] 0 2024-02-16T22:56:12.281 INFO:tasks.workunit.client.1.smithi192.stdout:2/248: mknod d22/d24/d2b/d2c/c51 0 2024-02-16T22:56:12.283 INFO:tasks.workunit.client.0.smithi064.stdout:9/156: dwrite d3/fc [0,4194304] 0 2024-02-16T22:56:12.285 INFO:tasks.workunit.client.0.smithi064.stdout:7/107: rmdir d5/d6 39 2024-02-16T22:56:12.290 INFO:tasks.workunit.client.1.smithi192.stdout:0/187: write d1/f36 [1386106,75092] 0 2024-02-16T22:56:12.293 INFO:tasks.workunit.client.0.smithi064.stdout:1/119: rename d2/df/d11/f29 to d2/df/d14/f2d 0 2024-02-16T22:56:12.297 INFO:tasks.workunit.client.0.smithi064.stdout:3/181: unlink d4/f5 0 2024-02-16T22:56:12.297 INFO:tasks.workunit.client.0.smithi064.stdout:3/182: chown d4/cb 23084 1 2024-02-16T22:56:12.298 INFO:tasks.workunit.client.1.smithi192.stdout:3/219: creat da/d11/f50 x:0 0 0 2024-02-16T22:56:12.299 INFO:tasks.workunit.client.1.smithi192.stdout:4/228: write d1/f19 [1593303,60299] 0 2024-02-16T22:56:12.299 INFO:tasks.workunit.client.1.smithi192.stdout:1/199: symlink d3/de/d27/d34/l3f 0 2024-02-16T22:56:12.301 INFO:tasks.workunit.client.0.smithi064.stdout:5/122: creat d6/d18/d19/f2e x:0 0 0 2024-02-16T22:56:12.302 INFO:tasks.workunit.client.1.smithi192.stdout:1/200: read d3/de/f35 [200913,76118] 0 2024-02-16T22:56:12.302 INFO:tasks.workunit.client.0.smithi064.stdout:5/123: write d6/d18/d19/f26 [4767998,92748] 0 2024-02-16T22:56:12.302 INFO:tasks.workunit.client.1.smithi192.stdout:6/233: dwrite da/d27/f2f [0,4194304] 0 2024-02-16T22:56:12.303 INFO:tasks.workunit.client.0.smithi064.stdout:5/124: write d6/d18/d19/f1f [994555,26703] 0 2024-02-16T22:56:12.305 INFO:tasks.workunit.client.0.smithi064.stdout:5/125: write d6/f17 [337044,49824] 0 2024-02-16T22:56:12.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/249: mkdir d22/d24/d39/d42/d52 0 2024-02-16T22:56:12.306 INFO:tasks.workunit.client.1.smithi192.stdout:2/250: dread - d22/f41 zero size 2024-02-16T22:56:12.308 INFO:tasks.workunit.client.1.smithi192.stdout:0/188: mknod d1/d3/de/c38 0 2024-02-16T22:56:12.309 INFO:tasks.workunit.client.1.smithi192.stdout:0/189: write d1/d3/de/f21 [673176,85646] 0 2024-02-16T22:56:12.310 INFO:tasks.workunit.client.0.smithi064.stdout:2/122: rename l1 to d7/l1c 0 2024-02-16T22:56:12.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/201: dread d3/f2d [0,4194304] 0 2024-02-16T22:56:12.310 INFO:tasks.workunit.client.1.smithi192.stdout:5/248: sync 2024-02-16T22:56:12.312 INFO:tasks.workunit.client.1.smithi192.stdout:4/229: creat d1/d26/d34/f50 x:0 0 0 2024-02-16T22:56:12.313 INFO:tasks.workunit.client.0.smithi064.stdout:7/108: dwrite d5/d6/f18 [0,4194304] 0 2024-02-16T22:56:12.317 INFO:tasks.workunit.client.1.smithi192.stdout:6/234: creat da/df/d18/f43 x:0 0 0 2024-02-16T22:56:12.319 INFO:tasks.workunit.client.1.smithi192.stdout:2/251: symlink d22/d24/d39/l53 0 2024-02-16T22:56:12.321 INFO:tasks.workunit.client.0.smithi064.stdout:8/132: rename c4 to df/d22/c27 0 2024-02-16T22:56:12.325 INFO:tasks.workunit.client.0.smithi064.stdout:6/166: link d0/d5/f11 d0/d1d/d21/f2d 0 2024-02-16T22:56:12.326 INFO:tasks.workunit.client.1.smithi192.stdout:4/230: creat d1/d8/d11/f51 x:0 0 0 2024-02-16T22:56:12.327 INFO:tasks.workunit.client.1.smithi192.stdout:6/235: dwrite da/df/d18/f43 [0,4194304] 0 2024-02-16T22:56:12.329 INFO:tasks.workunit.client.1.smithi192.stdout:0/190: dwrite d1/fa [0,4194304] 0 2024-02-16T22:56:12.329 INFO:tasks.workunit.client.1.smithi192.stdout:4/231: truncate d1/d8/d11/f4e 351374 0 2024-02-16T22:56:12.329 INFO:tasks.workunit.client.0.smithi064.stdout:8/133: dwrite df/f24 [0,4194304] 0 2024-02-16T22:56:12.330 INFO:tasks.workunit.client.1.smithi192.stdout:0/191: write d1/fa [3562929,130169] 0 2024-02-16T22:56:12.331 INFO:tasks.workunit.client.0.smithi064.stdout:2/123: symlink d7/de/l1d 0 2024-02-16T22:56:12.332 INFO:tasks.workunit.client.1.smithi192.stdout:9/302: getdents d7/d19 0 2024-02-16T22:56:12.333 INFO:tasks.workunit.client.0.smithi064.stdout:2/124: write d7/f8 [3562442,91790] 0 2024-02-16T22:56:12.336 INFO:tasks.workunit.client.0.smithi064.stdout:0/119: rename da/dc/de/d12/f19 to da/dc/f1e 0 2024-02-16T22:56:12.336 INFO:tasks.workunit.client.0.smithi064.stdout:6/167: dwrite d0/d6/d10/f1c [0,4194304] 0 2024-02-16T22:56:12.338 INFO:tasks.workunit.client.1.smithi192.stdout:3/220: creat da/f51 x:0 0 0 2024-02-16T22:56:12.339 INFO:tasks.workunit.client.0.smithi064.stdout:9/157: write d3/d11/d2b/f27 [3633598,110620] 0 2024-02-16T22:56:12.339 INFO:tasks.workunit.client.0.smithi064.stdout:7/109: unlink d5/d6/cf 0 2024-02-16T22:56:12.340 INFO:tasks.workunit.client.1.smithi192.stdout:3/221: rename da to da/d11/d2b/d32/d52 22 2024-02-16T22:56:12.341 INFO:tasks.workunit.client.0.smithi064.stdout:6/168: fdatasync d0/d6/da/f25 0 2024-02-16T22:56:12.341 INFO:tasks.workunit.client.1.smithi192.stdout:9/303: dwrite d7/d15/f58 [0,4194304] 0 2024-02-16T22:56:12.342 INFO:tasks.workunit.client.1.smithi192.stdout:3/222: write da/d11/f38 [2367237,65681] 0 2024-02-16T22:56:12.342 INFO:tasks.workunit.client.0.smithi064.stdout:7/110: write d5/d6/f9 [3439038,86350] 0 2024-02-16T22:56:12.343 INFO:tasks.workunit.client.0.smithi064.stdout:6/169: fsync d0/d12/f1f 0 2024-02-16T22:56:12.346 INFO:tasks.workunit.client.0.smithi064.stdout:2/125: dread d7/f13 [0,4194304] 0 2024-02-16T22:56:12.347 INFO:tasks.workunit.client.0.smithi064.stdout:2/126: dread - d7/f1a zero size 2024-02-16T22:56:12.349 INFO:tasks.workunit.client.1.smithi192.stdout:2/252: creat d22/d24/d39/d42/f54 x:0 0 0 2024-02-16T22:56:12.351 INFO:tasks.workunit.client.0.smithi064.stdout:0/120: dwrite da/dc/de/f16 [0,4194304] 0 2024-02-16T22:56:12.351 INFO:tasks.workunit.client.1.smithi192.stdout:1/202: mkdir d3/d40 0 2024-02-16T22:56:12.354 INFO:tasks.workunit.client.0.smithi064.stdout:2/127: fsync d7/f19 0 2024-02-16T22:56:12.355 INFO:tasks.workunit.client.1.smithi192.stdout:5/249: symlink d6/d12/d15/d46/l47 0 2024-02-16T22:56:12.356 INFO:tasks.workunit.client.0.smithi064.stdout:2/128: chown d7/de 16555010 1 2024-02-16T22:56:12.357 INFO:tasks.workunit.client.0.smithi064.stdout:6/170: chown d0/d12/f1f 212992104 1 2024-02-16T22:56:12.358 INFO:tasks.workunit.client.0.smithi064.stdout:8/134: fdatasync df/f24 0 2024-02-16T22:56:12.358 INFO:tasks.workunit.client.0.smithi064.stdout:1/120: dwrite d2/df/d11/d1a/d1d/f24 [0,4194304] 0 2024-02-16T22:56:12.361 INFO:tasks.workunit.client.1.smithi192.stdout:8/203: getdents d5/db/de/df 0 2024-02-16T22:56:12.364 INFO:tasks.workunit.client.0.smithi064.stdout:0/121: dwrite da/dc/de/f17 [0,4194304] 0 2024-02-16T22:56:12.368 INFO:tasks.workunit.client.0.smithi064.stdout:3/183: rename d4/d1f/f24 to d4/d14/d39/f3b 0 2024-02-16T22:56:12.369 INFO:tasks.workunit.client.0.smithi064.stdout:9/158: rmdir d3/d11/d2b/d2a 39 2024-02-16T22:56:12.370 INFO:tasks.workunit.client.1.smithi192.stdout:0/192: write d1/fd [669315,118151] 0 2024-02-16T22:56:12.374 INFO:tasks.workunit.client.1.smithi192.stdout:9/304: readlink d7/d15/d36/l48 0 2024-02-16T22:56:12.374 INFO:tasks.workunit.client.1.smithi192.stdout:4/232: dread d1/d8/f9 [0,4194304] 0 2024-02-16T22:56:12.375 INFO:tasks.workunit.client.0.smithi064.stdout:7/111: unlink d5/d6/le 0 2024-02-16T22:56:12.376 INFO:tasks.workunit.client.1.smithi192.stdout:3/223: mkdir da/d11/d20/d39/d53 0 2024-02-16T22:56:12.378 INFO:tasks.workunit.client.1.smithi192.stdout:2/253: symlink d22/d24/d2b/d2c/d3f/l55 0 2024-02-16T22:56:12.379 INFO:tasks.workunit.client.0.smithi064.stdout:5/126: getdents d6/d18/d19 0 2024-02-16T22:56:12.380 INFO:tasks.workunit.client.1.smithi192.stdout:1/203: rename d3/l11 to d3/de/d27/d34/l41 0 2024-02-16T22:56:12.381 INFO:tasks.workunit.client.1.smithi192.stdout:5/250: symlink d6/d12/d1d/d3c/l48 0 2024-02-16T22:56:12.383 INFO:tasks.workunit.client.0.smithi064.stdout:2/129: mkdir d7/d14/d1e 0 2024-02-16T22:56:12.384 INFO:tasks.workunit.client.1.smithi192.stdout:8/204: creat d5/db/de/f37 x:0 0 0 2024-02-16T22:56:12.385 INFO:tasks.workunit.client.1.smithi192.stdout:8/205: chown d5/db/dd/f1c 2125906 1 2024-02-16T22:56:12.385 INFO:tasks.workunit.client.1.smithi192.stdout:7/274: sync 2024-02-16T22:56:12.385 INFO:tasks.workunit.client.0.smithi064.stdout:6/171: unlink d0/d5/c29 0 2024-02-16T22:56:12.386 INFO:tasks.workunit.client.1.smithi192.stdout:6/236: mknod da/d40/c44 0 2024-02-16T22:56:12.389 INFO:tasks.workunit.client.0.smithi064.stdout:1/121: mknod d2/df/c2e 0 2024-02-16T22:56:12.390 INFO:tasks.workunit.client.0.smithi064.stdout:1/122: write d2/df/d14/f16 [1101380,97834] 0 2024-02-16T22:56:12.391 INFO:tasks.workunit.client.1.smithi192.stdout:6/237: dread da/d27/f2f [0,4194304] 0 2024-02-16T22:56:12.393 INFO:tasks.workunit.client.1.smithi192.stdout:6/238: write da/df/d12/d15/d1b/f37 [1132395,67303] 0 2024-02-16T22:56:12.393 INFO:tasks.workunit.client.0.smithi064.stdout:3/184: stat d4/l23 0 2024-02-16T22:56:12.394 INFO:tasks.workunit.client.1.smithi192.stdout:1/204: creat d3/de/d27/f42 x:0 0 0 2024-02-16T22:56:12.395 INFO:tasks.workunit.client.0.smithi064.stdout:9/159: creat d3/d9/f3c x:0 0 0 2024-02-16T22:56:12.396 INFO:tasks.workunit.client.1.smithi192.stdout:0/193: dwrite d1/f15 [4194304,4194304] 0 2024-02-16T22:56:12.396 INFO:tasks.workunit.client.1.smithi192.stdout:5/251: mknod d6/d12/d15/d46/c49 0 2024-02-16T22:56:12.398 INFO:tasks.workunit.client.1.smithi192.stdout:5/252: read f2 [989597,80936] 0 2024-02-16T22:56:12.400 INFO:tasks.workunit.client.1.smithi192.stdout:7/275: mkdir d1/d19/d4c/d60 0 2024-02-16T22:56:12.407 INFO:tasks.workunit.client.1.smithi192.stdout:9/305: truncate d7/f8 453231 0 2024-02-16T22:56:12.408 INFO:tasks.workunit.client.0.smithi064.stdout:1/123: creat d2/df/d14/f2f x:0 0 0 2024-02-16T22:56:12.412 INFO:tasks.workunit.client.0.smithi064.stdout:7/112: rmdir d5 39 2024-02-16T22:56:12.415 INFO:tasks.workunit.client.0.smithi064.stdout:1/124: mkdir d2/df/d11/d1a/d30 0 2024-02-16T22:56:12.416 INFO:tasks.workunit.client.0.smithi064.stdout:3/185: creat d4/d1f/d34/f3c x:0 0 0 2024-02-16T22:56:12.416 INFO:tasks.workunit.client.1.smithi192.stdout:4/233: dread d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:12.418 INFO:tasks.workunit.client.0.smithi064.stdout:2/130: creat d7/de/f1f x:0 0 0 2024-02-16T22:56:12.420 INFO:tasks.workunit.client.0.smithi064.stdout:8/135: write fe [150742,100932] 0 2024-02-16T22:56:12.421 INFO:tasks.workunit.client.1.smithi192.stdout:5/253: mknod d6/d12/d2d/c4a 0 2024-02-16T22:56:12.425 INFO:tasks.workunit.client.0.smithi064.stdout:1/125: rename d2/c6 to d2/df/d11/d25/c31 0 2024-02-16T22:56:12.426 INFO:tasks.workunit.client.0.smithi064.stdout:1/126: write d2/df/d14/d18/f28 [393145,103545] 0 2024-02-16T22:56:12.437 INFO:tasks.workunit.client.1.smithi192.stdout:7/276: fdatasync d1/d2/de/f34 0 2024-02-16T22:56:12.444 INFO:tasks.workunit.client.1.smithi192.stdout:7/277: dwrite d1/d2/f58 [0,4194304] 0 2024-02-16T22:56:12.449 INFO:tasks.workunit.client.0.smithi064.stdout:3/186: symlink d4/d13/d1b/l3d 0 2024-02-16T22:56:12.449 INFO:tasks.workunit.client.0.smithi064.stdout:0/122: write f6 [343175,92184] 0 2024-02-16T22:56:12.451 INFO:tasks.workunit.client.0.smithi064.stdout:3/187: truncate d4/d13/d1b/f1c 751743 0 2024-02-16T22:56:12.455 INFO:tasks.workunit.client.0.smithi064.stdout:8/136: mkdir df/d28 0 2024-02-16T22:56:12.457 INFO:tasks.workunit.client.1.smithi192.stdout:3/224: link da/d11/f3c da/d11/f54 0 2024-02-16T22:56:12.457 INFO:tasks.workunit.client.0.smithi064.stdout:6/172: getdents d0/d6/da 0 2024-02-16T22:56:12.458 INFO:tasks.workunit.client.1.smithi192.stdout:1/205: symlink d3/l43 0 2024-02-16T22:56:12.461 INFO:tasks.workunit.client.0.smithi064.stdout:6/173: dread d0/d12/f1f [0,4194304] 0 2024-02-16T22:56:12.461 INFO:tasks.workunit.client.0.smithi064.stdout:1/127: rename d2/df/d14/d18/f19 to d2/df/d11/d1a/d30/f32 0 2024-02-16T22:56:12.464 INFO:tasks.workunit.client.1.smithi192.stdout:2/254: truncate f4 1860650 0 2024-02-16T22:56:12.464 INFO:tasks.workunit.client.0.smithi064.stdout:2/131: getdents d7/d14/d1e 0 2024-02-16T22:56:12.464 INFO:tasks.workunit.client.0.smithi064.stdout:2/132: dread - d7/f12 zero size 2024-02-16T22:56:12.467 INFO:tasks.workunit.client.1.smithi192.stdout:4/234: rename d1/d8/d11/f2b to d1/d15/d23/f52 0 2024-02-16T22:56:12.469 INFO:tasks.workunit.client.1.smithi192.stdout:8/206: creat d5/f38 x:0 0 0 2024-02-16T22:56:12.470 INFO:tasks.workunit.client.0.smithi064.stdout:7/113: symlink d5/l1a 0 2024-02-16T22:56:12.470 INFO:tasks.workunit.client.0.smithi064.stdout:7/114: chown d5/d6/l16 818287 1 2024-02-16T22:56:12.471 INFO:tasks.workunit.client.0.smithi064.stdout:7/115: chown d5/d6/cd 0 1 2024-02-16T22:56:12.476 INFO:tasks.workunit.client.0.smithi064.stdout:8/137: dwrite df/f23 [0,4194304] 0 2024-02-16T22:56:12.482 INFO:tasks.workunit.client.0.smithi064.stdout:1/128: symlink d2/df/l33 0 2024-02-16T22:56:12.483 INFO:tasks.workunit.client.0.smithi064.stdout:1/129: write d2/df/d14/f16 [1503017,112072] 0 2024-02-16T22:56:12.486 INFO:tasks.workunit.client.1.smithi192.stdout:3/225: mknod da/d11/d2b/c55 0 2024-02-16T22:56:12.487 INFO:tasks.workunit.client.1.smithi192.stdout:3/226: dread - da/d11/f54 zero size 2024-02-16T22:56:12.489 INFO:tasks.workunit.client.0.smithi064.stdout:2/133: symlink d7/l20 0 2024-02-16T22:56:12.491 INFO:tasks.workunit.client.0.smithi064.stdout:3/188: creat d4/d13/d35/f3e x:0 0 0 2024-02-16T22:56:12.492 INFO:tasks.workunit.client.1.smithi192.stdout:2/255: mknod d22/d24/d2b/d3e/c56 0 2024-02-16T22:56:12.492 INFO:tasks.workunit.client.1.smithi192.stdout:3/227: dread f3 [0,4194304] 0 2024-02-16T22:56:12.493 INFO:tasks.workunit.client.1.smithi192.stdout:3/228: readlink da/d11/l29 0 2024-02-16T22:56:12.493 INFO:tasks.workunit.client.0.smithi064.stdout:7/116: rmdir d5 39 2024-02-16T22:56:12.495 INFO:tasks.workunit.client.0.smithi064.stdout:2/134: dwrite d7/de/f15 [0,4194304] 0 2024-02-16T22:56:12.495 INFO:tasks.workunit.client.0.smithi064.stdout:2/135: chown d7/de 3 1 2024-02-16T22:56:12.496 INFO:tasks.workunit.client.1.smithi192.stdout:5/254: rename d6/d12/c2f to d6/d12/d15/c4b 0 2024-02-16T22:56:12.502 INFO:tasks.workunit.client.1.smithi192.stdout:8/207: mkdir d5/db/dd/d39 0 2024-02-16T22:56:12.503 INFO:tasks.workunit.client.0.smithi064.stdout:2/136: dwrite d7/f1a [0,4194304] 0 2024-02-16T22:56:12.506 INFO:tasks.workunit.client.0.smithi064.stdout:3/189: mkdir d4/d13/d1b/d3f 0 2024-02-16T22:56:12.516 INFO:tasks.workunit.client.1.smithi192.stdout:5/255: creat d6/d12/f4c x:0 0 0 2024-02-16T22:56:12.521 INFO:tasks.workunit.client.0.smithi064.stdout:2/137: rename d7/f19 to d7/d14/d1e/f21 0 2024-02-16T22:56:12.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/306: write d7/d19/d28/d43/f4d [1408808,118232] 0 2024-02-16T22:56:12.524 INFO:tasks.workunit.client.0.smithi064.stdout:7/117: dread d5/d6/fa [0,4194304] 0 2024-02-16T22:56:12.524 INFO:tasks.workunit.client.1.smithi192.stdout:3/229: creat da/df/d42/d4e/f56 x:0 0 0 2024-02-16T22:56:12.525 INFO:tasks.workunit.client.1.smithi192.stdout:3/230: write da/df/f3a [251470,83220] 0 2024-02-16T22:56:12.526 INFO:tasks.workunit.client.1.smithi192.stdout:3/231: write da/f22 [2921131,100178] 0 2024-02-16T22:56:12.531 INFO:tasks.workunit.client.1.smithi192.stdout:9/307: creat d7/d19/d37/d5a/f63 x:0 0 0 2024-02-16T22:56:12.531 INFO:tasks.workunit.client.1.smithi192.stdout:3/232: dwrite da/df/d42/d4e/f56 [0,4194304] 0 2024-02-16T22:56:12.535 INFO:tasks.workunit.client.1.smithi192.stdout:9/308: symlink d7/l64 0 2024-02-16T22:56:12.538 INFO:tasks.workunit.client.1.smithi192.stdout:6/239: truncate da/df/d12/d15/f33 827401 0 2024-02-16T22:56:12.541 INFO:tasks.workunit.client.0.smithi064.stdout:3/190: link d4/d13/c19 d4/d13/d35/c40 0 2024-02-16T22:56:12.545 INFO:tasks.workunit.client.0.smithi064.stdout:7/118: mknod d5/c1b 0 2024-02-16T22:56:12.545 INFO:tasks.workunit.client.0.smithi064.stdout:7/119: readlink d5/d6/l17 0 2024-02-16T22:56:12.545 INFO:tasks.workunit.client.0.smithi064.stdout:3/191: dread d4/f18 [0,4194304] 0 2024-02-16T22:56:12.546 INFO:tasks.workunit.client.1.smithi192.stdout:3/233: symlink da/d11/d2b/d43/l57 0 2024-02-16T22:56:12.546 INFO:tasks.workunit.client.1.smithi192.stdout:3/234: chown f4 7 1 2024-02-16T22:56:12.547 INFO:tasks.workunit.client.0.smithi064.stdout:3/192: chown d4/f17 862476 1 2024-02-16T22:56:12.547 INFO:tasks.workunit.client.0.smithi064.stdout:7/120: write d5/d6/f9 [586459,20538] 0 2024-02-16T22:56:12.547 INFO:tasks.workunit.client.0.smithi064.stdout:3/193: fdatasync d4/fd 0 2024-02-16T22:56:12.550 INFO:tasks.workunit.client.0.smithi064.stdout:7/121: dread d5/d6/f9 [4194304,4194304] 0 2024-02-16T22:56:12.550 INFO:tasks.workunit.client.0.smithi064.stdout:7/122: readlink d5/l14 0 2024-02-16T22:56:12.557 INFO:tasks.workunit.client.1.smithi192.stdout:2/256: dread f4 [0,4194304] 0 2024-02-16T22:56:12.589 INFO:tasks.workunit.client.1.smithi192.stdout:0/194: truncate d1/d3/d29/f1a 3672211 0 2024-02-16T22:56:12.590 INFO:tasks.workunit.client.1.smithi192.stdout:9/309: creat d7/d15/d36/d54/f65 x:0 0 0 2024-02-16T22:56:12.595 INFO:tasks.workunit.client.0.smithi064.stdout:7/123: fsync d5/f10 0 2024-02-16T22:56:12.597 INFO:tasks.workunit.client.0.smithi064.stdout:7/124: dread d5/d6/fa [0,4194304] 0 2024-02-16T22:56:12.599 INFO:tasks.workunit.client.0.smithi064.stdout:3/194: chown d4/d13/c19 3218891 1 2024-02-16T22:56:12.600 INFO:tasks.workunit.client.0.smithi064.stdout:7/125: dread d5/d6/fb [0,4194304] 0 2024-02-16T22:56:12.601 INFO:tasks.workunit.client.1.smithi192.stdout:0/195: mkdir d1/d3/d39 0 2024-02-16T22:56:12.602 INFO:tasks.workunit.client.0.smithi064.stdout:0/123: truncate f6 809299 0 2024-02-16T22:56:12.602 INFO:tasks.workunit.client.0.smithi064.stdout:0/124: truncate da/dc/de/f16 4679243 0 2024-02-16T22:56:12.603 INFO:tasks.workunit.client.0.smithi064.stdout:0/125: stat da/dc/ld 0 2024-02-16T22:56:12.603 INFO:tasks.workunit.client.1.smithi192.stdout:9/310: mknod d7/d19/d5b/c66 0 2024-02-16T22:56:12.605 INFO:tasks.workunit.client.1.smithi192.stdout:3/235: creat da/d11/f58 x:0 0 0 2024-02-16T22:56:12.605 INFO:tasks.workunit.client.1.smithi192.stdout:0/196: dread d1/d3/de/f21 [0,4194304] 0 2024-02-16T22:56:12.605 INFO:tasks.workunit.client.1.smithi192.stdout:9/311: write d7/d19/d37/d39/f44 [1297757,55357] 0 2024-02-16T22:56:12.612 INFO:tasks.workunit.client.1.smithi192.stdout:9/312: symlink d7/d19/d28/d43/l67 0 2024-02-16T22:56:12.614 INFO:tasks.workunit.client.1.smithi192.stdout:0/197: mknod d1/d2/c3a 0 2024-02-16T22:56:12.615 INFO:tasks.workunit.client.1.smithi192.stdout:8/208: rmdir d5/db/dd 39 2024-02-16T22:56:12.615 INFO:tasks.workunit.client.0.smithi064.stdout:6/174: dwrite d0/d5/f11 [0,4194304] 0 2024-02-16T22:56:12.617 INFO:tasks.workunit.client.1.smithi192.stdout:7/278: write d1/d2/f6 [1423552,59739] 0 2024-02-16T22:56:12.617 INFO:tasks.workunit.client.1.smithi192.stdout:0/198: mknod d1/d3/de/c3b 0 2024-02-16T22:56:12.619 INFO:tasks.workunit.client.1.smithi192.stdout:7/279: stat d1/d19/f3a 0 2024-02-16T22:56:12.619 INFO:tasks.workunit.client.1.smithi192.stdout:0/199: chown d1/d3/fc 363904480 1 2024-02-16T22:56:12.619 INFO:tasks.workunit.client.0.smithi064.stdout:7/126: rename d5/c1b to d5/c1c 0 2024-02-16T22:56:12.620 INFO:tasks.workunit.client.1.smithi192.stdout:9/313: dwrite d7/d15/f33 [0,4194304] 0 2024-02-16T22:56:12.621 INFO:tasks.workunit.client.1.smithi192.stdout:8/209: rmdir d5/d16 39 2024-02-16T22:56:12.623 INFO:tasks.workunit.client.1.smithi192.stdout:7/280: write d1/d2/f4 [4206878,15043] 0 2024-02-16T22:56:12.623 INFO:tasks.workunit.client.0.smithi064.stdout:7/127: dread d5/f10 [0,4194304] 0 2024-02-16T22:56:12.623 INFO:tasks.workunit.client.0.smithi064.stdout:6/175: symlink d0/d6/da/l2e 0 2024-02-16T22:56:12.626 INFO:tasks.workunit.client.1.smithi192.stdout:9/314: rename d7/c1b to d7/d15/d36/d54/c68 0 2024-02-16T22:56:12.627 INFO:tasks.workunit.client.0.smithi064.stdout:3/195: getdents d4/d1f/d34 0 2024-02-16T22:56:12.629 INFO:tasks.workunit.client.0.smithi064.stdout:3/196: chown d4/d13/d1b/d27/f2c 769313 1 2024-02-16T22:56:12.630 INFO:tasks.workunit.client.0.smithi064.stdout:3/197: truncate d4/d13/f28 558850 0 2024-02-16T22:56:12.631 INFO:tasks.workunit.client.0.smithi064.stdout:3/198: write d4/d14/f2a [538944,121684] 0 2024-02-16T22:56:12.634 INFO:tasks.workunit.client.0.smithi064.stdout:6/176: symlink d0/d6/d10/l2f 0 2024-02-16T22:56:12.635 INFO:tasks.workunit.client.0.smithi064.stdout:1/130: truncate d2/df/f1c 130347 0 2024-02-16T22:56:12.637 INFO:tasks.workunit.client.0.smithi064.stdout:7/128: link d5/f10 d5/d6/f1d 0 2024-02-16T22:56:12.637 INFO:tasks.workunit.client.0.smithi064.stdout:7/129: stat d5/d6/f1d 0 2024-02-16T22:56:12.637 INFO:tasks.workunit.client.0.smithi064.stdout:7/130: stat d5/d6/f1d 0 2024-02-16T22:56:12.638 INFO:tasks.workunit.client.1.smithi192.stdout:8/210: mkdir d5/d3a 0 2024-02-16T22:56:12.638 INFO:tasks.workunit.client.1.smithi192.stdout:8/211: chown d5/f25 2737991 1 2024-02-16T22:56:12.639 INFO:tasks.workunit.client.1.smithi192.stdout:0/200: dread d1/d3/f12 [0,4194304] 0 2024-02-16T22:56:12.639 INFO:tasks.workunit.client.0.smithi064.stdout:3/199: rmdir d4/d1f 39 2024-02-16T22:56:12.639 INFO:tasks.workunit.client.1.smithi192.stdout:0/201: stat d1/d3/f12 0 2024-02-16T22:56:12.641 INFO:tasks.workunit.client.0.smithi064.stdout:2/138: rmdir d7 39 2024-02-16T22:56:12.642 INFO:tasks.workunit.client.0.smithi064.stdout:6/177: mknod d0/d5/df/d17/c30 0 2024-02-16T22:56:12.645 INFO:tasks.workunit.client.0.smithi064.stdout:1/131: creat d2/df/d11/d1a/f34 x:0 0 0 2024-02-16T22:56:12.651 INFO:tasks.workunit.client.0.smithi064.stdout:7/131: mknod d5/d6/c1e 0 2024-02-16T22:56:12.651 INFO:tasks.workunit.client.0.smithi064.stdout:1/132: dread d2/df/d11/d1a/d1d/f24 [0,4194304] 0 2024-02-16T22:56:12.651 INFO:tasks.workunit.client.0.smithi064.stdout:1/133: fsync d2/df/d11/f27 0 2024-02-16T22:56:12.652 INFO:tasks.workunit.client.1.smithi192.stdout:8/212: dwrite d5/d16/f18 [4194304,4194304] 0 2024-02-16T22:56:12.653 INFO:tasks.workunit.client.0.smithi064.stdout:6/178: fdatasync d0/d1b/f23 0 2024-02-16T22:56:12.655 INFO:tasks.workunit.client.1.smithi192.stdout:8/213: dwrite d5/d16/f18 [4194304,4194304] 0 2024-02-16T22:56:12.662 INFO:tasks.workunit.client.0.smithi064.stdout:6/179: dwrite d0/d12/f1f [0,4194304] 0 2024-02-16T22:56:12.663 INFO:tasks.workunit.client.1.smithi192.stdout:7/281: dread d1/d2/f29 [0,4194304] 0 2024-02-16T22:56:12.666 INFO:tasks.workunit.client.0.smithi064.stdout:2/139: mknod d7/d14/c22 0 2024-02-16T22:56:12.666 INFO:tasks.workunit.client.1.smithi192.stdout:4/235: link d1/d15/c3e d1/c53 0 2024-02-16T22:56:12.669 INFO:tasks.workunit.client.1.smithi192.stdout:7/282: dwrite d1/f11 [0,4194304] 0 2024-02-16T22:56:12.671 INFO:tasks.workunit.client.0.smithi064.stdout:1/134: creat d2/df/d11/f35 x:0 0 0 2024-02-16T22:56:12.672 INFO:tasks.workunit.client.0.smithi064.stdout:6/180: dwrite d0/d6/d10/f14 [0,4194304] 0 2024-02-16T22:56:12.674 INFO:tasks.workunit.client.1.smithi192.stdout:4/236: dwrite d1/d8/d11/f4e [0,4194304] 0 2024-02-16T22:56:12.676 INFO:tasks.workunit.client.1.smithi192.stdout:4/237: dread - d1/d15/d23/f4a zero size 2024-02-16T22:56:12.677 INFO:tasks.workunit.client.1.smithi192.stdout:4/238: chown d1/d15/d23 8 1 2024-02-16T22:56:12.683 INFO:tasks.workunit.client.1.smithi192.stdout:6/240: dwrite da/df/f1d [0,4194304] 0 2024-02-16T22:56:12.686 INFO:tasks.workunit.client.1.smithi192.stdout:4/239: dwrite d1/d15/f4f [0,4194304] 0 2024-02-16T22:56:12.688 INFO:tasks.workunit.client.1.smithi192.stdout:7/283: mkdir d1/d2/d2d/d61 0 2024-02-16T22:56:12.689 INFO:tasks.workunit.client.0.smithi064.stdout:2/140: mknod d7/c23 0 2024-02-16T22:56:12.698 INFO:tasks.workunit.client.0.smithi064.stdout:0/126: rename f6 to da/dc/de/d12/d1d/f1f 0 2024-02-16T22:56:12.702 INFO:tasks.workunit.client.1.smithi192.stdout:4/240: creat d1/d26/f54 x:0 0 0 2024-02-16T22:56:12.704 INFO:tasks.workunit.client.0.smithi064.stdout:3/200: rename d4/d14/d39 to d4/d13/d41 0 2024-02-16T22:56:12.705 INFO:tasks.workunit.client.0.smithi064.stdout:0/127: dwrite da/dc/de/f16 [0,4194304] 0 2024-02-16T22:56:12.706 INFO:tasks.workunit.client.0.smithi064.stdout:3/201: write d4/d13/f26 [222106,72175] 0 2024-02-16T22:56:12.707 INFO:tasks.workunit.client.1.smithi192.stdout:7/284: rename d1/f5e to d1/d2/de/d2c/d49/d54/f62 0 2024-02-16T22:56:12.708 INFO:tasks.workunit.client.1.smithi192.stdout:7/285: dread d1/d2/f25 [0,4194304] 0 2024-02-16T22:56:12.711 INFO:tasks.workunit.client.0.smithi064.stdout:2/141: creat d7/de/f24 x:0 0 0 2024-02-16T22:56:12.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/241: creat d1/d15/d48/f55 x:0 0 0 2024-02-16T22:56:12.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/242: chown d1/d15/d23/c21 4663859 1 2024-02-16T22:56:12.717 INFO:tasks.workunit.client.0.smithi064.stdout:0/128: fsync da/dc/f1e 0 2024-02-16T22:56:12.717 INFO:tasks.workunit.client.0.smithi064.stdout:2/142: dwrite d7/f1a [0,4194304] 0 2024-02-16T22:56:12.718 INFO:tasks.workunit.client.1.smithi192.stdout:9/315: chown d7/d15/d36/d54/c68 20 1 2024-02-16T22:56:12.724 INFO:tasks.workunit.client.1.smithi192.stdout:8/214: creat d5/db/dd/d39/f3b x:0 0 0 2024-02-16T22:56:12.726 INFO:tasks.workunit.client.1.smithi192.stdout:9/316: dwrite d7/d15/f20 [0,4194304] 0 2024-02-16T22:56:12.728 INFO:tasks.workunit.client.1.smithi192.stdout:9/317: write d7/d15/d36/d38/d3f/f59 [82115,62634] 0 2024-02-16T22:56:12.729 INFO:tasks.workunit.client.1.smithi192.stdout:7/286: creat d1/d19/d2a/f63 x:0 0 0 2024-02-16T22:56:12.729 INFO:tasks.workunit.client.1.smithi192.stdout:8/215: dread d5/db/dd/f1c [0,4194304] 0 2024-02-16T22:56:12.729 INFO:tasks.workunit.client.0.smithi064.stdout:7/132: chown d5/c1c 30769897 1 2024-02-16T22:56:12.729 INFO:tasks.workunit.client.0.smithi064.stdout:4/120: sync 2024-02-16T22:56:12.730 INFO:tasks.workunit.client.1.smithi192.stdout:9/318: stat d7/d15/f33 0 2024-02-16T22:56:12.730 INFO:tasks.workunit.client.0.smithi064.stdout:4/121: write f3 [4372225,55962] 0 2024-02-16T22:56:12.731 INFO:tasks.workunit.client.1.smithi192.stdout:8/216: truncate d5/d9/f33 647444 0 2024-02-16T22:56:12.731 INFO:tasks.workunit.client.1.smithi192.stdout:9/319: fsync d7/d19/d28/f3e 0 2024-02-16T22:56:12.732 INFO:tasks.workunit.client.0.smithi064.stdout:2/143: mkdir d7/d14/d25 0 2024-02-16T22:56:12.732 INFO:tasks.workunit.client.1.smithi192.stdout:8/217: write d5/db/dd/f2e [631169,64550] 0 2024-02-16T22:56:12.733 INFO:tasks.workunit.client.1.smithi192.stdout:8/218: readlink d5/d26/l31 0 2024-02-16T22:56:12.736 INFO:tasks.workunit.client.1.smithi192.stdout:4/243: link d1/d15/d48/f55 d1/d15/d48/f56 0 2024-02-16T22:56:12.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/244: chown d1/d26/f37 6 1 2024-02-16T22:56:12.741 INFO:tasks.workunit.client.1.smithi192.stdout:1/206: sync 2024-02-16T22:56:12.741 INFO:tasks.workunit.client.0.smithi064.stdout:1/135: rename d2/df/d14/f2f to d2/f36 0 2024-02-16T22:56:12.741 INFO:tasks.workunit.client.1.smithi192.stdout:1/207: dread - d3/f26 zero size 2024-02-16T22:56:12.742 INFO:tasks.workunit.client.1.smithi192.stdout:9/320: mknod d7/d19/d5b/c69 0 2024-02-16T22:56:12.743 INFO:tasks.workunit.client.0.smithi064.stdout:7/133: creat d5/d6/f1f x:0 0 0 2024-02-16T22:56:12.750 INFO:tasks.workunit.client.0.smithi064.stdout:3/202: creat d4/f42 x:0 0 0 2024-02-16T22:56:12.751 INFO:tasks.workunit.client.1.smithi192.stdout:0/202: truncate d1/f15 1503766 0 2024-02-16T22:56:12.752 INFO:tasks.workunit.client.1.smithi192.stdout:8/219: write d5/db/de/df/f2b [1996773,14262] 0 2024-02-16T22:56:12.753 INFO:tasks.workunit.client.1.smithi192.stdout:9/321: creat d7/d15/f6a x:0 0 0 2024-02-16T22:56:12.755 INFO:tasks.workunit.client.1.smithi192.stdout:4/245: symlink d1/d15/d48/d4c/l57 0 2024-02-16T22:56:12.758 INFO:tasks.workunit.client.0.smithi064.stdout:5/127: sync 2024-02-16T22:56:12.758 INFO:tasks.workunit.client.0.smithi064.stdout:9/160: sync 2024-02-16T22:56:12.758 INFO:tasks.workunit.client.0.smithi064.stdout:8/138: sync 2024-02-16T22:56:12.759 INFO:tasks.workunit.client.1.smithi192.stdout:4/246: dwrite d1/f30 [0,4194304] 0 2024-02-16T22:56:12.759 INFO:tasks.workunit.client.0.smithi064.stdout:5/128: read d6/d18/d19/f26 [1120932,44371] 0 2024-02-16T22:56:12.760 INFO:tasks.workunit.client.1.smithi192.stdout:4/247: write d1/d15/d29/d3c/f43 [2844665,8165] 0 2024-02-16T22:56:12.761 INFO:tasks.workunit.client.1.smithi192.stdout:6/241: write da/df/d12/d15/d1b/f1e [868337,49427] 0 2024-02-16T22:56:12.761 INFO:tasks.workunit.client.1.smithi192.stdout:4/248: write d1/fd [3182298,13649] 0 2024-02-16T22:56:12.762 INFO:tasks.workunit.client.0.smithi064.stdout:8/139: dread df/f24 [0,4194304] 0 2024-02-16T22:56:12.763 INFO:tasks.workunit.client.0.smithi064.stdout:8/140: chown l7 209207 1 2024-02-16T22:56:12.763 INFO:tasks.workunit.client.0.smithi064.stdout:8/141: read - df/f19 zero size 2024-02-16T22:56:12.765 INFO:tasks.workunit.client.1.smithi192.stdout:0/203: creat d1/d3/de/f3c x:0 0 0 2024-02-16T22:56:12.765 INFO:tasks.workunit.client.1.smithi192.stdout:0/204: readlink d1/d2/l2e 0 2024-02-16T22:56:12.768 INFO:tasks.workunit.client.0.smithi064.stdout:3/203: mkdir d4/d13/d1b/d43 0 2024-02-16T22:56:12.768 INFO:tasks.workunit.client.0.smithi064.stdout:8/142: dread df/f23 [0,4194304] 0 2024-02-16T22:56:12.770 INFO:tasks.workunit.client.1.smithi192.stdout:8/220: chown d5/f1e 48676 1 2024-02-16T22:56:12.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/322: unlink d7/d19/l2d 0 2024-02-16T22:56:12.778 INFO:tasks.workunit.client.0.smithi064.stdout:9/161: creat d3/d11/d15/d23/f3d x:0 0 0 2024-02-16T22:56:12.783 INFO:tasks.workunit.client.0.smithi064.stdout:0/129: getdents da/dc 0 2024-02-16T22:56:12.783 INFO:tasks.workunit.client.1.smithi192.stdout:4/249: mkdir d1/d8/d11/d58 0 2024-02-16T22:56:12.784 INFO:tasks.workunit.client.1.smithi192.stdout:0/205: symlink d1/d3/de/l3d 0 2024-02-16T22:56:12.785 INFO:tasks.workunit.client.1.smithi192.stdout:8/221: symlink d5/d9/l3c 0 2024-02-16T22:56:12.786 INFO:tasks.workunit.client.0.smithi064.stdout:3/204: mkdir d4/d13/d1b/d27/d44 0 2024-02-16T22:56:12.786 INFO:tasks.workunit.client.0.smithi064.stdout:3/205: readlink d4/l2e 0 2024-02-16T22:56:12.787 INFO:tasks.workunit.client.1.smithi192.stdout:8/222: read d5/d9/f17 [2495829,31741] 0 2024-02-16T22:56:12.787 INFO:tasks.workunit.client.1.smithi192.stdout:8/223: chown d5/d16/l30 4842 1 2024-02-16T22:56:12.787 INFO:tasks.workunit.client.0.smithi064.stdout:3/206: chown d4/d13/d1b/d27/f36 7787 1 2024-02-16T22:56:12.787 INFO:tasks.workunit.client.1.smithi192.stdout:9/323: symlink d7/d15/d36/l6b 0 2024-02-16T22:56:12.788 INFO:tasks.workunit.client.1.smithi192.stdout:8/224: truncate d5/d16/f18 8891847 0 2024-02-16T22:56:12.789 INFO:tasks.workunit.client.1.smithi192.stdout:6/242: mknod da/c45 0 2024-02-16T22:56:12.790 INFO:tasks.workunit.client.0.smithi064.stdout:6/181: rename d0/d6/d10/c16 to d0/d6/d19/c31 0 2024-02-16T22:56:12.791 INFO:tasks.workunit.client.0.smithi064.stdout:3/207: dread d4/f12 [0,4194304] 0 2024-02-16T22:56:12.793 INFO:tasks.workunit.client.1.smithi192.stdout:9/324: dwrite d7/d19/d28/d43/f57 [0,4194304] 0 2024-02-16T22:56:12.794 INFO:tasks.workunit.client.1.smithi192.stdout:0/206: chown d1/f2a 1694319 1 2024-02-16T22:56:12.794 INFO:tasks.workunit.client.1.smithi192.stdout:8/225: dwrite d5/db/dd/f2e [0,4194304] 0 2024-02-16T22:56:12.795 INFO:tasks.workunit.client.0.smithi064.stdout:7/134: creat d5/f20 x:0 0 0 2024-02-16T22:56:12.795 INFO:tasks.workunit.client.0.smithi064.stdout:7/135: chown f0 1 1 2024-02-16T22:56:12.797 INFO:tasks.workunit.client.0.smithi064.stdout:4/122: truncate f3 1943891 0 2024-02-16T22:56:12.799 INFO:tasks.workunit.client.1.smithi192.stdout:0/207: dread d1/fd [4194304,4194304] 0 2024-02-16T22:56:12.799 INFO:tasks.workunit.client.1.smithi192.stdout:1/208: truncate d3/f16 2082804 0 2024-02-16T22:56:12.801 INFO:tasks.workunit.client.0.smithi064.stdout:9/162: mkdir d3/d9/d1e/d3e 0 2024-02-16T22:56:12.805 INFO:tasks.workunit.client.1.smithi192.stdout:0/208: dwrite d1/d3/f12 [0,4194304] 0 2024-02-16T22:56:12.805 INFO:tasks.workunit.client.1.smithi192.stdout:6/243: mkdir da/df/d18/d46 0 2024-02-16T22:56:12.807 INFO:tasks.workunit.client.0.smithi064.stdout:9/163: dwrite f1 [0,4194304] 0 2024-02-16T22:56:12.807 INFO:tasks.workunit.client.0.smithi064.stdout:0/130: stat da/f1b 0 2024-02-16T22:56:12.810 INFO:tasks.workunit.client.1.smithi192.stdout:4/250: link d1/f30 d1/d8/d11/f59 0 2024-02-16T22:56:12.814 INFO:tasks.workunit.client.0.smithi064.stdout:8/143: creat df/d28/f29 x:0 0 0 2024-02-16T22:56:12.815 INFO:tasks.workunit.client.0.smithi064.stdout:8/144: write df/f19 [1001241,79222] 0 2024-02-16T22:56:12.816 INFO:tasks.workunit.client.0.smithi064.stdout:5/129: write d6/f23 [544816,14152] 0 2024-02-16T22:56:12.818 INFO:tasks.workunit.client.0.smithi064.stdout:8/145: dread fe [0,4194304] 0 2024-02-16T22:56:12.818 INFO:tasks.workunit.client.0.smithi064.stdout:8/146: write fa [1100251,62455] 0 2024-02-16T22:56:12.819 INFO:tasks.workunit.client.1.smithi192.stdout:9/325: chown d7/d19/d28/f2c 31313789 1 2024-02-16T22:56:12.820 INFO:tasks.workunit.client.0.smithi064.stdout:2/144: rename d7/l20 to d7/d14/l26 0 2024-02-16T22:56:12.821 INFO:tasks.workunit.client.1.smithi192.stdout:9/326: truncate d7/d15/d36/d38/f4b 749561 0 2024-02-16T22:56:12.825 INFO:tasks.workunit.client.0.smithi064.stdout:8/147: dwrite f9 [0,4194304] 0 2024-02-16T22:56:12.829 INFO:tasks.workunit.client.0.smithi064.stdout:7/136: fdatasync d5/d6/f1d 0 2024-02-16T22:56:12.830 INFO:tasks.workunit.client.0.smithi064.stdout:7/137: write d5/f20 [747411,98823] 0 2024-02-16T22:56:12.831 INFO:tasks.workunit.client.1.smithi192.stdout:5/256: sync 2024-02-16T22:56:12.831 INFO:tasks.workunit.client.1.smithi192.stdout:2/257: sync 2024-02-16T22:56:12.832 INFO:tasks.workunit.client.1.smithi192.stdout:6/244: unlink f6 0 2024-02-16T22:56:12.834 INFO:tasks.workunit.client.0.smithi064.stdout:4/123: mkdir dc/d10/d18/d22 0 2024-02-16T22:56:12.840 INFO:tasks.workunit.client.0.smithi064.stdout:9/164: rmdir d3/d11/d2b 39 2024-02-16T22:56:12.840 INFO:tasks.workunit.client.0.smithi064.stdout:9/165: write d3/f18 [349011,111282] 0 2024-02-16T22:56:12.841 INFO:tasks.workunit.client.1.smithi192.stdout:4/251: mknod d1/d15/c5a 0 2024-02-16T22:56:12.842 INFO:tasks.workunit.client.1.smithi192.stdout:4/252: write d1/d26/f31 [1608873,54568] 0 2024-02-16T22:56:12.843 INFO:tasks.workunit.client.0.smithi064.stdout:0/131: creat da/dc/de/f20 x:0 0 0 2024-02-16T22:56:12.846 INFO:tasks.workunit.client.0.smithi064.stdout:5/130: symlink d6/d18/l2f 0 2024-02-16T22:56:12.847 INFO:tasks.workunit.client.0.smithi064.stdout:5/131: chown d6/d1c/l2b 10135650 1 2024-02-16T22:56:12.850 INFO:tasks.workunit.client.1.smithi192.stdout:4/253: dread d1/d26/f37 [0,4194304] 0 2024-02-16T22:56:12.852 INFO:tasks.workunit.client.1.smithi192.stdout:5/257: creat d6/d20/f4d x:0 0 0 2024-02-16T22:56:12.853 INFO:tasks.workunit.client.1.smithi192.stdout:2/258: rmdir d22/d24/d2b/d3e 39 2024-02-16T22:56:12.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/236: sync 2024-02-16T22:56:12.854 INFO:tasks.workunit.client.1.smithi192.stdout:7/287: sync 2024-02-16T22:56:12.855 INFO:tasks.workunit.client.0.smithi064.stdout:8/148: mkdir df/d2a 0 2024-02-16T22:56:12.859 INFO:tasks.workunit.client.0.smithi064.stdout:3/208: creat d4/d1f/d34/f45 x:0 0 0 2024-02-16T22:56:12.859 INFO:tasks.workunit.client.1.smithi192.stdout:7/288: write d1/d2/de/f4d [389189,56180] 0 2024-02-16T22:56:12.860 INFO:tasks.workunit.client.1.smithi192.stdout:0/209: write d1/d3/de/f21 [1631595,118108] 0 2024-02-16T22:56:12.860 INFO:tasks.workunit.client.1.smithi192.stdout:8/226: dwrite d5/db/dd/f1c [0,4194304] 0 2024-02-16T22:56:12.861 INFO:tasks.workunit.client.1.smithi192.stdout:7/289: write d1/d19/d2a/f63 [82188,23458] 0 2024-02-16T22:56:12.862 INFO:tasks.workunit.client.1.smithi192.stdout:0/210: write d1/d3/de/d22/f26 [227743,112172] 0 2024-02-16T22:56:12.863 INFO:tasks.workunit.client.1.smithi192.stdout:9/327: mknod d7/d15/d36/d38/c6c 0 2024-02-16T22:56:12.864 INFO:tasks.workunit.client.0.smithi064.stdout:4/124: creat dc/dd/d16/f23 x:0 0 0 2024-02-16T22:56:12.865 INFO:tasks.workunit.client.1.smithi192.stdout:0/211: chown d1/fd 1416799 1 2024-02-16T22:56:12.865 INFO:tasks.workunit.client.1.smithi192.stdout:6/245: dwrite da/df/d12/d15/f31 [0,4194304] 0 2024-02-16T22:56:12.865 INFO:tasks.workunit.client.1.smithi192.stdout:0/212: stat d1/fa 0 2024-02-16T22:56:12.865 INFO:tasks.workunit.client.0.smithi064.stdout:4/125: write dc/de/f1c [65165,2483] 0 2024-02-16T22:56:12.867 INFO:tasks.workunit.client.1.smithi192.stdout:0/213: read d1/f36 [604259,5058] 0 2024-02-16T22:56:12.867 INFO:tasks.workunit.client.1.smithi192.stdout:4/254: mknod d1/d26/d34/c5b 0 2024-02-16T22:56:12.868 INFO:tasks.workunit.client.1.smithi192.stdout:0/214: write d1/d3/fc [6813674,32191] 0 2024-02-16T22:56:12.869 INFO:tasks.workunit.client.1.smithi192.stdout:4/255: truncate d1/d15/d23/f52 519206 0 2024-02-16T22:56:12.871 INFO:tasks.workunit.client.1.smithi192.stdout:5/258: unlink d6/d12/f4c 0 2024-02-16T22:56:12.875 INFO:tasks.workunit.client.1.smithi192.stdout:3/237: creat da/d11/d35/f59 x:0 0 0 2024-02-16T22:56:12.878 INFO:tasks.workunit.client.1.smithi192.stdout:3/238: fdatasync da/df/d42/f47 0 2024-02-16T22:56:12.878 INFO:tasks.workunit.client.1.smithi192.stdout:0/215: dwrite d1/f2a [0,4194304] 0 2024-02-16T22:56:12.878 INFO:tasks.workunit.client.0.smithi064.stdout:2/145: creat d7/d14/d25/f27 x:0 0 0 2024-02-16T22:56:12.882 INFO:tasks.workunit.client.0.smithi064.stdout:6/182: creat d0/d6/d19/f32 x:0 0 0 2024-02-16T22:56:12.883 INFO:tasks.workunit.client.1.smithi192.stdout:5/259: dwrite d6/d12/d2d/f43 [0,4194304] 0 2024-02-16T22:56:12.886 INFO:tasks.workunit.client.0.smithi064.stdout:3/209: creat d4/d14/f46 x:0 0 0 2024-02-16T22:56:12.887 INFO:tasks.workunit.client.1.smithi192.stdout:3/239: dwrite da/f22 [0,4194304] 0 2024-02-16T22:56:12.888 INFO:tasks.workunit.client.1.smithi192.stdout:8/227: mknod d5/db/de/d21/c3d 0 2024-02-16T22:56:12.888 INFO:tasks.workunit.client.0.smithi064.stdout:7/138: dwrite f0 [0,4194304] 0 2024-02-16T22:56:12.890 INFO:tasks.workunit.client.0.smithi064.stdout:3/210: write d4/d14/f46 [671518,112413] 0 2024-02-16T22:56:12.893 INFO:tasks.workunit.client.1.smithi192.stdout:7/290: creat d1/d19/f64 x:0 0 0 2024-02-16T22:56:12.897 INFO:tasks.workunit.client.0.smithi064.stdout:0/132: dwrite da/dc/de/d12/d1d/f1f [0,4194304] 0 2024-02-16T22:56:12.898 INFO:tasks.workunit.client.1.smithi192.stdout:6/246: write da/df/d12/d15/f2d [659779,40711] 0 2024-02-16T22:56:12.899 INFO:tasks.workunit.client.0.smithi064.stdout:0/133: write da/dc/f1e [728208,69017] 0 2024-02-16T22:56:12.900 INFO:tasks.workunit.client.0.smithi064.stdout:0/134: fdatasync da/dc/de/f17 0 2024-02-16T22:56:12.909 INFO:tasks.workunit.client.1.smithi192.stdout:1/209: getdents d3/de 0 2024-02-16T22:56:12.910 INFO:tasks.workunit.client.0.smithi064.stdout:5/132: creat d6/d18/d19/d1a/f30 x:0 0 0 2024-02-16T22:56:12.911 INFO:tasks.workunit.client.0.smithi064.stdout:0/135: dwrite da/dc/f1e [0,4194304] 0 2024-02-16T22:56:12.913 INFO:tasks.workunit.client.0.smithi064.stdout:1/136: rename d2/df/d11/f35 to d2/f37 0 2024-02-16T22:56:12.916 INFO:tasks.workunit.client.0.smithi064.stdout:1/137: truncate d2/df/d14/f2d 246513 0 2024-02-16T22:56:12.918 INFO:tasks.workunit.client.0.smithi064.stdout:2/146: creat d7/d14/d1e/f28 x:0 0 0 2024-02-16T22:56:12.919 INFO:tasks.workunit.client.0.smithi064.stdout:2/147: chown d7/d14/l1b 16263 1 2024-02-16T22:56:12.920 INFO:tasks.workunit.client.0.smithi064.stdout:0/136: dwrite da/f10 [0,4194304] 0 2024-02-16T22:56:12.920 INFO:tasks.workunit.client.1.smithi192.stdout:4/256: rename d1/d15/d48/f56 to d1/d15/d23/f5c 0 2024-02-16T22:56:12.920 INFO:tasks.workunit.client.1.smithi192.stdout:4/257: readlink d1/d8/d11/l33 0 2024-02-16T22:56:12.928 INFO:tasks.workunit.client.0.smithi064.stdout:9/166: rmdir d3/d11/d2b/d2a 39 2024-02-16T22:56:12.930 INFO:tasks.workunit.client.0.smithi064.stdout:4/126: creat dc/d10/d18/d22/f24 x:0 0 0 2024-02-16T22:56:12.931 INFO:tasks.workunit.client.0.smithi064.stdout:4/127: chown dc/d10/d18/d22/f24 38028 1 2024-02-16T22:56:12.937 INFO:tasks.workunit.client.1.smithi192.stdout:1/210: fdatasync d3/de/d27/f39 0 2024-02-16T22:56:12.938 INFO:tasks.workunit.client.0.smithi064.stdout:6/183: rename d0/d6/d19 to d0/d6/d10/d33 0 2024-02-16T22:56:12.938 INFO:tasks.workunit.client.1.smithi192.stdout:1/211: write d3/de/f1b [963975,23156] 0 2024-02-16T22:56:12.940 INFO:tasks.workunit.client.1.smithi192.stdout:2/259: link d22/d24/d2b/l35 d22/d24/d39/d42/d52/l57 0 2024-02-16T22:56:12.941 INFO:tasks.workunit.client.0.smithi064.stdout:4/128: dwrite dc/d10/f12 [0,4194304] 0 2024-02-16T22:56:12.944 INFO:tasks.workunit.client.1.smithi192.stdout:3/240: mknod da/d11/c5a 0 2024-02-16T22:56:12.951 INFO:tasks.workunit.client.0.smithi064.stdout:6/184: symlink d0/d6/d10/l34 0 2024-02-16T22:56:12.952 INFO:tasks.workunit.client.0.smithi064.stdout:6/185: dread - d0/d5/f27 zero size 2024-02-16T22:56:12.952 INFO:tasks.workunit.client.0.smithi064.stdout:6/186: fsync d0/d6/d10/f14 0 2024-02-16T22:56:12.953 INFO:tasks.workunit.client.1.smithi192.stdout:2/260: dread fa [0,4194304] 0 2024-02-16T22:56:12.954 INFO:tasks.workunit.client.0.smithi064.stdout:9/167: mkdir d3/d11/d2b/d2a/d2e/d31/d3f 0 2024-02-16T22:56:12.954 INFO:tasks.workunit.client.0.smithi064.stdout:6/187: chown d0/d6/d10/d33/f32 9837800 1 2024-02-16T22:56:12.956 INFO:tasks.workunit.client.0.smithi064.stdout:4/129: mknod dc/d10/d18/c25 0 2024-02-16T22:56:12.959 INFO:tasks.workunit.client.0.smithi064.stdout:5/133: creat d6/f31 x:0 0 0 2024-02-16T22:56:12.961 INFO:tasks.workunit.client.1.smithi192.stdout:4/258: creat d1/d8/d11/d58/f5d x:0 0 0 2024-02-16T22:56:12.962 INFO:tasks.workunit.client.1.smithi192.stdout:1/212: dwrite d3/f19 [8388608,4194304] 0 2024-02-16T22:56:12.964 INFO:tasks.workunit.client.0.smithi064.stdout:9/168: dwrite d3/d9/f3c [0,4194304] 0 2024-02-16T22:56:12.965 INFO:tasks.workunit.client.0.smithi064.stdout:6/188: dwrite d0/d5/f27 [0,4194304] 0 2024-02-16T22:56:12.971 INFO:tasks.workunit.client.1.smithi192.stdout:0/216: creat d1/f3e x:0 0 0 2024-02-16T22:56:12.974 INFO:tasks.workunit.client.0.smithi064.stdout:3/211: dwrite d4/f18 [0,4194304] 0 2024-02-16T22:56:12.977 INFO:tasks.workunit.client.0.smithi064.stdout:7/139: link d5/l14 d5/l21 0 2024-02-16T22:56:12.982 INFO:tasks.workunit.client.0.smithi064.stdout:7/140: write d5/d6/f18 [2292006,89647] 0 2024-02-16T22:56:12.983 INFO:tasks.workunit.client.1.smithi192.stdout:7/291: truncate d1/d2/de/f24 762484 0 2024-02-16T22:56:12.986 INFO:tasks.workunit.client.1.smithi192.stdout:6/247: creat da/df/d12/f47 x:0 0 0 2024-02-16T22:56:12.986 INFO:tasks.workunit.client.0.smithi064.stdout:6/189: dwrite d0/d6/d10/f1c [0,4194304] 0 2024-02-16T22:56:12.987 INFO:tasks.workunit.client.0.smithi064.stdout:4/130: symlink dc/d10/d18/l26 0 2024-02-16T22:56:12.988 INFO:tasks.workunit.client.1.smithi192.stdout:9/328: getdents d7/d19/d37/d5c 0 2024-02-16T22:56:12.990 INFO:tasks.workunit.client.0.smithi064.stdout:3/212: dwrite d4/d14/f22 [0,4194304] 0 2024-02-16T22:56:12.992 INFO:tasks.workunit.client.0.smithi064.stdout:1/138: getdents d2/df/d14/d18 0 2024-02-16T22:56:12.995 INFO:tasks.workunit.client.1.smithi192.stdout:8/228: rename d5/db/dd to d5/d3e 0 2024-02-16T22:56:12.996 INFO:tasks.workunit.client.0.smithi064.stdout:5/134: creat d6/d25/f32 x:0 0 0 2024-02-16T22:56:12.997 INFO:tasks.workunit.client.1.smithi192.stdout:4/259: mkdir d1/d26/d5e 0 2024-02-16T22:56:12.998 INFO:tasks.workunit.client.0.smithi064.stdout:2/148: getdents d7/de 0 2024-02-16T22:56:12.998 INFO:tasks.workunit.client.0.smithi064.stdout:2/149: readlink l3 0 2024-02-16T22:56:13.002 INFO:tasks.workunit.client.0.smithi064.stdout:9/169: mknod d3/d11/d2b/d2a/d2e/c40 0 2024-02-16T22:56:13.005 INFO:tasks.workunit.client.1.smithi192.stdout:0/217: creat d1/d3/d29/f3f x:0 0 0 2024-02-16T22:56:13.006 INFO:tasks.workunit.client.1.smithi192.stdout:5/260: getdents d6/d12/d15 0 2024-02-16T22:56:13.007 INFO:tasks.workunit.client.1.smithi192.stdout:3/241: creat da/d11/d20/d39/d53/f5b x:0 0 0 2024-02-16T22:56:13.009 INFO:tasks.workunit.client.1.smithi192.stdout:7/292: mknod d1/d1f/c65 0 2024-02-16T22:56:13.010 INFO:tasks.workunit.client.0.smithi064.stdout:7/141: creat d5/d6/f22 x:0 0 0 2024-02-16T22:56:13.012 INFO:tasks.workunit.client.0.smithi064.stdout:6/190: fsync d0/d1b/f23 0 2024-02-16T22:56:13.012 INFO:tasks.workunit.client.0.smithi064.stdout:6/191: chown d0/f22 2662914 1 2024-02-16T22:56:13.013 INFO:tasks.workunit.client.1.smithi192.stdout:9/329: creat d7/d15/f6d x:0 0 0 2024-02-16T22:56:13.014 INFO:tasks.workunit.client.0.smithi064.stdout:4/131: rename dc/d10/d18/c21 to dc/c27 0 2024-02-16T22:56:13.015 INFO:tasks.workunit.client.0.smithi064.stdout:4/132: readlink dc/d10/d18/l26 0 2024-02-16T22:56:13.015 INFO:tasks.workunit.client.1.smithi192.stdout:7/293: dwrite d1/d19/f1d [4194304,4194304] 0 2024-02-16T22:56:13.019 INFO:tasks.workunit.client.1.smithi192.stdout:8/229: creat d5/d26/f3f x:0 0 0 2024-02-16T22:56:13.020 INFO:tasks.workunit.client.1.smithi192.stdout:8/230: write d5/db/de/df/f2b [2590222,40962] 0 2024-02-16T22:56:13.021 INFO:tasks.workunit.client.1.smithi192.stdout:8/231: write d5/d3e/d39/f3b [1047270,66464] 0 2024-02-16T22:56:13.024 INFO:tasks.workunit.client.0.smithi064.stdout:4/133: dwrite dc/dd/d16/f23 [0,4194304] 0 2024-02-16T22:56:13.026 INFO:tasks.workunit.client.0.smithi064.stdout:4/134: chown l9 11101827 1 2024-02-16T22:56:13.026 INFO:tasks.workunit.client.0.smithi064.stdout:2/150: symlink d7/de/l29 0 2024-02-16T22:56:13.030 INFO:tasks.workunit.client.0.smithi064.stdout:1/139: dread d2/df/d11/f27 [0,4194304] 0 2024-02-16T22:56:13.033 INFO:tasks.workunit.client.0.smithi064.stdout:4/135: dwrite dc/f19 [0,4194304] 0 2024-02-16T22:56:13.036 INFO:tasks.workunit.client.1.smithi192.stdout:3/242: mknod da/d11/d2b/d43/c5c 0 2024-02-16T22:56:13.041 INFO:tasks.workunit.client.1.smithi192.stdout:6/248: symlink da/df/d18/d46/l48 0 2024-02-16T22:56:13.043 INFO:tasks.workunit.client.1.smithi192.stdout:9/330: mkdir d7/df/d6e 0 2024-02-16T22:56:13.045 INFO:tasks.workunit.client.1.smithi192.stdout:3/243: dwrite da/df/f2e [0,4194304] 0 2024-02-16T22:56:13.046 INFO:tasks.workunit.client.1.smithi192.stdout:3/244: dread - da/d11/f54 zero size 2024-02-16T22:56:13.047 INFO:tasks.workunit.client.1.smithi192.stdout:2/261: rename c13 to d22/d24/d2b/d3e/d4f/c58 0 2024-02-16T22:56:13.050 INFO:tasks.workunit.client.1.smithi192.stdout:7/294: mknod d1/d2/d2d/c66 0 2024-02-16T22:56:13.053 INFO:tasks.workunit.client.1.smithi192.stdout:0/218: write d1/d3/de/f10 [3453970,88106] 0 2024-02-16T22:56:13.056 INFO:tasks.workunit.client.1.smithi192.stdout:7/295: dread d1/d1f/f4f [0,4194304] 0 2024-02-16T22:56:13.057 INFO:tasks.workunit.client.0.smithi064.stdout:9/170: link d3/d11/d2b/f27 d3/d9/f41 0 2024-02-16T22:56:13.058 INFO:tasks.workunit.client.0.smithi064.stdout:9/171: truncate d3/d11/d2b/f29 1660319 0 2024-02-16T22:56:13.060 INFO:tasks.workunit.client.0.smithi064.stdout:1/140: symlink d2/df/d11/d25/l38 0 2024-02-16T22:56:13.060 INFO:tasks.workunit.client.0.smithi064.stdout:6/192: write d0/f22 [1601726,3577] 0 2024-02-16T22:56:13.063 INFO:tasks.workunit.client.0.smithi064.stdout:9/172: dread d3/d11/f28 [0,4194304] 0 2024-02-16T22:56:13.064 INFO:tasks.workunit.client.1.smithi192.stdout:1/213: rmdir d3/de/d3e 0 2024-02-16T22:56:13.067 INFO:tasks.workunit.client.1.smithi192.stdout:1/214: chown d3/de/c18 12 1 2024-02-16T22:56:13.071 INFO:tasks.workunit.client.0.smithi064.stdout:5/135: creat d6/f33 x:0 0 0 2024-02-16T22:56:13.072 INFO:tasks.workunit.client.0.smithi064.stdout:5/136: chown d6/ld 745 1 2024-02-16T22:56:13.073 INFO:tasks.workunit.client.0.smithi064.stdout:5/137: fdatasync d6/ff 0 2024-02-16T22:56:13.074 INFO:tasks.workunit.client.0.smithi064.stdout:1/141: unlink d2/f9 0 2024-02-16T22:56:13.075 INFO:tasks.workunit.client.0.smithi064.stdout:2/151: dwrite d7/f13 [0,4194304] 0 2024-02-16T22:56:13.076 INFO:tasks.workunit.client.0.smithi064.stdout:2/152: chown l4 270660055 1 2024-02-16T22:56:13.077 INFO:tasks.workunit.client.0.smithi064.stdout:6/193: unlink d0/c24 0 2024-02-16T22:56:13.085 INFO:tasks.workunit.client.0.smithi064.stdout:9/173: unlink d3/d9/d1e/l22 0 2024-02-16T22:56:13.089 INFO:tasks.workunit.client.1.smithi192.stdout:3/245: write da/f1a [460527,635] 0 2024-02-16T22:56:13.092 INFO:tasks.workunit.client.1.smithi192.stdout:0/219: symlink d1/d3/de/l40 0 2024-02-16T22:56:13.093 INFO:tasks.workunit.client.0.smithi064.stdout:3/213: getdents d4/d13/d35 0 2024-02-16T22:56:13.095 INFO:tasks.workunit.client.1.smithi192.stdout:3/246: dwrite f9 [0,4194304] 0 2024-02-16T22:56:13.096 INFO:tasks.workunit.client.0.smithi064.stdout:4/136: truncate dc/dd/d16/f23 1834784 0 2024-02-16T22:56:13.097 INFO:tasks.workunit.client.1.smithi192.stdout:4/260: link d1/d26/d34/l47 d1/d8/d11/l5f 0 2024-02-16T22:56:13.105 INFO:tasks.workunit.client.0.smithi064.stdout:4/137: dwrite dc/f19 [0,4194304] 0 2024-02-16T22:56:13.107 INFO:tasks.workunit.client.0.smithi064.stdout:7/142: getdents d5/d6 0 2024-02-16T22:56:13.108 INFO:tasks.workunit.client.0.smithi064.stdout:7/143: chown d5/d6/f22 1677 1 2024-02-16T22:56:13.110 INFO:tasks.workunit.client.0.smithi064.stdout:4/138: read dc/f17 [206253,48908] 0 2024-02-16T22:56:13.110 INFO:tasks.workunit.client.0.smithi064.stdout:9/174: mkdir d3/d9/d1e/d42 0 2024-02-16T22:56:13.110 INFO:tasks.workunit.client.0.smithi064.stdout:1/142: write d2/df/d11/f27 [2487631,73515] 0 2024-02-16T22:56:13.111 INFO:tasks.workunit.client.0.smithi064.stdout:9/175: fdatasync d3/d9/f3c 0 2024-02-16T22:56:13.111 INFO:tasks.workunit.client.1.smithi192.stdout:9/331: mkdir d7/d19/d5e/d6f 0 2024-02-16T22:56:13.112 INFO:tasks.workunit.client.0.smithi064.stdout:9/176: stat d3/d11/d15/d23 0 2024-02-16T22:56:13.112 INFO:tasks.workunit.client.0.smithi064.stdout:3/214: rename d4/d13/l29 to d4/d13/d1b/d27/l47 0 2024-02-16T22:56:13.113 INFO:tasks.workunit.client.1.smithi192.stdout:9/332: readlink d7/df/l1f 0 2024-02-16T22:56:13.114 INFO:tasks.workunit.client.1.smithi192.stdout:6/249: truncate da/df/d12/d15/f31 3418809 0 2024-02-16T22:56:13.116 INFO:tasks.workunit.client.1.smithi192.stdout:2/262: symlink d22/d24/d39/d42/d52/l59 0 2024-02-16T22:56:13.117 INFO:tasks.workunit.client.1.smithi192.stdout:0/220: rmdir d1/d3/d29 39 2024-02-16T22:56:13.119 INFO:tasks.workunit.client.1.smithi192.stdout:1/215: dwrite d3/de/d27/f30 [0,4194304] 0 2024-02-16T22:56:13.122 INFO:tasks.workunit.client.1.smithi192.stdout:7/296: rename d1/d2/f3b to d1/d2/de/d2c/d49/d54/f67 0 2024-02-16T22:56:13.122 INFO:tasks.workunit.client.0.smithi064.stdout:2/153: dwrite d7/de/f1f [0,4194304] 0 2024-02-16T22:56:13.124 INFO:tasks.workunit.client.1.smithi192.stdout:9/333: dwrite d7/d19/d37/d5a/f63 [0,4194304] 0 2024-02-16T22:56:13.124 INFO:tasks.workunit.client.0.smithi064.stdout:2/154: write d7/f1a [4069932,93146] 0 2024-02-16T22:56:13.127 INFO:tasks.workunit.client.1.smithi192.stdout:3/247: creat da/d11/d20/d39/f5d x:0 0 0 2024-02-16T22:56:13.130 INFO:tasks.workunit.client.1.smithi192.stdout:4/261: creat d1/d26/d34/f60 x:0 0 0 2024-02-16T22:56:13.133 INFO:tasks.workunit.client.0.smithi064.stdout:4/139: mkdir dc/de/d28 0 2024-02-16T22:56:13.135 INFO:tasks.workunit.client.0.smithi064.stdout:2/155: dwrite d7/d14/d1e/f28 [0,4194304] 0 2024-02-16T22:56:13.136 INFO:tasks.workunit.client.0.smithi064.stdout:6/194: truncate d0/f22 1386718 0 2024-02-16T22:56:13.136 INFO:tasks.workunit.client.1.smithi192.stdout:5/261: getdents d6/d12/d1d/d3c 0 2024-02-16T22:56:13.139 INFO:tasks.workunit.client.0.smithi064.stdout:1/143: creat d2/df/d14/f39 x:0 0 0 2024-02-16T22:56:13.143 INFO:tasks.workunit.client.0.smithi064.stdout:3/215: mkdir d4/d13/d1b/d48 0 2024-02-16T22:56:13.146 INFO:tasks.workunit.client.0.smithi064.stdout:4/140: dwrite dc/d10/d18/d22/f24 [0,4194304] 0 2024-02-16T22:56:13.148 INFO:tasks.workunit.client.0.smithi064.stdout:6/195: dread d0/d6/d10/f14 [0,4194304] 0 2024-02-16T22:56:13.155 INFO:tasks.workunit.client.0.smithi064.stdout:5/138: creat d6/f34 x:0 0 0 2024-02-16T22:56:13.157 INFO:tasks.workunit.client.0.smithi064.stdout:5/139: fsync d6/d18/d19/d1a/f30 0 2024-02-16T22:56:13.158 INFO:tasks.workunit.client.0.smithi064.stdout:6/196: dwrite d0/d6/d10/f1c [0,4194304] 0 2024-02-16T22:56:13.159 INFO:tasks.workunit.client.0.smithi064.stdout:5/140: chown d6/d18/d19/d1a 2 1 2024-02-16T22:56:13.159 INFO:tasks.workunit.client.1.smithi192.stdout:0/221: rename d1/d2/l2e to d1/d3/d18/l41 0 2024-02-16T22:56:13.162 INFO:tasks.workunit.client.1.smithi192.stdout:2/263: write f4 [902601,88445] 0 2024-02-16T22:56:13.164 INFO:tasks.workunit.client.1.smithi192.stdout:7/297: creat d1/d2/d37/f68 x:0 0 0 2024-02-16T22:56:13.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/248: creat da/d11/d2b/f5e x:0 0 0 2024-02-16T22:56:13.166 INFO:tasks.workunit.client.1.smithi192.stdout:5/262: dread d6/d12/f1e [0,4194304] 0 2024-02-16T22:56:13.167 INFO:tasks.workunit.client.0.smithi064.stdout:5/141: dwrite d6/ff [0,4194304] 0 2024-02-16T22:56:13.168 INFO:tasks.workunit.client.0.smithi064.stdout:5/142: stat d6/f24 0 2024-02-16T22:56:13.168 INFO:tasks.workunit.client.1.smithi192.stdout:8/232: getdents d5 0 2024-02-16T22:56:13.170 INFO:tasks.workunit.client.1.smithi192.stdout:5/263: readlink d6/d12/d2d/l3f 0 2024-02-16T22:56:13.173 INFO:tasks.workunit.client.0.smithi064.stdout:5/143: dread d6/f17 [0,4194304] 0 2024-02-16T22:56:13.174 INFO:tasks.workunit.client.0.smithi064.stdout:5/144: fsync d6/d1c/f20 0 2024-02-16T22:56:13.174 INFO:tasks.workunit.client.1.smithi192.stdout:6/250: symlink da/df/d12/l49 0 2024-02-16T22:56:13.175 INFO:tasks.workunit.client.1.smithi192.stdout:3/249: dwrite da/d11/d20/f30 [0,4194304] 0 2024-02-16T22:56:13.175 INFO:tasks.workunit.client.0.smithi064.stdout:5/145: write d6/d18/d19/d1a/f30 [724647,104913] 0 2024-02-16T22:56:13.176 INFO:tasks.workunit.client.1.smithi192.stdout:3/250: stat da/d11/f2f 0 2024-02-16T22:56:13.180 INFO:tasks.workunit.client.0.smithi064.stdout:8/149: sync 2024-02-16T22:56:13.181 INFO:tasks.workunit.client.1.smithi192.stdout:1/216: rename d3/f26 to d3/de/d27/f44 0 2024-02-16T22:56:13.183 INFO:tasks.workunit.client.1.smithi192.stdout:0/222: creat d1/d2/f42 x:0 0 0 2024-02-16T22:56:13.185 INFO:tasks.workunit.client.0.smithi064.stdout:9/177: truncate d3/d9/f36 141803 0 2024-02-16T22:56:13.187 INFO:tasks.workunit.client.0.smithi064.stdout:8/150: dwrite f8 [0,4194304] 0 2024-02-16T22:56:13.188 INFO:tasks.workunit.client.1.smithi192.stdout:9/334: fsync d7/f8 0 2024-02-16T22:56:13.192 INFO:tasks.workunit.client.0.smithi064.stdout:7/144: link d5/d6/fb d5/f23 0 2024-02-16T22:56:13.193 INFO:tasks.workunit.client.1.smithi192.stdout:8/233: symlink d5/d3e/d39/l40 0 2024-02-16T22:56:13.193 INFO:tasks.workunit.client.0.smithi064.stdout:2/156: mknod d7/de/c2a 0 2024-02-16T22:56:13.195 INFO:tasks.workunit.client.1.smithi192.stdout:5/264: unlink f2 0 2024-02-16T22:56:13.195 INFO:tasks.workunit.client.1.smithi192.stdout:5/265: chown d6/d8 149642931 1 2024-02-16T22:56:13.196 INFO:tasks.workunit.client.1.smithi192.stdout:5/266: fsync d6/d20/f4d 0 2024-02-16T22:56:13.196 INFO:tasks.workunit.client.1.smithi192.stdout:5/267: truncate d6/f33 410167 0 2024-02-16T22:56:13.197 INFO:tasks.workunit.client.1.smithi192.stdout:4/262: rename d1/d26/f37 to d1/d15/d29/f61 0 2024-02-16T22:56:13.199 INFO:tasks.workunit.client.1.smithi192.stdout:1/217: creat d3/de/d27/d34/f45 x:0 0 0 2024-02-16T22:56:13.200 INFO:tasks.workunit.client.1.smithi192.stdout:4/263: write d1/d15/d23/f4a [167232,113022] 0 2024-02-16T22:56:13.202 INFO:tasks.workunit.client.1.smithi192.stdout:9/335: mkdir d7/d19/d28/d43/d70 0 2024-02-16T22:56:13.203 INFO:tasks.workunit.client.1.smithi192.stdout:4/264: fdatasync d1/d8/fc 0 2024-02-16T22:56:13.204 INFO:tasks.workunit.client.1.smithi192.stdout:7/298: mkdir d1/d69 0 2024-02-16T22:56:13.206 INFO:tasks.workunit.client.1.smithi192.stdout:5/268: dwrite d6/f33 [0,4194304] 0 2024-02-16T22:56:13.206 INFO:tasks.workunit.client.1.smithi192.stdout:4/265: chown d1/f35 313765 1 2024-02-16T22:56:13.206 INFO:tasks.workunit.client.1.smithi192.stdout:2/264: link d22/d24/c2f d22/d24/d2b/d3e/c5a 0 2024-02-16T22:56:13.207 INFO:tasks.workunit.client.1.smithi192.stdout:5/269: readlink d6/d12/l2a 0 2024-02-16T22:56:13.208 INFO:tasks.workunit.client.1.smithi192.stdout:8/234: link d5/d3e/d39/f3b d5/d3e/f41 0 2024-02-16T22:56:13.211 INFO:tasks.workunit.client.1.smithi192.stdout:4/266: write d1/d8/d11/f4e [4104373,84078] 0 2024-02-16T22:56:13.212 INFO:tasks.workunit.client.1.smithi192.stdout:0/223: dwrite d1/d3/d29/f3f [0,4194304] 0 2024-02-16T22:56:13.214 INFO:tasks.workunit.client.1.smithi192.stdout:1/218: creat d3/f46 x:0 0 0 2024-02-16T22:56:13.214 INFO:tasks.workunit.client.1.smithi192.stdout:9/336: symlink d7/d19/d28/d43/d70/l71 0 2024-02-16T22:56:13.215 INFO:tasks.workunit.client.1.smithi192.stdout:0/224: chown d1/d3/de/d22/d37 3724765 1 2024-02-16T22:56:13.217 INFO:tasks.workunit.client.1.smithi192.stdout:5/270: rename d6/d12/d15 to d6/d12/d4e 0 2024-02-16T22:56:13.217 INFO:tasks.workunit.client.1.smithi192.stdout:8/235: symlink d5/d26/l42 0 2024-02-16T22:56:13.217 INFO:tasks.workunit.client.1.smithi192.stdout:9/337: read d7/d19/d28/d43/f4d [1070965,45387] 0 2024-02-16T22:56:13.218 INFO:tasks.workunit.client.1.smithi192.stdout:2/265: rename d22/d24/c50 to d22/d24/d2b/d3e/d4f/c5b 0 2024-02-16T22:56:13.219 INFO:tasks.workunit.client.1.smithi192.stdout:9/338: dread - d7/d19/d28/f53 zero size 2024-02-16T22:56:13.221 INFO:tasks.workunit.client.1.smithi192.stdout:2/266: fdatasync d22/d24/d2b/d2c/f37 0 2024-02-16T22:56:13.221 INFO:tasks.workunit.client.1.smithi192.stdout:1/219: mkdir d3/de/d27/d34/d36/d47 0 2024-02-16T22:56:13.224 INFO:tasks.workunit.client.1.smithi192.stdout:0/225: dwrite d1/f2a [0,4194304] 0 2024-02-16T22:56:13.224 INFO:tasks.workunit.client.1.smithi192.stdout:2/267: truncate d22/d24/f49 769310 0 2024-02-16T22:56:13.226 INFO:tasks.workunit.client.1.smithi192.stdout:2/268: dread - d22/f3c zero size 2024-02-16T22:56:13.226 INFO:tasks.workunit.client.1.smithi192.stdout:0/226: chown d1/d3/d18 936 1 2024-02-16T22:56:13.230 INFO:tasks.workunit.client.1.smithi192.stdout:7/299: dread d1/d2/de/f24 [0,4194304] 0 2024-02-16T22:56:13.231 INFO:tasks.workunit.client.1.smithi192.stdout:0/227: mkdir d1/d3/d29/d43 0 2024-02-16T22:56:13.231 INFO:tasks.workunit.client.1.smithi192.stdout:2/269: write d22/d24/d2b/d2c/d3f/f4e [614258,18461] 0 2024-02-16T22:56:13.233 INFO:tasks.workunit.client.1.smithi192.stdout:1/220: dwrite d3/f19 [12582912,4194304] 0 2024-02-16T22:56:13.234 INFO:tasks.workunit.client.1.smithi192.stdout:7/300: fsync d1/d2/de/f24 0 2024-02-16T22:56:13.235 INFO:tasks.workunit.client.1.smithi192.stdout:7/301: stat d1/d2/c39 0 2024-02-16T22:56:13.237 INFO:tasks.workunit.client.1.smithi192.stdout:1/221: chown d3/de/d27/d34/l41 0 1 2024-02-16T22:56:13.249 INFO:tasks.workunit.client.0.smithi064.stdout:8/151: creat df/d22/f2b x:0 0 0 2024-02-16T22:56:13.249 INFO:tasks.workunit.client.0.smithi064.stdout:6/197: mknod d0/d1d/d21/c35 0 2024-02-16T22:56:13.251 INFO:tasks.workunit.client.0.smithi064.stdout:2/157: symlink d7/d14/d1e/l2b 0 2024-02-16T22:56:13.251 INFO:tasks.workunit.client.0.smithi064.stdout:4/141: dread f5 [4194304,4194304] 0 2024-02-16T22:56:13.253 INFO:tasks.workunit.client.1.smithi192.stdout:6/251: dread da/df/d12/d15/f2d [0,4194304] 0 2024-02-16T22:56:13.257 INFO:tasks.workunit.client.0.smithi064.stdout:6/198: creat d0/d1b/f36 x:0 0 0 2024-02-16T22:56:13.259 INFO:tasks.workunit.client.1.smithi192.stdout:6/252: creat da/df/d18/d3f/f4a x:0 0 0 2024-02-16T22:56:13.260 INFO:tasks.workunit.client.0.smithi064.stdout:7/145: mknod d5/c24 0 2024-02-16T22:56:13.262 INFO:tasks.workunit.client.0.smithi064.stdout:7/146: chown d5/l1a 11189928 1 2024-02-16T22:56:13.264 INFO:tasks.workunit.client.0.smithi064.stdout:4/142: rename f5 to dc/d1d/f29 0 2024-02-16T22:56:13.264 INFO:tasks.workunit.client.0.smithi064.stdout:7/147: dread - d5/d6/f19 zero size 2024-02-16T22:56:13.265 INFO:tasks.workunit.client.1.smithi192.stdout:6/253: rmdir da/df/d12/d15/d1b 39 2024-02-16T22:56:13.265 INFO:tasks.workunit.client.0.smithi064.stdout:1/144: getdents d2/df/d11/d1a 0 2024-02-16T22:56:13.266 INFO:tasks.workunit.client.0.smithi064.stdout:9/178: link d3/d9/ff d3/d11/d2b/d2a/d2e/d31/d3f/f43 0 2024-02-16T22:56:13.266 INFO:tasks.workunit.client.0.smithi064.stdout:6/199: dwrite d0/d6/d10/d33/f32 [0,4194304] 0 2024-02-16T22:56:13.268 INFO:tasks.workunit.client.1.smithi192.stdout:6/254: creat da/d27/f4b x:0 0 0 2024-02-16T22:56:13.270 INFO:tasks.workunit.client.0.smithi064.stdout:8/152: link f8 df/d2a/f2c 0 2024-02-16T22:56:13.272 INFO:tasks.workunit.client.1.smithi192.stdout:6/255: rename da/df/f1d to da/d40/f4c 0 2024-02-16T22:56:13.274 INFO:tasks.workunit.client.1.smithi192.stdout:6/256: read da/fb [1733234,117559] 0 2024-02-16T22:56:13.275 INFO:tasks.workunit.client.0.smithi064.stdout:1/145: dwrite d2/df/d11/d1a/d1d/f24 [0,4194304] 0 2024-02-16T22:56:13.275 INFO:tasks.workunit.client.1.smithi192.stdout:6/257: readlink da/l14 0 2024-02-16T22:56:13.277 INFO:tasks.workunit.client.0.smithi064.stdout:6/200: dread d0/d6/d10/f1c [4194304,4194304] 0 2024-02-16T22:56:13.277 INFO:tasks.workunit.client.0.smithi064.stdout:4/143: symlink dc/de/l2a 0 2024-02-16T22:56:13.280 INFO:tasks.workunit.client.1.smithi192.stdout:6/258: mknod da/df/d12/d15/d1b/c4d 0 2024-02-16T22:56:13.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:12 smithi192 ceph-mon[36254]: pgmap v11: 65 pgs: 65 active+clean; 391 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 776 KiB/s rd, 27 MiB/s wr, 238 op/s 2024-02-16T22:56:13.285 INFO:tasks.workunit.client.1.smithi192.stdout:6/259: write da/df/d12/f39 [132531,96517] 0 2024-02-16T22:56:13.285 INFO:tasks.workunit.client.0.smithi064.stdout:4/144: dwrite dc/f19 [4194304,4194304] 0 2024-02-16T22:56:13.285 INFO:tasks.workunit.client.0.smithi064.stdout:6/201: mkdir d0/d1d/d21/d37 0 2024-02-16T22:56:13.287 INFO:tasks.workunit.client.0.smithi064.stdout:6/202: write d0/d6/da/f25 [672621,124978] 0 2024-02-16T22:56:13.290 INFO:tasks.workunit.client.0.smithi064.stdout:8/153: link df/d22/f2b df/d2a/f2d 0 2024-02-16T22:56:13.291 INFO:tasks.workunit.client.0.smithi064.stdout:7/148: creat d5/f25 x:0 0 0 2024-02-16T22:56:13.291 INFO:tasks.workunit.client.0.smithi064.stdout:1/146: symlink d2/df/d14/d2a/l3a 0 2024-02-16T22:56:13.292 INFO:tasks.workunit.client.0.smithi064.stdout:9/179: link d3/l13 d3/d9/d1e/d3e/l44 0 2024-02-16T22:56:13.292 INFO:tasks.workunit.client.1.smithi192.stdout:6/260: dwrite da/f34 [0,4194304] 0 2024-02-16T22:56:13.292 INFO:tasks.workunit.client.0.smithi064.stdout:8/154: chown df/l10 984333776 1 2024-02-16T22:56:13.293 INFO:tasks.workunit.client.0.smithi064.stdout:8/155: dread - df/d22/f2b zero size 2024-02-16T22:56:13.294 INFO:tasks.workunit.client.0.smithi064.stdout:9/180: chown d3/d11/d2b/d2a/d2e/c40 1878857444 1 2024-02-16T22:56:13.306 INFO:tasks.workunit.client.0.smithi064.stdout:1/147: rename d2/f17 to d2/df/d11/f3b 0 2024-02-16T22:56:13.310 INFO:tasks.workunit.client.0.smithi064.stdout:1/148: dread d2/df/d14/d18/f28 [0,4194304] 0 2024-02-16T22:56:13.311 INFO:tasks.workunit.client.0.smithi064.stdout:8/156: rename df/c1f to df/d2a/c2e 0 2024-02-16T22:56:13.312 INFO:tasks.workunit.client.0.smithi064.stdout:9/181: mknod d3/d11/d2b/d2a/d2e/d31/d3f/c45 0 2024-02-16T22:56:13.315 INFO:tasks.workunit.client.0.smithi064.stdout:7/149: creat d5/f26 x:0 0 0 2024-02-16T22:56:13.315 INFO:tasks.workunit.client.0.smithi064.stdout:7/150: chown d5 1754 1 2024-02-16T22:56:13.318 INFO:tasks.workunit.client.0.smithi064.stdout:1/149: rename d2/f37 to d2/df/d14/d2c/f3c 0 2024-02-16T22:56:13.320 INFO:tasks.workunit.client.0.smithi064.stdout:1/150: fdatasync d2/df/d11/d1a/d1d/f24 0 2024-02-16T22:56:13.321 INFO:tasks.workunit.client.0.smithi064.stdout:9/182: chown d3/c5 1492877 1 2024-02-16T22:56:13.321 INFO:tasks.workunit.client.0.smithi064.stdout:1/151: chown d2/df/d11/d1a/d1d/l2b 11384 1 2024-02-16T22:56:13.321 INFO:tasks.workunit.client.0.smithi064.stdout:9/183: fsync d3/fc 0 2024-02-16T22:56:13.322 INFO:tasks.workunit.client.0.smithi064.stdout:1/152: chown d2/df/d11/d25/l38 1674854562 1 2024-02-16T22:56:13.329 INFO:tasks.workunit.client.0.smithi064.stdout:1/153: mkdir d2/df/d11/d1a/d30/d3d 0 2024-02-16T22:56:13.330 INFO:tasks.workunit.client.0.smithi064.stdout:1/154: chown d2/df/d14/d18/f26 10 1 2024-02-16T22:56:13.330 INFO:tasks.workunit.client.0.smithi064.stdout:7/151: creat d5/f27 x:0 0 0 2024-02-16T22:56:13.331 INFO:tasks.workunit.client.0.smithi064.stdout:1/155: chown l1 1011 1 2024-02-16T22:56:13.331 INFO:tasks.workunit.client.1.smithi192.stdout:4/267: write d1/f4 [1836073,47754] 0 2024-02-16T22:56:13.332 INFO:tasks.workunit.client.0.smithi064.stdout:7/152: rmdir d5/d6 39 2024-02-16T22:56:13.332 INFO:tasks.workunit.client.0.smithi064.stdout:1/156: write d2/df/d14/f16 [1728256,10305] 0 2024-02-16T22:56:13.332 INFO:tasks.workunit.client.1.smithi192.stdout:4/268: write d1/f4 [5854776,9670] 0 2024-02-16T22:56:13.333 INFO:tasks.workunit.client.1.smithi192.stdout:4/269: chown d1/d15/d29/c2c 32418244 1 2024-02-16T22:56:13.334 INFO:tasks.workunit.client.1.smithi192.stdout:4/270: chown d1/d26/c32 7299 1 2024-02-16T22:56:13.336 INFO:tasks.workunit.client.0.smithi064.stdout:1/157: dread d2/df/d11/d1a/d1d/f24 [0,4194304] 0 2024-02-16T22:56:13.337 INFO:tasks.workunit.client.1.smithi192.stdout:4/271: getdents d1/d26/d5e 0 2024-02-16T22:56:13.340 INFO:tasks.workunit.client.1.smithi192.stdout:8/236: write f2 [1239240,64713] 0 2024-02-16T22:56:13.340 INFO:tasks.workunit.client.1.smithi192.stdout:8/237: fsync d5/d16/f27 0 2024-02-16T22:56:13.342 INFO:tasks.workunit.client.0.smithi064.stdout:1/158: dwrite d2/df/d14/d18/f26 [0,4194304] 0 2024-02-16T22:56:13.342 INFO:tasks.workunit.client.1.smithi192.stdout:4/272: rename d1/c53 to d1/d8/d11/c62 0 2024-02-16T22:56:13.344 INFO:tasks.workunit.client.1.smithi192.stdout:7/302: write d1/d2/de/f28 [1457270,120670] 0 2024-02-16T22:56:13.348 INFO:tasks.workunit.client.1.smithi192.stdout:8/238: dwrite d5/f38 [0,4194304] 0 2024-02-16T22:56:13.351 INFO:tasks.workunit.client.1.smithi192.stdout:7/303: chown d1/d2/l50 5 1 2024-02-16T22:56:13.353 INFO:tasks.workunit.client.1.smithi192.stdout:8/239: mkdir d5/d3e/d39/d43 0 2024-02-16T22:56:13.354 INFO:tasks.workunit.client.1.smithi192.stdout:4/273: dwrite d1/f5 [4194304,4194304] 0 2024-02-16T22:56:13.355 INFO:tasks.workunit.client.0.smithi064.stdout:6/203: write d0/d6/d10/d33/f32 [4480330,45069] 0 2024-02-16T22:56:13.360 INFO:tasks.workunit.client.0.smithi064.stdout:6/204: dread d0/d5/f27 [0,4194304] 0 2024-02-16T22:56:13.362 INFO:tasks.workunit.client.1.smithi192.stdout:8/240: dwrite d5/db/de/df/f1b [0,4194304] 0 2024-02-16T22:56:13.366 INFO:tasks.workunit.client.1.smithi192.stdout:7/304: dread d1/d2/de/f34 [0,4194304] 0 2024-02-16T22:56:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:12 smithi064 ceph-mon[29226]: pgmap v11: 65 pgs: 65 active+clean; 391 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 776 KiB/s rd, 27 MiB/s wr, 238 op/s 2024-02-16T22:56:13.369 INFO:tasks.workunit.client.0.smithi064.stdout:8/157: rename df/d22 to df/d28/d2f 0 2024-02-16T22:56:13.372 INFO:tasks.workunit.client.1.smithi192.stdout:8/241: creat d5/d9/f44 x:0 0 0 2024-02-16T22:56:13.372 INFO:tasks.workunit.client.1.smithi192.stdout:8/242: chown d5/d9/f33 0 1 2024-02-16T22:56:13.375 INFO:tasks.workunit.client.1.smithi192.stdout:7/305: mkdir d1/d2/de/d2c/d49/d54/d6a 0 2024-02-16T22:56:13.377 INFO:tasks.workunit.client.1.smithi192.stdout:7/306: dread d1/d2/f25 [0,4194304] 0 2024-02-16T22:56:13.378 INFO:tasks.workunit.client.0.smithi064.stdout:7/153: rename d5/c24 to d5/d6/c28 0 2024-02-16T22:56:13.380 INFO:tasks.workunit.client.0.smithi064.stdout:7/154: read d5/d6/fb [680965,26549] 0 2024-02-16T22:56:13.380 INFO:tasks.workunit.client.1.smithi192.stdout:8/243: creat d5/d16/f45 x:0 0 0 2024-02-16T22:56:13.381 INFO:tasks.workunit.client.0.smithi064.stdout:4/145: dwrite dc/f17 [0,4194304] 0 2024-02-16T22:56:13.382 INFO:tasks.workunit.client.1.smithi192.stdout:3/251: sync 2024-02-16T22:56:13.383 INFO:tasks.workunit.client.1.smithi192.stdout:7/307: creat d1/d19/d4c/f6b x:0 0 0 2024-02-16T22:56:13.384 INFO:tasks.workunit.client.1.smithi192.stdout:3/252: fdatasync da/d11/f38 0 2024-02-16T22:56:13.384 INFO:tasks.workunit.client.1.smithi192.stdout:7/308: stat d1/d69 0 2024-02-16T22:56:13.386 INFO:tasks.workunit.client.1.smithi192.stdout:6/261: dwrite da/df/d12/f13 [0,4194304] 0 2024-02-16T22:56:13.388 INFO:tasks.workunit.client.1.smithi192.stdout:3/253: write da/d11/f38 [351199,64249] 0 2024-02-16T22:56:13.389 INFO:tasks.workunit.client.1.smithi192.stdout:6/262: stat da/df/d12/f47 0 2024-02-16T22:56:13.389 INFO:tasks.workunit.client.1.smithi192.stdout:3/254: chown da/d11/d35/f59 2030 1 2024-02-16T22:56:13.395 INFO:tasks.workunit.client.1.smithi192.stdout:7/309: mknod d1/d2/de/c6c 0 2024-02-16T22:56:13.399 INFO:tasks.workunit.client.1.smithi192.stdout:6/263: creat da/d40/f4e x:0 0 0 2024-02-16T22:56:13.401 INFO:tasks.workunit.client.0.smithi064.stdout:9/184: dwrite d3/d11/f1f [0,4194304] 0 2024-02-16T22:56:13.401 INFO:tasks.workunit.client.1.smithi192.stdout:3/255: creat da/d11/d2b/f5f x:0 0 0 2024-02-16T22:56:13.402 INFO:tasks.workunit.client.1.smithi192.stdout:3/256: read da/df/f1f [233168,24453] 0 2024-02-16T22:56:13.403 INFO:tasks.workunit.client.1.smithi192.stdout:7/310: chown d1/d2/f1b 91 1 2024-02-16T22:56:13.407 INFO:tasks.workunit.client.1.smithi192.stdout:7/311: rmdir d1/d2/d37 39 2024-02-16T22:56:13.412 INFO:tasks.workunit.client.1.smithi192.stdout:6/264: symlink da/l4f 0 2024-02-16T22:56:13.412 INFO:tasks.workunit.client.1.smithi192.stdout:3/257: link da/d11/f58 da/d11/d2b/d32/f60 0 2024-02-16T22:56:13.412 INFO:tasks.workunit.client.1.smithi192.stdout:4/274: truncate d1/d15/d23/f4a 55644 0 2024-02-16T22:56:13.412 INFO:tasks.workunit.client.1.smithi192.stdout:3/258: rename da/df/d42 to da/df/d42/d4e/d61 22 2024-02-16T22:56:13.412 INFO:tasks.workunit.client.1.smithi192.stdout:3/259: fsync da/d11/d20/d39/f5d 0 2024-02-16T22:56:13.418 INFO:tasks.workunit.client.0.smithi064.stdout:7/155: link d5/d6/fb d5/f29 0 2024-02-16T22:56:13.418 INFO:tasks.workunit.client.0.smithi064.stdout:0/137: sync 2024-02-16T22:56:13.419 INFO:tasks.workunit.client.0.smithi064.stdout:6/205: write d0/d6/d10/f1c [1022267,65212] 0 2024-02-16T22:56:13.420 INFO:tasks.workunit.client.0.smithi064.stdout:8/158: rmdir df/d28/d2f 39 2024-02-16T22:56:13.420 INFO:tasks.workunit.client.0.smithi064.stdout:0/138: readlink da/dc/ld 0 2024-02-16T22:56:13.422 INFO:tasks.workunit.client.0.smithi064.stdout:8/159: fsync df/f24 0 2024-02-16T22:56:13.423 INFO:tasks.workunit.client.0.smithi064.stdout:6/206: stat d0/d1d/d21/c35 0 2024-02-16T22:56:13.424 INFO:tasks.workunit.client.0.smithi064.stdout:0/139: chown da/f1b 7499980 1 2024-02-16T22:56:13.425 INFO:tasks.workunit.client.0.smithi064.stdout:0/140: chown da/f18 2503 1 2024-02-16T22:56:13.427 INFO:tasks.workunit.client.1.smithi192.stdout:6/265: creat da/d27/f50 x:0 0 0 2024-02-16T22:56:13.428 INFO:tasks.workunit.client.0.smithi064.stdout:1/159: dwrite d2/df/f1e [0,4194304] 0 2024-02-16T22:56:13.431 INFO:tasks.workunit.client.1.smithi192.stdout:3/260: creat da/d11/d20/d39/d53/f62 x:0 0 0 2024-02-16T22:56:13.432 INFO:tasks.workunit.client.0.smithi064.stdout:4/146: dread dc/ff [0,4194304] 0 2024-02-16T22:56:13.432 INFO:tasks.workunit.client.0.smithi064.stdout:4/147: readlink l6 0 2024-02-16T22:56:13.433 INFO:tasks.workunit.client.0.smithi064.stdout:4/148: write dc/f17 [1053435,91824] 0 2024-02-16T22:56:13.434 INFO:tasks.workunit.client.1.smithi192.stdout:7/312: symlink d1/d69/l6d 0 2024-02-16T22:56:13.435 INFO:tasks.workunit.client.0.smithi064.stdout:6/207: symlink d0/d5/l38 0 2024-02-16T22:56:13.436 INFO:tasks.workunit.client.1.smithi192.stdout:6/266: truncate da/d27/f2f 2212226 0 2024-02-16T22:56:13.437 INFO:tasks.workunit.client.0.smithi064.stdout:8/160: symlink df/d2a/l30 0 2024-02-16T22:56:13.438 INFO:tasks.workunit.client.0.smithi064.stdout:8/161: fdatasync df/f1a 0 2024-02-16T22:56:13.438 INFO:tasks.workunit.client.1.smithi192.stdout:4/275: creat d1/d26/d5e/f63 x:0 0 0 2024-02-16T22:56:13.440 INFO:tasks.workunit.client.1.smithi192.stdout:5/271: sync 2024-02-16T22:56:13.440 INFO:tasks.workunit.client.1.smithi192.stdout:1/222: sync 2024-02-16T22:56:13.440 INFO:tasks.workunit.client.1.smithi192.stdout:9/339: sync 2024-02-16T22:56:13.441 INFO:tasks.workunit.client.1.smithi192.stdout:2/270: sync 2024-02-16T22:56:13.441 INFO:tasks.workunit.client.1.smithi192.stdout:0/228: sync 2024-02-16T22:56:13.441 INFO:tasks.workunit.client.0.smithi064.stdout:9/185: rename d3/d9/d1e/d3e/l44 to d3/d11/d15/l46 0 2024-02-16T22:56:13.441 INFO:tasks.workunit.client.1.smithi192.stdout:2/271: chown f1 7990 1 2024-02-16T22:56:13.442 INFO:tasks.workunit.client.0.smithi064.stdout:7/156: symlink d5/l2a 0 2024-02-16T22:56:13.442 INFO:tasks.workunit.client.1.smithi192.stdout:2/272: dread - d22/f41 zero size 2024-02-16T22:56:13.444 INFO:tasks.workunit.client.1.smithi192.stdout:4/276: write d1/d15/d23/f52 [1391014,101536] 0 2024-02-16T22:56:13.445 INFO:tasks.workunit.client.1.smithi192.stdout:3/261: mkdir da/d11/d63 0 2024-02-16T22:56:13.447 INFO:tasks.workunit.client.0.smithi064.stdout:4/149: creat dc/d1d/f2b x:0 0 0 2024-02-16T22:56:13.448 INFO:tasks.workunit.client.0.smithi064.stdout:4/150: chown c7 1346 1 2024-02-16T22:56:13.450 INFO:tasks.workunit.client.1.smithi192.stdout:0/229: dwrite d1/d3/fc [4194304,4194304] 0 2024-02-16T22:56:13.450 INFO:tasks.workunit.client.1.smithi192.stdout:7/313: symlink d1/l6e 0 2024-02-16T22:56:13.451 INFO:tasks.workunit.client.1.smithi192.stdout:8/244: dread d5/db/de/df/f2d [0,4194304] 0 2024-02-16T22:56:13.452 INFO:tasks.workunit.client.1.smithi192.stdout:4/277: dread d1/d15/d23/f3b [0,4194304] 0 2024-02-16T22:56:13.454 INFO:tasks.workunit.client.0.smithi064.stdout:4/151: dwrite dc/f19 [4194304,4194304] 0 2024-02-16T22:56:13.455 INFO:tasks.workunit.client.1.smithi192.stdout:9/340: rename d7/d19/d28/d43/f4d to d7/d19/d28/d43/d70/f72 0 2024-02-16T22:56:13.459 INFO:tasks.workunit.client.1.smithi192.stdout:7/314: dwrite d1/d19/f64 [0,4194304] 0 2024-02-16T22:56:13.460 INFO:tasks.workunit.client.1.smithi192.stdout:1/223: dread d3/de/ff [0,4194304] 0 2024-02-16T22:56:13.460 INFO:tasks.workunit.client.0.smithi064.stdout:8/162: rename df/d28 to df/d31 0 2024-02-16T22:56:13.462 INFO:tasks.workunit.client.0.smithi064.stdout:8/163: stat df/c16 0 2024-02-16T22:56:13.463 INFO:tasks.workunit.client.1.smithi192.stdout:2/273: truncate d22/f27 1901057 0 2024-02-16T22:56:13.464 INFO:tasks.workunit.client.0.smithi064.stdout:4/152: dwrite dc/d10/d18/d22/f24 [0,4194304] 0 2024-02-16T22:56:13.465 INFO:tasks.workunit.client.0.smithi064.stdout:4/153: fsync dc/de/f1c 0 2024-02-16T22:56:13.466 INFO:tasks.workunit.client.0.smithi064.stdout:4/154: chown dc/d10 1474452 1 2024-02-16T22:56:13.466 INFO:tasks.workunit.client.1.smithi192.stdout:2/274: truncate d22/d24/d2b/d2c/f2d 208855 0 2024-02-16T22:56:13.471 INFO:tasks.workunit.client.1.smithi192.stdout:3/262: creat da/d11/d2b/d43/f64 x:0 0 0 2024-02-16T22:56:13.471 INFO:tasks.workunit.client.1.smithi192.stdout:3/263: stat da/df/f2e 0 2024-02-16T22:56:13.476 INFO:tasks.workunit.client.1.smithi192.stdout:6/267: creat da/df/d12/f51 x:0 0 0 2024-02-16T22:56:13.481 INFO:tasks.workunit.client.0.smithi064.stdout:6/208: symlink d0/d5/df/d17/d18/l39 0 2024-02-16T22:56:13.482 INFO:tasks.workunit.client.0.smithi064.stdout:6/209: write d0/d6/f15 [7728244,73066] 0 2024-02-16T22:56:13.485 INFO:tasks.workunit.client.1.smithi192.stdout:6/268: dwrite da/df/d12/d15/f3e [0,4194304] 0 2024-02-16T22:56:13.485 INFO:tasks.workunit.client.1.smithi192.stdout:9/341: mknod d7/d15/d36/c73 0 2024-02-16T22:56:13.487 INFO:tasks.workunit.client.0.smithi064.stdout:8/164: rename df/f23 to df/f32 0 2024-02-16T22:56:13.489 INFO:tasks.workunit.client.1.smithi192.stdout:7/315: creat d1/d2/de/d2c/d49/f6f x:0 0 0 2024-02-16T22:56:13.490 INFO:tasks.workunit.client.1.smithi192.stdout:1/224: symlink d3/de/d27/d34/l48 0 2024-02-16T22:56:13.491 INFO:tasks.workunit.client.1.smithi192.stdout:0/230: dread d1/d3/d29/f1a [0,4194304] 0 2024-02-16T22:56:13.491 INFO:tasks.workunit.client.0.smithi064.stdout:4/155: creat dc/f2c x:0 0 0 2024-02-16T22:56:13.492 INFO:tasks.workunit.client.0.smithi064.stdout:4/156: stat dc/de/d28 0 2024-02-16T22:56:13.493 INFO:tasks.workunit.client.1.smithi192.stdout:5/272: mkdir d6/d12/d4e/d46/d4f 0 2024-02-16T22:56:13.493 INFO:tasks.workunit.client.0.smithi064.stdout:4/157: write dc/de/f1c [222982,22875] 0 2024-02-16T22:56:13.494 INFO:tasks.workunit.client.0.smithi064.stdout:9/186: mknod d3/c47 0 2024-02-16T22:56:13.494 INFO:tasks.workunit.client.1.smithi192.stdout:5/273: write d6/d12/f32 [1534069,33912] 0 2024-02-16T22:56:13.495 INFO:tasks.workunit.client.1.smithi192.stdout:1/225: dread d3/de/d27/f39 [0,4194304] 0 2024-02-16T22:56:13.496 INFO:tasks.workunit.client.1.smithi192.stdout:2/275: mknod d22/d24/d39/d42/c5c 0 2024-02-16T22:56:13.497 INFO:tasks.workunit.client.1.smithi192.stdout:1/226: truncate d3/de/d27/f44 101231 0 2024-02-16T22:56:13.497 INFO:tasks.workunit.client.0.smithi064.stdout:0/141: truncate f3 743789 0 2024-02-16T22:56:13.499 INFO:tasks.workunit.client.0.smithi064.stdout:4/158: dwrite dc/f19 [0,4194304] 0 2024-02-16T22:56:13.503 INFO:tasks.workunit.client.1.smithi192.stdout:3/264: symlink da/d11/d2b/l65 0 2024-02-16T22:56:13.504 INFO:tasks.workunit.client.1.smithi192.stdout:3/265: truncate da/d11/d2b/f5f 32069 0 2024-02-16T22:56:13.504 INFO:tasks.workunit.client.1.smithi192.stdout:3/266: fsync da/f14 0 2024-02-16T22:56:13.508 INFO:tasks.workunit.client.1.smithi192.stdout:8/245: rename d5/db/l28 to d5/d16/l46 0 2024-02-16T22:56:13.510 INFO:tasks.workunit.client.1.smithi192.stdout:6/269: unlink da/l4f 0 2024-02-16T22:56:13.511 INFO:tasks.workunit.client.1.smithi192.stdout:6/270: read da/df/d18/f29 [2270949,103123] 0 2024-02-16T22:56:13.511 INFO:tasks.workunit.client.1.smithi192.stdout:7/316: creat d1/d19/d4c/f70 x:0 0 0 2024-02-16T22:56:13.512 INFO:tasks.workunit.client.0.smithi064.stdout:9/187: fdatasync d3/f10 0 2024-02-16T22:56:13.513 INFO:tasks.workunit.client.1.smithi192.stdout:7/317: fdatasync d1/d2/de/f34 0 2024-02-16T22:56:13.513 INFO:tasks.workunit.client.1.smithi192.stdout:7/318: chown d1/d2/d2d/c4e 47765 1 2024-02-16T22:56:13.514 INFO:tasks.workunit.client.0.smithi064.stdout:9/188: dread - d3/d11/d15/d23/f3d zero size 2024-02-16T22:56:13.515 INFO:tasks.workunit.client.0.smithi064.stdout:9/189: write d3/d11/d15/d23/f3d [147873,115151] 0 2024-02-16T22:56:13.516 INFO:tasks.workunit.client.0.smithi064.stdout:1/160: getdents d2/df 0 2024-02-16T22:56:13.517 INFO:tasks.workunit.client.1.smithi192.stdout:0/231: rmdir d1/d2 39 2024-02-16T22:56:13.518 INFO:tasks.workunit.client.1.smithi192.stdout:7/319: dwrite d1/d2/f58 [4194304,4194304] 0 2024-02-16T22:56:13.519 INFO:tasks.workunit.client.0.smithi064.stdout:0/142: mknod da/dc/de/d12/d1d/c21 0 2024-02-16T22:56:13.519 INFO:tasks.workunit.client.0.smithi064.stdout:0/143: chown da 131 1 2024-02-16T22:56:13.519 INFO:tasks.workunit.client.0.smithi064.stdout:0/144: chown da/dc/de/d12/d1d/f1f 15 1 2024-02-16T22:56:13.522 INFO:tasks.workunit.client.1.smithi192.stdout:5/274: symlink d6/d12/d4e/d46/l50 0 2024-02-16T22:56:13.524 INFO:tasks.workunit.client.1.smithi192.stdout:7/320: dwrite d1/d19/d4c/f70 [0,4194304] 0 2024-02-16T22:56:13.538 INFO:tasks.workunit.client.0.smithi064.stdout:1/161: rmdir d2/df/d11/d1a/d30 39 2024-02-16T22:56:13.540 INFO:tasks.workunit.client.0.smithi064.stdout:9/190: dwrite d3/d11/f28 [0,4194304] 0 2024-02-16T22:56:13.541 INFO:tasks.workunit.client.0.smithi064.stdout:0/145: chown da/dc/de/l14 1803075 1 2024-02-16T22:56:13.544 INFO:tasks.workunit.client.0.smithi064.stdout:7/157: link d5/d6/l17 d5/l2b 0 2024-02-16T22:56:13.547 INFO:tasks.workunit.client.1.smithi192.stdout:6/271: rename da/df/d18/d3f to da/df/d18/d52 0 2024-02-16T22:56:13.547 INFO:tasks.workunit.client.0.smithi064.stdout:0/146: dread da/f10 [4194304,4194304] 0 2024-02-16T22:56:13.548 INFO:tasks.workunit.client.0.smithi064.stdout:6/210: link d0/d6/d10/d33/l20 d0/d5/df/l3a 0 2024-02-16T22:56:13.548 INFO:tasks.workunit.client.0.smithi064.stdout:9/191: dwrite d3/d9/f3c [0,4194304] 0 2024-02-16T22:56:13.549 INFO:tasks.workunit.client.0.smithi064.stdout:0/147: fdatasync da/dc/f13 0 2024-02-16T22:56:13.553 INFO:tasks.workunit.client.0.smithi064.stdout:8/165: rename df/d31/d2f/c27 to df/d31/d2f/c33 0 2024-02-16T22:56:13.555 INFO:tasks.workunit.client.0.smithi064.stdout:8/166: write df/d31/f29 [313955,112184] 0 2024-02-16T22:56:13.556 INFO:tasks.workunit.client.0.smithi064.stdout:8/167: fdatasync df/f1a 0 2024-02-16T22:56:13.558 INFO:tasks.workunit.client.1.smithi192.stdout:0/232: rmdir d1 39 2024-02-16T22:56:13.558 INFO:tasks.workunit.client.0.smithi064.stdout:6/211: dwrite d0/d1b/f36 [0,4194304] 0 2024-02-16T22:56:13.559 INFO:tasks.workunit.client.0.smithi064.stdout:6/212: chown d0/d5/df 589058877 1 2024-02-16T22:56:13.560 INFO:tasks.workunit.client.0.smithi064.stdout:1/162: mkdir d2/df/d14/d18/d3e 0 2024-02-16T22:56:13.561 INFO:tasks.workunit.client.0.smithi064.stdout:1/163: read - d2/df/d14/d2c/f3c zero size 2024-02-16T22:56:13.562 INFO:tasks.workunit.client.1.smithi192.stdout:5/275: truncate d6/d20/f4d 361972 0 2024-02-16T22:56:13.566 INFO:tasks.workunit.client.0.smithi064.stdout:7/158: rename d5/c1c to d5/d6/c2c 0 2024-02-16T22:56:13.567 INFO:tasks.workunit.client.0.smithi064.stdout:7/159: write d5/f20 [1323919,100444] 0 2024-02-16T22:56:13.569 INFO:tasks.workunit.client.0.smithi064.stdout:9/192: creat d3/d9/f48 x:0 0 0 2024-02-16T22:56:13.574 INFO:tasks.workunit.client.0.smithi064.stdout:4/159: getdents dc/de 0 2024-02-16T22:56:13.576 INFO:tasks.workunit.client.0.smithi064.stdout:8/168: rename df/d2a/f2c to df/d2a/f34 0 2024-02-16T22:56:13.577 INFO:tasks.workunit.client.1.smithi192.stdout:2/276: dwrite f5 [0,4194304] 0 2024-02-16T22:56:13.577 INFO:tasks.workunit.client.0.smithi064.stdout:6/213: fdatasync d0/d1b/f23 0 2024-02-16T22:56:13.577 INFO:tasks.workunit.client.1.smithi192.stdout:5/276: symlink d6/d12/l51 0 2024-02-16T22:56:13.577 INFO:tasks.workunit.client.0.smithi064.stdout:6/214: dread d0/d1b/f23 [0,4194304] 0 2024-02-16T22:56:13.578 INFO:tasks.workunit.client.1.smithi192.stdout:5/277: write d6/d12/d2d/f43 [4685800,50865] 0 2024-02-16T22:56:13.592 INFO:tasks.workunit.client.1.smithi192.stdout:9/342: sync 2024-02-16T22:56:13.616 INFO:tasks.workunit.client.0.smithi064.stdout:3/216: sync 2024-02-16T22:56:13.617 INFO:tasks.workunit.client.1.smithi192.stdout:7/321: link d1/d19/f1d d1/d2/de/d2c/d49/f71 0 2024-02-16T22:56:13.618 INFO:tasks.workunit.client.1.smithi192.stdout:7/322: dread - d1/d19/d4c/f6b zero size 2024-02-16T22:56:13.619 INFO:tasks.workunit.client.0.smithi064.stdout:6/215: dwrite d0/d5/f2b [0,4194304] 0 2024-02-16T22:56:13.620 INFO:tasks.workunit.client.0.smithi064.stdout:7/160: truncate d5/d6/f1d 1552294 0 2024-02-16T22:56:13.621 INFO:tasks.workunit.client.0.smithi064.stdout:7/161: chown d5/f27 227 1 2024-02-16T22:56:13.621 INFO:tasks.workunit.client.1.smithi192.stdout:4/278: rename d1/d26/f54 to d1/d8/f64 0 2024-02-16T22:56:13.623 INFO:tasks.workunit.client.0.smithi064.stdout:9/193: creat d3/d9/d1e/d3e/f49 x:0 0 0 2024-02-16T22:56:13.624 INFO:tasks.workunit.client.1.smithi192.stdout:1/227: write d3/f16 [233318,89919] 0 2024-02-16T22:56:13.624 INFO:tasks.workunit.client.0.smithi064.stdout:6/216: dread d0/d6/d10/f14 [0,4194304] 0 2024-02-16T22:56:13.625 INFO:tasks.workunit.client.0.smithi064.stdout:6/217: chown d0/d6/d10 12540271 1 2024-02-16T22:56:13.628 INFO:tasks.workunit.client.0.smithi064.stdout:8/169: mknod df/d31/d2f/c35 0 2024-02-16T22:56:13.630 INFO:tasks.workunit.client.1.smithi192.stdout:5/278: fsync d6/fd 0 2024-02-16T22:56:13.630 INFO:tasks.workunit.client.1.smithi192.stdout:0/233: dwrite d1/f24 [0,4194304] 0 2024-02-16T22:56:13.631 INFO:tasks.workunit.client.1.smithi192.stdout:5/279: chown d6/d12/d4e/d46/c49 409281 1 2024-02-16T22:56:13.631 INFO:tasks.workunit.client.0.smithi064.stdout:4/160: dwrite dc/d1d/f29 [4194304,4194304] 0 2024-02-16T22:56:13.631 INFO:tasks.workunit.client.1.smithi192.stdout:7/323: truncate d1/d1f/f4f 799353 0 2024-02-16T22:56:13.632 INFO:tasks.workunit.client.1.smithi192.stdout:5/280: chown d6/d12/l2c 16925321 1 2024-02-16T22:56:13.639 INFO:tasks.workunit.client.0.smithi064.stdout:1/164: mknod d2/df/d11/d1a/d30/c3f 0 2024-02-16T22:56:13.640 INFO:tasks.workunit.client.0.smithi064.stdout:1/165: chown d2/df/f1c 16381 1 2024-02-16T22:56:13.640 INFO:tasks.workunit.client.1.smithi192.stdout:3/267: sync 2024-02-16T22:56:13.641 INFO:tasks.workunit.client.1.smithi192.stdout:4/279: symlink d1/d15/l65 0 2024-02-16T22:56:13.648 INFO:tasks.workunit.client.0.smithi064.stdout:6/218: creat d0/d6/f3b x:0 0 0 2024-02-16T22:56:13.649 INFO:tasks.workunit.client.1.smithi192.stdout:7/324: mknod d1/d69/c72 0 2024-02-16T22:56:13.650 INFO:tasks.workunit.client.1.smithi192.stdout:7/325: stat d1/d2/f25 0 2024-02-16T22:56:13.652 INFO:tasks.workunit.client.0.smithi064.stdout:4/161: symlink dc/dd/d16/l2d 0 2024-02-16T22:56:13.652 INFO:tasks.workunit.client.1.smithi192.stdout:3/268: dread da/f12 [0,4194304] 0 2024-02-16T22:56:13.653 INFO:tasks.workunit.client.1.smithi192.stdout:3/269: write da/df/f2e [982221,75831] 0 2024-02-16T22:56:13.654 INFO:tasks.workunit.client.0.smithi064.stdout:8/170: dread fb [0,4194304] 0 2024-02-16T22:56:13.656 INFO:tasks.workunit.client.0.smithi064.stdout:3/217: creat d4/d13/d1b/d43/f49 x:0 0 0 2024-02-16T22:56:13.657 INFO:tasks.workunit.client.1.smithi192.stdout:4/280: mkdir d1/d26/d66 0 2024-02-16T22:56:13.659 INFO:tasks.workunit.client.1.smithi192.stdout:3/270: dwrite da/d11/f2f [0,4194304] 0 2024-02-16T22:56:13.660 INFO:tasks.workunit.client.0.smithi064.stdout:4/162: dwrite dc/f2c [0,4194304] 0 2024-02-16T22:56:13.660 INFO:tasks.workunit.client.1.smithi192.stdout:3/271: fsync da/df/f49 0 2024-02-16T22:56:13.662 INFO:tasks.workunit.client.1.smithi192.stdout:0/234: unlink d1/d2/l32 0 2024-02-16T22:56:13.663 INFO:tasks.workunit.client.1.smithi192.stdout:0/235: truncate d1/d2/f42 911047 0 2024-02-16T22:56:13.664 INFO:tasks.workunit.client.0.smithi064.stdout:4/163: dread dc/ff [0,4194304] 0 2024-02-16T22:56:13.666 INFO:tasks.workunit.client.1.smithi192.stdout:5/281: symlink d6/d12/d4e/d46/d4f/l52 0 2024-02-16T22:56:13.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/246: rename d5/d3e/d39 to d5/db/d47 0 2024-02-16T22:56:13.671 INFO:tasks.workunit.client.1.smithi192.stdout:6/272: rename da/df/d18 to da/df/d18/d52/d53 22 2024-02-16T22:56:13.672 INFO:tasks.workunit.client.0.smithi064.stdout:3/218: unlink d4/d13/d1b/d27/f2c 0 2024-02-16T22:56:13.672 INFO:tasks.workunit.client.0.smithi064.stdout:3/219: stat d4/d13/d1b/f1c 0 2024-02-16T22:56:13.672 INFO:tasks.workunit.client.0.smithi064.stdout:7/162: link d5/d6/cd d5/d6/c2d 0 2024-02-16T22:56:13.674 INFO:tasks.workunit.client.1.smithi192.stdout:1/228: dwrite d3/de/f35 [0,4194304] 0 2024-02-16T22:56:13.674 INFO:tasks.workunit.client.1.smithi192.stdout:8/247: write d5/d9/f44 [7914,104564] 0 2024-02-16T22:56:13.675 INFO:tasks.workunit.client.0.smithi064.stdout:6/219: fdatasync d0/f22 0 2024-02-16T22:56:13.677 INFO:tasks.workunit.client.1.smithi192.stdout:4/281: mknod d1/d8/d11/d58/c67 0 2024-02-16T22:56:13.679 INFO:tasks.workunit.client.1.smithi192.stdout:0/236: mknod d1/d2/c44 0 2024-02-16T22:56:13.683 INFO:tasks.workunit.client.1.smithi192.stdout:7/326: mknod d1/d2/de/d2c/d49/d54/d6a/c73 0 2024-02-16T22:56:13.685 INFO:tasks.workunit.client.0.smithi064.stdout:3/220: symlink d4/d13/d1b/d43/l4a 0 2024-02-16T22:56:13.686 INFO:tasks.workunit.client.1.smithi192.stdout:2/277: rename d22/d24/d39/d3d/c46 to d22/d24/c5d 0 2024-02-16T22:56:13.689 INFO:tasks.workunit.client.1.smithi192.stdout:8/248: fsync d5/d9/f17 0 2024-02-16T22:56:13.692 INFO:tasks.workunit.client.0.smithi064.stdout:6/220: rename d0/d1d/d21/d37 to d0/d3c 0 2024-02-16T22:56:13.692 INFO:tasks.workunit.client.1.smithi192.stdout:2/278: dwrite fe [4194304,4194304] 0 2024-02-16T22:56:13.693 INFO:tasks.workunit.client.1.smithi192.stdout:1/229: mknod d3/de/d27/d34/c49 0 2024-02-16T22:56:13.694 INFO:tasks.workunit.client.0.smithi064.stdout:1/166: getdents d2/df/d14/d2a 0 2024-02-16T22:56:13.694 INFO:tasks.workunit.client.1.smithi192.stdout:2/279: write d22/d24/f49 [1731649,48479] 0 2024-02-16T22:56:13.694 INFO:tasks.workunit.client.1.smithi192.stdout:1/230: read d3/de/f1b [624030,93637] 0 2024-02-16T22:56:13.694 INFO:tasks.workunit.client.0.smithi064.stdout:1/167: chown d2/df/f1e 298244609 1 2024-02-16T22:56:13.698 INFO:tasks.workunit.client.0.smithi064.stdout:8/171: creat df/d31/f36 x:0 0 0 2024-02-16T22:56:13.700 INFO:tasks.workunit.client.1.smithi192.stdout:1/231: dwrite d3/de/f1a [0,4194304] 0 2024-02-16T22:56:13.701 INFO:tasks.workunit.client.1.smithi192.stdout:0/237: creat d1/d3/de/d22/f45 x:0 0 0 2024-02-16T22:56:13.703 INFO:tasks.workunit.client.0.smithi064.stdout:1/168: dread d2/df/d14/d18/f26 [0,4194304] 0 2024-02-16T22:56:13.706 INFO:tasks.workunit.client.0.smithi064.stdout:3/221: dread d4/d13/f16 [0,4194304] 0 2024-02-16T22:56:13.707 INFO:tasks.workunit.client.0.smithi064.stdout:6/221: creat d0/d5/f3d x:0 0 0 2024-02-16T22:56:13.709 INFO:tasks.workunit.client.0.smithi064.stdout:8/172: dwrite f1 [0,4194304] 0 2024-02-16T22:56:13.717 INFO:tasks.workunit.client.1.smithi192.stdout:8/249: unlink d5/d16/c20 0 2024-02-16T22:56:13.719 INFO:tasks.workunit.client.1.smithi192.stdout:8/250: dread d5/d9/f44 [0,4194304] 0 2024-02-16T22:56:13.719 INFO:tasks.workunit.client.1.smithi192.stdout:8/251: dread - d5/d16/f45 zero size 2024-02-16T22:56:13.720 INFO:tasks.workunit.client.1.smithi192.stdout:8/252: write d5/f38 [2541596,65402] 0 2024-02-16T22:56:13.720 INFO:tasks.workunit.client.0.smithi064.stdout:1/169: mkdir d2/df/d14/d18/d40 0 2024-02-16T22:56:13.723 INFO:tasks.workunit.client.0.smithi064.stdout:8/173: rename fb to df/f37 0 2024-02-16T22:56:13.723 INFO:tasks.workunit.client.0.smithi064.stdout:8/174: chown l0 27207470 1 2024-02-16T22:56:13.725 INFO:tasks.workunit.client.1.smithi192.stdout:1/232: creat d3/de/d27/f4a x:0 0 0 2024-02-16T22:56:13.726 INFO:tasks.workunit.client.0.smithi064.stdout:7/163: link d5/l21 d5/d6/l2e 0 2024-02-16T22:56:13.726 INFO:tasks.workunit.client.1.smithi192.stdout:1/233: chown d3/de/d27/f30 550334596 1 2024-02-16T22:56:13.727 INFO:tasks.workunit.client.0.smithi064.stdout:1/170: dread d2/df/f1c [0,4194304] 0 2024-02-16T22:56:13.727 INFO:tasks.workunit.client.1.smithi192.stdout:0/238: symlink d1/d3/de/d22/l46 0 2024-02-16T22:56:13.728 INFO:tasks.workunit.client.0.smithi064.stdout:3/222: creat d4/d13/d1b/d27/d44/f4b x:0 0 0 2024-02-16T22:56:13.728 INFO:tasks.workunit.client.1.smithi192.stdout:7/327: creat d1/d2/d2d/d61/f74 x:0 0 0 2024-02-16T22:56:13.730 INFO:tasks.workunit.client.1.smithi192.stdout:9/343: rename d7/d19/d28/d43/d70/f72 to d7/d19/d28/f74 0 2024-02-16T22:56:13.731 INFO:tasks.workunit.client.1.smithi192.stdout:9/344: write d7/d19/d28/f52 [920397,123467] 0 2024-02-16T22:56:13.733 INFO:tasks.workunit.client.0.smithi064.stdout:8/175: dwrite f8 [0,4194304] 0 2024-02-16T22:56:13.766 INFO:tasks.workunit.client.1.smithi192.stdout:6/273: sync 2024-02-16T22:56:13.769 INFO:tasks.workunit.client.0.smithi064.stdout:6/222: link d0/d5/f11 d0/d6/d10/f3e 0 2024-02-16T22:56:13.771 INFO:tasks.workunit.client.0.smithi064.stdout:1/171: creat d2/df/d11/d1a/d30/f41 x:0 0 0 2024-02-16T22:56:13.772 INFO:tasks.workunit.client.1.smithi192.stdout:8/253: mknod d5/d16/c48 0 2024-02-16T22:56:13.773 INFO:tasks.workunit.client.0.smithi064.stdout:3/223: creat d4/d14/f4c x:0 0 0 2024-02-16T22:56:13.774 INFO:tasks.workunit.client.0.smithi064.stdout:8/176: mkdir df/d31/d2f/d38 0 2024-02-16T22:56:13.775 INFO:tasks.workunit.client.0.smithi064.stdout:8/177: chown df/f1a 3 1 2024-02-16T22:56:13.775 INFO:tasks.workunit.client.1.smithi192.stdout:8/254: dread d5/d3e/f1c [0,4194304] 0 2024-02-16T22:56:13.775 INFO:tasks.workunit.client.1.smithi192.stdout:4/282: creat d1/d8/f68 x:0 0 0 2024-02-16T22:56:13.776 INFO:tasks.workunit.client.0.smithi064.stdout:8/178: chown df/d31/d2f/f2b 0 1 2024-02-16T22:56:13.776 INFO:tasks.workunit.client.0.smithi064.stdout:6/223: rmdir d0/d6/da 39 2024-02-16T22:56:13.778 INFO:tasks.workunit.client.0.smithi064.stdout:6/224: truncate d0/d6/d10/d33/f2a 493542 0 2024-02-16T22:56:13.781 INFO:tasks.workunit.client.0.smithi064.stdout:3/224: mkdir d4/d13/d4d 0 2024-02-16T22:56:13.782 INFO:tasks.workunit.client.0.smithi064.stdout:3/225: read d4/d13/f16 [3876116,116486] 0 2024-02-16T22:56:13.782 INFO:tasks.workunit.client.1.smithi192.stdout:7/328: truncate d1/d2/de/f24 599685 0 2024-02-16T22:56:13.785 INFO:tasks.workunit.client.0.smithi064.stdout:7/164: rename d5/f20 to d5/f2f 0 2024-02-16T22:56:13.785 INFO:tasks.workunit.client.0.smithi064.stdout:7/165: fsync d5/d6/f19 0 2024-02-16T22:56:13.786 INFO:tasks.workunit.client.0.smithi064.stdout:3/226: dread d4/d14/f46 [0,4194304] 0 2024-02-16T22:56:13.787 INFO:tasks.workunit.client.1.smithi192.stdout:3/272: rename da/d11/d2b/d32 to da/df/d42/d66 0 2024-02-16T22:56:13.788 INFO:tasks.workunit.client.0.smithi064.stdout:3/227: write d4/f18 [2902726,53592] 0 2024-02-16T22:56:13.788 INFO:tasks.workunit.client.0.smithi064.stdout:3/228: stat d4/d13 0 2024-02-16T22:56:13.789 INFO:tasks.workunit.client.0.smithi064.stdout:6/225: fdatasync d0/d1b/f23 0 2024-02-16T22:56:13.794 INFO:tasks.workunit.client.0.smithi064.stdout:8/179: dread df/f1a [0,4194304] 0 2024-02-16T22:56:13.795 INFO:tasks.workunit.client.0.smithi064.stdout:7/166: dwrite d5/f25 [0,4194304] 0 2024-02-16T22:56:13.796 INFO:tasks.workunit.client.1.smithi192.stdout:6/274: creat da/d40/f54 x:0 0 0 2024-02-16T22:56:13.798 INFO:tasks.workunit.client.0.smithi064.stdout:3/229: dwrite d4/f11 [0,4194304] 0 2024-02-16T22:56:13.803 INFO:tasks.workunit.client.0.smithi064.stdout:8/180: dwrite df/f24 [0,4194304] 0 2024-02-16T22:56:13.805 INFO:tasks.workunit.client.0.smithi064.stdout:3/230: dread d4/d13/f16 [0,4194304] 0 2024-02-16T22:56:13.805 INFO:tasks.workunit.client.1.smithi192.stdout:8/255: mkdir d5/d3e/d49 0 2024-02-16T22:56:13.812 INFO:tasks.workunit.client.0.smithi064.stdout:1/172: creat d2/f42 x:0 0 0 2024-02-16T22:56:13.813 INFO:tasks.workunit.client.1.smithi192.stdout:1/234: creat d3/de/d27/d34/d36/d47/f4b x:0 0 0 2024-02-16T22:56:13.815 INFO:tasks.workunit.client.1.smithi192.stdout:0/239: mkdir d1/d3/de/d22/d37/d47 0 2024-02-16T22:56:13.815 INFO:tasks.workunit.client.1.smithi192.stdout:0/240: readlink d1/d3/de/l3d 0 2024-02-16T22:56:13.817 INFO:tasks.workunit.client.0.smithi064.stdout:7/167: creat d5/d6/f30 x:0 0 0 2024-02-16T22:56:13.821 INFO:tasks.workunit.client.1.smithi192.stdout:0/241: dwrite d1/d3/de/f10 [0,4194304] 0 2024-02-16T22:56:13.822 INFO:tasks.workunit.client.1.smithi192.stdout:0/242: read - d1/d3/de/d22/f31 zero size 2024-02-16T22:56:13.823 INFO:tasks.workunit.client.0.smithi064.stdout:7/168: dread d5/f25 [0,4194304] 0 2024-02-16T22:56:13.823 INFO:tasks.workunit.client.0.smithi064.stdout:7/169: fdatasync d5/d6/f30 0 2024-02-16T22:56:13.827 INFO:tasks.workunit.client.1.smithi192.stdout:5/282: rename d6/d8/c22 to d6/d12/d2d/c53 0 2024-02-16T22:56:13.831 INFO:tasks.workunit.client.1.smithi192.stdout:5/283: dread d6/d12/d2d/f43 [0,4194304] 0 2024-02-16T22:56:13.832 INFO:tasks.workunit.client.1.smithi192.stdout:3/273: creat da/df/d42/d66/f67 x:0 0 0 2024-02-16T22:56:13.833 INFO:tasks.workunit.client.1.smithi192.stdout:9/345: getdents d7/d19/d5e/d6f 0 2024-02-16T22:56:13.833 INFO:tasks.workunit.client.1.smithi192.stdout:3/274: dread - da/d11/d20/d39/f5d zero size 2024-02-16T22:56:13.836 INFO:tasks.workunit.client.1.smithi192.stdout:6/275: unlink da/fb 0 2024-02-16T22:56:13.837 INFO:tasks.workunit.client.0.smithi064.stdout:8/181: creat df/d31/d2f/d38/f39 x:0 0 0 2024-02-16T22:56:13.838 INFO:tasks.workunit.client.1.smithi192.stdout:6/276: fdatasync da/df/d12/d15/d1b/f37 0 2024-02-16T22:56:13.839 INFO:tasks.workunit.client.0.smithi064.stdout:8/182: read f3 [1687975,35908] 0 2024-02-16T22:56:13.840 INFO:tasks.workunit.client.1.smithi192.stdout:9/346: dwrite f0 [0,4194304] 0 2024-02-16T22:56:13.842 INFO:tasks.workunit.client.1.smithi192.stdout:8/256: creat d5/db/d47/f4a x:0 0 0 2024-02-16T22:56:13.843 INFO:tasks.workunit.client.1.smithi192.stdout:8/257: chown d5 1216 1 2024-02-16T22:56:13.845 INFO:tasks.workunit.client.1.smithi192.stdout:2/280: link d22/d24/d2b/d3e/c5a d22/d24/d2b/d3e/d4f/c5e 0 2024-02-16T22:56:13.845 INFO:tasks.workunit.client.0.smithi064.stdout:7/170: dread d5/d6/f18 [0,4194304] 0 2024-02-16T22:56:13.847 INFO:tasks.workunit.client.1.smithi192.stdout:8/258: dread d5/db/de/df/f1b [0,4194304] 0 2024-02-16T22:56:13.848 INFO:tasks.workunit.client.1.smithi192.stdout:1/235: mknod d3/de/c4c 0 2024-02-16T22:56:13.848 INFO:tasks.workunit.client.1.smithi192.stdout:8/259: chown d5/db/de/d21/f2f 18860 1 2024-02-16T22:56:13.850 INFO:tasks.workunit.client.1.smithi192.stdout:7/329: write d1/d19/f1d [8553472,86581] 0 2024-02-16T22:56:13.852 INFO:tasks.workunit.client.0.smithi064.stdout:8/183: rename fe to df/d31/d2f/f3a 0 2024-02-16T22:56:13.853 INFO:tasks.workunit.client.0.smithi064.stdout:8/184: chown df/f1a 0 1 2024-02-16T22:56:13.854 INFO:tasks.workunit.client.0.smithi064.stdout:6/226: getdents d0/d1d 0 2024-02-16T22:56:13.857 INFO:tasks.workunit.client.0.smithi064.stdout:6/227: write d0/d1b/f36 [4999700,10693] 0 2024-02-16T22:56:13.860 INFO:tasks.workunit.client.1.smithi192.stdout:5/284: rmdir d6/d12/d1d/d3c 39 2024-02-16T22:56:13.860 INFO:tasks.workunit.client.0.smithi064.stdout:3/231: dwrite d4/f17 [0,4194304] 0 2024-02-16T22:56:13.862 INFO:tasks.workunit.client.0.smithi064.stdout:1/173: creat d2/f43 x:0 0 0 2024-02-16T22:56:13.864 INFO:tasks.workunit.client.0.smithi064.stdout:8/185: dwrite df/d2a/f34 [0,4194304] 0 2024-02-16T22:56:13.866 INFO:tasks.workunit.client.1.smithi192.stdout:5/285: dwrite d6/fd [0,4194304] 0 2024-02-16T22:56:13.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/275: symlink da/d11/d20/l68 0 2024-02-16T22:56:13.867 INFO:tasks.workunit.client.1.smithi192.stdout:5/286: chown d6/d12/l2c 41667714 1 2024-02-16T22:56:13.868 INFO:tasks.workunit.client.1.smithi192.stdout:3/276: write da/d11/f2f [2182799,44490] 0 2024-02-16T22:56:13.870 INFO:tasks.workunit.client.1.smithi192.stdout:4/283: write d1/d15/d23/f4a [287815,109651] 0 2024-02-16T22:56:13.871 INFO:tasks.workunit.client.1.smithi192.stdout:4/284: fdatasync d1/f5 0 2024-02-16T22:56:13.873 INFO:tasks.workunit.client.1.smithi192.stdout:6/277: rmdir da/df/d18/d46 39 2024-02-16T22:56:13.873 INFO:tasks.workunit.client.1.smithi192.stdout:4/285: fsync d1/d26/f28 0 2024-02-16T22:56:13.874 INFO:tasks.workunit.client.0.smithi064.stdout:6/228: truncate d0/d6/da/f25 1781020 0 2024-02-16T22:56:13.874 INFO:tasks.workunit.client.1.smithi192.stdout:9/347: mknod d7/d15/d36/d38/d3f/c75 0 2024-02-16T22:56:13.874 INFO:tasks.workunit.client.1.smithi192.stdout:4/286: write d1/f4 [6279947,8462] 0 2024-02-16T22:56:13.875 INFO:tasks.workunit.client.1.smithi192.stdout:9/348: write f0 [3794175,34866] 0 2024-02-16T22:56:13.876 INFO:tasks.workunit.client.0.smithi064.stdout:8/186: rmdir df 39 2024-02-16T22:56:13.877 INFO:tasks.workunit.client.1.smithi192.stdout:4/287: write d1/d15/d23/f52 [1735044,84637] 0 2024-02-16T22:56:13.879 INFO:tasks.workunit.client.1.smithi192.stdout:4/288: dread - d1/d15/d23/f5c zero size 2024-02-16T22:56:13.880 INFO:tasks.workunit.client.0.smithi064.stdout:1/174: truncate d2/df/d14/d18/f28 443212 0 2024-02-16T22:56:13.880 INFO:tasks.workunit.client.0.smithi064.stdout:1/175: dread - d2/df/d11/f3b zero size 2024-02-16T22:56:13.881 INFO:tasks.workunit.client.1.smithi192.stdout:2/281: chown d22/f27 12 1 2024-02-16T22:56:13.882 INFO:tasks.workunit.client.0.smithi064.stdout:1/176: fdatasync d2/df/d11/d1a/d1d/f24 0 2024-02-16T22:56:13.882 INFO:tasks.workunit.client.1.smithi192.stdout:4/289: fdatasync d1/d26/d34/f50 0 2024-02-16T22:56:13.882 INFO:tasks.workunit.client.1.smithi192.stdout:6/278: dwrite da/df/f1c [4194304,4194304] 0 2024-02-16T22:56:13.887 INFO:tasks.workunit.client.1.smithi192.stdout:1/236: write d3/de/d27/f3b [3034266,119570] 0 2024-02-16T22:56:13.887 INFO:tasks.workunit.client.1.smithi192.stdout:7/330: rename d1/l1c to d1/d2/de/d2c/d49/l75 0 2024-02-16T22:56:13.887 INFO:tasks.workunit.client.1.smithi192.stdout:6/279: dread - da/d27/f50 zero size 2024-02-16T22:56:13.889 INFO:tasks.workunit.client.0.smithi064.stdout:8/187: symlink df/d31/d2f/l3b 0 2024-02-16T22:56:13.890 INFO:tasks.workunit.client.0.smithi064.stdout:6/229: dread d0/d6/da/f25 [0,4194304] 0 2024-02-16T22:56:13.893 INFO:tasks.workunit.client.0.smithi064.stdout:6/230: write d0/d6/f3b [703559,97937] 0 2024-02-16T22:56:13.896 INFO:tasks.workunit.client.0.smithi064.stdout:5/146: sync 2024-02-16T22:56:13.897 INFO:tasks.workunit.client.0.smithi064.stdout:1/177: dwrite d2/f36 [0,4194304] 0 2024-02-16T22:56:13.897 INFO:tasks.workunit.client.0.smithi064.stdout:7/171: write d5/d6/fb [2368841,85066] 0 2024-02-16T22:56:13.898 INFO:tasks.workunit.client.0.smithi064.stdout:7/172: readlink d5/l11 0 2024-02-16T22:56:13.899 INFO:tasks.workunit.client.0.smithi064.stdout:7/173: chown d5/l11 92410 1 2024-02-16T22:56:13.904 INFO:tasks.workunit.client.0.smithi064.stdout:8/188: dread df/f32 [0,4194304] 0 2024-02-16T22:56:13.905 INFO:tasks.workunit.client.0.smithi064.stdout:6/231: rename d0/d12/f1f to d0/d5/df/d17/d18/f3f 0 2024-02-16T22:56:13.907 INFO:tasks.workunit.client.0.smithi064.stdout:6/232: chown d0/d6/f15 464 1 2024-02-16T22:56:13.909 INFO:tasks.workunit.client.0.smithi064.stdout:5/147: dread d6/f24 [0,4194304] 0 2024-02-16T22:56:13.910 INFO:tasks.workunit.client.0.smithi064.stdout:7/174: dwrite d5/f27 [0,4194304] 0 2024-02-16T22:56:13.913 INFO:tasks.workunit.client.0.smithi064.stdout:3/232: write d4/d14/f46 [311992,89821] 0 2024-02-16T22:56:13.917 INFO:tasks.workunit.client.0.smithi064.stdout:1/178: fdatasync d2/df/d14/d18/f28 0 2024-02-16T22:56:13.921 INFO:tasks.workunit.client.1.smithi192.stdout:9/349: truncate d7/f12 1072947 0 2024-02-16T22:56:13.922 INFO:tasks.workunit.client.1.smithi192.stdout:0/243: truncate d1/d3/de/d22/f28 1680943 0 2024-02-16T22:56:13.923 INFO:tasks.workunit.client.1.smithi192.stdout:9/350: dread d7/d15/d36/d38/d3f/f59 [0,4194304] 0 2024-02-16T22:56:13.927 INFO:tasks.workunit.client.1.smithi192.stdout:0/244: dwrite d1/d3/d29/f3f [0,4194304] 0 2024-02-16T22:56:13.927 INFO:tasks.workunit.client.1.smithi192.stdout:0/245: fsync d1/f2a 0 2024-02-16T22:56:13.928 INFO:tasks.workunit.client.1.smithi192.stdout:5/287: write d6/d12/f18 [7848959,78462] 0 2024-02-16T22:56:13.929 INFO:tasks.workunit.client.0.smithi064.stdout:5/148: rename d6/d18/d19/c1e to d6/d25/c35 0 2024-02-16T22:56:13.931 INFO:tasks.workunit.client.1.smithi192.stdout:5/288: write d6/d12/d1d/f39 [1095509,29887] 0 2024-02-16T22:56:13.931 INFO:tasks.workunit.client.1.smithi192.stdout:8/260: unlink d5/d16/c32 0 2024-02-16T22:56:13.932 INFO:tasks.workunit.client.0.smithi064.stdout:6/233: dwrite d0/d5/df/d17/d18/f3f [4194304,4194304] 0 2024-02-16T22:56:13.932 INFO:tasks.workunit.client.0.smithi064.stdout:3/233: mkdir d4/d13/d1b/d27/d4e 0 2024-02-16T22:56:13.933 INFO:tasks.workunit.client.0.smithi064.stdout:7/175: dwrite d5/f25 [0,4194304] 0 2024-02-16T22:56:13.935 INFO:tasks.workunit.client.1.smithi192.stdout:2/282: creat d22/d24/d39/d42/f5f x:0 0 0 2024-02-16T22:56:13.936 INFO:tasks.workunit.client.1.smithi192.stdout:8/261: read d5/f1e [1620705,91716] 0 2024-02-16T22:56:13.937 INFO:tasks.workunit.client.1.smithi192.stdout:0/246: dwrite d1/d3/de/f10 [0,4194304] 0 2024-02-16T22:56:13.940 INFO:tasks.workunit.client.1.smithi192.stdout:8/262: chown d5/db/d47/f4a 1693162 1 2024-02-16T22:56:13.940 INFO:tasks.workunit.client.1.smithi192.stdout:4/290: symlink d1/d15/d29/l69 0 2024-02-16T22:56:13.945 INFO:tasks.workunit.client.0.smithi064.stdout:1/179: write d2/df/d11/d1a/d30/f32 [3624181,63068] 0 2024-02-16T22:56:13.945 INFO:tasks.workunit.client.1.smithi192.stdout:7/331: fdatasync d1/d2/f1e 0 2024-02-16T22:56:13.947 INFO:tasks.workunit.client.0.smithi064.stdout:5/149: fsync d6/f17 0 2024-02-16T22:56:13.948 INFO:tasks.workunit.client.1.smithi192.stdout:2/283: dwrite d22/f4a [0,4194304] 0 2024-02-16T22:56:13.951 INFO:tasks.workunit.client.0.smithi064.stdout:3/234: mkdir d4/d13/d1b/d4f 0 2024-02-16T22:56:13.951 INFO:tasks.workunit.client.1.smithi192.stdout:0/247: dwrite d1/f24 [0,4194304] 0 2024-02-16T22:56:13.952 INFO:tasks.workunit.client.1.smithi192.stdout:3/277: sync 2024-02-16T22:56:13.957 INFO:tasks.workunit.client.0.smithi064.stdout:1/180: dwrite d2/f43 [0,4194304] 0 2024-02-16T22:56:13.965 INFO:tasks.workunit.client.0.smithi064.stdout:3/235: unlink d4/c21 0 2024-02-16T22:56:13.967 INFO:tasks.workunit.client.1.smithi192.stdout:5/289: mknod d6/d12/d4e/d46/d4f/c54 0 2024-02-16T22:56:13.971 INFO:tasks.workunit.client.0.smithi064.stdout:6/234: link d0/d6/d10/d33/f32 d0/d6/d10/f40 0 2024-02-16T22:56:13.972 INFO:tasks.workunit.client.1.smithi192.stdout:4/291: mknod d1/d15/c6a 0 2024-02-16T22:56:13.976 INFO:tasks.workunit.client.0.smithi064.stdout:3/236: mknod d4/d13/d1b/c50 0 2024-02-16T22:56:13.979 INFO:tasks.workunit.client.1.smithi192.stdout:2/284: stat d22/d24/d2b/d3e/d4f/c5e 0 2024-02-16T22:56:13.979 INFO:tasks.workunit.client.1.smithi192.stdout:2/285: readlink l1e 0 2024-02-16T22:56:13.979 INFO:tasks.workunit.client.0.smithi064.stdout:6/235: dwrite d0/d5/f2b [4194304,4194304] 0 2024-02-16T22:56:13.980 INFO:tasks.workunit.client.1.smithi192.stdout:2/286: dread - d22/d24/d2b/f38 zero size 2024-02-16T22:56:13.982 INFO:tasks.workunit.client.1.smithi192.stdout:3/278: creat da/d11/d20/f69 x:0 0 0 2024-02-16T22:56:13.984 INFO:tasks.workunit.client.0.smithi064.stdout:5/150: rename d6/c16 to d6/c36 0 2024-02-16T22:56:13.986 INFO:tasks.workunit.client.0.smithi064.stdout:6/236: fdatasync d0/d6/da/f25 0 2024-02-16T22:56:13.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.986+0000 7f984646b700 1 -- 172.21.15.64:0/2808256741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98400fafa0 msgr2=0x7f98400fb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:13.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.986+0000 7f984646b700 1 --2- 172.21.15.64:0/2808256741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98400fafa0 0x7f98400fb3b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f983c004940 tx=0x7f983c009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:13.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.986+0000 7f984646b700 1 -- 172.21.15.64:0/2808256741 shutdown_connections 2024-02-16T22:56:13.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.986+0000 7f984646b700 1 --2- 172.21.15.64:0/2808256741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98400fe030 0x7f98400fe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:13.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.986+0000 7f984646b700 1 --2- 172.21.15.64:0/2808256741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98400fafa0 0x7f98400fb3b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:13.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.986+0000 7f984646b700 1 -- 172.21.15.64:0/2808256741 >> 172.21.15.64:0/2808256741 conn(0x7f98400f6a20 msgr2=0x7f98400f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:13.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.986+0000 7f984646b700 1 -- 172.21.15.64:0/2808256741 shutdown_connections 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.986+0000 7f984646b700 1 -- 172.21.15.64:0/2808256741 wait complete. 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.987+0000 7f984646b700 1 Processor -- start 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.987+0000 7f984646b700 1 -- start start 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.987+0000 7f984646b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98400fafa0 0x7f984010b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.987+0000 7f984646b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98400fe030 0x7f984010b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.987+0000 7f984646b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984010bf80 con 0x7f98400fafa0 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.987+0000 7f984646b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984010c820 con 0x7f98400fe030 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.988+0000 7f9844c68700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98400fe030 0x7f984010b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.988+0000 7f9844c68700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98400fe030 0x7f984010b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:39406/0 (socket says 172.21.15.64:39406) 2024-02-16T22:56:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.988+0000 7f9844c68700 1 -- 172.21.15.64:0/1218176620 learned_addr learned my addr 172.21.15.64:0/1218176620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:13.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.988+0000 7f9845469700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98400fafa0 0x7f984010b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:13.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.989+0000 7f9844c68700 1 -- 172.21.15.64:0/1218176620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98400fafa0 msgr2=0x7f984010b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:13.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.989+0000 7f9844c68700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98400fafa0 0x7f984010b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:13.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.989+0000 7f9844c68700 1 -- 172.21.15.64:0/1218176620 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c005040 con 0x7f98400fe030 2024-02-16T22:56:13.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.990+0000 7f9844c68700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98400fe030 0x7f984010b930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f983000eb40 tx=0x7f983000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.990+0000 7f9835ffb700 1 -- 172.21.15.64:0/1218176620 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9830007cc0 con 0x7f98400fe030 2024-02-16T22:56:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.990+0000 7f9835ffb700 1 -- 172.21.15.64:0/1218176620 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f983000f040 con 0x7f98400fe030 2024-02-16T22:56:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.990+0000 7f9835ffb700 1 -- 172.21.15.64:0/1218176620 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983000ca00 con 0x7f98400fe030 2024-02-16T22:56:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.990+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984010caa0 con 0x7f98400fe030 2024-02-16T22:56:13.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.991+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984010cfe0 con 0x7f98400fe030 2024-02-16T22:56:13.992 INFO:tasks.workunit.client.0.smithi064.stdout:3/237: rename d4/d13/d1b/d3f to d4/d13/d1b/d27/d4e/d51 0 2024-02-16T22:56:13.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.992+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984004bed0 con 0x7f98400fe030 2024-02-16T22:56:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.993+0000 7f9835ffb700 1 -- 172.21.15.64:0/1218176620 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f983000cb60 con 0x7f98400fe030 2024-02-16T22:56:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.993+0000 7f9835ffb700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f982c06b3b0 0x7f982c06d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.993+0000 7f9835ffb700 1 -- 172.21.15.64:0/1218176620 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f98300898f0 con 0x7f98400fe030 2024-02-16T22:56:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.994+0000 7f9845469700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f982c06b3b0 0x7f982c06d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.994+0000 7f9845469700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f982c06b3b0 0x7f982c06d860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f983c013740 tx=0x7f983c009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:13.996 INFO:tasks.workunit.client.0.smithi064.stdout:6/237: fdatasync d0/d1b/f23 0 2024-02-16T22:56:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:13.998+0000 7f9835ffb700 1 -- 172.21.15.64:0/1218176620 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f983004f620 con 0x7f98400fe030 2024-02-16T22:56:14.002 INFO:tasks.workunit.client.0.smithi064.stdout:6/238: dwrite d0/d6/d10/f14 [4194304,4194304] 0 2024-02-16T22:56:14.005 INFO:tasks.workunit.client.0.smithi064.stdout:3/238: rmdir d4/d13/d1b/d27/d44 39 2024-02-16T22:56:14.006 INFO:tasks.workunit.client.0.smithi064.stdout:7/176: dwrite d5/d6/f9 [0,4194304] 0 2024-02-16T22:56:14.007 INFO:tasks.workunit.client.1.smithi192.stdout:1/237: sync 2024-02-16T22:56:14.008 INFO:tasks.workunit.client.1.smithi192.stdout:2/287: creat d22/d24/d39/d42/d52/f60 x:0 0 0 2024-02-16T22:56:14.008 INFO:tasks.workunit.client.0.smithi064.stdout:3/239: read - d4/d1f/d34/f45 zero size 2024-02-16T22:56:14.009 INFO:tasks.workunit.client.1.smithi192.stdout:1/238: write d3/de/d27/f39 [4129410,30884] 0 2024-02-16T22:56:14.010 INFO:tasks.workunit.client.0.smithi064.stdout:3/240: chown d4/d1f/f32 9375 1 2024-02-16T22:56:14.011 INFO:tasks.workunit.client.1.smithi192.stdout:1/239: write d3/de/d27/d34/f45 [83000,61920] 0 2024-02-16T22:56:14.011 INFO:tasks.workunit.client.0.smithi064.stdout:3/241: write d4/d14/f2a [1556525,33087] 0 2024-02-16T22:56:14.015 INFO:tasks.workunit.client.1.smithi192.stdout:0/248: creat d1/d3/de/d22/d37/d47/f48 x:0 0 0 2024-02-16T22:56:14.015 INFO:tasks.workunit.client.0.smithi064.stdout:6/239: creat d0/f41 x:0 0 0 2024-02-16T22:56:14.018 INFO:tasks.workunit.client.0.smithi064.stdout:5/151: link d6/f29 d6/f37 0 2024-02-16T22:56:14.019 INFO:tasks.workunit.client.1.smithi192.stdout:2/288: dwrite d22/d24/d2b/d2c/d3f/f4e [0,4194304] 0 2024-02-16T22:56:14.020 INFO:tasks.workunit.client.0.smithi064.stdout:5/152: fdatasync d6/d25/f32 0 2024-02-16T22:56:14.023 INFO:tasks.workunit.client.0.smithi064.stdout:1/181: dwrite d2/df/f1c [0,4194304] 0 2024-02-16T22:56:14.026 INFO:tasks.workunit.client.1.smithi192.stdout:4/292: mkdir d1/d6b 0 2024-02-16T22:56:14.029 INFO:tasks.workunit.client.1.smithi192.stdout:7/332: link d1/d19/d2a/c2b d1/d19/d2a/c76 0 2024-02-16T22:56:14.033 INFO:tasks.workunit.client.1.smithi192.stdout:6/280: getdents da/df/d12/d15/d1b 0 2024-02-16T22:56:14.036 INFO:tasks.workunit.client.1.smithi192.stdout:9/351: truncate d7/df/f16 115233 0 2024-02-16T22:56:14.039 INFO:tasks.workunit.client.0.smithi064.stdout:7/177: mknod d5/c31 0 2024-02-16T22:56:14.039 INFO:tasks.workunit.client.1.smithi192.stdout:6/281: dwrite da/f34 [0,4194304] 0 2024-02-16T22:56:14.040 INFO:tasks.workunit.client.1.smithi192.stdout:3/279: mkdir da/d6a 0 2024-02-16T22:56:14.040 INFO:tasks.workunit.client.1.smithi192.stdout:6/282: stat da/df/d12/d15/d1b/f1e 0 2024-02-16T22:56:14.041 INFO:tasks.workunit.client.1.smithi192.stdout:3/280: chown da/d11 497 1 2024-02-16T22:56:14.041 INFO:tasks.workunit.client.1.smithi192.stdout:1/240: symlink d3/de/d27/l4d 0 2024-02-16T22:56:14.042 INFO:tasks.workunit.client.0.smithi064.stdout:3/242: symlink d4/d13/d1b/d48/l52 0 2024-02-16T22:56:14.042 INFO:tasks.workunit.client.1.smithi192.stdout:0/249: mknod d1/d3/de/d22/d37/d47/c49 0 2024-02-16T22:56:14.043 INFO:tasks.workunit.client.1.smithi192.stdout:6/283: truncate da/df/d18/d52/f4a 806475 0 2024-02-16T22:56:14.044 INFO:tasks.workunit.client.1.smithi192.stdout:6/284: write da/df/d12/d15/f21 [1217723,31253] 0 2024-02-16T22:56:14.048 INFO:tasks.workunit.client.1.smithi192.stdout:2/289: mknod d22/d24/d2b/d3e/c61 0 2024-02-16T22:56:14.048 INFO:tasks.workunit.client.0.smithi064.stdout:7/178: unlink d5/f26 0 2024-02-16T22:56:14.050 INFO:tasks.workunit.client.0.smithi064.stdout:3/243: dwrite d4/d1f/d34/f3c [0,4194304] 0 2024-02-16T22:56:14.050 INFO:tasks.workunit.client.0.smithi064.stdout:7/179: fsync d5/d6/f22 0 2024-02-16T22:56:14.051 INFO:tasks.workunit.client.1.smithi192.stdout:9/352: symlink d7/df/l76 0 2024-02-16T22:56:14.055 INFO:tasks.workunit.client.1.smithi192.stdout:3/281: rename da/df/d42 to da/d11/d63/d6b 0 2024-02-16T22:56:14.060 INFO:tasks.workunit.client.0.smithi064.stdout:2/158: sync 2024-02-16T22:56:14.061 INFO:tasks.workunit.client.0.smithi064.stdout:7/180: unlink d5/f29 0 2024-02-16T22:56:14.064 INFO:tasks.workunit.client.0.smithi064.stdout:6/240: getdents d0/d1d/d21 0 2024-02-16T22:56:14.065 INFO:tasks.workunit.client.0.smithi064.stdout:6/241: dread - d0/d5/f3d zero size 2024-02-16T22:56:14.067 INFO:tasks.workunit.client.0.smithi064.stdout:6/242: dread d0/d6/d10/d33/f2a [0,4194304] 0 2024-02-16T22:56:14.067 INFO:tasks.workunit.client.0.smithi064.stdout:5/153: link d6/c7 d6/c38 0 2024-02-16T22:56:14.068 INFO:tasks.workunit.client.0.smithi064.stdout:6/243: read - d0/f41 zero size 2024-02-16T22:56:14.068 INFO:tasks.workunit.client.0.smithi064.stdout:5/154: readlink d6/l11 0 2024-02-16T22:56:14.069 INFO:tasks.workunit.client.0.smithi064.stdout:6/244: chown d0/d6/d10/l2f 30 1 2024-02-16T22:56:14.076 INFO:tasks.workunit.client.0.smithi064.stdout:5/155: dwrite d6/d1c/f20 [0,4194304] 0 2024-02-16T22:56:14.076 INFO:tasks.workunit.client.1.smithi192.stdout:1/241: creat d3/de/d27/d34/f4e x:0 0 0 2024-02-16T22:56:14.076 INFO:tasks.workunit.client.1.smithi192.stdout:5/290: link d6/d12/d4e/d46/l47 d6/d12/d1d/d3c/l55 0 2024-02-16T22:56:14.076 INFO:tasks.workunit.client.1.smithi192.stdout:3/282: fdatasync da/f4c 0 2024-02-16T22:56:14.076 INFO:tasks.workunit.client.1.smithi192.stdout:8/263: sync 2024-02-16T22:56:14.077 INFO:tasks.workunit.client.1.smithi192.stdout:6/285: rmdir da/df/d18 39 2024-02-16T22:56:14.077 INFO:tasks.workunit.client.0.smithi064.stdout:5/156: write d6/d25/f32 [171894,119074] 0 2024-02-16T22:56:14.077 INFO:tasks.workunit.client.0.smithi064.stdout:2/159: creat d7/d14/f2c x:0 0 0 2024-02-16T22:56:14.078 INFO:tasks.workunit.client.1.smithi192.stdout:9/353: dwrite d7/d19/d28/f52 [0,4194304] 0 2024-02-16T22:56:14.079 INFO:tasks.workunit.client.0.smithi064.stdout:5/157: write d6/f23 [5190066,68791] 0 2024-02-16T22:56:14.087 INFO:tasks.workunit.client.0.smithi064.stdout:6/245: truncate d0/f22 2290428 0 2024-02-16T22:56:14.088 INFO:tasks.workunit.client.1.smithi192.stdout:8/264: dwrite d5/db/de/df/f2b [0,4194304] 0 2024-02-16T22:56:14.088 INFO:tasks.workunit.client.1.smithi192.stdout:8/265: read - d5/d16/f45 zero size 2024-02-16T22:56:14.089 INFO:tasks.workunit.client.1.smithi192.stdout:8/266: fsync d5/d16/f27 0 2024-02-16T22:56:14.094 INFO:tasks.workunit.client.0.smithi064.stdout:1/182: truncate d2/f3 4237055 0 2024-02-16T22:56:14.097 INFO:tasks.workunit.client.0.smithi064.stdout:9/194: sync 2024-02-16T22:56:14.097 INFO:tasks.workunit.client.0.smithi064.stdout:0/148: sync 2024-02-16T22:56:14.097 INFO:tasks.workunit.client.0.smithi064.stdout:8/189: sync 2024-02-16T22:56:14.097 INFO:tasks.workunit.client.0.smithi064.stdout:4/164: sync 2024-02-16T22:56:14.099 INFO:tasks.workunit.client.0.smithi064.stdout:9/195: stat d3/d11/d2b/d2a/d2e/d31 0 2024-02-16T22:56:14.100 INFO:tasks.workunit.client.0.smithi064.stdout:0/149: readlink da/dc/ld 0 2024-02-16T22:56:14.101 INFO:tasks.workunit.client.1.smithi192.stdout:2/290: unlink d22/d24/d2b/d3e/d4f/c58 0 2024-02-16T22:56:14.102 INFO:tasks.workunit.client.1.smithi192.stdout:2/291: write d22/f3c [522375,37871] 0 2024-02-16T22:56:14.104 INFO:tasks.workunit.client.0.smithi064.stdout:1/183: dread d2/df/d14/f16 [0,4194304] 0 2024-02-16T22:56:14.105 INFO:tasks.workunit.client.0.smithi064.stdout:4/165: dwrite dc/f19 [0,4194304] 0 2024-02-16T22:56:14.107 INFO:tasks.workunit.client.1.smithi192.stdout:2/292: dwrite f5 [0,4194304] 0 2024-02-16T22:56:14.109 INFO:tasks.workunit.client.1.smithi192.stdout:2/293: write d22/d24/d2b/d2c/f37 [74597,74109] 0 2024-02-16T22:56:14.114 INFO:tasks.workunit.client.1.smithi192.stdout:9/354: mkdir d7/d15/d36/d77 0 2024-02-16T22:56:14.116 INFO:tasks.workunit.client.1.smithi192.stdout:1/242: rmdir d3 39 2024-02-16T22:56:14.116 INFO:tasks.workunit.client.1.smithi192.stdout:4/293: getdents d1/d8/d11/d58 0 2024-02-16T22:56:14.120 INFO:tasks.workunit.client.0.smithi064.stdout:3/244: dwrite d4/d14/f30 [0,4194304] 0 2024-02-16T22:56:14.121 INFO:tasks.workunit.client.0.smithi064.stdout:3/245: dread - d4/d14/f4c zero size 2024-02-16T22:56:14.122 INFO:tasks.workunit.client.0.smithi064.stdout:9/196: fdatasync d3/f10 0 2024-02-16T22:56:14.122 INFO:tasks.workunit.client.0.smithi064.stdout:9/197: stat c2 0 2024-02-16T22:56:14.125 INFO:tasks.workunit.client.0.smithi064.stdout:0/150: creat da/dc/de/d12/d1d/f22 x:0 0 0 2024-02-16T22:56:14.126 INFO:tasks.workunit.client.1.smithi192.stdout:6/286: creat da/df/d18/d52/f55 x:0 0 0 2024-02-16T22:56:14.128 INFO:tasks.workunit.client.1.smithi192.stdout:7/333: getdents d1/d2/de/d2c 0 2024-02-16T22:56:14.131 INFO:tasks.workunit.client.1.smithi192.stdout:5/291: dwrite d6/d12/d2d/f43 [0,4194304] 0 2024-02-16T22:56:14.133 INFO:tasks.workunit.client.1.smithi192.stdout:5/292: truncate d6/d12/d1d/f34 711273 0 2024-02-16T22:56:14.134 INFO:tasks.workunit.client.0.smithi064.stdout:4/166: creat dc/dd/d16/f2e x:0 0 0 2024-02-16T22:56:14.136 INFO:tasks.workunit.client.1.smithi192.stdout:2/294: symlink d22/d24/d2b/d2c/l62 0 2024-02-16T22:56:14.136 INFO:tasks.workunit.client.1.smithi192.stdout:2/295: stat fa 0 2024-02-16T22:56:14.138 INFO:tasks.workunit.client.0.smithi064.stdout:5/158: write d6/d18/d19/f26 [2194112,84389] 0 2024-02-16T22:56:14.139 INFO:tasks.workunit.client.0.smithi064.stdout:5/159: readlink d6/d1c/l2b 0 2024-02-16T22:56:14.143 INFO:tasks.workunit.client.1.smithi192.stdout:0/250: sync 2024-02-16T22:56:14.144 INFO:tasks.workunit.client.1.smithi192.stdout:0/251: fdatasync d1/d3/de/d22/f45 0 2024-02-16T22:56:14.144 INFO:tasks.workunit.client.0.smithi064.stdout:4/167: dwrite dc/d10/d18/d22/f24 [0,4194304] 0 2024-02-16T22:56:14.147 INFO:tasks.workunit.client.1.smithi192.stdout:9/355: dwrite d7/d15/d36/d38/f4b [0,4194304] 0 2024-02-16T22:56:14.150 INFO:tasks.workunit.client.0.smithi064.stdout:6/246: dwrite d0/d6/d10/d33/f32 [0,4194304] 0 2024-02-16T22:56:14.151 INFO:tasks.workunit.client.0.smithi064.stdout:3/246: mknod d4/d13/d1b/d43/c53 0 2024-02-16T22:56:14.152 INFO:tasks.workunit.client.1.smithi192.stdout:8/267: creat d5/d3e/d49/f4b x:0 0 0 2024-02-16T22:56:14.153 INFO:tasks.workunit.client.0.smithi064.stdout:1/184: write d2/df/d14/d18/f26 [4636170,112946] 0 2024-02-16T22:56:14.153 INFO:tasks.workunit.client.1.smithi192.stdout:8/268: rename d5 to d5/d3e/d49/d4c 22 2024-02-16T22:56:14.155 INFO:tasks.workunit.client.0.smithi064.stdout:4/168: dwrite dc/d10/f12 [0,4194304] 0 2024-02-16T22:56:14.158 INFO:tasks.workunit.client.0.smithi064.stdout:0/151: symlink da/dc/l23 0 2024-02-16T22:56:14.158 INFO:tasks.workunit.client.0.smithi064.stdout:1/185: write d2/df/f1c [4146342,64343] 0 2024-02-16T22:56:14.161 INFO:tasks.workunit.client.1.smithi192.stdout:2/296: symlink d22/d24/d39/l63 0 2024-02-16T22:56:14.162 INFO:tasks.workunit.client.1.smithi192.stdout:2/297: stat d22/d24/f49 0 2024-02-16T22:56:14.164 INFO:tasks.workunit.client.0.smithi064.stdout:5/160: unlink d6/f33 0 2024-02-16T22:56:14.167 INFO:tasks.workunit.client.0.smithi064.stdout:2/160: rename d7/l1c to d7/de/l2d 0 2024-02-16T22:56:14.171 INFO:tasks.workunit.client.1.smithi192.stdout:0/252: symlink d1/d3/de/d22/d37/d47/l4a 0 2024-02-16T22:56:14.174 INFO:tasks.workunit.client.1.smithi192.stdout:9/356: mknod d7/d15/d36/d54/c78 0 2024-02-16T22:56:14.175 INFO:tasks.workunit.client.1.smithi192.stdout:9/357: write d7/d15/f20 [4816861,50058] 0 2024-02-16T22:56:14.175 INFO:tasks.workunit.client.1.smithi192.stdout:6/287: symlink da/l56 0 2024-02-16T22:56:14.176 INFO:tasks.workunit.client.0.smithi064.stdout:9/198: dwrite d3/d9/f41 [0,4194304] 0 2024-02-16T22:56:14.180 INFO:tasks.workunit.client.1.smithi192.stdout:8/269: rename d5/d3e/d49 to d5/d3a/d4d 0 2024-02-16T22:56:14.182 INFO:tasks.workunit.client.1.smithi192.stdout:5/293: mkdir d6/d12/d1d/d56 0 2024-02-16T22:56:14.183 INFO:tasks.workunit.client.1.smithi192.stdout:3/283: getdents da/d11/d20/d39 0 2024-02-16T22:56:14.184 INFO:tasks.workunit.client.0.smithi064.stdout:4/169: mknod dc/de/c2f 0 2024-02-16T22:56:14.186 INFO:tasks.workunit.client.1.smithi192.stdout:1/243: getdents d3/d40 0 2024-02-16T22:56:14.187 INFO:tasks.workunit.client.1.smithi192.stdout:3/284: write da/d11/d2b/f5e [988234,24054] 0 2024-02-16T22:56:14.189 INFO:tasks.workunit.client.1.smithi192.stdout:2/298: unlink d22/d24/d2b/d2c/d3f/f4e 0 2024-02-16T22:56:14.190 INFO:tasks.workunit.client.1.smithi192.stdout:8/270: dwrite d5/f38 [0,4194304] 0 2024-02-16T22:56:14.191 INFO:tasks.workunit.client.0.smithi064.stdout:0/152: fsync f9 0 2024-02-16T22:56:14.191 INFO:tasks.workunit.client.1.smithi192.stdout:8/271: chown d5/d3e 6463 1 2024-02-16T22:56:14.192 INFO:tasks.workunit.client.1.smithi192.stdout:4/294: link d1/d15/d29/c46 d1/d15/d48/c6c 0 2024-02-16T22:56:14.192 INFO:tasks.workunit.client.0.smithi064.stdout:4/170: dwrite dc/f2c [0,4194304] 0 2024-02-16T22:56:14.199 INFO:tasks.workunit.client.0.smithi064.stdout:5/161: mkdir d6/d18/d39 0 2024-02-16T22:56:14.203 INFO:tasks.workunit.client.0.smithi064.stdout:8/190: rename l0 to df/d2a/l3c 0 2024-02-16T22:56:14.204 INFO:tasks.workunit.client.0.smithi064.stdout:2/161: fdatasync d7/f8 0 2024-02-16T22:56:14.207 INFO:tasks.workunit.client.0.smithi064.stdout:3/247: mknod d4/c54 0 2024-02-16T22:56:14.208 INFO:tasks.workunit.client.0.smithi064.stdout:3/248: chown d4/d1f/f32 9133559 1 2024-02-16T22:56:14.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/294: dread d6/d12/d1d/f36 [0,4194304] 0 2024-02-16T22:56:14.209 INFO:tasks.workunit.client.0.smithi064.stdout:3/249: chown d4/d13/d1b/d27/f36 360 1 2024-02-16T22:56:14.210 INFO:tasks.workunit.client.0.smithi064.stdout:9/199: rmdir d3/d11/d2b/d2a/d2e 39 2024-02-16T22:56:14.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/244: symlink d3/de/d27/d34/d36/l4f 0 2024-02-16T22:56:14.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/245: chown d3/de/c1d 84626 1 2024-02-16T22:56:14.214 INFO:tasks.workunit.client.1.smithi192.stdout:1/246: dread - d3/de/f32 zero size 2024-02-16T22:56:14.217 INFO:tasks.workunit.client.0.smithi064.stdout:1/186: symlink d2/df/d14/d18/d40/l44 0 2024-02-16T22:56:14.219 INFO:tasks.workunit.client.1.smithi192.stdout:4/295: symlink d1/d26/l6d 0 2024-02-16T22:56:14.220 INFO:tasks.workunit.client.1.smithi192.stdout:1/247: dwrite d3/de/d27/f30 [0,4194304] 0 2024-02-16T22:56:14.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/296: write d1/d8/d11/f2a [3676956,118247] 0 2024-02-16T22:56:14.221 INFO:tasks.workunit.client.0.smithi064.stdout:8/191: creat df/f3d x:0 0 0 2024-02-16T22:56:14.223 INFO:tasks.workunit.client.0.smithi064.stdout:4/171: dread f3 [0,4194304] 0 2024-02-16T22:56:14.223 INFO:tasks.workunit.client.1.smithi192.stdout:0/253: unlink d1/d3/cf 0 2024-02-16T22:56:14.226 INFO:tasks.workunit.client.1.smithi192.stdout:9/358: mknod d7/d15/d36/d38/c79 0 2024-02-16T22:56:14.226 INFO:tasks.workunit.client.0.smithi064.stdout:7/181: link d5/d6/fb d5/f32 0 2024-02-16T22:56:14.229 INFO:tasks.workunit.client.1.smithi192.stdout:1/248: dwrite d3/de/d27/f4a [0,4194304] 0 2024-02-16T22:56:14.231 INFO:tasks.workunit.client.0.smithi064.stdout:8/192: dread fa [0,4194304] 0 2024-02-16T22:56:14.231 INFO:tasks.workunit.client.0.smithi064.stdout:3/250: creat d4/d13/f55 x:0 0 0 2024-02-16T22:56:14.235 INFO:tasks.workunit.client.0.smithi064.stdout:9/200: creat d3/d9/f4a x:0 0 0 2024-02-16T22:56:14.237 INFO:tasks.workunit.client.1.smithi192.stdout:5/295: creat d6/d12/d4e/d46/f57 x:0 0 0 2024-02-16T22:56:14.241 INFO:tasks.workunit.client.0.smithi064.stdout:2/162: dread d7/f8 [4194304,4194304] 0 2024-02-16T22:56:14.242 INFO:tasks.workunit.client.0.smithi064.stdout:4/172: symlink dc/d10/l30 0 2024-02-16T22:56:14.242 INFO:tasks.workunit.client.0.smithi064.stdout:2/163: write d7/d14/f2c [420036,123138] 0 2024-02-16T22:56:14.243 INFO:tasks.workunit.client.1.smithi192.stdout:8/272: link d5/d9/f33 d5/d16/d34/f4e 0 2024-02-16T22:56:14.243 INFO:tasks.workunit.client.0.smithi064.stdout:2/164: read d7/d14/f2c [498564,29584] 0 2024-02-16T22:56:14.245 INFO:tasks.workunit.client.0.smithi064.stdout:2/165: read - d7/d14/d25/f27 zero size 2024-02-16T22:56:14.245 INFO:tasks.workunit.client.1.smithi192.stdout:6/288: dwrite da/df/d12/d15/f31 [0,4194304] 0 2024-02-16T22:56:14.247 INFO:tasks.workunit.client.0.smithi064.stdout:6/247: getdents d0/d1b 0 2024-02-16T22:56:14.250 INFO:tasks.workunit.client.1.smithi192.stdout:0/254: rename d1/d3/de/c1e to d1/d3/de/d22/d37/c4b 0 2024-02-16T22:56:14.252 INFO:tasks.workunit.client.1.smithi192.stdout:0/255: dread - d1/d3/de/d22/d37/d47/f48 zero size 2024-02-16T22:56:14.254 INFO:tasks.workunit.client.0.smithi064.stdout:2/166: dwrite d7/d14/d25/f27 [0,4194304] 0 2024-02-16T22:56:14.255 INFO:tasks.workunit.client.0.smithi064.stdout:2/167: truncate d7/f12 461626 0 2024-02-16T22:56:14.257 INFO:tasks.workunit.client.1.smithi192.stdout:3/285: truncate da/d11/f4b 2900990 0 2024-02-16T22:56:14.259 INFO:tasks.workunit.client.1.smithi192.stdout:7/334: link d1/d2/d2d/c5b d1/c77 0 2024-02-16T22:56:14.260 INFO:tasks.workunit.client.0.smithi064.stdout:0/153: rename c1 to da/c24 0 2024-02-16T22:56:14.261 INFO:tasks.workunit.client.1.smithi192.stdout:2/299: sync 2024-02-16T22:56:14.263 INFO:tasks.workunit.client.1.smithi192.stdout:5/296: mkdir d6/d12/d4e/d58 0 2024-02-16T22:56:14.263 INFO:tasks.workunit.client.0.smithi064.stdout:7/182: mkdir d5/d33 0 2024-02-16T22:56:14.264 INFO:tasks.workunit.client.0.smithi064.stdout:7/183: read d5/f27 [3641758,105228] 0 2024-02-16T22:56:14.268 INFO:tasks.workunit.client.1.smithi192.stdout:8/273: write d5/db/de/df/f2d [2490575,121300] 0 2024-02-16T22:56:14.269 INFO:tasks.workunit.client.1.smithi192.stdout:3/286: dwrite da/f4c [0,4194304] 0 2024-02-16T22:56:14.272 INFO:tasks.workunit.client.1.smithi192.stdout:9/359: creat d7/d19/d5e/d6f/f7a x:0 0 0 2024-02-16T22:56:14.273 INFO:tasks.workunit.client.0.smithi064.stdout:2/168: creat d7/d14/f2e x:0 0 0 2024-02-16T22:56:14.274 INFO:tasks.workunit.client.1.smithi192.stdout:7/335: dwrite d1/d2/f58 [8388608,4194304] 0 2024-02-16T22:56:14.274 INFO:tasks.workunit.client.0.smithi064.stdout:4/173: dread dc/dd/d16/f23 [0,4194304] 0 2024-02-16T22:56:14.274 INFO:tasks.workunit.client.1.smithi192.stdout:3/287: write da/d11/f38 [2310428,100687] 0 2024-02-16T22:56:14.275 INFO:tasks.workunit.client.0.smithi064.stdout:4/174: fsync dc/f2c 0 2024-02-16T22:56:14.275 INFO:tasks.workunit.client.1.smithi192.stdout:9/360: readlink d7/d15/l5f 0 2024-02-16T22:56:14.278 INFO:tasks.workunit.client.0.smithi064.stdout:5/162: rename d6/d18/l2f to d6/d25/l3a 0 2024-02-16T22:56:14.278 INFO:tasks.workunit.client.1.smithi192.stdout:4/297: dwrite d1/d15/d48/f55 [0,4194304] 0 2024-02-16T22:56:14.280 INFO:tasks.workunit.client.1.smithi192.stdout:4/298: chown d1/d15/d29/c46 1 1 2024-02-16T22:56:14.282 INFO:tasks.workunit.client.0.smithi064.stdout:0/154: symlink da/dc/de/l25 0 2024-02-16T22:56:14.283 INFO:tasks.workunit.client.1.smithi192.stdout:2/300: dread f1 [0,4194304] 0 2024-02-16T22:56:14.285 INFO:tasks.workunit.client.0.smithi064.stdout:5/163: dwrite f0 [0,4194304] 0 2024-02-16T22:56:14.286 INFO:tasks.workunit.client.1.smithi192.stdout:2/301: stat d22/d47 0 2024-02-16T22:56:14.286 INFO:tasks.workunit.client.0.smithi064.stdout:5/164: dread - d6/f37 zero size 2024-02-16T22:56:14.290 INFO:tasks.workunit.client.1.smithi192.stdout:2/302: dread d22/f31 [0,4194304] 0 2024-02-16T22:56:14.291 INFO:tasks.workunit.client.1.smithi192.stdout:7/336: dwrite d1/d19/d4c/f70 [0,4194304] 0 2024-02-16T22:56:14.293 INFO:tasks.workunit.client.1.smithi192.stdout:5/297: symlink d6/d12/d4e/l59 0 2024-02-16T22:56:14.293 INFO:tasks.workunit.client.1.smithi192.stdout:7/337: readlink d1/l6e 0 2024-02-16T22:56:14.293 INFO:tasks.workunit.client.1.smithi192.stdout:3/288: dwrite da/d11/d63/d6b/d4e/f56 [4194304,4194304] 0 2024-02-16T22:56:14.297 INFO:tasks.workunit.client.0.smithi064.stdout:6/248: link d0/d5/f3d d0/d5/f42 0 2024-02-16T22:56:14.298 INFO:tasks.workunit.client.0.smithi064.stdout:8/193: dwrite df/d31/d2f/f3a [0,4194304] 0 2024-02-16T22:56:14.300 INFO:tasks.workunit.client.0.smithi064.stdout:9/201: rename d3/d11/d2b/d2a/f30 to d3/d11/d2b/d2a/f4b 0 2024-02-16T22:56:14.303 INFO:tasks.workunit.client.0.smithi064.stdout:8/194: dread f1 [0,4194304] 0 2024-02-16T22:56:14.305 INFO:tasks.workunit.client.1.smithi192.stdout:8/274: creat d5/d16/f4f x:0 0 0 2024-02-16T22:56:14.308 INFO:tasks.workunit.client.0.smithi064.stdout:7/184: write d5/d6/fa [1567544,92535] 0 2024-02-16T22:56:14.311 INFO:tasks.workunit.client.1.smithi192.stdout:6/289: truncate da/df/f1c 5986449 0 2024-02-16T22:56:14.313 INFO:tasks.workunit.client.1.smithi192.stdout:1/249: creat d3/f50 x:0 0 0 2024-02-16T22:56:14.316 INFO:tasks.workunit.client.0.smithi064.stdout:1/187: rename d2/df/f1f to d2/df/d14/d18/d3e/f45 0 2024-02-16T22:56:14.321 INFO:tasks.workunit.client.0.smithi064.stdout:5/165: rmdir d6/d1c 39 2024-02-16T22:56:14.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/338: write d1/d2/f46 [1633706,10128] 0 2024-02-16T22:56:14.323 INFO:tasks.workunit.client.1.smithi192.stdout:4/299: dread d1/fd [0,4194304] 0 2024-02-16T22:56:14.324 INFO:tasks.workunit.client.1.smithi192.stdout:5/298: symlink d6/d12/d2d/l5a 0 2024-02-16T22:56:14.324 INFO:tasks.workunit.client.0.smithi064.stdout:6/249: mknod d0/d1d/c43 0 2024-02-16T22:56:14.325 INFO:tasks.workunit.client.1.smithi192.stdout:5/299: chown d6/d12/d4e/l59 1208560 1 2024-02-16T22:56:14.325 INFO:tasks.workunit.client.1.smithi192.stdout:5/300: chown d6/d8/l21 26 1 2024-02-16T22:56:14.326 INFO:tasks.workunit.client.1.smithi192.stdout:3/289: unlink da/d11/d35/f59 0 2024-02-16T22:56:14.327 INFO:tasks.workunit.client.1.smithi192.stdout:8/275: mknod d5/db/de/d21/c50 0 2024-02-16T22:56:14.330 INFO:tasks.workunit.client.1.smithi192.stdout:3/290: dread da/df/f3a [0,4194304] 0 2024-02-16T22:56:14.330 INFO:tasks.workunit.client.1.smithi192.stdout:0/256: rename d1/d3/de/d22/f26 to d1/d3/f4c 0 2024-02-16T22:56:14.333 INFO:tasks.workunit.client.0.smithi064.stdout:8/195: unlink df/c1e 0 2024-02-16T22:56:14.336 INFO:tasks.workunit.client.0.smithi064.stdout:7/185: creat d5/d6/f34 x:0 0 0 2024-02-16T22:56:14.338 INFO:tasks.workunit.client.0.smithi064.stdout:4/175: symlink dc/de/d28/l31 0 2024-02-16T22:56:14.340 INFO:tasks.workunit.client.0.smithi064.stdout:8/196: dwrite df/f19 [0,4194304] 0 2024-02-16T22:56:14.343 INFO:tasks.workunit.client.1.smithi192.stdout:4/300: symlink d1/d26/l6e 0 2024-02-16T22:56:14.345 INFO:tasks.workunit.client.0.smithi064.stdout:0/155: mkdir da/d26 0 2024-02-16T22:56:14.350 INFO:tasks.workunit.client.1.smithi192.stdout:5/301: mkdir d6/d12/d4e/d5b 0 2024-02-16T22:56:14.350 INFO:tasks.workunit.client.1.smithi192.stdout:5/302: chown d6/d12/d4e/d46/d4f 1637 1 2024-02-16T22:56:14.351 INFO:tasks.workunit.client.0.smithi064.stdout:5/166: creat d6/d18/d19/d1a/f3b x:0 0 0 2024-02-16T22:56:14.353 INFO:tasks.workunit.client.0.smithi064.stdout:6/250: mknod d0/d5/c44 0 2024-02-16T22:56:14.357 INFO:tasks.workunit.client.1.smithi192.stdout:3/291: fdatasync da/d11/d63/d6b/d66/f60 0 2024-02-16T22:56:14.358 INFO:tasks.workunit.client.0.smithi064.stdout:9/202: truncate d3/fc 1087837 0 2024-02-16T22:56:14.360 INFO:tasks.workunit.client.1.smithi192.stdout:1/250: rename d3/de/d27/c3a to d3/de/d27/c51 0 2024-02-16T22:56:14.361 INFO:tasks.workunit.client.0.smithi064.stdout:2/169: creat d7/de/f2f x:0 0 0 2024-02-16T22:56:14.364 INFO:tasks.workunit.client.1.smithi192.stdout:2/303: creat d22/d24/d2b/f64 x:0 0 0 2024-02-16T22:56:14.365 INFO:tasks.workunit.client.1.smithi192.stdout:2/304: write d22/d24/d2b/d2c/f2d [232919,61315] 0 2024-02-16T22:56:14.368 INFO:tasks.workunit.client.0.smithi064.stdout:4/176: rename dc/d10/d18 to dc/dd/d32 0 2024-02-16T22:56:14.370 INFO:tasks.workunit.client.1.smithi192.stdout:5/303: mkdir d6/d12/d2d/d5c 0 2024-02-16T22:56:14.370 INFO:tasks.workunit.client.1.smithi192.stdout:9/361: dwrite d7/f12 [0,4194304] 0 2024-02-16T22:56:14.371 INFO:tasks.workunit.client.1.smithi192.stdout:2/305: dwrite d22/d24/f49 [0,4194304] 0 2024-02-16T22:56:14.371 INFO:tasks.workunit.client.0.smithi064.stdout:8/197: mkdir df/d31/d2f/d3e 0 2024-02-16T22:56:14.373 INFO:tasks.workunit.client.1.smithi192.stdout:9/362: write d7/d19/d37/f3d [5455957,100527] 0 2024-02-16T22:56:14.375 INFO:tasks.workunit.client.0.smithi064.stdout:1/188: symlink d2/df/d11/d1a/d30/d3d/l46 0 2024-02-16T22:56:14.376 INFO:tasks.workunit.client.0.smithi064.stdout:1/189: write d2/f42 [1005408,128915] 0 2024-02-16T22:56:14.380 INFO:tasks.workunit.client.1.smithi192.stdout:3/292: mknod da/d11/d63/d6b/c6c 0 2024-02-16T22:56:14.380 INFO:tasks.workunit.client.1.smithi192.stdout:3/293: chown da/d11/l13 3 1 2024-02-16T22:56:14.381 INFO:tasks.workunit.client.0.smithi064.stdout:6/251: symlink d0/d5/l45 0 2024-02-16T22:56:14.391 INFO:tasks.workunit.client.0.smithi064.stdout:7/186: mkdir d5/d33/d35 0 2024-02-16T22:56:14.395 INFO:tasks.workunit.client.0.smithi064.stdout:4/177: rename dc/d1d/f29 to dc/f33 0 2024-02-16T22:56:14.395 INFO:tasks.workunit.client.0.smithi064.stdout:4/178: write dc/d10/f12 [4709260,52013] 0 2024-02-16T22:56:14.400 INFO:tasks.workunit.client.0.smithi064.stdout:8/198: chown df/d2a/l3c 3 1 2024-02-16T22:56:14.402 INFO:tasks.workunit.client.1.smithi192.stdout:2/306: mkdir d22/d24/d39/d42/d65 0 2024-02-16T22:56:14.402 INFO:tasks.workunit.client.0.smithi064.stdout:0/156: write da/f18 [2347911,40237] 0 2024-02-16T22:56:14.403 INFO:tasks.workunit.client.0.smithi064.stdout:0/157: dread - da/dc/f13 zero size 2024-02-16T22:56:14.403 INFO:tasks.workunit.client.0.smithi064.stdout:1/190: mknod d2/df/d14/d18/d40/c47 0 2024-02-16T22:56:14.407 INFO:tasks.workunit.client.1.smithi192.stdout:6/290: rename da/d27 to da/df/d12/d57 0 2024-02-16T22:56:14.408 INFO:tasks.workunit.client.1.smithi192.stdout:6/291: write da/df/d12/f51 [481497,2534] 0 2024-02-16T22:56:14.411 INFO:tasks.workunit.client.0.smithi064.stdout:6/252: mknod d0/d1d/c46 0 2024-02-16T22:56:14.411 INFO:tasks.workunit.client.1.smithi192.stdout:9/363: creat d7/d15/d36/f7b x:0 0 0 2024-02-16T22:56:14.412 INFO:tasks.workunit.client.1.smithi192.stdout:7/339: sync 2024-02-16T22:56:14.412 INFO:tasks.workunit.client.1.smithi192.stdout:3/294: creat da/d11/d63/d6b/d4e/f6d x:0 0 0 2024-02-16T22:56:14.413 INFO:tasks.workunit.client.1.smithi192.stdout:3/295: dread - da/d11/f50 zero size 2024-02-16T22:56:14.414 INFO:tasks.workunit.client.0.smithi064.stdout:9/203: mknod d3/d11/c4c 0 2024-02-16T22:56:14.414 INFO:tasks.workunit.client.1.smithi192.stdout:1/251: getdents d3/d40 0 2024-02-16T22:56:14.415 INFO:tasks.workunit.client.0.smithi064.stdout:9/204: truncate d3/d9/f48 299149 0 2024-02-16T22:56:14.416 INFO:tasks.workunit.client.1.smithi192.stdout:4/301: truncate d1/fd 2567035 0 2024-02-16T22:56:14.417 INFO:tasks.workunit.client.1.smithi192.stdout:1/252: truncate d3/f22 4202301 0 2024-02-16T22:56:14.418 INFO:tasks.workunit.client.1.smithi192.stdout:1/253: chown d3/de/d27/c28 661 1 2024-02-16T22:56:14.418 INFO:tasks.workunit.client.1.smithi192.stdout:3/296: dread da/d11/d63/d6b/d4e/f56 [4194304,4194304] 0 2024-02-16T22:56:14.420 INFO:tasks.workunit.client.1.smithi192.stdout:3/297: readlink da/lb 0 2024-02-16T22:56:14.420 INFO:tasks.workunit.client.1.smithi192.stdout:1/254: fsync d3/de/d27/d34/f4e 0 2024-02-16T22:56:14.421 INFO:tasks.workunit.client.1.smithi192.stdout:3/298: stat da/d11/d2b/c55 0 2024-02-16T22:56:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.421+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f984010d390 con 0x7f982c06b3b0 2024-02-16T22:56:14.423 INFO:tasks.workunit.client.1.smithi192.stdout:2/307: dwrite f1 [0,4194304] 0 2024-02-16T22:56:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.422+0000 7f9835ffb700 1 -- 172.21.15.64:0/1218176620 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f984010d390 con 0x7f982c06b3b0 2024-02-16T22:56:14.423 INFO:tasks.workunit.client.1.smithi192.stdout:2/308: chown f19 62075 1 2024-02-16T22:56:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f982c06b3b0 msgr2=0x7f982c06d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f982c06b3b0 0x7f982c06d860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f983c013740 tx=0x7f983c009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98400fe030 msgr2=0x7f984010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98400fe030 0x7f984010b930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f983000eb40 tx=0x7f983000b040 comp rx=0 tx=0).stop 2024-02-16T22:56:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 shutdown_connections 2024-02-16T22:56:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98400fafa0 0x7f984010b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f982c06b3b0 0x7f982c06d860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 --2- 172.21.15.64:0/1218176620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98400fe030 0x7f984010b930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:14.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 >> 172.21.15.64:0/1218176620 conn(0x7f98400f6a20 msgr2=0x7f98400ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:14.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 shutdown_connections 2024-02-16T22:56:14.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.425+0000 7f984646b700 1 -- 172.21.15.64:0/1218176620 wait complete. 2024-02-16T22:56:14.438 INFO:tasks.workunit.client.0.smithi064.stdout:4/179: symlink dc/dd/d32/d22/l34 0 2024-02-16T22:56:14.439 INFO:tasks.workunit.client.0.smithi064.stdout:4/180: chown dc/f19 876 1 2024-02-16T22:56:14.440 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:56:14.442 INFO:tasks.workunit.client.0.smithi064.stdout:2/170: truncate d7/f13 1904919 0 2024-02-16T22:56:14.446 INFO:tasks.workunit.client.0.smithi064.stdout:7/187: write d5/f32 [2710372,77541] 0 2024-02-16T22:56:14.448 INFO:tasks.workunit.client.0.smithi064.stdout:5/167: creat d6/f3c x:0 0 0 2024-02-16T22:56:14.450 INFO:tasks.workunit.client.1.smithi192.stdout:4/302: mkdir d1/d26/d6f 0 2024-02-16T22:56:14.451 INFO:tasks.workunit.client.0.smithi064.stdout:6/253: write d0/d5/f27 [2777186,39289] 0 2024-02-16T22:56:14.452 INFO:tasks.workunit.client.0.smithi064.stdout:9/205: readlink d3/d11/d15/l46 0 2024-02-16T22:56:14.454 INFO:tasks.workunit.client.1.smithi192.stdout:6/292: mkdir da/df/d58 0 2024-02-16T22:56:14.458 INFO:tasks.workunit.client.1.smithi192.stdout:1/255: mkdir d3/de/d27/d34/d52 0 2024-02-16T22:56:14.460 INFO:tasks.workunit.client.0.smithi064.stdout:4/181: mknod dc/dd/c35 0 2024-02-16T22:56:14.461 INFO:tasks.workunit.client.1.smithi192.stdout:3/299: mknod da/d11/d63/d6b/c6e 0 2024-02-16T22:56:14.463 INFO:tasks.workunit.client.0.smithi064.stdout:1/191: dwrite d2/df/f13 [0,4194304] 0 2024-02-16T22:56:14.465 INFO:tasks.workunit.client.0.smithi064.stdout:2/171: rename d7/d14/l1b to d7/l30 0 2024-02-16T22:56:14.471 INFO:tasks.workunit.client.1.smithi192.stdout:8/276: rename d5/d3e/f2e to d5/db/f51 0 2024-02-16T22:56:14.472 INFO:tasks.workunit.client.0.smithi064.stdout:6/254: unlink d0/d5/c44 0 2024-02-16T22:56:14.473 INFO:tasks.workunit.client.0.smithi064.stdout:2/172: dwrite d7/de/f15 [4194304,4194304] 0 2024-02-16T22:56:14.475 INFO:tasks.workunit.client.1.smithi192.stdout:6/293: symlink da/df/d18/d52/l59 0 2024-02-16T22:56:14.476 INFO:tasks.workunit.client.1.smithi192.stdout:6/294: read da/df/d18/f29 [66373,74977] 0 2024-02-16T22:56:14.476 INFO:tasks.workunit.client.1.smithi192.stdout:6/295: stat da/df/d18 0 2024-02-16T22:56:14.478 INFO:tasks.workunit.client.0.smithi064.stdout:4/182: creat dc/d1d/f36 x:0 0 0 2024-02-16T22:56:14.479 INFO:tasks.workunit.client.0.smithi064.stdout:4/183: write dc/f19 [1714455,110392] 0 2024-02-16T22:56:14.480 INFO:tasks.workunit.client.1.smithi192.stdout:2/309: mkdir d22/d24/d39/d42/d65/d66 0 2024-02-16T22:56:14.482 INFO:tasks.workunit.client.1.smithi192.stdout:9/364: write d7/df/f16 [703983,27157] 0 2024-02-16T22:56:14.483 INFO:tasks.workunit.client.1.smithi192.stdout:2/310: write f8 [2896750,83916] 0 2024-02-16T22:56:14.484 INFO:tasks.workunit.client.1.smithi192.stdout:9/365: truncate d7/d15/d36/f7b 590533 0 2024-02-16T22:56:14.487 INFO:tasks.workunit.client.1.smithi192.stdout:8/277: fdatasync d5/db/d47/f3b 0 2024-02-16T22:56:14.488 INFO:tasks.workunit.client.0.smithi064.stdout:4/184: dwrite dc/d1d/f36 [0,4194304] 0 2024-02-16T22:56:14.489 INFO:tasks.workunit.client.1.smithi192.stdout:1/256: mknod d3/c53 0 2024-02-16T22:56:14.489 INFO:tasks.workunit.client.0.smithi064.stdout:5/168: symlink d6/d18/d39/l3d 0 2024-02-16T22:56:14.490 INFO:tasks.workunit.client.0.smithi064.stdout:5/169: chown d6/d25/c28 87114927 1 2024-02-16T22:56:14.491 INFO:tasks.workunit.client.1.smithi192.stdout:9/366: dwrite d7/d15/f6d [0,4194304] 0 2024-02-16T22:56:14.495 INFO:tasks.workunit.client.0.smithi064.stdout:2/173: unlink d7/c23 0 2024-02-16T22:56:14.496 INFO:tasks.workunit.client.0.smithi064.stdout:2/174: stat d7/d14/d1e/l2b 0 2024-02-16T22:56:14.497 INFO:tasks.workunit.client.1.smithi192.stdout:1/257: dwrite d3/de/d27/d34/f45 [0,4194304] 0 2024-02-16T22:56:14.497 INFO:tasks.workunit.client.0.smithi064.stdout:8/199: link fa df/d31/f3f 0 2024-02-16T22:56:14.499 INFO:tasks.workunit.client.1.smithi192.stdout:0/257: rename d1/d3/de/d22/d37/c4b to d1/d3/d29/c4d 0 2024-02-16T22:56:14.500 INFO:tasks.workunit.client.1.smithi192.stdout:0/258: read - d1/d3/de/f3c zero size 2024-02-16T22:56:14.502 INFO:tasks.workunit.client.0.smithi064.stdout:1/192: fsync d2/f3 0 2024-02-16T22:56:14.504 INFO:tasks.workunit.client.0.smithi064.stdout:1/193: chown d2/df/d11/d1a/d30/d3d/l46 3570 1 2024-02-16T22:56:14.506 INFO:tasks.workunit.client.1.smithi192.stdout:3/300: dread da/d11/d20/f24 [0,4194304] 0 2024-02-16T22:56:14.507 INFO:tasks.workunit.client.0.smithi064.stdout:1/194: write d2/df/d14/f39 [31776,28711] 0 2024-02-16T22:56:14.508 INFO:tasks.workunit.client.0.smithi064.stdout:1/195: write d2/df/d14/f39 [392801,101202] 0 2024-02-16T22:56:14.508 INFO:tasks.workunit.client.1.smithi192.stdout:0/259: dwrite d1/f36 [4194304,4194304] 0 2024-02-16T22:56:14.511 INFO:tasks.workunit.client.0.smithi064.stdout:4/185: mknod dc/d10/c37 0 2024-02-16T22:56:14.513 INFO:tasks.workunit.client.0.smithi064.stdout:1/196: dread d2/df/f1e [0,4194304] 0 2024-02-16T22:56:14.515 INFO:tasks.workunit.client.0.smithi064.stdout:1/197: read d2/df/f1c [2386465,86326] 0 2024-02-16T22:56:14.522 INFO:tasks.workunit.client.0.smithi064.stdout:2/175: rmdir d7/de 39 2024-02-16T22:56:14.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/367: symlink d7/d15/d36/l7c 0 2024-02-16T22:56:14.525 INFO:tasks.workunit.client.0.smithi064.stdout:8/200: rename df/c17 to df/c40 0 2024-02-16T22:56:14.527 INFO:tasks.workunit.client.0.smithi064.stdout:5/170: dread d6/d18/d19/f26 [0,4194304] 0 2024-02-16T22:56:14.527 INFO:tasks.workunit.client.1.smithi192.stdout:2/311: mkdir d22/d24/d2b/d67 0 2024-02-16T22:56:14.529 INFO:tasks.workunit.client.1.smithi192.stdout:6/296: write da/df/d12/d57/f2f [3032868,55420] 0 2024-02-16T22:56:14.532 INFO:tasks.workunit.client.1.smithi192.stdout:5/304: rename d6/d12/d1d/f39 to d6/d12/d4e/d46/f5d 0 2024-02-16T22:56:14.534 INFO:tasks.workunit.client.1.smithi192.stdout:3/301: mknod da/d11/d63/d6b/d4e/c6f 0 2024-02-16T22:56:14.535 INFO:tasks.workunit.client.1.smithi192.stdout:6/297: dwrite da/d40/f54 [0,4194304] 0 2024-02-16T22:56:14.535 INFO:tasks.workunit.client.0.smithi064.stdout:4/186: dread dc/f33 [0,4194304] 0 2024-02-16T22:56:14.538 INFO:tasks.workunit.client.1.smithi192.stdout:0/260: unlink d1/d3/c17 0 2024-02-16T22:56:14.538 INFO:tasks.workunit.client.1.smithi192.stdout:0/261: stat d1/d3/de/d22/f45 0 2024-02-16T22:56:14.539 INFO:tasks.workunit.client.1.smithi192.stdout:8/278: creat d5/db/d47/d43/f52 x:0 0 0 2024-02-16T22:56:14.539 INFO:tasks.workunit.client.1.smithi192.stdout:0/262: chown d1/d3/de/l3d 13 1 2024-02-16T22:56:14.542 INFO:tasks.workunit.client.1.smithi192.stdout:9/368: creat d7/d19/d5e/d6f/f7d x:0 0 0 2024-02-16T22:56:14.543 INFO:tasks.workunit.client.1.smithi192.stdout:2/312: creat d22/d24/d2b/d3e/f68 x:0 0 0 2024-02-16T22:56:14.544 INFO:tasks.workunit.client.1.smithi192.stdout:0/263: dwrite d1/fa [0,4194304] 0 2024-02-16T22:56:14.545 INFO:tasks.workunit.client.1.smithi192.stdout:1/258: creat d3/de/d27/d34/d52/f54 x:0 0 0 2024-02-16T22:56:14.545 INFO:tasks.workunit.client.1.smithi192.stdout:0/264: readlink d1/d3/d29/l33 0 2024-02-16T22:56:14.545 INFO:tasks.workunit.client.1.smithi192.stdout:1/259: chown d3/de/d27/f39 2759 1 2024-02-16T22:56:14.546 INFO:tasks.workunit.client.1.smithi192.stdout:4/303: sync 2024-02-16T22:56:14.546 INFO:tasks.workunit.client.0.smithi064.stdout:9/206: getdents d3/d11/d2b/d2a/d2e 0 2024-02-16T22:56:14.548 INFO:tasks.workunit.client.0.smithi064.stdout:9/207: chown d3/d11/d2b/d2a/d2e/c40 368611 1 2024-02-16T22:56:14.549 INFO:tasks.workunit.client.1.smithi192.stdout:7/340: rename d1/d2/f4 to d1/d19/f78 0 2024-02-16T22:56:14.551 INFO:tasks.workunit.client.0.smithi064.stdout:4/187: creat dc/d10/f38 x:0 0 0 2024-02-16T22:56:14.552 INFO:tasks.workunit.client.1.smithi192.stdout:1/260: dwrite d3/de/d27/f39 [0,4194304] 0 2024-02-16T22:56:14.552 INFO:tasks.workunit.client.1.smithi192.stdout:3/302: creat da/d11/d35/f70 x:0 0 0 2024-02-16T22:56:14.556 INFO:tasks.workunit.client.1.smithi192.stdout:6/298: rmdir da/df/d18 39 2024-02-16T22:56:14.557 INFO:tasks.workunit.client.0.smithi064.stdout:5/171: symlink d6/l3e 0 2024-02-16T22:56:14.558 INFO:tasks.workunit.client.0.smithi064.stdout:5/172: fdatasync d6/fa 0 2024-02-16T22:56:14.561 INFO:tasks.workunit.client.1.smithi192.stdout:3/303: dwrite da/d11/d20/f69 [0,4194304] 0 2024-02-16T22:56:14.561 INFO:tasks.workunit.client.1.smithi192.stdout:8/279: mknod d5/db/de/d21/c53 0 2024-02-16T22:56:14.563 INFO:tasks.workunit.client.0.smithi064.stdout:2/176: symlink d7/de/l31 0 2024-02-16T22:56:14.565 INFO:tasks.workunit.client.1.smithi192.stdout:9/369: creat d7/d19/d37/f7e x:0 0 0 2024-02-16T22:56:14.566 INFO:tasks.workunit.client.0.smithi064.stdout:4/188: mknod dc/dd/d32/d22/c39 0 2024-02-16T22:56:14.567 INFO:tasks.workunit.client.0.smithi064.stdout:4/189: write dc/de/f1c [724403,52641] 0 2024-02-16T22:56:14.569 INFO:tasks.workunit.client.1.smithi192.stdout:0/265: rmdir d1 39 2024-02-16T22:56:14.570 INFO:tasks.workunit.client.0.smithi064.stdout:8/201: creat df/d31/f41 x:0 0 0 2024-02-16T22:56:14.573 INFO:tasks.workunit.client.1.smithi192.stdout:4/304: creat d1/d26/d5e/f70 x:0 0 0 2024-02-16T22:56:14.574 INFO:tasks.workunit.client.0.smithi064.stdout:4/190: creat dc/dd/d16/f3a x:0 0 0 2024-02-16T22:56:14.575 INFO:tasks.workunit.client.0.smithi064.stdout:9/208: dread d3/d9/f21 [0,4194304] 0 2024-02-16T22:56:14.577 INFO:tasks.workunit.client.1.smithi192.stdout:2/313: rename d22/d24/d2b/d3e to d22/d24/d39/d42/d65/d69 0 2024-02-16T22:56:14.578 INFO:tasks.workunit.client.1.smithi192.stdout:5/305: creat d6/d12/d4e/d5b/f5e x:0 0 0 2024-02-16T22:56:14.580 INFO:tasks.workunit.client.0.smithi064.stdout:9/209: unlink d3/d11/d15/d23/f25 0 2024-02-16T22:56:14.589 INFO:tasks.workunit.client.1.smithi192.stdout:7/341: symlink d1/l79 0 2024-02-16T22:56:14.590 INFO:tasks.workunit.client.1.smithi192.stdout:1/261: mkdir d3/d40/d55 0 2024-02-16T22:56:14.594 INFO:tasks.workunit.client.1.smithi192.stdout:0/266: dwrite d1/d3/de/d22/f31 [0,4194304] 0 2024-02-16T22:56:14.595 INFO:tasks.workunit.client.1.smithi192.stdout:9/370: mknod d7/d15/d36/d77/c7f 0 2024-02-16T22:56:14.596 INFO:tasks.workunit.client.1.smithi192.stdout:9/371: fdatasync d7/df/f16 0 2024-02-16T22:56:14.596 INFO:tasks.workunit.client.0.smithi064.stdout:9/210: dread d3/d11/d2b/f26 [0,4194304] 0 2024-02-16T22:56:14.597 INFO:tasks.workunit.client.1.smithi192.stdout:4/305: symlink d1/l71 0 2024-02-16T22:56:14.598 INFO:tasks.workunit.client.0.smithi064.stdout:4/191: getdents dc/dd/d16 0 2024-02-16T22:56:14.598 INFO:tasks.workunit.client.1.smithi192.stdout:7/342: dwrite d1/d19/d4c/f6b [0,4194304] 0 2024-02-16T22:56:14.598 INFO:tasks.workunit.client.1.smithi192.stdout:9/372: dread - d7/d19/d28/f53 zero size 2024-02-16T22:56:14.599 INFO:tasks.workunit.client.0.smithi064.stdout:2/177: truncate d7/de/f15 3243729 0 2024-02-16T22:56:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.603+0000 7f6a0abcb700 1 -- 172.21.15.64:0/4148064557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a040fe0e0 msgr2=0x7f6a040fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.603+0000 7f6a0abcb700 1 --2- 172.21.15.64:0/4148064557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a040fe0e0 0x7f6a040fe550 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f69f4004880 tx=0x7f69f4009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.604+0000 7f6a0abcb700 1 -- 172.21.15.64:0/4148064557 shutdown_connections 2024-02-16T22:56:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.604+0000 7f6a0abcb700 1 --2- 172.21.15.64:0/4148064557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a040fe0e0 0x7f6a040fe550 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.604+0000 7f6a0abcb700 1 --2- 172.21.15.64:0/4148064557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a040fcef0 0x7f6a040fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.604+0000 7f6a0abcb700 1 -- 172.21.15.64:0/4148064557 >> 172.21.15.64:0/4148064557 conn(0x7f6a040f8620 msgr2=0x7f6a040faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.604+0000 7f6a0abcb700 1 -- 172.21.15.64:0/4148064557 shutdown_connections 2024-02-16T22:56:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.604+0000 7f6a0abcb700 1 -- 172.21.15.64:0/4148064557 wait complete. 2024-02-16T22:56:14.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.605+0000 7f6a0abcb700 1 Processor -- start 2024-02-16T22:56:14.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.605+0000 7f6a0abcb700 1 -- start start 2024-02-16T22:56:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.605+0000 7f6a0abcb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a040fcef0 0x7f6a04113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.605+0000 7f6a0abcb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a040fe0e0 0x7f6a041144c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.606+0000 7f6a0abcb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a04114ac0 con 0x7f6a040fe0e0 2024-02-16T22:56:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.606+0000 7f6a0abcb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a04115360 con 0x7f6a040fcef0 2024-02-16T22:56:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.606+0000 7f6a08967700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a040fcef0 0x7f6a04113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.606+0000 7f6a08967700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a040fcef0 0x7f6a04113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:39432/0 (socket says 172.21.15.64:39432) 2024-02-16T22:56:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.606+0000 7f6a08967700 1 -- 172.21.15.64:0/118887879 learned_addr learned my addr 172.21.15.64:0/118887879 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.606+0000 7f69fbfff700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a040fe0e0 0x7f6a041144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:14.608 INFO:tasks.workunit.client.1.smithi192.stdout:5/306: unlink d6/d12/d4e/d46/l47 0 2024-02-16T22:56:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.607+0000 7f6a08967700 1 -- 172.21.15.64:0/118887879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a040fe0e0 msgr2=0x7f6a041144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.607+0000 7f6a08967700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a040fe0e0 0x7f6a041144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.607+0000 7f6a08967700 1 -- 172.21.15.64:0/118887879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4005040 con 0x7f6a040fcef0 2024-02-16T22:56:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.607+0000 7f6a08967700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a040fcef0 0x7f6a04113f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000eb40 tx=0x7f6a0000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.607+0000 7f69f97fa700 1 -- 172.21.15.64:0/118887879 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a00007cc0 con 0x7f6a040fcef0 2024-02-16T22:56:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.607+0000 7f69f97fa700 1 -- 172.21.15.64:0/118887879 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a0000f040 con 0x7f6a040fcef0 2024-02-16T22:56:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.607+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a04115640 con 0x7f6a040fcef0 2024-02-16T22:56:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.607+0000 7f69f97fa700 1 -- 172.21.15.64:0/118887879 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a00007cc0 con 0x7f6a040fcef0 2024-02-16T22:56:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.608+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a04115b30 con 0x7f6a040fcef0 2024-02-16T22:56:14.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.609+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0404bed0 con 0x7f6a040fcef0 2024-02-16T22:56:14.610 INFO:tasks.workunit.client.1.smithi192.stdout:5/307: read d6/d20/f4d [289466,3730] 0 2024-02-16T22:56:14.610 INFO:tasks.workunit.client.1.smithi192.stdout:4/306: dwrite d1/d26/d5e/f70 [0,4194304] 0 2024-02-16T22:56:14.610 INFO:tasks.workunit.client.1.smithi192.stdout:6/299: creat da/df/d12/f5a x:0 0 0 2024-02-16T22:56:14.610 INFO:tasks.workunit.client.1.smithi192.stdout:5/308: write d6/d12/f18 [4965659,77594] 0 2024-02-16T22:56:14.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.610+0000 7f69f97fa700 1 -- 172.21.15.64:0/118887879 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f6a00009070 con 0x7f6a040fcef0 2024-02-16T22:56:14.611 INFO:tasks.workunit.client.1.smithi192.stdout:8/280: link d5/d16/d34/f4e d5/d9/f54 0 2024-02-16T22:56:14.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.611+0000 7f69f97fa700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f69ec06b380 0x7f69ec06d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:14.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.611+0000 7f69f97fa700 1 -- 172.21.15.64:0/118887879 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f6a000894c0 con 0x7f6a040fcef0 2024-02-16T22:56:14.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.611+0000 7f69fbfff700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f69ec06b380 0x7f69ec06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:14.612 INFO:tasks.workunit.client.1.smithi192.stdout:3/304: truncate da/d11/d63/d6b/d4e/f56 4283677 0 2024-02-16T22:56:14.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.612+0000 7f69fbfff700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f69ec06b380 0x7f69ec06d830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009fd0 tx=0x7f69f4009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:14.613 INFO:tasks.workunit.client.0.smithi064.stdout:9/211: symlink d3/d11/d2b/d38/l4d 0 2024-02-16T22:56:14.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:14.620+0000 7f69f97fa700 1 -- 172.21.15.64:0/118887879 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f6a0004f620 con 0x7f6a040fcef0 2024-02-16T22:56:14.621 INFO:tasks.workunit.client.1.smithi192.stdout:3/305: dwrite da/d11/d20/d39/f5d [0,4194304] 0 2024-02-16T22:56:14.621 INFO:tasks.workunit.client.1.smithi192.stdout:0/267: creat d1/d3/de/f4e x:0 0 0 2024-02-16T22:56:14.625 INFO:tasks.workunit.client.1.smithi192.stdout:9/373: symlink d7/d19/l80 0 2024-02-16T22:56:14.627 INFO:tasks.workunit.client.0.smithi064.stdout:2/178: dread d7/f1a [0,4194304] 0 2024-02-16T22:56:14.628 INFO:tasks.workunit.client.0.smithi064.stdout:9/212: creat d3/d11/d2b/d2a/d2e/d31/f4e x:0 0 0 2024-02-16T22:56:14.630 INFO:tasks.workunit.client.1.smithi192.stdout:0/268: dwrite d1/d3/d29/f27 [0,4194304] 0 2024-02-16T22:56:14.631 INFO:tasks.workunit.client.1.smithi192.stdout:0/269: readlink d1/d3/de/l40 0 2024-02-16T22:56:14.631 INFO:tasks.workunit.client.1.smithi192.stdout:0/270: stat d1/d3/de/d22/d37/d47/l4a 0 2024-02-16T22:56:14.632 INFO:tasks.workunit.client.0.smithi064.stdout:9/213: dread d3/d11/d2b/f29 [0,4194304] 0 2024-02-16T22:56:14.632 INFO:tasks.workunit.client.1.smithi192.stdout:0/271: write d1/f24 [3749942,52588] 0 2024-02-16T22:56:14.633 INFO:tasks.workunit.client.1.smithi192.stdout:0/272: readlink d1/d3/de/d22/l46 0 2024-02-16T22:56:14.633 INFO:tasks.workunit.client.1.smithi192.stdout:7/343: rename d1/d2/de/f4d to d1/d2/de/d2c/d49/d54/f7a 0 2024-02-16T22:56:14.633 INFO:tasks.workunit.client.1.smithi192.stdout:7/344: fdatasync d1/d2/f1e 0 2024-02-16T22:56:14.634 INFO:tasks.workunit.client.1.smithi192.stdout:0/273: truncate d1/d3/f4c 1182478 0 2024-02-16T22:56:14.634 INFO:tasks.workunit.client.1.smithi192.stdout:7/345: chown d1/d2/de/l10 1119111 1 2024-02-16T22:56:14.636 INFO:tasks.workunit.client.0.smithi064.stdout:2/179: dwrite d7/f12 [0,4194304] 0 2024-02-16T22:56:14.639 INFO:tasks.workunit.client.0.smithi064.stdout:3/251: sync 2024-02-16T22:56:14.639 INFO:tasks.workunit.client.1.smithi192.stdout:7/346: dread d1/d19/f64 [0,4194304] 0 2024-02-16T22:56:14.647 INFO:tasks.workunit.client.0.smithi064.stdout:9/214: symlink d3/d11/d2b/d2a/d2e/d31/d3f/l4f 0 2024-02-16T22:56:14.648 INFO:tasks.workunit.client.0.smithi064.stdout:3/252: dwrite d4/d13/f28 [0,4194304] 0 2024-02-16T22:56:14.649 INFO:tasks.workunit.client.1.smithi192.stdout:4/307: mknod d1/d26/d34/c72 0 2024-02-16T22:56:14.651 INFO:tasks.workunit.client.0.smithi064.stdout:3/253: write d4/d1f/d34/f45 [717593,22291] 0 2024-02-16T22:56:14.652 INFO:tasks.workunit.client.0.smithi064.stdout:4/192: truncate dc/f19 5822554 0 2024-02-16T22:56:14.653 INFO:tasks.workunit.client.0.smithi064.stdout:2/180: creat d7/d14/d25/f32 x:0 0 0 2024-02-16T22:56:14.653 INFO:tasks.workunit.client.0.smithi064.stdout:2/181: chown d7/de/l29 336 1 2024-02-16T22:56:14.658 INFO:tasks.workunit.client.1.smithi192.stdout:8/281: symlink d5/d3a/d4d/l55 0 2024-02-16T22:56:14.660 INFO:tasks.workunit.client.1.smithi192.stdout:3/306: mkdir da/d11/d63/d6b/d71 0 2024-02-16T22:56:14.660 INFO:tasks.workunit.client.1.smithi192.stdout:3/307: fsync da/d11/f3f 0 2024-02-16T22:56:14.662 INFO:tasks.workunit.client.0.smithi064.stdout:3/254: dwrite d4/d14/f30 [4194304,4194304] 0 2024-02-16T22:56:14.662 INFO:tasks.workunit.client.0.smithi064.stdout:4/193: dwrite dc/de/f1c [0,4194304] 0 2024-02-16T22:56:14.664 INFO:tasks.workunit.client.0.smithi064.stdout:3/255: readlink d4/l7 0 2024-02-16T22:56:14.664 INFO:tasks.workunit.client.0.smithi064.stdout:4/194: truncate dc/f33 8469665 0 2024-02-16T22:56:14.667 INFO:tasks.workunit.client.0.smithi064.stdout:9/215: write d3/d11/d2b/f29 [2103972,115715] 0 2024-02-16T22:56:14.667 INFO:tasks.workunit.client.0.smithi064.stdout:4/195: write dc/d1d/f36 [3646264,119492] 0 2024-02-16T22:56:14.671 INFO:tasks.workunit.client.0.smithi064.stdout:9/216: write d3/f18 [5188999,30750] 0 2024-02-16T22:56:14.677 INFO:tasks.workunit.client.1.smithi192.stdout:2/314: getdents d22/d24/d39/d42/d65/d69 0 2024-02-16T22:56:14.678 INFO:tasks.workunit.client.1.smithi192.stdout:1/262: sync 2024-02-16T22:56:14.679 INFO:tasks.workunit.client.1.smithi192.stdout:1/263: write d3/f19 [16542895,111458] 0 2024-02-16T22:56:14.679 INFO:tasks.workunit.client.0.smithi064.stdout:2/182: chown d7/cd 13080194 1 2024-02-16T22:56:14.681 INFO:tasks.workunit.client.0.smithi064.stdout:3/256: creat d4/d1f/d34/f56 x:0 0 0 2024-02-16T22:56:14.682 INFO:tasks.workunit.client.0.smithi064.stdout:3/257: chown d4/d1f/l33 386515 1 2024-02-16T22:56:14.683 INFO:tasks.workunit.client.0.smithi064.stdout:3/258: chown d4 26518364 1 2024-02-16T22:56:14.684 INFO:tasks.workunit.client.1.smithi192.stdout:1/264: dwrite d3/f50 [0,4194304] 0 2024-02-16T22:56:14.689 INFO:tasks.workunit.client.1.smithi192.stdout:1/265: dwrite d3/f22 [0,4194304] 0 2024-02-16T22:56:14.695 INFO:tasks.workunit.client.1.smithi192.stdout:7/347: mknod d1/d2/d2d/d55/c7b 0 2024-02-16T22:56:14.699 INFO:tasks.workunit.client.0.smithi064.stdout:2/183: creat d7/d14/f33 x:0 0 0 2024-02-16T22:56:14.705 INFO:tasks.workunit.client.1.smithi192.stdout:4/308: rename d1/d8/d11/l5f to d1/d15/d29/d3c/l73 0 2024-02-16T22:56:14.706 INFO:tasks.workunit.client.1.smithi192.stdout:6/300: mknod da/df/d58/c5b 0 2024-02-16T22:56:14.707 INFO:tasks.workunit.client.1.smithi192.stdout:5/309: symlink d6/d12/d1d/l5f 0 2024-02-16T22:56:14.708 INFO:tasks.workunit.client.0.smithi064.stdout:9/217: mknod d3/d11/d2b/c50 0 2024-02-16T22:56:14.710 INFO:tasks.workunit.client.1.smithi192.stdout:9/374: dread d7/d15/d36/d38/f47 [0,4194304] 0 2024-02-16T22:56:14.710 INFO:tasks.workunit.client.1.smithi192.stdout:8/282: mknod d5/d3a/c56 0 2024-02-16T22:56:14.711 INFO:tasks.workunit.client.0.smithi064.stdout:4/196: dwrite dc/dd/d16/f23 [0,4194304] 0 2024-02-16T22:56:14.712 INFO:tasks.workunit.client.0.smithi064.stdout:3/259: rename d4/d13/d1b/l3d to d4/d13/d1b/d27/d44/l57 0 2024-02-16T22:56:14.713 INFO:tasks.workunit.client.0.smithi064.stdout:3/260: chown d4/d13/d1b/d43/l4a 15 1 2024-02-16T22:56:14.715 INFO:tasks.workunit.client.1.smithi192.stdout:3/308: unlink da/d11/d63/d6b/d4e/c6f 0 2024-02-16T22:56:14.718 INFO:tasks.workunit.client.0.smithi064.stdout:9/218: creat d3/d11/d15/f51 x:0 0 0 2024-02-16T22:56:14.720 INFO:tasks.workunit.client.0.smithi064.stdout:2/184: truncate d7/f13 737603 0 2024-02-16T22:56:14.726 INFO:tasks.workunit.client.1.smithi192.stdout:0/274: creat d1/d3/d39/f4f x:0 0 0 2024-02-16T22:56:14.728 INFO:tasks.workunit.client.0.smithi064.stdout:2/185: dwrite d7/f8 [0,4194304] 0 2024-02-16T22:56:14.729 INFO:tasks.workunit.client.0.smithi064.stdout:4/197: rename dc/c27 to dc/dd/d32/c3b 0 2024-02-16T22:56:14.730 INFO:tasks.workunit.client.1.smithi192.stdout:1/266: creat d3/de/d27/d34/d52/f56 x:0 0 0 2024-02-16T22:56:14.734 INFO:tasks.workunit.client.0.smithi064.stdout:2/186: mkdir d7/d14/d1e/d34 0 2024-02-16T22:56:14.736 INFO:tasks.workunit.client.1.smithi192.stdout:1/267: dwrite d3/f22 [4194304,4194304] 0 2024-02-16T22:56:14.736 INFO:tasks.workunit.client.1.smithi192.stdout:7/348: rmdir d1/d19/d4c 39 2024-02-16T22:56:14.742 INFO:tasks.workunit.client.0.smithi064.stdout:2/187: dwrite d7/d14/d1e/f28 [0,4194304] 0 2024-02-16T22:56:14.742 INFO:tasks.workunit.client.0.smithi064.stdout:9/219: dread d3/d11/f28 [0,4194304] 0 2024-02-16T22:56:14.742 INFO:tasks.workunit.client.1.smithi192.stdout:2/315: rename l1d to d22/d24/d2b/d2c/l6a 0 2024-02-16T22:56:14.744 INFO:tasks.workunit.client.1.smithi192.stdout:7/349: dwrite d1/d2/f25 [0,4194304] 0 2024-02-16T22:56:14.747 INFO:tasks.workunit.client.1.smithi192.stdout:4/309: mknod d1/d26/c74 0 2024-02-16T22:56:14.754 INFO:tasks.workunit.client.1.smithi192.stdout:9/375: mkdir d7/d19/d5e/d81 0 2024-02-16T22:56:14.756 INFO:tasks.workunit.client.1.smithi192.stdout:3/309: creat da/d11/d20/f72 x:0 0 0 2024-02-16T22:56:14.761 INFO:tasks.workunit.client.1.smithi192.stdout:0/275: unlink d1/d2/f4 0 2024-02-16T22:56:14.765 INFO:tasks.workunit.client.0.smithi064.stdout:3/261: rename d4/d14/f4c to d4/d13/d1b/d27/f58 0 2024-02-16T22:56:14.766 INFO:tasks.workunit.client.0.smithi064.stdout:3/262: write d4/d13/d1b/f1c [777574,103192] 0 2024-02-16T22:56:14.766 INFO:tasks.workunit.client.0.smithi064.stdout:4/198: write dc/de/f1c [5145898,26664] 0 2024-02-16T22:56:14.768 INFO:tasks.workunit.client.0.smithi064.stdout:4/199: chown dc/dd/d32/c25 62 1 2024-02-16T22:56:14.768 INFO:tasks.workunit.client.1.smithi192.stdout:1/268: rmdir d3/de/d27 39 2024-02-16T22:56:14.772 INFO:tasks.workunit.client.1.smithi192.stdout:2/316: symlink d22/d24/d2b/d2c/d3f/l6b 0 2024-02-16T22:56:14.773 INFO:tasks.workunit.client.1.smithi192.stdout:7/350: creat d1/d19/d2a/f7c x:0 0 0 2024-02-16T22:56:14.773 INFO:tasks.workunit.client.1.smithi192.stdout:7/351: chown d1/d2/f29 62017 1 2024-02-16T22:56:14.774 INFO:tasks.workunit.client.0.smithi064.stdout:3/263: symlink d4/d13/d35/l59 0 2024-02-16T22:56:14.775 INFO:tasks.workunit.client.1.smithi192.stdout:4/310: mknod d1/d15/d29/c75 0 2024-02-16T22:56:14.777 INFO:tasks.workunit.client.0.smithi064.stdout:2/188: creat d7/f35 x:0 0 0 2024-02-16T22:56:14.778 INFO:tasks.workunit.client.1.smithi192.stdout:8/283: sync 2024-02-16T22:56:14.778 INFO:tasks.workunit.client.1.smithi192.stdout:8/284: dread - d5/d26/f3f zero size 2024-02-16T22:56:14.779 INFO:tasks.workunit.client.1.smithi192.stdout:8/285: chown d5/db/d47/d43/f52 1073 1 2024-02-16T22:56:14.779 INFO:tasks.workunit.client.0.smithi064.stdout:3/264: dwrite d4/f1e [4194304,4194304] 0 2024-02-16T22:56:14.780 INFO:tasks.workunit.client.1.smithi192.stdout:8/286: write d5/db/d47/f4a [882495,61960] 0 2024-02-16T22:56:14.780 INFO:tasks.workunit.client.0.smithi064.stdout:9/220: write d3/d11/d2b/d2a/d2e/d31/d3f/f43 [599034,38965] 0 2024-02-16T22:56:14.782 INFO:tasks.workunit.client.1.smithi192.stdout:9/376: unlink d7/d19/l25 0 2024-02-16T22:56:14.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:14 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:14.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:14 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:14 smithi192 ceph-mon[36254]: pgmap v12: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 8.3 MiB/s rd, 116 MiB/s wr, 357 op/s 2024-02-16T22:56:14.785 INFO:tasks.workunit.client.1.smithi192.stdout:3/310: symlink da/d11/d20/l73 0 2024-02-16T22:56:14.787 INFO:tasks.workunit.client.1.smithi192.stdout:3/311: write da/d11/d2b/d43/f64 [79928,118096] 0 2024-02-16T22:56:14.787 INFO:tasks.workunit.client.1.smithi192.stdout:0/276: chown d1/d3/d29/c4d 13707183 1 2024-02-16T22:56:14.788 INFO:tasks.workunit.client.0.smithi064.stdout:2/189: write d7/f1a [1915730,106402] 0 2024-02-16T22:56:14.789 INFO:tasks.workunit.client.1.smithi192.stdout:0/277: fsync d1/d3/de/d22/d37/d47/f48 0 2024-02-16T22:56:14.790 INFO:tasks.workunit.client.1.smithi192.stdout:2/317: truncate fa 2750087 0 2024-02-16T22:56:14.792 INFO:tasks.workunit.client.0.smithi064.stdout:3/265: mknod d4/d14/c5a 0 2024-02-16T22:56:14.794 INFO:tasks.workunit.client.1.smithi192.stdout:4/311: unlink d1/d8/fe 0 2024-02-16T22:56:14.796 INFO:tasks.workunit.client.1.smithi192.stdout:6/301: link da/df/d12/d57/f2c da/df/d12/d15/f5c 0 2024-02-16T22:56:14.796 INFO:tasks.workunit.client.0.smithi064.stdout:2/190: creat d7/d14/f36 x:0 0 0 2024-02-16T22:56:14.797 INFO:tasks.workunit.client.1.smithi192.stdout:6/302: write da/df/d12/d15/f21 [596032,40170] 0 2024-02-16T22:56:14.801 INFO:tasks.workunit.client.0.smithi064.stdout:3/266: rename d4/d13/d1b/d27/d4e to d4/d1f/d5b 0 2024-02-16T22:56:14.802 INFO:tasks.workunit.client.1.smithi192.stdout:8/287: mknod d5/db/d47/c57 0 2024-02-16T22:56:14.804 INFO:tasks.workunit.client.1.smithi192.stdout:8/288: read d5/d16/f27 [2603163,78434] 0 2024-02-16T22:56:14.804 INFO:tasks.workunit.client.1.smithi192.stdout:9/377: mknod d7/d15/d36/d77/c82 0 2024-02-16T22:56:14.805 INFO:tasks.workunit.client.0.smithi064.stdout:2/191: mkdir d7/d14/d25/d37 0 2024-02-16T22:56:14.805 INFO:tasks.workunit.client.1.smithi192.stdout:8/289: write d5/db/de/f37 [968418,28256] 0 2024-02-16T22:56:14.805 INFO:tasks.workunit.client.1.smithi192.stdout:9/378: chown d7/d15/d36/l48 514690 1 2024-02-16T22:56:14.805 INFO:tasks.workunit.client.1.smithi192.stdout:9/379: readlink d7/l35 0 2024-02-16T22:56:14.806 INFO:tasks.workunit.client.1.smithi192.stdout:3/312: fdatasync da/f12 0 2024-02-16T22:56:14.807 INFO:tasks.workunit.client.1.smithi192.stdout:9/380: read d7/d19/d28/d43/f57 [2697591,32350] 0 2024-02-16T22:56:14.809 INFO:tasks.workunit.client.0.smithi064.stdout:9/221: creat d3/d11/d2b/f52 x:0 0 0 2024-02-16T22:56:14.810 INFO:tasks.workunit.client.0.smithi064.stdout:9/222: chown d3/d11/d15 173 1 2024-02-16T22:56:14.810 INFO:tasks.workunit.client.0.smithi064.stdout:4/200: truncate dc/d1d/f36 1784507 0 2024-02-16T22:56:14.811 INFO:tasks.workunit.client.1.smithi192.stdout:0/278: symlink d1/d3/d18/l50 0 2024-02-16T22:56:14.813 INFO:tasks.workunit.client.0.smithi064.stdout:2/192: mkdir d7/d14/d1e/d38 0 2024-02-16T22:56:14.813 INFO:tasks.workunit.client.1.smithi192.stdout:1/269: dwrite d3/de/d27/f44 [0,4194304] 0 2024-02-16T22:56:14.814 INFO:tasks.workunit.client.1.smithi192.stdout:1/270: dread - d3/de/f32 zero size 2024-02-16T22:56:14.814 INFO:tasks.workunit.client.0.smithi064.stdout:2/193: rename d7/d14/d25 to d7/d14/d25/d37/d39 22 2024-02-16T22:56:14.817 INFO:tasks.workunit.client.0.smithi064.stdout:3/267: mkdir d4/d13/d4d/d5c 0 2024-02-16T22:56:14.818 INFO:tasks.workunit.client.1.smithi192.stdout:7/352: symlink d1/d2/d37/l7d 0 2024-02-16T22:56:14.819 INFO:tasks.workunit.client.1.smithi192.stdout:7/353: fdatasync d1/d19/f3a 0 2024-02-16T22:56:14.820 INFO:tasks.workunit.client.1.smithi192.stdout:7/354: truncate d1/f4b 70603 0 2024-02-16T22:56:14.820 INFO:tasks.workunit.client.1.smithi192.stdout:7/355: fdatasync d1/d2/f6 0 2024-02-16T22:56:14.824 INFO:tasks.workunit.client.0.smithi064.stdout:3/268: dwrite d4/f18 [0,4194304] 0 2024-02-16T22:56:14.825 INFO:tasks.workunit.client.0.smithi064.stdout:3/269: fsync d4/d13/d1b/d43/f49 0 2024-02-16T22:56:14.826 INFO:tasks.workunit.client.0.smithi064.stdout:4/201: symlink dc/de/l3c 0 2024-02-16T22:56:14.831 INFO:tasks.workunit.client.1.smithi192.stdout:4/312: creat d1/d26/d34/f76 x:0 0 0 2024-02-16T22:56:14.835 INFO:tasks.workunit.client.0.smithi064.stdout:4/202: dwrite dc/dd/d16/f2e [0,4194304] 0 2024-02-16T22:56:14.837 INFO:tasks.workunit.client.0.smithi064.stdout:4/203: write dc/dd/d32/d22/f24 [796132,115307] 0 2024-02-16T22:56:14.837 INFO:tasks.workunit.client.1.smithi192.stdout:5/310: link c4 d6/d12/d1d/c60 0 2024-02-16T22:56:14.848 INFO:tasks.workunit.client.1.smithi192.stdout:3/313: symlink da/d11/d2b/l74 0 2024-02-16T22:56:14.849 INFO:tasks.workunit.client.1.smithi192.stdout:2/318: dwrite d22/f4a [4194304,4194304] 0 2024-02-16T22:56:14.850 INFO:tasks.workunit.client.1.smithi192.stdout:2/319: dread - d22/d24/d39/d42/f5f zero size 2024-02-16T22:56:14.852 INFO:tasks.workunit.client.0.smithi064.stdout:9/223: dwrite d3/d9/f12 [4194304,4194304] 0 2024-02-16T22:56:14.852 INFO:tasks.workunit.client.1.smithi192.stdout:9/381: creat d7/f83 x:0 0 0 2024-02-16T22:56:14.853 INFO:tasks.workunit.client.0.smithi064.stdout:4/204: creat dc/dd/d32/f3d x:0 0 0 2024-02-16T22:56:14.856 INFO:tasks.workunit.client.1.smithi192.stdout:3/314: dwrite f3 [0,4194304] 0 2024-02-16T22:56:14.856 INFO:tasks.workunit.client.0.smithi064.stdout:4/205: truncate dc/dd/d32/d22/f24 5223426 0 2024-02-16T22:56:14.857 INFO:tasks.workunit.client.0.smithi064.stdout:3/270: creat d4/d13/d1b/d4f/f5d x:0 0 0 2024-02-16T22:56:14.857 INFO:tasks.workunit.client.1.smithi192.stdout:3/315: write da/d11/d2b/f5e [545351,26927] 0 2024-02-16T22:56:14.858 INFO:tasks.workunit.client.1.smithi192.stdout:3/316: write f9 [2979110,113124] 0 2024-02-16T22:56:14.862 INFO:tasks.workunit.client.0.smithi064.stdout:9/224: dwrite d3/d11/d2b/d2a/d2e/f39 [0,4194304] 0 2024-02-16T22:56:14.864 INFO:tasks.workunit.client.0.smithi064.stdout:0/158: sync 2024-02-16T22:56:14.866 INFO:tasks.workunit.client.0.smithi064.stdout:2/194: truncate d7/f1a 3487356 0 2024-02-16T22:56:14.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:14 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:14.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:14 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:14.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:14 smithi064 ceph-mon[29226]: pgmap v12: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 8.3 MiB/s rd, 116 MiB/s wr, 357 op/s 2024-02-16T22:56:14.869 INFO:tasks.workunit.client.0.smithi064.stdout:4/206: unlink l6 0 2024-02-16T22:56:14.869 INFO:tasks.workunit.client.0.smithi064.stdout:0/159: read da/dc/de/f17 [143465,94590] 0 2024-02-16T22:56:14.874 INFO:tasks.workunit.client.1.smithi192.stdout:7/356: fsync d1/d2/f1b 0 2024-02-16T22:56:14.876 INFO:tasks.workunit.client.1.smithi192.stdout:4/313: unlink d1/d15/d23/f3b 0 2024-02-16T22:56:14.876 INFO:tasks.workunit.client.0.smithi064.stdout:9/225: mknod d3/d11/d2b/d2a/d2e/d31/c53 0 2024-02-16T22:56:14.877 INFO:tasks.workunit.client.1.smithi192.stdout:4/314: read d1/f4 [4901350,68993] 0 2024-02-16T22:56:14.877 INFO:tasks.workunit.client.1.smithi192.stdout:6/303: fdatasync da/df/f1c 0 2024-02-16T22:56:14.880 INFO:tasks.workunit.client.1.smithi192.stdout:8/290: creat d5/d16/d34/f58 x:0 0 0 2024-02-16T22:56:14.881 INFO:tasks.workunit.client.1.smithi192.stdout:8/291: truncate d5/d26/f3f 626180 0 2024-02-16T22:56:14.881 INFO:tasks.workunit.client.0.smithi064.stdout:0/160: rename da/dc/f13 to da/dc/de/d12/f27 0 2024-02-16T22:56:14.883 INFO:tasks.workunit.client.0.smithi064.stdout:3/271: link d4/d13/d1b/d43/f49 d4/d13/d1b/d27/d44/f5e 0 2024-02-16T22:56:14.885 INFO:tasks.workunit.client.0.smithi064.stdout:2/195: creat d7/d14/d25/d37/f3a x:0 0 0 2024-02-16T22:56:14.890 INFO:tasks.workunit.client.1.smithi192.stdout:2/320: creat d22/d24/d2b/d2c/d3f/f6c x:0 0 0 2024-02-16T22:56:14.891 INFO:tasks.workunit.client.0.smithi064.stdout:3/272: dwrite d4/d13/f26 [0,4194304] 0 2024-02-16T22:56:14.893 INFO:tasks.workunit.client.0.smithi064.stdout:0/161: unlink da/dc/de/d12/d1d/f22 0 2024-02-16T22:56:14.894 INFO:tasks.workunit.client.1.smithi192.stdout:9/382: mknod d7/d15/d36/d54/c84 0 2024-02-16T22:56:14.898 INFO:tasks.workunit.client.1.smithi192.stdout:1/271: dwrite d3/f2d [0,4194304] 0 2024-02-16T22:56:14.903 INFO:tasks.workunit.client.1.smithi192.stdout:9/383: dwrite d7/d15/f6a [0,4194304] 0 2024-02-16T22:56:14.904 INFO:tasks.workunit.client.1.smithi192.stdout:3/317: chown l7 1 1 2024-02-16T22:56:14.908 INFO:tasks.workunit.client.0.smithi064.stdout:3/273: creat d4/d13/d1b/d27/d44/f5f x:0 0 0 2024-02-16T22:56:14.909 INFO:tasks.workunit.client.0.smithi064.stdout:3/274: stat d4/d1f/d34/f45 0 2024-02-16T22:56:14.912 INFO:tasks.workunit.client.1.smithi192.stdout:5/311: write d6/d8/f3e [853893,63969] 0 2024-02-16T22:56:14.916 INFO:tasks.workunit.client.0.smithi064.stdout:3/275: dwrite d4/d13/f55 [0,4194304] 0 2024-02-16T22:56:14.918 INFO:tasks.workunit.client.1.smithi192.stdout:7/357: truncate d1/d2/de/f20 536261 0 2024-02-16T22:56:14.925 INFO:tasks.workunit.client.0.smithi064.stdout:4/207: truncate dc/d1d/f36 713030 0 2024-02-16T22:56:14.926 INFO:tasks.workunit.client.0.smithi064.stdout:4/208: truncate dc/de/f1c 6219003 0 2024-02-16T22:56:14.926 INFO:tasks.workunit.client.0.smithi064.stdout:4/209: stat dc/de/f1c 0 2024-02-16T22:56:14.927 INFO:tasks.workunit.client.0.smithi064.stdout:4/210: chown dc/de 5 1 2024-02-16T22:56:14.927 INFO:tasks.workunit.client.1.smithi192.stdout:4/315: creat d1/d26/f77 x:0 0 0 2024-02-16T22:56:14.931 INFO:tasks.workunit.client.0.smithi064.stdout:9/226: getdents d3/d9/d1e/d3e 0 2024-02-16T22:56:14.932 INFO:tasks.workunit.client.0.smithi064.stdout:9/227: chown d3/d11/d2b/d2a/d2e/d31/f4e 8145817 1 2024-02-16T22:56:14.932 INFO:tasks.workunit.client.1.smithi192.stdout:8/292: truncate d5/db/d47/f3b 172647 0 2024-02-16T22:56:14.935 INFO:tasks.workunit.client.0.smithi064.stdout:4/211: mkdir dc/de/d28/d3e 0 2024-02-16T22:56:14.935 INFO:tasks.workunit.client.0.smithi064.stdout:0/162: write da/fb [629680,78880] 0 2024-02-16T22:56:14.936 INFO:tasks.workunit.client.1.smithi192.stdout:2/321: symlink d22/d24/d39/d42/d65/d69/d4f/l6d 0 2024-02-16T22:56:14.937 INFO:tasks.workunit.client.1.smithi192.stdout:2/322: chown d22/d24/d2b/d2c/f2e 254 1 2024-02-16T22:56:14.938 INFO:tasks.workunit.client.0.smithi064.stdout:2/196: truncate d7/f10 2346339 0 2024-02-16T22:56:14.938 INFO:tasks.workunit.client.0.smithi064.stdout:2/197: stat d7/de 0 2024-02-16T22:56:14.945 INFO:tasks.workunit.client.0.smithi064.stdout:3/276: symlink d4/d14/d2b/l60 0 2024-02-16T22:56:14.945 INFO:tasks.workunit.client.0.smithi064.stdout:4/212: dwrite dc/dd/d32/d22/f24 [0,4194304] 0 2024-02-16T22:56:14.946 INFO:tasks.workunit.client.1.smithi192.stdout:1/272: unlink d3/de/l31 0 2024-02-16T22:56:14.947 INFO:tasks.workunit.client.0.smithi064.stdout:4/213: write dc/f17 [3193621,31829] 0 2024-02-16T22:56:14.955 INFO:tasks.workunit.client.1.smithi192.stdout:9/384: fdatasync d7/d15/d36/d38/f49 0 2024-02-16T22:56:14.959 INFO:tasks.workunit.client.1.smithi192.stdout:3/318: rmdir da/d11/d35 39 2024-02-16T22:56:14.960 INFO:tasks.workunit.client.0.smithi064.stdout:2/198: stat d7/de/f24 0 2024-02-16T22:56:14.963 INFO:tasks.workunit.client.1.smithi192.stdout:0/279: rename d1/d3/de/f21 to d1/f51 0 2024-02-16T22:56:14.963 INFO:tasks.workunit.client.1.smithi192.stdout:2/323: rename d22/d24/d39/d42 to d22/d24/d39/d42/d65/d69/d4f/d6e 22 2024-02-16T22:56:14.965 INFO:tasks.workunit.client.1.smithi192.stdout:2/324: truncate d22/d24/d39/d42/f54 668385 0 2024-02-16T22:56:14.967 INFO:tasks.workunit.client.1.smithi192.stdout:2/325: write d22/d24/d39/d3d/f40 [1143483,29406] 0 2024-02-16T22:56:14.972 INFO:tasks.workunit.client.1.smithi192.stdout:2/326: dwrite f8 [0,4194304] 0 2024-02-16T22:56:14.973 INFO:tasks.workunit.client.1.smithi192.stdout:2/327: readlink d22/d24/d39/l63 0 2024-02-16T22:56:14.973 INFO:tasks.workunit.client.0.smithi064.stdout:3/277: write d4/d13/d1b/d27/f58 [871324,102139] 0 2024-02-16T22:56:14.976 INFO:tasks.workunit.client.0.smithi064.stdout:3/278: truncate d4/d13/d1b/d43/f49 996253 0 2024-02-16T22:56:14.976 INFO:tasks.workunit.client.0.smithi064.stdout:3/279: stat d4/f11 0 2024-02-16T22:56:14.976 INFO:tasks.workunit.client.1.smithi192.stdout:4/316: truncate d1/d15/d23/f1a 4592660 0 2024-02-16T22:56:14.978 INFO:tasks.workunit.client.0.smithi064.stdout:4/214: rename dc/dd/d16/l20 to dc/dd/d32/l3f 0 2024-02-16T22:56:14.978 INFO:tasks.workunit.client.0.smithi064.stdout:9/228: write d3/d9/f21 [1860506,97008] 0 2024-02-16T22:56:14.979 INFO:tasks.workunit.client.1.smithi192.stdout:2/328: dwrite d22/d24/d39/d42/d65/d69/f68 [0,4194304] 0 2024-02-16T22:56:14.980 INFO:tasks.workunit.client.1.smithi192.stdout:6/304: mknod da/df/d12/c5d 0 2024-02-16T22:56:14.980 INFO:tasks.workunit.client.0.smithi064.stdout:9/229: chown d3/d11/d2b/d2a/d2e/d31/c53 1265258 1 2024-02-16T22:56:14.981 INFO:tasks.workunit.client.0.smithi064.stdout:0/163: creat da/d26/f28 x:0 0 0 2024-02-16T22:56:14.982 INFO:tasks.workunit.client.0.smithi064.stdout:0/164: chown c7 1265107 1 2024-02-16T22:56:14.984 INFO:tasks.workunit.client.0.smithi064.stdout:0/165: write da/dc/de/f16 [1009032,79625] 0 2024-02-16T22:56:14.985 INFO:tasks.workunit.client.1.smithi192.stdout:1/273: creat d3/de/d27/d34/d52/f57 x:0 0 0 2024-02-16T22:56:14.985 INFO:tasks.workunit.client.0.smithi064.stdout:0/166: write da/dc/de/f20 [447263,59733] 0 2024-02-16T22:56:14.988 INFO:tasks.workunit.client.1.smithi192.stdout:6/305: dread da/df/d12/d15/f2d [0,4194304] 0 2024-02-16T22:56:14.989 INFO:tasks.workunit.client.1.smithi192.stdout:6/306: dread da/df/d12/d15/f2d [0,4194304] 0 2024-02-16T22:56:14.992 INFO:tasks.workunit.client.1.smithi192.stdout:3/319: mkdir da/d11/d63/d6b/d66/d75 0 2024-02-16T22:56:14.995 INFO:tasks.workunit.client.1.smithi192.stdout:5/312: write d6/d12/d1d/f36 [2016362,76227] 0 2024-02-16T22:56:14.997 INFO:tasks.workunit.client.0.smithi064.stdout:2/199: mknod d7/d14/d1e/d38/c3b 0 2024-02-16T22:56:14.998 INFO:tasks.workunit.client.0.smithi064.stdout:7/188: sync 2024-02-16T22:56:14.998 INFO:tasks.workunit.client.0.smithi064.stdout:6/255: sync 2024-02-16T22:56:15.000 INFO:tasks.workunit.client.0.smithi064.stdout:9/230: creat d3/d9/d1e/d42/f54 x:0 0 0 2024-02-16T22:56:15.000 INFO:tasks.workunit.client.1.smithi192.stdout:5/313: dwrite d6/d12/d2d/f43 [0,4194304] 0 2024-02-16T22:56:15.001 INFO:tasks.workunit.client.1.smithi192.stdout:5/314: chown d6/d20 6562416 1 2024-02-16T22:56:15.010 INFO:tasks.workunit.client.0.smithi064.stdout:7/189: dwrite d5/d6/f34 [0,4194304] 0 2024-02-16T22:56:15.011 INFO:tasks.workunit.client.1.smithi192.stdout:4/317: write d1/d15/f45 [3573087,4181] 0 2024-02-16T22:56:15.012 INFO:tasks.workunit.client.1.smithi192.stdout:4/318: dread - d1/d26/d34/f50 zero size 2024-02-16T22:56:15.012 INFO:tasks.workunit.client.0.smithi064.stdout:9/231: dwrite d3/d9/d1e/d3e/f49 [0,4194304] 0 2024-02-16T22:56:15.013 INFO:tasks.workunit.client.1.smithi192.stdout:2/329: symlink d22/l6f 0 2024-02-16T22:56:15.023 INFO:tasks.workunit.client.0.smithi064.stdout:7/190: rename d5/d6/c13 to d5/d33/c36 0 2024-02-16T22:56:15.027 INFO:tasks.workunit.client.0.smithi064.stdout:2/200: getdents d7/d14 0 2024-02-16T22:56:15.028 INFO:tasks.workunit.client.0.smithi064.stdout:0/167: dread da/f1b [0,4194304] 0 2024-02-16T22:56:15.028 INFO:tasks.workunit.client.0.smithi064.stdout:2/201: truncate d7/de/f2f 1035888 0 2024-02-16T22:56:15.030 INFO:tasks.workunit.client.1.smithi192.stdout:9/385: rename d7/l9 to d7/d19/d37/d39/l85 0 2024-02-16T22:56:15.034 INFO:tasks.workunit.client.0.smithi064.stdout:9/232: rename d3/d11/c1c to d3/d11/d2b/d2a/d2e/d31/d3f/c55 0 2024-02-16T22:56:15.036 INFO:tasks.workunit.client.0.smithi064.stdout:4/215: dread dc/d1d/f36 [0,4194304] 0 2024-02-16T22:56:15.037 INFO:tasks.workunit.client.1.smithi192.stdout:8/293: dwrite d5/d9/f33 [0,4194304] 0 2024-02-16T22:56:15.038 INFO:tasks.workunit.client.0.smithi064.stdout:4/216: dread - dc/d10/f38 zero size 2024-02-16T22:56:15.038 INFO:tasks.workunit.client.1.smithi192.stdout:7/358: creat d1/d2/f7e x:0 0 0 2024-02-16T22:56:15.039 INFO:tasks.workunit.client.0.smithi064.stdout:2/202: read d7/f13 [505607,97288] 0 2024-02-16T22:56:15.039 INFO:tasks.workunit.client.0.smithi064.stdout:4/217: write dc/d1d/f2b [87715,16577] 0 2024-02-16T22:56:15.040 INFO:tasks.workunit.client.0.smithi064.stdout:2/203: readlink d7/de/l31 0 2024-02-16T22:56:15.041 INFO:tasks.workunit.client.0.smithi064.stdout:0/168: creat da/dc/de/d12/f29 x:0 0 0 2024-02-16T22:56:15.044 INFO:tasks.workunit.client.0.smithi064.stdout:7/191: unlink d5/d6/cd 0 2024-02-16T22:56:15.046 INFO:tasks.workunit.client.1.smithi192.stdout:2/330: symlink d22/d24/d2b/d2c/l70 0 2024-02-16T22:56:15.047 INFO:tasks.workunit.client.1.smithi192.stdout:1/274: symlink d3/de/l58 0 2024-02-16T22:56:15.049 INFO:tasks.workunit.client.0.smithi064.stdout:4/218: mknod dc/dd/d32/d22/c40 0 2024-02-16T22:56:15.050 INFO:tasks.workunit.client.0.smithi064.stdout:4/219: fsync dc/dd/d32/d22/f24 0 2024-02-16T22:56:15.050 INFO:tasks.workunit.client.0.smithi064.stdout:2/204: mknod d7/d14/d25/c3c 0 2024-02-16T22:56:15.052 INFO:tasks.workunit.client.1.smithi192.stdout:3/320: creat da/d11/d63/d6b/d66/d75/f76 x:0 0 0 2024-02-16T22:56:15.053 INFO:tasks.workunit.client.1.smithi192.stdout:1/275: dwrite d3/de/d27/f30 [4194304,4194304] 0 2024-02-16T22:56:15.054 INFO:tasks.workunit.client.0.smithi064.stdout:7/192: rename d5/l2a to d5/d6/l37 0 2024-02-16T22:56:15.057 INFO:tasks.workunit.client.1.smithi192.stdout:9/386: unlink d7/l35 0 2024-02-16T22:56:15.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.057+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a04115fd0 con 0x7f69ec06b380 2024-02-16T22:56:15.059 INFO:tasks.workunit.client.0.smithi064.stdout:6/256: truncate d0/d6/d10/f1c 3253931 0 2024-02-16T22:56:15.059 INFO:tasks.workunit.client.1.smithi192.stdout:1/276: dwrite d3/f16 [0,4194304] 0 2024-02-16T22:56:15.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.060+0000 7f69f97fa700 1 -- 172.21.15.64:0/118887879 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f6a04115fd0 con 0x7f69ec06b380 2024-02-16T22:56:15.064 INFO:tasks.workunit.client.1.smithi192.stdout:9/387: dwrite d7/d15/f6d [0,4194304] 0 2024-02-16T22:56:15.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f69ec06b380 msgr2=0x7f69ec06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:15.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f69ec06b380 0x7f69ec06d830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009fd0 tx=0x7f69f4009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:15.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a040fcef0 msgr2=0x7f6a04113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:15.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a040fcef0 0x7f6a04113f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000eb40 tx=0x7f6a0000b040 comp rx=0 tx=0).stop 2024-02-16T22:56:15.065 INFO:tasks.workunit.client.0.smithi064.stdout:4/220: unlink dc/de/f1c 0 2024-02-16T22:56:15.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 shutdown_connections 2024-02-16T22:56:15.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f69ec06b380 0x7f69ec06d830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a040fcef0 0x7f6a04113f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 --2- 172.21.15.64:0/118887879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a040fe0e0 0x7f6a041144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.064+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 >> 172.21.15.64:0/118887879 conn(0x7f6a040f8620 msgr2=0x7f6a041013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:15.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.065+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 shutdown_connections 2024-02-16T22:56:15.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.065+0000 7f6a0abcb700 1 -- 172.21.15.64:0/118887879 wait complete. 2024-02-16T22:56:15.067 INFO:tasks.workunit.client.1.smithi192.stdout:9/388: chown f1 3 1 2024-02-16T22:56:15.067 INFO:tasks.workunit.client.1.smithi192.stdout:9/389: stat d7/df/f16 0 2024-02-16T22:56:15.073 INFO:tasks.workunit.client.1.smithi192.stdout:7/359: fsync d1/d2/de/d2c/d49/d54/f7a 0 2024-02-16T22:56:15.074 INFO:tasks.workunit.client.1.smithi192.stdout:0/280: dwrite d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:15.074 INFO:tasks.workunit.client.0.smithi064.stdout:9/233: creat d3/d11/d2b/f56 x:0 0 0 2024-02-16T22:56:15.075 INFO:tasks.workunit.client.0.smithi064.stdout:9/234: chown d3/d11/c4c 21754550 1 2024-02-16T22:56:15.080 INFO:tasks.workunit.client.1.smithi192.stdout:2/331: rename f8 to d22/d24/d39/d42/d52/f71 0 2024-02-16T22:56:15.082 INFO:tasks.workunit.client.1.smithi192.stdout:2/332: fdatasync d22/d24/d2b/d2c/f2e 0 2024-02-16T22:56:15.084 INFO:tasks.workunit.client.0.smithi064.stdout:9/235: dwrite d3/d9/d1e/d3e/f49 [0,4194304] 0 2024-02-16T22:56:15.086 INFO:tasks.workunit.client.1.smithi192.stdout:6/307: creat da/df/f5e x:0 0 0 2024-02-16T22:56:15.086 INFO:tasks.workunit.client.0.smithi064.stdout:9/236: write d3/d9/d1e/d42/f54 [844392,27771] 0 2024-02-16T22:56:15.090 INFO:tasks.workunit.client.0.smithi064.stdout:4/221: rmdir dc/de/d28 39 2024-02-16T22:56:15.096 INFO:tasks.workunit.client.0.smithi064.stdout:6/257: dread d0/d6/f15 [0,4194304] 0 2024-02-16T22:56:15.098 INFO:tasks.workunit.client.0.smithi064.stdout:6/258: write d0/f41 [166554,48869] 0 2024-02-16T22:56:15.099 INFO:tasks.workunit.client.0.smithi064.stdout:7/193: mknod d5/c38 0 2024-02-16T22:56:15.101 INFO:tasks.workunit.client.1.smithi192.stdout:1/277: creat d3/d40/f59 x:0 0 0 2024-02-16T22:56:15.102 INFO:tasks.workunit.client.1.smithi192.stdout:1/278: write d3/de/d27/f44 [323337,99259] 0 2024-02-16T22:56:15.103 INFO:tasks.workunit.client.1.smithi192.stdout:1/279: fdatasync d3/de/f35 0 2024-02-16T22:56:15.105 INFO:tasks.workunit.client.1.smithi192.stdout:5/315: creat d6/f61 x:0 0 0 2024-02-16T22:56:15.105 INFO:tasks.workunit.client.1.smithi192.stdout:9/390: creat d7/d19/f86 x:0 0 0 2024-02-16T22:56:15.107 INFO:tasks.workunit.client.1.smithi192.stdout:9/391: write d7/d15/f6a [3572256,71083] 0 2024-02-16T22:56:15.107 INFO:tasks.workunit.client.0.smithi064.stdout:2/205: dwrite d7/d14/f2c [0,4194304] 0 2024-02-16T22:56:15.107 INFO:tasks.workunit.client.0.smithi064.stdout:9/237: symlink d3/d11/d2b/d2a/d2e/l57 0 2024-02-16T22:56:15.111 INFO:tasks.workunit.client.1.smithi192.stdout:8/294: dwrite d5/d9/f44 [0,4194304] 0 2024-02-16T22:56:15.112 INFO:tasks.workunit.client.1.smithi192.stdout:8/295: write d5/d26/f3f [1223733,1470] 0 2024-02-16T22:56:15.119 INFO:tasks.workunit.client.1.smithi192.stdout:4/319: creat d1/d8/f78 x:0 0 0 2024-02-16T22:56:15.119 INFO:tasks.workunit.client.1.smithi192.stdout:4/320: fdatasync d1/d8/f3a 0 2024-02-16T22:56:15.120 INFO:tasks.workunit.client.0.smithi064.stdout:0/169: getdents da 0 2024-02-16T22:56:15.120 INFO:tasks.workunit.client.1.smithi192.stdout:7/360: creat d1/d69/f7f x:0 0 0 2024-02-16T22:56:15.121 INFO:tasks.workunit.client.1.smithi192.stdout:0/281: symlink d1/d3/de/l52 0 2024-02-16T22:56:15.123 INFO:tasks.workunit.client.0.smithi064.stdout:0/170: dread da/dc/de/d12/d1d/f1f [0,4194304] 0 2024-02-16T22:56:15.128 INFO:tasks.workunit.client.1.smithi192.stdout:2/333: rename d22/d24/f34 to d22/f72 0 2024-02-16T22:56:15.134 INFO:tasks.workunit.client.1.smithi192.stdout:3/321: creat da/d11/d63/d6b/d71/f77 x:0 0 0 2024-02-16T22:56:15.136 INFO:tasks.workunit.client.0.smithi064.stdout:6/259: mknod d0/d6/d10/d33/c47 0 2024-02-16T22:56:15.137 INFO:tasks.workunit.client.0.smithi064.stdout:6/260: chown d0/d1d/d21/f2d 1 1 2024-02-16T22:56:15.138 INFO:tasks.workunit.client.1.smithi192.stdout:6/308: dwrite da/df/d12/d15/d1b/f1e [0,4194304] 0 2024-02-16T22:56:15.138 INFO:tasks.workunit.client.0.smithi064.stdout:0/171: creat da/dc/de/d12/f2a x:0 0 0 2024-02-16T22:56:15.140 INFO:tasks.workunit.client.0.smithi064.stdout:2/206: mknod d7/d14/d1e/d34/c3d 0 2024-02-16T22:56:15.143 INFO:tasks.workunit.client.0.smithi064.stdout:4/222: write dc/f19 [5385644,23473] 0 2024-02-16T22:56:15.143 INFO:tasks.workunit.client.1.smithi192.stdout:0/282: dread d1/d3/fc [4194304,4194304] 0 2024-02-16T22:56:15.143 INFO:tasks.workunit.client.0.smithi064.stdout:0/172: read da/dc/de/f17 [1848871,12878] 0 2024-02-16T22:56:15.145 INFO:tasks.workunit.client.0.smithi064.stdout:0/173: truncate da/d26/f28 980423 0 2024-02-16T22:56:15.148 INFO:tasks.workunit.client.0.smithi064.stdout:2/207: dread d7/d14/d1e/f28 [0,4194304] 0 2024-02-16T22:56:15.151 INFO:tasks.workunit.client.1.smithi192.stdout:5/316: creat d6/d12/d4e/d46/d4f/f62 x:0 0 0 2024-02-16T22:56:15.152 INFO:tasks.workunit.client.0.smithi064.stdout:4/223: dwrite dc/dd/d16/f3a [0,4194304] 0 2024-02-16T22:56:15.153 INFO:tasks.workunit.client.0.smithi064.stdout:4/224: readlink dc/de/l3c 0 2024-02-16T22:56:15.155 INFO:tasks.workunit.client.0.smithi064.stdout:6/261: unlink d0/d1d/c43 0 2024-02-16T22:56:15.155 INFO:tasks.workunit.client.0.smithi064.stdout:4/225: write dc/dd/d16/f23 [1513561,48896] 0 2024-02-16T22:56:15.156 INFO:tasks.workunit.client.0.smithi064.stdout:6/262: chown d0/d6/da 18492 1 2024-02-16T22:56:15.158 INFO:tasks.workunit.client.1.smithi192.stdout:5/317: dwrite d6/d8/f3e [0,4194304] 0 2024-02-16T22:56:15.160 INFO:tasks.workunit.client.0.smithi064.stdout:4/226: dread dc/d10/f14 [0,4194304] 0 2024-02-16T22:56:15.161 INFO:tasks.workunit.client.0.smithi064.stdout:4/227: write dc/d10/f14 [660518,48673] 0 2024-02-16T22:56:15.161 INFO:tasks.workunit.client.0.smithi064.stdout:0/174: creat da/dc/de/d12/d1d/f2b x:0 0 0 2024-02-16T22:56:15.162 INFO:tasks.workunit.client.1.smithi192.stdout:8/296: symlink d5/db/d47/l59 0 2024-02-16T22:56:15.162 INFO:tasks.workunit.client.1.smithi192.stdout:8/297: write d5/db/de/df/f2b [1574523,61271] 0 2024-02-16T22:56:15.168 INFO:tasks.workunit.client.0.smithi064.stdout:2/208: mkdir d7/d14/d3e 0 2024-02-16T22:56:15.170 INFO:tasks.workunit.client.0.smithi064.stdout:4/228: dwrite dc/dd/d32/f3d [0,4194304] 0 2024-02-16T22:56:15.171 INFO:tasks.workunit.client.1.smithi192.stdout:2/334: readlink l1c 0 2024-02-16T22:56:15.172 INFO:tasks.workunit.client.1.smithi192.stdout:8/298: dwrite d5/d26/f3f [0,4194304] 0 2024-02-16T22:56:15.172 INFO:tasks.workunit.client.0.smithi064.stdout:6/263: readlink d0/d5/df/l3a 0 2024-02-16T22:56:15.173 INFO:tasks.workunit.client.0.smithi064.stdout:6/264: chown d0/d6/d10 9991708 1 2024-02-16T22:56:15.174 INFO:tasks.workunit.client.1.smithi192.stdout:3/322: creat da/d11/d63/d6b/d71/f78 x:0 0 0 2024-02-16T22:56:15.174 INFO:tasks.workunit.client.1.smithi192.stdout:5/318: dread d6/d12/d4e/d46/f5d [0,4194304] 0 2024-02-16T22:56:15.180 INFO:tasks.workunit.client.0.smithi064.stdout:6/265: read d0/d5/f11 [3891630,99180] 0 2024-02-16T22:56:15.180 INFO:tasks.workunit.client.0.smithi064.stdout:7/194: getdents d5/d6 0 2024-02-16T22:56:15.181 INFO:tasks.workunit.client.0.smithi064.stdout:4/229: dwrite dc/d1d/f2b [0,4194304] 0 2024-02-16T22:56:15.183 INFO:tasks.workunit.client.0.smithi064.stdout:0/175: mkdir da/dc/de/d12/d2c 0 2024-02-16T22:56:15.183 INFO:tasks.workunit.client.0.smithi064.stdout:4/230: read - dc/d10/f38 zero size 2024-02-16T22:56:15.184 INFO:tasks.workunit.client.1.smithi192.stdout:0/283: read d1/f1f [596121,8859] 0 2024-02-16T22:56:15.187 INFO:tasks.workunit.client.0.smithi064.stdout:2/209: mkdir d7/d14/d1e/d34/d3f 0 2024-02-16T22:56:15.189 INFO:tasks.workunit.client.0.smithi064.stdout:2/210: write d7/d14/f2c [5117687,69482] 0 2024-02-16T22:56:15.190 INFO:tasks.workunit.client.1.smithi192.stdout:6/309: write da/df/d12/d15/f5c [986111,32] 0 2024-02-16T22:56:15.192 INFO:tasks.workunit.client.0.smithi064.stdout:2/211: dread - d7/d14/f33 zero size 2024-02-16T22:56:15.194 INFO:tasks.workunit.client.0.smithi064.stdout:4/231: dwrite dc/d10/f12 [0,4194304] 0 2024-02-16T22:56:15.197 INFO:tasks.workunit.client.0.smithi064.stdout:6/266: mkdir d0/d5/df/d17/d48 0 2024-02-16T22:56:15.200 INFO:tasks.workunit.client.1.smithi192.stdout:2/335: mknod d22/d24/d39/d42/d52/c73 0 2024-02-16T22:56:15.201 INFO:tasks.workunit.client.1.smithi192.stdout:8/299: creat d5/d9/f5a x:0 0 0 2024-02-16T22:56:15.203 INFO:tasks.workunit.client.1.smithi192.stdout:8/300: read d5/db/de/f37 [944804,42101] 0 2024-02-16T22:56:15.204 INFO:tasks.workunit.client.1.smithi192.stdout:9/392: truncate f1 2383091 0 2024-02-16T22:56:15.206 INFO:tasks.workunit.client.1.smithi192.stdout:8/301: truncate d5/d16/d34/f58 327757 0 2024-02-16T22:56:15.207 INFO:tasks.workunit.client.1.smithi192.stdout:3/323: rmdir da/d11/d2b 39 2024-02-16T22:56:15.208 INFO:tasks.workunit.client.1.smithi192.stdout:3/324: write da/df/f23 [5011133,62236] 0 2024-02-16T22:56:15.210 INFO:tasks.workunit.client.1.smithi192.stdout:3/325: write da/d11/f50 [331030,49871] 0 2024-02-16T22:56:15.210 INFO:tasks.workunit.client.0.smithi064.stdout:4/232: creat dc/d1d/f41 x:0 0 0 2024-02-16T22:56:15.212 INFO:tasks.workunit.client.0.smithi064.stdout:4/233: write dc/dd/d32/f3d [4264910,71074] 0 2024-02-16T22:56:15.213 INFO:tasks.workunit.client.0.smithi064.stdout:2/212: dread d7/d14/d1e/f28 [0,4194304] 0 2024-02-16T22:56:15.216 INFO:tasks.workunit.client.0.smithi064.stdout:6/267: creat d0/d12/f49 x:0 0 0 2024-02-16T22:56:15.217 INFO:tasks.workunit.client.0.smithi064.stdout:6/268: fsync d0/d6/f3b 0 2024-02-16T22:56:15.219 INFO:tasks.workunit.client.0.smithi064.stdout:4/234: dread dc/dd/d16/f3a [0,4194304] 0 2024-02-16T22:56:15.219 INFO:tasks.workunit.client.0.smithi064.stdout:0/176: creat da/dc/de/d12/d2c/f2d x:0 0 0 2024-02-16T22:56:15.219 INFO:tasks.workunit.client.1.smithi192.stdout:2/336: dread f10 [0,4194304] 0 2024-02-16T22:56:15.219 INFO:tasks.workunit.client.1.smithi192.stdout:0/284: truncate d1/d3/fc 5338909 0 2024-02-16T22:56:15.219 INFO:tasks.workunit.client.1.smithi192.stdout:2/337: read d22/d24/f49 [226121,76837] 0 2024-02-16T22:56:15.220 INFO:tasks.workunit.client.0.smithi064.stdout:0/177: write da/dc/f1e [1613753,108370] 0 2024-02-16T22:56:15.222 INFO:tasks.workunit.client.1.smithi192.stdout:0/285: dread d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:15.223 INFO:tasks.workunit.client.1.smithi192.stdout:0/286: write d1/fa [833142,18257] 0 2024-02-16T22:56:15.226 INFO:tasks.workunit.client.0.smithi064.stdout:7/195: truncate d5/f2f 154840 0 2024-02-16T22:56:15.230 INFO:tasks.workunit.client.0.smithi064.stdout:8/202: sync 2024-02-16T22:56:15.230 INFO:tasks.workunit.client.0.smithi064.stdout:5/173: sync 2024-02-16T22:56:15.230 INFO:tasks.workunit.client.0.smithi064.stdout:1/198: sync 2024-02-16T22:56:15.233 INFO:tasks.workunit.client.0.smithi064.stdout:5/174: truncate d6/f3c 622997 0 2024-02-16T22:56:15.234 INFO:tasks.workunit.client.1.smithi192.stdout:9/393: unlink d7/d15/d36/d54/c68 0 2024-02-16T22:56:15.236 INFO:tasks.workunit.client.0.smithi064.stdout:4/235: rmdir dc/d10 39 2024-02-16T22:56:15.238 INFO:tasks.workunit.client.0.smithi064.stdout:8/203: dread df/f19 [0,4194304] 0 2024-02-16T22:56:15.239 INFO:tasks.workunit.client.1.smithi192.stdout:7/361: sync 2024-02-16T22:56:15.240 INFO:tasks.workunit.client.0.smithi064.stdout:4/236: read dc/dd/d16/f2e [31263,99776] 0 2024-02-16T22:56:15.240 INFO:tasks.workunit.client.1.smithi192.stdout:7/362: write d1/d2/f7e [608113,52412] 0 2024-02-16T22:56:15.240 INFO:tasks.workunit.client.1.smithi192.stdout:8/302: mknod d5/d9/c5b 0 2024-02-16T22:56:15.241 INFO:tasks.workunit.client.0.smithi064.stdout:6/269: dread d0/f22 [0,4194304] 0 2024-02-16T22:56:15.242 INFO:tasks.workunit.client.0.smithi064.stdout:1/199: dwrite d2/df/f13 [0,4194304] 0 2024-02-16T22:56:15.248 INFO:tasks.workunit.client.0.smithi064.stdout:7/196: dread f0 [0,4194304] 0 2024-02-16T22:56:15.249 INFO:tasks.workunit.client.0.smithi064.stdout:1/200: dread d2/df/f13 [0,4194304] 0 2024-02-16T22:56:15.251 INFO:tasks.workunit.client.1.smithi192.stdout:3/326: read da/d11/d63/d6b/d4e/f56 [1439732,88293] 0 2024-02-16T22:56:15.252 INFO:tasks.workunit.client.1.smithi192.stdout:1/280: getdents d3/de/d27/d34 0 2024-02-16T22:56:15.253 INFO:tasks.workunit.client.1.smithi192.stdout:1/281: fsync d3/d40/f59 0 2024-02-16T22:56:15.255 INFO:tasks.workunit.client.1.smithi192.stdout:1/282: write d3/de/d27/d34/f4e [332584,25836] 0 2024-02-16T22:56:15.256 INFO:tasks.workunit.client.0.smithi064.stdout:1/201: dread d2/f42 [0,4194304] 0 2024-02-16T22:56:15.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.259+0000 7f0655cda700 1 -- 172.21.15.64:0/4140883939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 msgr2=0x7f0648093ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:15.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.259+0000 7f0655cda700 1 --2- 172.21.15.64:0/4140883939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 0x7f0648093ef0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0650060f80 tx=0x7f0650063f70 comp rx=0 tx=0).stop 2024-02-16T22:56:15.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.260+0000 7f0655cda700 1 -- 172.21.15.64:0/4140883939 shutdown_connections 2024-02-16T22:56:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.260+0000 7f0655cda700 1 --2- 172.21.15.64:0/4140883939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0648094cf0 0x7f0648095140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.260+0000 7f0655cda700 1 --2- 172.21.15.64:0/4140883939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 0x7f0648093ef0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.260+0000 7f0655cda700 1 -- 172.21.15.64:0/4140883939 >> 172.21.15.64:0/4140883939 conn(0x7f064808f1a0 msgr2=0x7f06480915d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:15.261 INFO:tasks.workunit.client.1.smithi192.stdout:1/283: dwrite d3/de/d27/f4a [0,4194304] 0 2024-02-16T22:56:15.261 INFO:tasks.workunit.client.0.smithi064.stdout:5/175: mknod d6/d18/c3f 0 2024-02-16T22:56:15.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.260+0000 7f0655cda700 1 -- 172.21.15.64:0/4140883939 shutdown_connections 2024-02-16T22:56:15.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.260+0000 7f0655cda700 1 -- 172.21.15.64:0/4140883939 wait complete. 2024-02-16T22:56:15.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0655cda700 1 Processor -- start 2024-02-16T22:56:15.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0655cda700 1 -- start start 2024-02-16T22:56:15.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0655cda700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 0x7f06480b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:15.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0655cda700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0648094cf0 0x7f06480b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0655cda700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06480b5eb0 con 0x7f0648093ae0 2024-02-16T22:56:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0655cda700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06480ba8a0 con 0x7f0648094cf0 2024-02-16T22:56:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0654cd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 0x7f06480b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0654cd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 0x7f06480b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43268/0 (socket says 172.21.15.64:43268) 2024-02-16T22:56:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.261+0000 7f0654cd8700 1 -- 172.21.15.64:0/4032785373 learned_addr learned my addr 172.21.15.64:0/4032785373 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.262+0000 7f064ffff700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0648094cf0 0x7f06480b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.262+0000 7f0654cd8700 1 -- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0648094cf0 msgr2=0x7f06480b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.262+0000 7f0654cd8700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0648094cf0 0x7f06480b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.262+0000 7f0654cd8700 1 -- 172.21.15.64:0/4032785373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06500677e0 con 0x7f0648093ae0 2024-02-16T22:56:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.262+0000 7f0654cd8700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 0x7f06480b52e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0650000eb0 tx=0x7f0650063f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.263+0000 7f064d7fa700 1 -- 172.21.15.64:0/4032785373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065006e070 con 0x7f0648093ae0 2024-02-16T22:56:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.263+0000 7f064d7fa700 1 -- 172.21.15.64:0/4032785373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0650064440 con 0x7f0648093ae0 2024-02-16T22:56:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.263+0000 7f064d7fa700 1 -- 172.21.15.64:0/4032785373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0650076a40 con 0x7f0648093ae0 2024-02-16T22:56:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.263+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0648004b80 con 0x7f0648093ae0 2024-02-16T22:56:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.264+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06480050c0 con 0x7f0648093ae0 2024-02-16T22:56:15.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.265+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064809c630 con 0x7f0648093ae0 2024-02-16T22:56:15.266 INFO:tasks.workunit.client.1.smithi192.stdout:2/338: mknod d22/d24/d39/d42/d65/c74 0 2024-02-16T22:56:15.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.266+0000 7f064d7fa700 1 -- 172.21.15.64:0/4032785373 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f065007c070 con 0x7f0648093ae0 2024-02-16T22:56:15.267 INFO:tasks.workunit.client.0.smithi064.stdout:8/204: mknod df/d31/d2f/c42 0 2024-02-16T22:56:15.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.267+0000 7f064d7fa700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f064006b400 0x7f064006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:15.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.267+0000 7f064d7fa700 1 -- 172.21.15.64:0/4032785373 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f06500ee390 con 0x7f0648093ae0 2024-02-16T22:56:15.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.268+0000 7f064ffff700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f064006b400 0x7f064006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:15.269 INFO:tasks.workunit.client.1.smithi192.stdout:4/321: getdents d1/d8 0 2024-02-16T22:56:15.269 INFO:tasks.workunit.client.1.smithi192.stdout:2/339: write d22/d24/d39/d42/d65/d69/f68 [101945,74239] 0 2024-02-16T22:56:15.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.269+0000 7f064ffff700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f064006b400 0x7f064006d8b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0644006fd0 tx=0x7f06440090f0 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:15.270 INFO:tasks.workunit.client.0.smithi064.stdout:6/270: mkdir d0/d5/df/d17/d4a 0 2024-02-16T22:56:15.272 INFO:tasks.workunit.client.0.smithi064.stdout:8/205: read - df/d31/d2f/f2b zero size 2024-02-16T22:56:15.272 INFO:tasks.workunit.client.0.smithi064.stdout:5/176: dwrite d6/f3c [0,4194304] 0 2024-02-16T22:56:15.273 INFO:tasks.workunit.client.0.smithi064.stdout:2/213: dwrite d7/f10 [0,4194304] 0 2024-02-16T22:56:15.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.274+0000 7f064d7fa700 1 -- 172.21.15.64:0/4032785373 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f06500bc010 con 0x7f0648093ae0 2024-02-16T22:56:15.275 INFO:tasks.workunit.client.0.smithi064.stdout:4/237: rename dc/f2c to dc/dd/f42 0 2024-02-16T22:56:15.276 INFO:tasks.workunit.client.1.smithi192.stdout:0/287: mkdir d1/d3/de/d22/d37/d47/d53 0 2024-02-16T22:56:15.286 INFO:tasks.workunit.client.1.smithi192.stdout:9/394: creat d7/d19/d37/d5c/f87 x:0 0 0 2024-02-16T22:56:15.287 INFO:tasks.workunit.client.0.smithi064.stdout:1/202: mkdir d2/df/d14/d18/d48 0 2024-02-16T22:56:15.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/395: write d7/d19/d5e/d6f/f7d [720597,65967] 0 2024-02-16T22:56:15.289 INFO:tasks.workunit.client.1.smithi192.stdout:8/303: symlink d5/d3a/l5c 0 2024-02-16T22:56:15.290 INFO:tasks.workunit.client.0.smithi064.stdout:5/177: dread d6/d18/d19/f26 [0,4194304] 0 2024-02-16T22:56:15.290 INFO:tasks.workunit.client.0.smithi064.stdout:2/214: dwrite d7/d14/f2e [0,4194304] 0 2024-02-16T22:56:15.290 INFO:tasks.workunit.client.1.smithi192.stdout:9/396: write d7/d19/d37/f34 [46555,56804] 0 2024-02-16T22:56:15.290 INFO:tasks.workunit.client.1.smithi192.stdout:7/363: creat d1/d2/de/d2c/d49/f80 x:0 0 0 2024-02-16T22:56:15.291 INFO:tasks.workunit.client.0.smithi064.stdout:2/215: fsync d7/d14/d25/d37/f3a 0 2024-02-16T22:56:15.291 INFO:tasks.workunit.client.1.smithi192.stdout:7/364: fdatasync d1/d19/d2a/f7c 0 2024-02-16T22:56:15.292 INFO:tasks.workunit.client.0.smithi064.stdout:8/206: dwrite f8 [0,4194304] 0 2024-02-16T22:56:15.292 INFO:tasks.workunit.client.1.smithi192.stdout:3/327: creat da/df/f79 x:0 0 0 2024-02-16T22:56:15.295 INFO:tasks.workunit.client.1.smithi192.stdout:0/288: dread d1/d3/d29/f3f [0,4194304] 0 2024-02-16T22:56:15.297 INFO:tasks.workunit.client.1.smithi192.stdout:7/365: dwrite d1/d2/f25 [0,4194304] 0 2024-02-16T22:56:15.298 INFO:tasks.workunit.client.0.smithi064.stdout:6/271: mknod d0/d1d/c4b 0 2024-02-16T22:56:15.300 INFO:tasks.workunit.client.1.smithi192.stdout:1/284: mkdir d3/de/d27/d34/d52/d5a 0 2024-02-16T22:56:15.301 INFO:tasks.workunit.client.1.smithi192.stdout:6/310: creat da/f5f x:0 0 0 2024-02-16T22:56:15.303 INFO:tasks.workunit.client.1.smithi192.stdout:0/289: read d1/f36 [2880810,50993] 0 2024-02-16T22:56:15.306 INFO:tasks.workunit.client.0.smithi064.stdout:7/197: mknod d5/d33/d35/c39 0 2024-02-16T22:56:15.306 INFO:tasks.workunit.client.1.smithi192.stdout:4/322: symlink d1/d26/d5e/l79 0 2024-02-16T22:56:15.307 INFO:tasks.workunit.client.1.smithi192.stdout:7/366: dwrite d1/d2/f1e [0,4194304] 0 2024-02-16T22:56:15.309 INFO:tasks.workunit.client.1.smithi192.stdout:7/367: chown d1/d2/d37/c45 5207886 1 2024-02-16T22:56:15.316 INFO:tasks.workunit.client.1.smithi192.stdout:8/304: rename f2 to d5/db/de/f5d 0 2024-02-16T22:56:15.318 INFO:tasks.workunit.client.0.smithi064.stdout:2/216: mknod d7/d14/c40 0 2024-02-16T22:56:15.319 INFO:tasks.workunit.client.1.smithi192.stdout:3/328: stat da/d11/d2b/c37 0 2024-02-16T22:56:15.320 INFO:tasks.workunit.client.1.smithi192.stdout:3/329: write da/d11/d20/f4a [753361,60192] 0 2024-02-16T22:56:15.320 INFO:tasks.workunit.client.0.smithi064.stdout:8/207: mknod df/d2a/c43 0 2024-02-16T22:56:15.320 INFO:tasks.workunit.client.1.smithi192.stdout:3/330: write da/f3d [4435478,122963] 0 2024-02-16T22:56:15.321 INFO:tasks.workunit.client.1.smithi192.stdout:5/319: truncate d6/f10 1959312 0 2024-02-16T22:56:15.322 INFO:tasks.workunit.client.1.smithi192.stdout:3/331: chown da/d11/d63/d6b/d66/d75/f76 0 1 2024-02-16T22:56:15.325 INFO:tasks.workunit.client.1.smithi192.stdout:5/320: dread d6/d12/d1d/f36 [0,4194304] 0 2024-02-16T22:56:15.325 INFO:tasks.workunit.client.0.smithi064.stdout:6/272: rename d0/d5/f2b to d0/d1b/f4c 0 2024-02-16T22:56:15.327 INFO:tasks.workunit.client.0.smithi064.stdout:7/198: creat d5/d33/d35/f3a x:0 0 0 2024-02-16T22:56:15.328 INFO:tasks.workunit.client.0.smithi064.stdout:7/199: chown d5/d6/c28 1986 1 2024-02-16T22:56:15.328 INFO:tasks.workunit.client.1.smithi192.stdout:6/311: write da/d40/f4c [1875014,117941] 0 2024-02-16T22:56:15.328 INFO:tasks.workunit.client.0.smithi064.stdout:7/200: chown d5/c31 3161 1 2024-02-16T22:56:15.331 INFO:tasks.workunit.client.0.smithi064.stdout:8/208: write df/f19 [288002,38982] 0 2024-02-16T22:56:15.335 INFO:tasks.workunit.client.1.smithi192.stdout:2/340: unlink d22/d24/d2b/d2c/l6a 0 2024-02-16T22:56:15.337 INFO:tasks.workunit.client.0.smithi064.stdout:4/238: rename dc/de/d28/d3e to dc/d1d/d43 0 2024-02-16T22:56:15.340 INFO:tasks.workunit.client.0.smithi064.stdout:9/238: sync 2024-02-16T22:56:15.340 INFO:tasks.workunit.client.0.smithi064.stdout:3/280: sync 2024-02-16T22:56:15.341 INFO:tasks.workunit.client.0.smithi064.stdout:9/239: readlink d3/l8 0 2024-02-16T22:56:15.342 INFO:tasks.workunit.client.1.smithi192.stdout:7/368: symlink d1/d1f/l81 0 2024-02-16T22:56:15.345 INFO:tasks.workunit.client.0.smithi064.stdout:8/209: creat df/d31/d2f/f44 x:0 0 0 2024-02-16T22:56:15.345 INFO:tasks.workunit.client.0.smithi064.stdout:4/239: dwrite dc/dd/d32/d22/f24 [0,4194304] 0 2024-02-16T22:56:15.348 INFO:tasks.workunit.client.0.smithi064.stdout:8/210: chown c5 1 1 2024-02-16T22:56:15.348 INFO:tasks.workunit.client.0.smithi064.stdout:7/201: dwrite f0 [0,4194304] 0 2024-02-16T22:56:15.350 INFO:tasks.workunit.client.1.smithi192.stdout:9/397: rename d7/d19/d37/d5c/c60 to d7/d19/d28/c88 0 2024-02-16T22:56:15.351 INFO:tasks.workunit.client.1.smithi192.stdout:9/398: chown d7/d15/d36/d38/f49 90641 1 2024-02-16T22:56:15.353 INFO:tasks.workunit.client.0.smithi064.stdout:3/281: dwrite d4/d14/f46 [0,4194304] 0 2024-02-16T22:56:15.355 INFO:tasks.workunit.client.0.smithi064.stdout:5/178: rename d6/l3e to d6/d1c/l40 0 2024-02-16T22:56:15.358 INFO:tasks.workunit.client.0.smithi064.stdout:5/179: write d6/d1c/f20 [179098,91874] 0 2024-02-16T22:56:15.362 INFO:tasks.workunit.client.0.smithi064.stdout:5/180: truncate d6/f31 616480 0 2024-02-16T22:56:15.363 INFO:tasks.workunit.client.1.smithi192.stdout:8/305: dread d5/db/d47/f3b [0,4194304] 0 2024-02-16T22:56:15.364 INFO:tasks.workunit.client.0.smithi064.stdout:3/282: dwrite d4/f18 [0,4194304] 0 2024-02-16T22:56:15.365 INFO:tasks.workunit.client.0.smithi064.stdout:8/211: truncate df/f15 178763 0 2024-02-16T22:56:15.367 INFO:tasks.workunit.client.0.smithi064.stdout:1/203: write d2/df/d14/d18/f28 [8265,41448] 0 2024-02-16T22:56:15.369 INFO:tasks.workunit.client.1.smithi192.stdout:5/321: mknod d6/d8/c63 0 2024-02-16T22:56:15.371 INFO:tasks.workunit.client.0.smithi064.stdout:7/202: rename d5/d6/f9 to d5/d6/f3b 0 2024-02-16T22:56:15.372 INFO:tasks.workunit.client.1.smithi192.stdout:1/285: mkdir d3/d5b 0 2024-02-16T22:56:15.372 INFO:tasks.workunit.client.0.smithi064.stdout:5/181: dwrite d6/f34 [0,4194304] 0 2024-02-16T22:56:15.374 INFO:tasks.workunit.client.0.smithi064.stdout:7/203: truncate d5/f27 4984979 0 2024-02-16T22:56:15.374 INFO:tasks.workunit.client.0.smithi064.stdout:5/182: write d6/f34 [1175543,74983] 0 2024-02-16T22:56:15.374 INFO:tasks.workunit.client.1.smithi192.stdout:1/286: write d3/de/d27/f44 [1071546,71144] 0 2024-02-16T22:56:15.376 INFO:tasks.workunit.client.0.smithi064.stdout:2/217: rename d7/de/l31 to d7/d14/d25/l41 0 2024-02-16T22:56:15.379 INFO:tasks.workunit.client.0.smithi064.stdout:4/240: symlink dc/l44 0 2024-02-16T22:56:15.380 INFO:tasks.workunit.client.1.smithi192.stdout:2/341: creat d22/d24/d2b/d2c/f75 x:0 0 0 2024-02-16T22:56:15.385 INFO:tasks.workunit.client.1.smithi192.stdout:3/332: rename da/d11/f3c to da/d11/d63/d6b/d66/f7a 0 2024-02-16T22:56:15.386 INFO:tasks.workunit.client.1.smithi192.stdout:2/342: read fe [4447496,118279] 0 2024-02-16T22:56:15.388 INFO:tasks.workunit.client.0.smithi064.stdout:1/204: creat d2/df/d14/d2c/f49 x:0 0 0 2024-02-16T22:56:15.389 INFO:tasks.workunit.client.0.smithi064.stdout:1/205: dread - d2/df/d14/d18/d3e/f45 zero size 2024-02-16T22:56:15.392 INFO:tasks.workunit.client.1.smithi192.stdout:3/333: dwrite da/d11/d63/d6b/d66/d75/f76 [0,4194304] 0 2024-02-16T22:56:15.392 INFO:tasks.workunit.client.1.smithi192.stdout:8/306: unlink d5/db/de/df/c24 0 2024-02-16T22:56:15.392 INFO:tasks.workunit.client.1.smithi192.stdout:3/334: read - da/d11/d63/d6b/f4d zero size 2024-02-16T22:56:15.394 INFO:tasks.workunit.client.1.smithi192.stdout:3/335: write da/f3d [2789390,5415] 0 2024-02-16T22:56:15.397 INFO:tasks.workunit.client.0.smithi064.stdout:4/241: symlink dc/dd/d16/l45 0 2024-02-16T22:56:15.399 INFO:tasks.workunit.client.1.smithi192.stdout:8/307: dwrite d5/db/d47/f4a [0,4194304] 0 2024-02-16T22:56:15.400 INFO:tasks.workunit.client.0.smithi064.stdout:3/283: mkdir d4/d13/d1b/d27/d61 0 2024-02-16T22:56:15.403 INFO:tasks.workunit.client.1.smithi192.stdout:0/290: link d1/d3/de/d22/d37/d47/c49 d1/d3/de/d22/d37/c54 0 2024-02-16T22:56:15.406 INFO:tasks.workunit.client.1.smithi192.stdout:1/287: dread d3/de/f35 [0,4194304] 0 2024-02-16T22:56:15.407 INFO:tasks.workunit.client.1.smithi192.stdout:7/369: rename d1/d2/f6 to d1/d1f/f82 0 2024-02-16T22:56:15.423 INFO:tasks.workunit.client.0.smithi064.stdout:2/218: dread d7/de/f11 [0,4194304] 0 2024-02-16T22:56:15.423 INFO:tasks.workunit.client.0.smithi064.stdout:2/219: dread - d7/de/f24 zero size 2024-02-16T22:56:15.423 INFO:tasks.workunit.client.0.smithi064.stdout:2/220: chown d7/d14/d1e/d34/d3f 163 1 2024-02-16T22:56:15.427 INFO:tasks.workunit.client.0.smithi064.stdout:7/204: link c2 d5/d33/d35/c3c 0 2024-02-16T22:56:15.436 INFO:tasks.workunit.client.0.smithi064.stdout:8/212: getdents df 0 2024-02-16T22:56:15.437 INFO:tasks.workunit.client.0.smithi064.stdout:8/213: chown df/f1b 1310 1 2024-02-16T22:56:15.440 INFO:tasks.workunit.client.0.smithi064.stdout:6/273: rename d0/d6/d10/d33/c31 to d0/d6/c4d 0 2024-02-16T22:56:15.442 INFO:tasks.workunit.client.1.smithi192.stdout:2/343: unlink d22/d24/c3a 0 2024-02-16T22:56:15.444 INFO:tasks.workunit.client.0.smithi064.stdout:8/214: fsync df/f32 0 2024-02-16T22:56:15.444 INFO:tasks.workunit.client.1.smithi192.stdout:8/308: creat d5/d16/d34/f5e x:0 0 0 2024-02-16T22:56:15.445 INFO:tasks.workunit.client.1.smithi192.stdout:4/323: write d1/d15/d23/f1a [127350,90333] 0 2024-02-16T22:56:15.446 INFO:tasks.workunit.client.1.smithi192.stdout:6/312: dread da/df/d12/d57/f3a [0,4194304] 0 2024-02-16T22:56:15.447 INFO:tasks.workunit.client.0.smithi064.stdout:1/206: write d2/f42 [1974011,102889] 0 2024-02-16T22:56:15.449 INFO:tasks.workunit.client.1.smithi192.stdout:1/288: dread d3/de/ff [0,4194304] 0 2024-02-16T22:56:15.451 INFO:tasks.workunit.client.1.smithi192.stdout:1/289: stat d3/de/d27/d34/f45 0 2024-02-16T22:56:15.457 INFO:tasks.workunit.client.0.smithi064.stdout:7/205: link d5/d6/f34 d5/d33/f3d 0 2024-02-16T22:56:15.457 INFO:tasks.workunit.client.0.smithi064.stdout:6/274: dread d0/d5/f11 [0,4194304] 0 2024-02-16T22:56:15.458 INFO:tasks.workunit.client.0.smithi064.stdout:4/242: fsync dc/f33 0 2024-02-16T22:56:15.459 INFO:tasks.workunit.client.0.smithi064.stdout:1/207: mknod d2/df/c4a 0 2024-02-16T22:56:15.459 INFO:tasks.workunit.client.1.smithi192.stdout:7/370: dread d1/f11 [0,4194304] 0 2024-02-16T22:56:15.461 INFO:tasks.workunit.client.0.smithi064.stdout:3/284: truncate d4/d13/f55 1863363 0 2024-02-16T22:56:15.462 INFO:tasks.workunit.client.0.smithi064.stdout:3/285: fsync d4/f42 0 2024-02-16T22:56:15.462 INFO:tasks.workunit.client.0.smithi064.stdout:8/215: symlink df/d31/d2f/d3e/l45 0 2024-02-16T22:56:15.463 INFO:tasks.workunit.client.0.smithi064.stdout:3/286: chown d4/d13/d1b/c50 3532 1 2024-02-16T22:56:15.464 INFO:tasks.workunit.client.1.smithi192.stdout:2/344: mknod d22/d24/d39/d42/d52/c76 0 2024-02-16T22:56:15.464 INFO:tasks.workunit.client.1.smithi192.stdout:2/345: chown d22/l4b 3352 1 2024-02-16T22:56:15.464 INFO:tasks.workunit.client.0.smithi064.stdout:2/221: truncate d7/d14/d1e/f21 2627129 0 2024-02-16T22:56:15.467 INFO:tasks.workunit.client.1.smithi192.stdout:0/291: symlink d1/d3/l55 0 2024-02-16T22:56:15.469 INFO:tasks.workunit.client.0.smithi064.stdout:6/275: truncate d0/d5/f11 2126243 0 2024-02-16T22:56:15.470 INFO:tasks.workunit.client.0.smithi064.stdout:4/243: rmdir dc/dd/d32 39 2024-02-16T22:56:15.472 INFO:tasks.workunit.client.0.smithi064.stdout:1/208: symlink d2/df/d14/d18/l4b 0 2024-02-16T22:56:15.473 INFO:tasks.workunit.client.0.smithi064.stdout:1/209: chown d2/df/d14/f2d 54 1 2024-02-16T22:56:15.473 INFO:tasks.workunit.client.0.smithi064.stdout:7/206: dread d5/f10 [0,4194304] 0 2024-02-16T22:56:15.474 INFO:tasks.workunit.client.1.smithi192.stdout:8/309: dwrite d5/db/de/f5d [0,4194304] 0 2024-02-16T22:56:15.474 INFO:tasks.workunit.client.1.smithi192.stdout:4/324: unlink d1/d15/d23/c21 0 2024-02-16T22:56:15.474 INFO:tasks.workunit.client.0.smithi064.stdout:1/210: chown d2/df/f13 487429 1 2024-02-16T22:56:15.475 INFO:tasks.workunit.client.1.smithi192.stdout:4/325: write d1/d26/f31 [122702,78868] 0 2024-02-16T22:56:15.475 INFO:tasks.workunit.client.0.smithi064.stdout:8/216: creat df/d31/d2f/d38/f46 x:0 0 0 2024-02-16T22:56:15.476 INFO:tasks.workunit.client.0.smithi064.stdout:1/211: dread - d2/df/d14/d2c/f49 zero size 2024-02-16T22:56:15.477 INFO:tasks.workunit.client.1.smithi192.stdout:4/326: write d1/d8/f3a [1445776,66539] 0 2024-02-16T22:56:15.477 INFO:tasks.workunit.client.0.smithi064.stdout:8/217: read df/d31/f29 [341844,13436] 0 2024-02-16T22:56:15.478 INFO:tasks.workunit.client.1.smithi192.stdout:9/399: sync 2024-02-16T22:56:15.479 INFO:tasks.workunit.client.1.smithi192.stdout:9/400: dread - d7/d19/d37/f55 zero size 2024-02-16T22:56:15.483 INFO:tasks.workunit.client.1.smithi192.stdout:1/290: dread d3/de/f1b [0,4194304] 0 2024-02-16T22:56:15.483 INFO:tasks.workunit.client.1.smithi192.stdout:7/371: mkdir d1/d2/de/d83 0 2024-02-16T22:56:15.486 INFO:tasks.workunit.client.0.smithi064.stdout:4/244: write dc/dd/d32/f1f [129117,42733] 0 2024-02-16T22:56:15.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/322: link d6/d8/l31 d6/d12/d1d/l64 0 2024-02-16T22:56:15.491 INFO:tasks.workunit.client.1.smithi192.stdout:1/291: dwrite d3/de/d27/d34/d52/f54 [0,4194304] 0 2024-02-16T22:56:15.494 INFO:tasks.workunit.client.0.smithi064.stdout:7/207: dread d5/d6/f3b [4194304,4194304] 0 2024-02-16T22:56:15.494 INFO:tasks.workunit.client.0.smithi064.stdout:4/245: dwrite dc/d1d/f41 [0,4194304] 0 2024-02-16T22:56:15.497 INFO:tasks.workunit.client.0.smithi064.stdout:1/212: rmdir d2/df/d14/d18/d3e 39 2024-02-16T22:56:15.499 INFO:tasks.workunit.client.0.smithi064.stdout:1/213: write d2/f43 [811454,50291] 0 2024-02-16T22:56:15.500 INFO:tasks.workunit.client.0.smithi064.stdout:0/178: sync 2024-02-16T22:56:15.501 INFO:tasks.workunit.client.0.smithi064.stdout:1/214: chown d2/df/d11/d1a/d1d/l2b 19978666 1 2024-02-16T22:56:15.502 INFO:tasks.workunit.client.0.smithi064.stdout:1/215: write d2/f36 [4543596,40889] 0 2024-02-16T22:56:15.503 INFO:tasks.workunit.client.0.smithi064.stdout:1/216: stat d2/f36 0 2024-02-16T22:56:15.507 INFO:tasks.workunit.client.1.smithi192.stdout:8/310: rename d5/d16/d34 to d5/d3e/d5f 0 2024-02-16T22:56:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:6/313: link da/df/d12/d15/d1b/f37 da/df/d12/d15/d1b/f60 0 2024-02-16T22:56:15.516 INFO:tasks.workunit.client.1.smithi192.stdout:6/314: write da/df/d12/d15/f3e [3659940,92711] 0 2024-02-16T22:56:15.517 INFO:tasks.workunit.client.0.smithi064.stdout:8/218: mkdir df/d31/d47 0 2024-02-16T22:56:15.517 INFO:tasks.workunit.client.0.smithi064.stdout:2/222: dwrite d7/de/f11 [0,4194304] 0 2024-02-16T22:56:15.519 INFO:tasks.workunit.client.1.smithi192.stdout:9/401: unlink d7/d19/d28/f61 0 2024-02-16T22:56:15.521 INFO:tasks.workunit.client.1.smithi192.stdout:3/336: sync 2024-02-16T22:56:15.521 INFO:tasks.workunit.client.1.smithi192.stdout:3/337: read - da/d11/f3f zero size 2024-02-16T22:56:15.523 INFO:tasks.workunit.client.1.smithi192.stdout:3/338: read - da/d11/f3f zero size 2024-02-16T22:56:15.524 INFO:tasks.workunit.client.1.smithi192.stdout:7/372: mknod d1/d2/de/d2c/d49/d54/c84 0 2024-02-16T22:56:15.524 INFO:tasks.workunit.client.1.smithi192.stdout:2/346: dwrite f19 [0,4194304] 0 2024-02-16T22:56:15.529 INFO:tasks.workunit.client.1.smithi192.stdout:1/292: symlink d3/de/d27/l5c 0 2024-02-16T22:56:15.532 INFO:tasks.workunit.client.1.smithi192.stdout:0/292: creat d1/d2/d34/f56 x:0 0 0 2024-02-16T22:56:15.537 INFO:tasks.workunit.client.1.smithi192.stdout:9/402: creat d7/d19/d5b/f89 x:0 0 0 2024-02-16T22:56:15.538 INFO:tasks.workunit.client.1.smithi192.stdout:7/373: creat d1/d1f/f85 x:0 0 0 2024-02-16T22:56:15.538 INFO:tasks.workunit.client.1.smithi192.stdout:3/339: creat da/d11/d63/d6b/d71/f7b x:0 0 0 2024-02-16T22:56:15.539 INFO:tasks.workunit.client.1.smithi192.stdout:2/347: rename l1c to d22/d24/d39/d42/l77 0 2024-02-16T22:56:15.541 INFO:tasks.workunit.client.1.smithi192.stdout:0/293: readlink d1/l23 0 2024-02-16T22:56:15.542 INFO:tasks.workunit.client.1.smithi192.stdout:0/294: chown d1/d2/c44 1 1 2024-02-16T22:56:15.542 INFO:tasks.workunit.client.0.smithi064.stdout:4/246: mkdir dc/dd/d32/d46 0 2024-02-16T22:56:15.542 INFO:tasks.workunit.client.0.smithi064.stdout:2/223: dread d7/de/f2f [0,4194304] 0 2024-02-16T22:56:15.545 INFO:tasks.workunit.client.0.smithi064.stdout:0/179: mkdir da/dc/de/d12/d2c/d2e 0 2024-02-16T22:56:15.545 INFO:tasks.workunit.client.1.smithi192.stdout:9/403: dread - d7/f2e zero size 2024-02-16T22:56:15.547 INFO:tasks.workunit.client.0.smithi064.stdout:8/219: mkdir df/d31/d2f/d38/d48 0 2024-02-16T22:56:15.549 INFO:tasks.workunit.client.0.smithi064.stdout:4/247: symlink dc/de/d28/l47 0 2024-02-16T22:56:15.549 INFO:tasks.workunit.client.0.smithi064.stdout:1/217: dread - d2/df/d14/d18/d3e/f45 zero size 2024-02-16T22:56:15.551 INFO:tasks.workunit.client.1.smithi192.stdout:7/374: mkdir d1/d2/d37/d86 0 2024-02-16T22:56:15.552 INFO:tasks.workunit.client.0.smithi064.stdout:6/276: getdents d0/d5/df 0 2024-02-16T22:56:15.553 INFO:tasks.workunit.client.0.smithi064.stdout:2/224: dwrite d7/f35 [0,4194304] 0 2024-02-16T22:56:15.555 INFO:tasks.workunit.client.0.smithi064.stdout:6/277: chown d0/d5/df/d17/d18/f3f 1 1 2024-02-16T22:56:15.559 INFO:tasks.workunit.client.1.smithi192.stdout:2/348: mkdir d22/d24/d2b/d2c/d78 0 2024-02-16T22:56:15.560 INFO:tasks.workunit.client.1.smithi192.stdout:2/349: write f19 [260310,38371] 0 2024-02-16T22:56:15.561 INFO:tasks.workunit.client.0.smithi064.stdout:0/180: creat da/dc/de/d12/d1d/f2f x:0 0 0 2024-02-16T22:56:15.562 INFO:tasks.workunit.client.0.smithi064.stdout:8/220: rmdir df/d2a 39 2024-02-16T22:56:15.567 INFO:tasks.workunit.client.1.smithi192.stdout:2/350: dwrite d22/f31 [0,4194304] 0 2024-02-16T22:56:15.569 INFO:tasks.workunit.client.1.smithi192.stdout:2/351: chown d22/d24/d39/d42/d65/d69/f68 1820 1 2024-02-16T22:56:15.570 INFO:tasks.workunit.client.1.smithi192.stdout:0/295: dwrite d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:15.570 INFO:tasks.workunit.client.1.smithi192.stdout:6/315: creat da/df/d18/d46/f61 x:0 0 0 2024-02-16T22:56:15.571 INFO:tasks.workunit.client.0.smithi064.stdout:4/248: symlink dc/d1d/d43/l48 0 2024-02-16T22:56:15.571 INFO:tasks.workunit.client.1.smithi192.stdout:2/352: dread - d22/d24/d39/f4c zero size 2024-02-16T22:56:15.572 INFO:tasks.workunit.client.1.smithi192.stdout:9/404: mkdir d7/d15/d36/d38/d3f/d8a 0 2024-02-16T22:56:15.575 INFO:tasks.workunit.client.0.smithi064.stdout:1/218: symlink d2/df/d14/d18/d40/l4c 0 2024-02-16T22:56:15.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/405: write d7/d15/f5d [597217,60495] 0 2024-02-16T22:56:15.577 INFO:tasks.workunit.client.0.smithi064.stdout:1/219: write d2/df/d14/d2c/f49 [7383,99326] 0 2024-02-16T22:56:15.578 INFO:tasks.workunit.client.1.smithi192.stdout:7/375: unlink d1/d2/de/d2c/d49/d54/f62 0 2024-02-16T22:56:15.578 INFO:tasks.workunit.client.0.smithi064.stdout:7/208: dwrite d5/d6/f34 [0,4194304] 0 2024-02-16T22:56:15.578 INFO:tasks.workunit.client.1.smithi192.stdout:7/376: chown d1/d2/d2d/d61 28970759 1 2024-02-16T22:56:15.581 INFO:tasks.workunit.client.0.smithi064.stdout:7/209: truncate d5/f23 3556480 0 2024-02-16T22:56:15.583 INFO:tasks.workunit.client.0.smithi064.stdout:1/220: truncate d2/df/d14/d2c/f3c 820052 0 2024-02-16T22:56:15.583 INFO:tasks.workunit.client.0.smithi064.stdout:1/221: chown d2/df/d11/d1a/d30 1503 1 2024-02-16T22:56:15.584 INFO:tasks.workunit.client.0.smithi064.stdout:2/225: rmdir d7/d14/d1e 39 2024-02-16T22:56:15.584 INFO:tasks.workunit.client.1.smithi192.stdout:5/323: dwrite d6/d12/f1e [4194304,4194304] 0 2024-02-16T22:56:15.592 INFO:tasks.workunit.client.1.smithi192.stdout:8/311: getdents d5/db/de/df 0 2024-02-16T22:56:15.593 INFO:tasks.workunit.client.1.smithi192.stdout:5/324: dwrite d6/d12/d2d/f43 [0,4194304] 0 2024-02-16T22:56:15.594 INFO:tasks.workunit.client.0.smithi064.stdout:0/181: creat da/d26/f30 x:0 0 0 2024-02-16T22:56:15.595 INFO:tasks.workunit.client.1.smithi192.stdout:4/327: write d1/fd [2588600,76727] 0 2024-02-16T22:56:15.596 INFO:tasks.workunit.client.1.smithi192.stdout:0/296: symlink d1/d3/d18/l57 0 2024-02-16T22:56:15.596 INFO:tasks.workunit.client.1.smithi192.stdout:4/328: write d1/d8/d11/f4e [4332985,97999] 0 2024-02-16T22:56:15.602 INFO:tasks.workunit.client.1.smithi192.stdout:2/353: symlink d22/d24/d39/d42/d65/d69/l79 0 2024-02-16T22:56:15.604 INFO:tasks.workunit.client.1.smithi192.stdout:0/297: dread d1/d3/d29/f1a [0,4194304] 0 2024-02-16T22:56:15.604 INFO:tasks.workunit.client.0.smithi064.stdout:1/222: creat d2/df/d11/d1a/d1d/f4d x:0 0 0 2024-02-16T22:56:15.606 INFO:tasks.workunit.client.1.smithi192.stdout:9/406: fdatasync d7/d15/f27 0 2024-02-16T22:56:15.606 INFO:tasks.workunit.client.0.smithi064.stdout:1/223: write d2/df/d11/d1a/f34 [422007,113816] 0 2024-02-16T22:56:15.609 INFO:tasks.workunit.client.1.smithi192.stdout:7/377: creat d1/d19/d2a/f87 x:0 0 0 2024-02-16T22:56:15.611 INFO:tasks.workunit.client.1.smithi192.stdout:1/293: dwrite d3/de/f1b [0,4194304] 0 2024-02-16T22:56:15.613 INFO:tasks.workunit.client.0.smithi064.stdout:0/182: creat da/dc/de/d12/d2c/f31 x:0 0 0 2024-02-16T22:56:15.615 INFO:tasks.workunit.client.0.smithi064.stdout:8/221: creat df/d31/d2f/d38/d48/f49 x:0 0 0 2024-02-16T22:56:15.617 INFO:tasks.workunit.client.0.smithi064.stdout:7/210: symlink d5/l3e 0 2024-02-16T22:56:15.618 INFO:tasks.workunit.client.1.smithi192.stdout:8/312: write d5/db/f51 [4022093,98421] 0 2024-02-16T22:56:15.619 INFO:tasks.workunit.client.0.smithi064.stdout:2/226: symlink d7/de/l42 0 2024-02-16T22:56:15.620 INFO:tasks.workunit.client.0.smithi064.stdout:2/227: truncate d7/d14/f2c 5205647 0 2024-02-16T22:56:15.624 INFO:tasks.workunit.client.1.smithi192.stdout:4/329: mknod d1/d15/d48/c7a 0 2024-02-16T22:56:15.624 INFO:tasks.workunit.client.1.smithi192.stdout:1/294: fsync d3/de/f1b 0 2024-02-16T22:56:15.627 INFO:tasks.workunit.client.0.smithi064.stdout:0/183: creat da/d26/f32 x:0 0 0 2024-02-16T22:56:15.628 INFO:tasks.workunit.client.1.smithi192.stdout:1/295: dread d3/de/d27/d34/f4e [0,4194304] 0 2024-02-16T22:56:15.630 INFO:tasks.workunit.client.1.smithi192.stdout:4/330: dwrite d1/d15/d48/f55 [4194304,4194304] 0 2024-02-16T22:56:15.631 INFO:tasks.workunit.client.0.smithi064.stdout:7/211: creat d5/d33/d35/f3f x:0 0 0 2024-02-16T22:56:15.635 INFO:tasks.workunit.client.1.smithi192.stdout:7/378: write d1/d19/d4c/f6b [3568738,16349] 0 2024-02-16T22:56:15.636 INFO:tasks.workunit.client.1.smithi192.stdout:7/379: write d1/d2/f1e [1151401,31648] 0 2024-02-16T22:56:15.637 INFO:tasks.workunit.client.1.smithi192.stdout:1/296: dwrite d3/f22 [4194304,4194304] 0 2024-02-16T22:56:15.639 INFO:tasks.workunit.client.1.smithi192.stdout:7/380: write d1/d2/de/d2c/d49/f71 [5937006,61291] 0 2024-02-16T22:56:15.640 INFO:tasks.workunit.client.0.smithi064.stdout:6/278: dread d0/d6/d10/f3e [0,4194304] 0 2024-02-16T22:56:15.643 INFO:tasks.workunit.client.0.smithi064.stdout:0/184: rmdir da/dc/de/d12/d2c 39 2024-02-16T22:56:15.644 INFO:tasks.workunit.client.0.smithi064.stdout:0/185: chown da/dc/de/c1a 8246833 1 2024-02-16T22:56:15.651 INFO:tasks.workunit.client.0.smithi064.stdout:6/279: dread d0/d6/da/f25 [0,4194304] 0 2024-02-16T22:56:15.652 INFO:tasks.workunit.client.0.smithi064.stdout:1/224: dwrite d2/df/d14/f16 [0,4194304] 0 2024-02-16T22:56:15.653 INFO:tasks.workunit.client.0.smithi064.stdout:1/225: chown d2/df/d11/d25 3 1 2024-02-16T22:56:15.658 INFO:tasks.workunit.client.0.smithi064.stdout:8/222: truncate df/f19 2966420 0 2024-02-16T22:56:15.664 INFO:tasks.workunit.client.1.smithi192.stdout:9/407: link d7/d19/d37/d5a/f63 d7/d19/d28/f8b 0 2024-02-16T22:56:15.664 INFO:tasks.workunit.client.0.smithi064.stdout:0/186: dread da/dc/f1e [0,4194304] 0 2024-02-16T22:56:15.666 INFO:tasks.workunit.client.0.smithi064.stdout:6/280: rename d0/d5/df/l3a to d0/d5/df/d17/d18/l4e 0 2024-02-16T22:56:15.667 INFO:tasks.workunit.client.1.smithi192.stdout:4/331: creat d1/d15/d48/d4c/f7b x:0 0 0 2024-02-16T22:56:15.668 INFO:tasks.workunit.client.1.smithi192.stdout:4/332: truncate d1/d26/f77 988947 0 2024-02-16T22:56:15.668 INFO:tasks.workunit.client.0.smithi064.stdout:7/212: unlink d5/d6/c1e 0 2024-02-16T22:56:15.669 INFO:tasks.workunit.client.1.smithi192.stdout:4/333: write d1/d15/f45 [5310631,19214] 0 2024-02-16T22:56:15.669 INFO:tasks.workunit.client.1.smithi192.stdout:4/334: readlink d1/l71 0 2024-02-16T22:56:15.669 INFO:tasks.workunit.client.0.smithi064.stdout:2/228: creat d7/d14/d1e/d34/d3f/f43 x:0 0 0 2024-02-16T22:56:15.670 INFO:tasks.workunit.client.1.smithi192.stdout:4/335: dread - d1/d26/d34/f76 zero size 2024-02-16T22:56:15.671 INFO:tasks.workunit.client.0.smithi064.stdout:8/223: chown df/d31/d2f/c26 826344 1 2024-02-16T22:56:15.672 INFO:tasks.workunit.client.0.smithi064.stdout:8/224: write df/d31/f36 [565152,116431] 0 2024-02-16T22:56:15.673 INFO:tasks.workunit.client.1.smithi192.stdout:1/297: mkdir d3/de/d27/d34/d36/d47/d5d 0 2024-02-16T22:56:15.674 INFO:tasks.workunit.client.0.smithi064.stdout:0/187: rename da/d26/f30 to da/d26/f33 0 2024-02-16T22:56:15.675 INFO:tasks.workunit.client.1.smithi192.stdout:3/340: link da/d11/d35/l36 da/d11/d2b/l7c 0 2024-02-16T22:56:15.676 INFO:tasks.workunit.client.0.smithi064.stdout:0/188: fdatasync da/dc/de/d12/f29 0 2024-02-16T22:56:15.679 INFO:tasks.workunit.client.1.smithi192.stdout:7/381: mkdir d1/d2/d2d/d61/d88 0 2024-02-16T22:56:15.681 INFO:tasks.workunit.client.1.smithi192.stdout:8/313: unlink d5/d9/lc 0 2024-02-16T22:56:15.685 INFO:tasks.workunit.client.0.smithi064.stdout:6/281: dwrite d0/d6/f15 [4194304,4194304] 0 2024-02-16T22:56:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.685+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0648005ae0 con 0x7f064006b400 2024-02-16T22:56:15.687 INFO:tasks.workunit.client.0.smithi064.stdout:2/229: mkdir d7/d14/d25/d37/d44 0 2024-02-16T22:56:15.689 INFO:tasks.workunit.client.1.smithi192.stdout:6/316: rename da/df/l2a to da/df/d18/l62 0 2024-02-16T22:56:15.690 INFO:tasks.workunit.client.0.smithi064.stdout:7/213: dwrite d5/d6/f30 [0,4194304] 0 2024-02-16T22:56:15.692 INFO:tasks.workunit.client.0.smithi064.stdout:8/225: creat df/d31/d2f/d3e/f4a x:0 0 0 2024-02-16T22:56:15.693 INFO:tasks.workunit.client.0.smithi064.stdout:8/226: chown df/d31/f29 0 1 2024-02-16T22:56:15.694 INFO:tasks.workunit.client.1.smithi192.stdout:9/408: creat d7/d19/d37/f8c x:0 0 0 2024-02-16T22:56:15.695 INFO:tasks.workunit.client.0.smithi064.stdout:0/189: creat da/dc/de/d12/d1d/f34 x:0 0 0 2024-02-16T22:56:15.698 INFO:tasks.workunit.client.1.smithi192.stdout:6/317: dwrite da/df/d12/d15/f5c [0,4194304] 0 2024-02-16T22:56:15.698 INFO:tasks.workunit.client.1.smithi192.stdout:4/336: symlink d1/d15/d29/l7c 0 2024-02-16T22:56:15.699 INFO:tasks.workunit.client.1.smithi192.stdout:4/337: chown d1/d26/d5e/f70 4443 1 2024-02-16T22:56:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.699+0000 7f064d7fa700 1 -- 172.21.15.64:0/4032785373 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f0648005ae0 con 0x7f064006b400 2024-02-16T22:56:15.700 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:56:15.700 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (6m) 2m ago 7m 25.1M - 0.25.0 c8568f914cd2 829b066b8c26 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (8m) 2m ago 8m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (7m) 2s ago 7m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (6m) 2m ago 7m 86.0M - 9.4.7 954c08fa6188 529226ab33c5 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (2m) 2m ago 2m 14.7M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (2m) 2m ago 2m 14.5M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (2m) 2s ago 2m 14.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (2m) 2s ago 2m 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:9283 running (9m) 2m ago 9m 480M - 17.2.7-513-gb88fae66 870a0944ccb8 7189c74ed7fc 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (30s) 2s ago 6m 487M - 19.0.0-1450-gf78a58c0 f081f847070a 05118581c81b 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (9m) 2m ago 9m 45.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (6m) 2s ago 6m 36.2M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (8m) 2m ago 7m 15.5M - 1.5.0 0da6a335fe13 c25d924fa381 2024-02-16T22:56:15.701 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 2s ago 6m 15.4M - 1.5.0 0da6a335fe13 c32798cb2969 2024-02-16T22:56:15.702 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (6m) 2m ago 6m 46.5M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:56:15.702 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (5m) 2m ago 5m 45.3M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:56:15.702 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (5m) 2m ago 5m 44.7M 4288M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:56:15.702 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (4m) 2s ago 4m 94.9M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:56:15.702 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (4m) 2s ago 4m 90.5M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:56:15.702 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (3m) 2s ago 3m 89.6M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:56:15.702 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (6m) 2m ago 7m 53.4M - 2.43.0 a07b618ecd1d dd20eff0542c 2024-02-16T22:56:15.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/298: creat d3/de/d27/d34/d36/d47/f5e x:0 0 0 2024-02-16T22:56:15.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/299: fdatasync d3/f50 0 2024-02-16T22:56:15.703 INFO:tasks.workunit.client.0.smithi064.stdout:6/282: mkdir d0/d5/df/d17/d18/d4f 0 2024-02-16T22:56:15.703 INFO:tasks.workunit.client.0.smithi064.stdout:6/283: dread - d0/d5/f42 zero size 2024-02-16T22:56:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.703+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f064006b400 msgr2=0x7f064006d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.703+0000 7f0655cda700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f064006b400 0x7f064006d8b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0644006fd0 tx=0x7f06440090f0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.703+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 msgr2=0x7f06480b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.703+0000 7f0655cda700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 0x7f06480b52e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0650000eb0 tx=0x7f0650063f70 comp rx=0 tx=0).stop 2024-02-16T22:56:15.705 INFO:tasks.workunit.client.1.smithi192.stdout:3/341: symlink da/d11/d63/d6b/l7d 0 2024-02-16T22:56:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.704+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 shutdown_connections 2024-02-16T22:56:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.704+0000 7f0655cda700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0648093ae0 0x7f06480b52e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.704+0000 7f0655cda700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f064006b400 0x7f064006d8b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.704+0000 7f0655cda700 1 --2- 172.21.15.64:0/4032785373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0648094cf0 0x7f06480b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.704+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 >> 172.21.15.64:0/4032785373 conn(0x7f064808f1a0 msgr2=0x7f0648097f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:15.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.704+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 shutdown_connections 2024-02-16T22:56:15.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.704+0000 7f0655cda700 1 -- 172.21.15.64:0/4032785373 wait complete. 2024-02-16T22:56:15.707 INFO:tasks.workunit.client.1.smithi192.stdout:7/382: mkdir d1/d2/de/d2c/d49/d89 0 2024-02-16T22:56:15.709 INFO:tasks.workunit.client.0.smithi064.stdout:7/214: chown d5/d6/f22 6169 1 2024-02-16T22:56:15.712 INFO:tasks.workunit.client.1.smithi192.stdout:5/325: rename d6/d12/f32 to d6/d12/d2d/d5c/f65 0 2024-02-16T22:56:15.714 INFO:tasks.workunit.client.1.smithi192.stdout:7/383: dwrite d1/d2/de/d2c/d49/d54/f7a [0,4194304] 0 2024-02-16T22:56:15.715 INFO:tasks.workunit.client.1.smithi192.stdout:5/326: dread d6/d12/d1d/f34 [0,4194304] 0 2024-02-16T22:56:15.716 INFO:tasks.workunit.client.0.smithi064.stdout:8/227: dread df/d31/f3f [0,4194304] 0 2024-02-16T22:56:15.717 INFO:tasks.workunit.client.0.smithi064.stdout:1/226: rename d2/df/d14/d18/d3e to d2/d4e 0 2024-02-16T22:56:15.718 INFO:tasks.workunit.client.1.smithi192.stdout:2/354: getdents d22/d24/d39/d42/d65/d69/d4f 0 2024-02-16T22:56:15.719 INFO:tasks.workunit.client.1.smithi192.stdout:2/355: write d22/d24/d39/d3d/f40 [1137487,127799] 0 2024-02-16T22:56:15.723 INFO:tasks.workunit.client.1.smithi192.stdout:9/409: creat d7/d19/d28/d43/d70/f8d x:0 0 0 2024-02-16T22:56:15.725 INFO:tasks.workunit.client.1.smithi192.stdout:4/338: creat d1/d15/d48/f7d x:0 0 0 2024-02-16T22:56:15.726 INFO:tasks.workunit.client.0.smithi064.stdout:2/230: fsync d7/f8 0 2024-02-16T22:56:15.729 INFO:tasks.workunit.client.0.smithi064.stdout:7/215: creat d5/d33/d35/f40 x:0 0 0 2024-02-16T22:56:15.732 INFO:tasks.workunit.client.1.smithi192.stdout:3/342: rmdir da/d11/d63/d6b/d71 39 2024-02-16T22:56:15.732 INFO:tasks.workunit.client.1.smithi192.stdout:3/343: write da/f51 [377591,42436] 0 2024-02-16T22:56:15.739 INFO:tasks.workunit.client.0.smithi064.stdout:6/284: dread d0/d1b/f4c [0,4194304] 0 2024-02-16T22:56:15.740 INFO:tasks.workunit.client.0.smithi064.stdout:1/227: rmdir d2/df/d14/d18/d40 39 2024-02-16T22:56:15.743 INFO:tasks.workunit.client.1.smithi192.stdout:0/298: rename d1/d3/d29/c4d to d1/d3/d29/d43/c58 0 2024-02-16T22:56:15.744 INFO:tasks.workunit.client.1.smithi192.stdout:0/299: stat d1/d3/de/d22/c25 0 2024-02-16T22:56:15.745 INFO:tasks.workunit.client.0.smithi064.stdout:2/231: unlink d7/d14/d1e/d38/c3b 0 2024-02-16T22:56:15.746 INFO:tasks.workunit.client.1.smithi192.stdout:4/339: dread d1/d15/f4f [0,4194304] 0 2024-02-16T22:56:15.748 INFO:tasks.workunit.client.1.smithi192.stdout:7/384: mkdir d1/d2/d2d/d61/d8a 0 2024-02-16T22:56:15.749 INFO:tasks.workunit.client.0.smithi064.stdout:0/190: mknod da/dc/de/d12/d2c/d2e/c35 0 2024-02-16T22:56:15.749 INFO:tasks.workunit.client.1.smithi192.stdout:7/385: truncate d1/d19/d4c/f6b 5036811 0 2024-02-16T22:56:15.754 INFO:tasks.workunit.client.0.smithi064.stdout:6/285: creat d0/d1b/f50 x:0 0 0 2024-02-16T22:56:15.754 INFO:tasks.workunit.client.1.smithi192.stdout:2/356: creat d22/d24/d39/d42/d65/d69/f7a x:0 0 0 2024-02-16T22:56:15.757 INFO:tasks.workunit.client.0.smithi064.stdout:0/191: dwrite da/d26/f28 [0,4194304] 0 2024-02-16T22:56:15.760 INFO:tasks.workunit.client.0.smithi064.stdout:8/228: dwrite fa [0,4194304] 0 2024-02-16T22:56:15.761 INFO:tasks.workunit.client.1.smithi192.stdout:1/300: getdents d3/d40/d55 0 2024-02-16T22:56:15.767 INFO:tasks.workunit.client.1.smithi192.stdout:8/314: link d5/d26/l42 d5/db/d47/d43/l60 0 2024-02-16T22:56:15.768 INFO:tasks.workunit.client.1.smithi192.stdout:8/315: chown d5/d26 24682 1 2024-02-16T22:56:15.769 INFO:tasks.workunit.client.1.smithi192.stdout:8/316: chown d5/db/de/df/f2b 21910 1 2024-02-16T22:56:15.774 INFO:tasks.workunit.client.1.smithi192.stdout:0/300: unlink d1/d3/d18/l50 0 2024-02-16T22:56:15.774 INFO:tasks.workunit.client.0.smithi064.stdout:2/232: mknod d7/c45 0 2024-02-16T22:56:15.775 INFO:tasks.workunit.client.1.smithi192.stdout:0/301: dread d1/d2/f42 [0,4194304] 0 2024-02-16T22:56:15.776 INFO:tasks.workunit.client.1.smithi192.stdout:0/302: chown d1/f3e 13331 1 2024-02-16T22:56:15.776 INFO:tasks.workunit.client.0.smithi064.stdout:2/233: write d7/f12 [2761459,121813] 0 2024-02-16T22:56:15.777 INFO:tasks.workunit.client.0.smithi064.stdout:2/234: truncate d7/d14/f33 873627 0 2024-02-16T22:56:15.778 INFO:tasks.workunit.client.0.smithi064.stdout:2/235: fdatasync d7/de/f1f 0 2024-02-16T22:56:15.779 INFO:tasks.workunit.client.0.smithi064.stdout:7/216: truncate d5/f32 2361552 0 2024-02-16T22:56:15.781 INFO:tasks.workunit.client.0.smithi064.stdout:0/192: write da/d26/f33 [17993,25322] 0 2024-02-16T22:56:15.781 INFO:tasks.workunit.client.1.smithi192.stdout:7/386: rmdir d1/d2/d2d/d55 39 2024-02-16T22:56:15.781 INFO:tasks.workunit.client.1.smithi192.stdout:9/410: dwrite d7/df/f22 [0,4194304] 0 2024-02-16T22:56:15.783 INFO:tasks.workunit.client.1.smithi192.stdout:3/344: write da/df/f3a [1005281,35020] 0 2024-02-16T22:56:15.783 INFO:tasks.workunit.client.0.smithi064.stdout:1/228: creat d2/f4f x:0 0 0 2024-02-16T22:56:15.786 INFO:tasks.workunit.client.1.smithi192.stdout:2/357: rmdir d22/d24/d2b/d2c 39 2024-02-16T22:56:15.790 INFO:tasks.workunit.client.0.smithi064.stdout:6/286: dwrite d0/f22 [0,4194304] 0 2024-02-16T22:56:15.791 INFO:tasks.workunit.client.1.smithi192.stdout:6/318: creat da/df/d18/f63 x:0 0 0 2024-02-16T22:56:15.791 INFO:tasks.workunit.client.0.smithi064.stdout:6/287: dread - d0/d1b/f50 zero size 2024-02-16T22:56:15.799 INFO:tasks.workunit.client.0.smithi064.stdout:6/288: dwrite d0/d5/f27 [0,4194304] 0 2024-02-16T22:56:15.799 INFO:tasks.workunit.client.0.smithi064.stdout:2/236: mkdir d7/d14/d25/d46 0 2024-02-16T22:56:15.802 INFO:tasks.workunit.client.0.smithi064.stdout:5/183: sync 2024-02-16T22:56:15.802 INFO:tasks.workunit.client.0.smithi064.stdout:9/240: sync 2024-02-16T22:56:15.803 INFO:tasks.workunit.client.0.smithi064.stdout:7/217: mkdir d5/d33/d35/d41 0 2024-02-16T22:56:15.806 INFO:tasks.workunit.client.0.smithi064.stdout:0/193: creat da/dc/de/d12/d2c/d2e/f36 x:0 0 0 2024-02-16T22:56:15.808 INFO:tasks.workunit.client.0.smithi064.stdout:8/229: mknod df/d31/c4b 0 2024-02-16T22:56:15.809 INFO:tasks.workunit.client.0.smithi064.stdout:0/194: read da/fb [4090520,27560] 0 2024-02-16T22:56:15.809 INFO:tasks.workunit.client.0.smithi064.stdout:2/237: dwrite d7/d14/f2e [4194304,4194304] 0 2024-02-16T22:56:15.810 INFO:tasks.workunit.client.1.smithi192.stdout:8/317: unlink d5/d3e/d5f/f58 0 2024-02-16T22:56:15.814 INFO:tasks.workunit.client.1.smithi192.stdout:0/303: mknod d1/d3/d29/d43/c59 0 2024-02-16T22:56:15.815 INFO:tasks.workunit.client.0.smithi064.stdout:0/195: dread da/d26/f33 [0,4194304] 0 2024-02-16T22:56:15.815 INFO:tasks.workunit.client.1.smithi192.stdout:4/340: link d1/d8/d11/f4e d1/d8/d11/d58/f7e 0 2024-02-16T22:56:15.817 INFO:tasks.workunit.client.0.smithi064.stdout:0/196: dread - da/dc/de/d12/d2c/d2e/f36 zero size 2024-02-16T22:56:15.818 INFO:tasks.workunit.client.0.smithi064.stdout:6/289: creat d0/d5/df/d17/d18/f51 x:0 0 0 2024-02-16T22:56:15.820 INFO:tasks.workunit.client.0.smithi064.stdout:5/184: creat d6/d18/f41 x:0 0 0 2024-02-16T22:56:15.824 INFO:tasks.workunit.client.0.smithi064.stdout:6/290: dread d0/d1d/d21/f2d [0,4194304] 0 2024-02-16T22:56:15.824 INFO:tasks.workunit.client.1.smithi192.stdout:5/327: symlink d6/d12/d1d/d3c/l66 0 2024-02-16T22:56:15.825 INFO:tasks.workunit.client.1.smithi192.stdout:5/328: chown d6/d20/l35 13213 1 2024-02-16T22:56:15.825 INFO:tasks.workunit.client.0.smithi064.stdout:7/218: dread d5/d6/f1d [0,4194304] 0 2024-02-16T22:56:15.828 INFO:tasks.workunit.client.1.smithi192.stdout:9/411: mkdir d7/d19/d28/d43/d70/d8e 0 2024-02-16T22:56:15.828 INFO:tasks.workunit.client.1.smithi192.stdout:9/412: dread - d7/d19/f86 zero size 2024-02-16T22:56:15.829 INFO:tasks.workunit.client.0.smithi064.stdout:5/185: dread d6/d18/d19/f26 [0,4194304] 0 2024-02-16T22:56:15.829 INFO:tasks.workunit.client.0.smithi064.stdout:5/186: chown d6/fc 39239 1 2024-02-16T22:56:15.831 INFO:tasks.workunit.client.1.smithi192.stdout:2/358: rmdir d22/d24/d39/d42 39 2024-02-16T22:56:15.838 INFO:tasks.workunit.client.1.smithi192.stdout:1/301: unlink d3/c29 0 2024-02-16T22:56:15.839 INFO:tasks.workunit.client.1.smithi192.stdout:4/341: dread d1/d26/f28 [4194304,4194304] 0 2024-02-16T22:56:15.839 INFO:tasks.workunit.client.1.smithi192.stdout:7/387: dread d1/d2/f1b [0,4194304] 0 2024-02-16T22:56:15.844 INFO:tasks.workunit.client.0.smithi064.stdout:0/197: mkdir da/dc/de/d37 0 2024-02-16T22:56:15.845 INFO:tasks.workunit.client.1.smithi192.stdout:0/304: symlink d1/d3/d18/l5a 0 2024-02-16T22:56:15.847 INFO:tasks.workunit.client.1.smithi192.stdout:5/329: mkdir d6/d12/d4e/d5b/d67 0 2024-02-16T22:56:15.849 INFO:tasks.workunit.client.1.smithi192.stdout:9/413: mknod d7/d15/d36/d77/c8f 0 2024-02-16T22:56:15.850 INFO:tasks.workunit.client.1.smithi192.stdout:2/359: mknod d22/c7b 0 2024-02-16T22:56:15.853 INFO:tasks.workunit.client.1.smithi192.stdout:5/330: dwrite d6/d12/d4e/d5b/f5e [0,4194304] 0 2024-02-16T22:56:15.854 INFO:tasks.workunit.client.0.smithi064.stdout:6/291: dwrite d0/d6/d10/d33/f2a [0,4194304] 0 2024-02-16T22:56:15.857 INFO:tasks.workunit.client.1.smithi192.stdout:1/302: symlink d3/de/d27/d34/d36/l5f 0 2024-02-16T22:56:15.857 INFO:tasks.workunit.client.0.smithi064.stdout:2/238: link d7/de/f11 d7/d14/d25/d37/f47 0 2024-02-16T22:56:15.857 INFO:tasks.workunit.client.1.smithi192.stdout:1/303: stat d3/d5b 0 2024-02-16T22:56:15.861 INFO:tasks.workunit.client.0.smithi064.stdout:7/219: dread d5/d6/f18 [0,4194304] 0 2024-02-16T22:56:15.863 INFO:tasks.workunit.client.0.smithi064.stdout:0/198: mkdir da/dc/de/d12/d2c/d2e/d38 0 2024-02-16T22:56:15.866 INFO:tasks.workunit.client.1.smithi192.stdout:0/305: mknod d1/d3/de/c5b 0 2024-02-16T22:56:15.866 INFO:tasks.workunit.client.1.smithi192.stdout:6/319: truncate da/df/d12/d57/f2c 309813 0 2024-02-16T22:56:15.867 INFO:tasks.workunit.client.0.smithi064.stdout:4/249: sync 2024-02-16T22:56:15.867 INFO:tasks.workunit.client.0.smithi064.stdout:3/287: sync 2024-02-16T22:56:15.868 INFO:tasks.workunit.client.1.smithi192.stdout:0/306: write d1/d3/de/d22/d37/d47/f48 [823044,115235] 0 2024-02-16T22:56:15.870 INFO:tasks.workunit.client.0.smithi064.stdout:5/187: rename d6/d25/c35 to d6/d1c/c42 0 2024-02-16T22:56:15.870 INFO:tasks.workunit.client.0.smithi064.stdout:8/230: write df/f19 [544149,29510] 0 2024-02-16T22:56:15.870 INFO:tasks.workunit.client.1.smithi192.stdout:0/307: dread d1/d3/d29/f1a [0,4194304] 0 2024-02-16T22:56:15.873 INFO:tasks.workunit.client.0.smithi064.stdout:6/292: fsync d0/d6/d10/f3e 0 2024-02-16T22:56:15.873 INFO:tasks.workunit.client.1.smithi192.stdout:9/414: mknod d7/d15/d36/d38/d3f/c90 0 2024-02-16T22:56:15.876 INFO:tasks.workunit.client.0.smithi064.stdout:2/239: readlink d7/de/l2d 0 2024-02-16T22:56:15.876 INFO:tasks.workunit.client.0.smithi064.stdout:3/288: dread d4/d13/f28 [0,4194304] 0 2024-02-16T22:56:15.876 INFO:tasks.workunit.client.1.smithi192.stdout:9/415: read d7/d19/d37/f3d [6380831,56471] 0 2024-02-16T22:56:15.876 INFO:tasks.workunit.client.0.smithi064.stdout:1/229: link d2/df/d14/d18/d40/l44 d2/df/d11/d25/l50 0 2024-02-16T22:56:15.877 INFO:tasks.workunit.client.0.smithi064.stdout:3/289: dread - d4/f42 zero size 2024-02-16T22:56:15.878 INFO:tasks.workunit.client.0.smithi064.stdout:3/290: dread - d4/d13/d41/f3b zero size 2024-02-16T22:56:15.879 INFO:tasks.workunit.client.1.smithi192.stdout:4/342: link d1/d15/d23/f5c d1/d6b/f7f 0 2024-02-16T22:56:15.879 INFO:tasks.workunit.client.1.smithi192.stdout:4/343: chown d1/d26/d66 623980 1 2024-02-16T22:56:15.880 INFO:tasks.workunit.client.0.smithi064.stdout:7/220: dread - d5/d6/f19 zero size 2024-02-16T22:56:15.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.879+0000 7f4b79372700 1 -- 172.21.15.64:0/1813670421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 msgr2=0x7f4b74104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:15.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.879+0000 7f4b79372700 1 --2- 172.21.15.64:0/1813670421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 0x7f4b74104ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4b64004940 tx=0x7f4b64009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:15.881 INFO:tasks.workunit.client.1.smithi192.stdout:4/344: write d1/d26/d5e/f63 [131714,86620] 0 2024-02-16T22:56:15.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.880+0000 7f4b79372700 1 -- 172.21.15.64:0/1813670421 shutdown_connections 2024-02-16T22:56:15.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.880+0000 7f4b79372700 1 --2- 172.21.15.64:0/1813670421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 0x7f4b74104ba0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.880+0000 7f4b79372700 1 --2- 172.21.15.64:0/1813670421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b74103540 0x7f4b74103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.880+0000 7f4b79372700 1 -- 172.21.15.64:0/1813670421 >> 172.21.15.64:0/1813670421 conn(0x7f4b740727d0 msgr2=0x7f4b74074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:15.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.880+0000 7f4b79372700 1 -- 172.21.15.64:0/1813670421 shutdown_connections 2024-02-16T22:56:15.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.880+0000 7f4b79372700 1 -- 172.21.15.64:0/1813670421 wait complete. 2024-02-16T22:56:15.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.881+0000 7f4b79372700 1 Processor -- start 2024-02-16T22:56:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.881+0000 7f4b79372700 1 -- start start 2024-02-16T22:56:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b79372700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b74103540 0x7f4b74113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b79372700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 0x7f4b74114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b79372700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74114a30 con 0x7f4b74104750 2024-02-16T22:56:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b79372700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741152d0 con 0x7f4b74103540 2024-02-16T22:56:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 0x7f4b74114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b74103540 0x7f4b74113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 0x7f4b74114430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43284/0 (socket says 172.21.15.64:43284) 2024-02-16T22:56:15.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b727fc700 1 -- 172.21.15.64:0/2704225952 learned_addr learned my addr 172.21.15.64:0/2704225952 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:15.885 INFO:tasks.workunit.client.1.smithi192.stdout:7/388: link d1/d1f/f82 d1/d2/de/d2c/f8b 0 2024-02-16T22:56:15.885 INFO:tasks.workunit.client.1.smithi192.stdout:5/331: dread d6/f29 [0,4194304] 0 2024-02-16T22:56:15.885 INFO:tasks.workunit.client.1.smithi192.stdout:5/332: stat d6/d8/f3e 0 2024-02-16T22:56:15.885 INFO:tasks.workunit.client.1.smithi192.stdout:5/333: stat d6/d12/d4e/d46/f5d 0 2024-02-16T22:56:15.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b727fc700 1 -- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b74103540 msgr2=0x7f4b74113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b727fc700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b74103540 0x7f4b74113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b727fc700 1 -- 172.21.15.64:0/2704225952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b64005040 con 0x7f4b74104750 2024-02-16T22:56:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.882+0000 7f4b727fc700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 0x7f4b74114430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4b640093b0 tx=0x7f4b64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.883+0000 7f4b5b7fe700 1 -- 172.21.15.64:0/2704225952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b6400e040 con 0x7f4b74104750 2024-02-16T22:56:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.883+0000 7f4b5b7fe700 1 -- 172.21.15.64:0/2704225952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b6400bcc0 con 0x7f4b74104750 2024-02-16T22:56:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.883+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b74115550 con 0x7f4b74104750 2024-02-16T22:56:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.883+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b74115a90 con 0x7f4b74104750 2024-02-16T22:56:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.883+0000 7f4b5b7fe700 1 -- 172.21.15.64:0/2704225952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b6401c630 con 0x7f4b74104750 2024-02-16T22:56:15.886 INFO:tasks.workunit.client.0.smithi064.stdout:9/241: getdents d3/d11 0 2024-02-16T22:56:15.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.884+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b600052e0 con 0x7f4b74104750 2024-02-16T22:56:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.885+0000 7f4b5b7fe700 1 -- 172.21.15.64:0/2704225952 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f4b6401c8a0 con 0x7f4b74104750 2024-02-16T22:56:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.885+0000 7f4b5b7fe700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f4b5c06b2e0 0x7f4b5c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.886+0000 7f4b5b7fe700 1 -- 172.21.15.64:0/2704225952 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f4b6408b110 con 0x7f4b74104750 2024-02-16T22:56:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.886+0000 7f4b72ffd700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f4b5c06b2e0 0x7f4b5c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:15.888 INFO:tasks.workunit.client.0.smithi064.stdout:0/199: symlink da/dc/de/d12/d2c/d2e/l39 0 2024-02-16T22:56:15.888 INFO:tasks.workunit.client.0.smithi064.stdout:0/200: chown da/f1b 516438 1 2024-02-16T22:56:15.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.887+0000 7f4b72ffd700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f4b5c06b2e0 0x7f4b5c06d790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009fd0 tx=0x7f4b6800b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:15.889 INFO:tasks.workunit.client.1.smithi192.stdout:0/308: write d1/f51 [389647,10253] 0 2024-02-16T22:56:15.892 INFO:tasks.workunit.client.1.smithi192.stdout:2/360: mkdir d22/d24/d39/d42/d52/d7c 0 2024-02-16T22:56:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:15.892+0000 7f4b5b7fe700 1 -- 172.21.15.64:0/2704225952 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f4b64058d10 con 0x7f4b74104750 2024-02-16T22:56:15.894 INFO:tasks.workunit.client.0.smithi064.stdout:4/250: symlink dc/d1d/l49 0 2024-02-16T22:56:15.896 INFO:tasks.workunit.client.0.smithi064.stdout:0/201: dwrite da/dc/de/d12/d2c/d2e/f36 [0,4194304] 0 2024-02-16T22:56:15.902 INFO:tasks.workunit.client.0.smithi064.stdout:5/188: mkdir d6/d1c/d43 0 2024-02-16T22:56:15.904 INFO:tasks.workunit.client.0.smithi064.stdout:0/202: dwrite da/dc/de/d12/d1d/f2b [0,4194304] 0 2024-02-16T22:56:15.904 INFO:tasks.workunit.client.1.smithi192.stdout:1/304: symlink d3/l60 0 2024-02-16T22:56:15.908 INFO:tasks.workunit.client.0.smithi064.stdout:9/242: dread d3/d9/f3c [0,4194304] 0 2024-02-16T22:56:15.909 INFO:tasks.workunit.client.0.smithi064.stdout:9/243: chown d3/d9/c2c 3713439 1 2024-02-16T22:56:15.911 INFO:tasks.workunit.client.1.smithi192.stdout:4/345: symlink d1/d15/d29/d3c/l80 0 2024-02-16T22:56:15.913 INFO:tasks.workunit.client.1.smithi192.stdout:4/346: fdatasync d1/d8/d11/d58/f5d 0 2024-02-16T22:56:15.914 INFO:tasks.workunit.client.0.smithi064.stdout:1/230: chown d2/df/d11/d25/c31 1498125 1 2024-02-16T22:56:15.915 INFO:tasks.workunit.client.1.smithi192.stdout:8/318: getdents d5/d3e 0 2024-02-16T22:56:15.915 INFO:tasks.workunit.client.0.smithi064.stdout:8/231: write df/d31/f29 [782005,83853] 0 2024-02-16T22:56:15.916 INFO:tasks.workunit.client.0.smithi064.stdout:3/291: creat d4/d13/f62 x:0 0 0 2024-02-16T22:56:15.917 INFO:tasks.workunit.client.1.smithi192.stdout:5/334: mknod d6/d12/d4e/d46/d4f/c68 0 2024-02-16T22:56:15.919 INFO:tasks.workunit.client.1.smithi192.stdout:0/309: mknod d1/d3/de/d22/c5c 0 2024-02-16T22:56:15.920 INFO:tasks.workunit.client.1.smithi192.stdout:0/310: fdatasync d1/f51 0 2024-02-16T22:56:15.920 INFO:tasks.workunit.client.1.smithi192.stdout:0/311: readlink d1/l23 0 2024-02-16T22:56:15.923 INFO:tasks.workunit.client.0.smithi064.stdout:7/221: dwrite d5/f27 [0,4194304] 0 2024-02-16T22:56:15.927 INFO:tasks.workunit.client.0.smithi064.stdout:7/222: chown d5/d33/d35/d41 2601 1 2024-02-16T22:56:15.928 INFO:tasks.workunit.client.0.smithi064.stdout:7/223: read d5/d6/f3b [7851059,68804] 0 2024-02-16T22:56:15.928 INFO:tasks.workunit.client.1.smithi192.stdout:0/312: dwrite d1/f3e [0,4194304] 0 2024-02-16T22:56:15.928 INFO:tasks.workunit.client.1.smithi192.stdout:1/305: rename d3/de/c15 to d3/de/d27/c61 0 2024-02-16T22:56:15.931 INFO:tasks.workunit.client.1.smithi192.stdout:9/416: creat d7/df/d6e/f91 x:0 0 0 2024-02-16T22:56:15.932 INFO:tasks.workunit.client.1.smithi192.stdout:0/313: dwrite d1/fa [0,4194304] 0 2024-02-16T22:56:15.933 INFO:tasks.workunit.client.1.smithi192.stdout:7/389: creat d1/d2/d37/d86/f8c x:0 0 0 2024-02-16T22:56:15.935 INFO:tasks.workunit.client.1.smithi192.stdout:0/314: write d1/d3/de/d22/f45 [1017156,88681] 0 2024-02-16T22:56:15.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/390: dwrite d1/d2/f1e [0,4194304] 0 2024-02-16T22:56:15.941 INFO:tasks.workunit.client.0.smithi064.stdout:6/293: unlink d0/d6/d10/f3e 0 2024-02-16T22:56:15.943 INFO:tasks.workunit.client.1.smithi192.stdout:8/319: rmdir d5/db/d47 39 2024-02-16T22:56:15.949 INFO:tasks.workunit.client.0.smithi064.stdout:2/240: mknod d7/d14/c48 0 2024-02-16T22:56:15.951 INFO:tasks.workunit.client.1.smithi192.stdout:2/361: creat d22/d47/f7d x:0 0 0 2024-02-16T22:56:15.952 INFO:tasks.workunit.client.1.smithi192.stdout:2/362: write d22/d47/f7d [377372,48991] 0 2024-02-16T22:56:15.955 INFO:tasks.workunit.client.0.smithi064.stdout:4/251: mkdir dc/d4a 0 2024-02-16T22:56:15.956 INFO:tasks.workunit.client.0.smithi064.stdout:6/294: dread d0/d6/f3b [0,4194304] 0 2024-02-16T22:56:15.956 INFO:tasks.workunit.client.1.smithi192.stdout:1/306: mknod d3/de/d27/d34/c62 0 2024-02-16T22:56:15.957 INFO:tasks.workunit.client.1.smithi192.stdout:2/363: dwrite f21 [0,4194304] 0 2024-02-16T22:56:15.957 INFO:tasks.workunit.client.0.smithi064.stdout:6/295: write d0/d5/f27 [557016,64935] 0 2024-02-16T22:56:15.958 INFO:tasks.workunit.client.1.smithi192.stdout:2/364: chown d22/d24/d39/d42/d65 64168470 1 2024-02-16T22:56:15.958 INFO:tasks.workunit.client.0.smithi064.stdout:7/224: mkdir d5/d33/d42 0 2024-02-16T22:56:15.959 INFO:tasks.workunit.client.1.smithi192.stdout:2/365: chown d22/d24/d39/f4c 24272 1 2024-02-16T22:56:15.960 INFO:tasks.workunit.client.1.smithi192.stdout:2/366: read - d22/d24/d2b/f38 zero size 2024-02-16T22:56:15.961 INFO:tasks.workunit.client.1.smithi192.stdout:9/417: mkdir d7/df/d6e/d92 0 2024-02-16T22:56:15.966 INFO:tasks.workunit.client.1.smithi192.stdout:4/347: mknod d1/c81 0 2024-02-16T22:56:15.967 INFO:tasks.workunit.client.1.smithi192.stdout:4/348: chown d1/l27 35 1 2024-02-16T22:56:15.970 INFO:tasks.workunit.client.1.smithi192.stdout:7/391: rename d1/d2/de/d2c/d49/d54/f7a to d1/d2/de/d83/f8d 0 2024-02-16T22:56:15.971 INFO:tasks.workunit.client.0.smithi064.stdout:3/292: mkdir d4/d13/d1b/d27/d61/d63 0 2024-02-16T22:56:15.972 INFO:tasks.workunit.client.0.smithi064.stdout:3/293: write d4/d13/d1b/d4f/f5d [800826,47090] 0 2024-02-16T22:56:15.979 INFO:tasks.workunit.client.1.smithi192.stdout:1/307: mkdir d3/de/d27/d34/d52/d63 0 2024-02-16T22:56:15.980 INFO:tasks.workunit.client.0.smithi064.stdout:3/294: dwrite d4/f17 [4194304,4194304] 0 2024-02-16T22:56:15.983 INFO:tasks.workunit.client.1.smithi192.stdout:9/418: symlink d7/d19/d5e/l93 0 2024-02-16T22:56:15.985 INFO:tasks.workunit.client.0.smithi064.stdout:0/203: link da/c24 da/dc/c3a 0 2024-02-16T22:56:15.987 INFO:tasks.workunit.client.0.smithi064.stdout:3/295: dwrite d4/d13/d1b/d4f/f5d [0,4194304] 0 2024-02-16T22:56:15.988 INFO:tasks.workunit.client.0.smithi064.stdout:0/204: fdatasync da/dc/de/d12/d2c/d2e/f36 0 2024-02-16T22:56:15.991 INFO:tasks.workunit.client.0.smithi064.stdout:8/232: creat df/f4c x:0 0 0 2024-02-16T22:56:15.994 INFO:tasks.workunit.client.0.smithi064.stdout:2/241: creat d7/d14/d25/d37/d44/f49 x:0 0 0 2024-02-16T22:56:15.996 INFO:tasks.workunit.client.1.smithi192.stdout:4/349: rename d1/d15/d29/c49 to d1/d15/c82 0 2024-02-16T22:56:16.001 INFO:tasks.workunit.client.1.smithi192.stdout:5/335: symlink d6/d12/d1d/d3c/l69 0 2024-02-16T22:56:16.002 INFO:tasks.workunit.client.1.smithi192.stdout:5/336: write d6/d12/d4e/d5b/f5e [590244,99812] 0 2024-02-16T22:56:16.003 INFO:tasks.workunit.client.0.smithi064.stdout:7/225: dread d5/f32 [0,4194304] 0 2024-02-16T22:56:16.004 INFO:tasks.workunit.client.1.smithi192.stdout:1/308: mknod d3/de/d27/d34/d52/c64 0 2024-02-16T22:56:16.006 INFO:tasks.workunit.client.0.smithi064.stdout:9/244: getdents d3/d11/d2b/d2a 0 2024-02-16T22:56:16.007 INFO:tasks.workunit.client.1.smithi192.stdout:2/367: write d22/f2a [879220,60937] 0 2024-02-16T22:56:16.012 INFO:tasks.workunit.client.0.smithi064.stdout:0/205: mkdir da/dc/d3b 0 2024-02-16T22:56:16.013 INFO:tasks.workunit.client.1.smithi192.stdout:7/392: rename d1/d19/c5d to d1/d19/d4c/c8e 0 2024-02-16T22:56:16.016 INFO:tasks.workunit.client.0.smithi064.stdout:3/296: rename d4/le to d4/d13/d1b/d27/d61/d63/l64 0 2024-02-16T22:56:16.017 INFO:tasks.workunit.client.0.smithi064.stdout:2/242: dread d7/f12 [0,4194304] 0 2024-02-16T22:56:16.018 INFO:tasks.workunit.client.0.smithi064.stdout:6/296: dwrite d0/d5/f3d [0,4194304] 0 2024-02-16T22:56:16.021 INFO:tasks.workunit.client.0.smithi064.stdout:3/297: truncate d4/d1f/d34/f45 1014771 0 2024-02-16T22:56:16.024 INFO:tasks.workunit.client.0.smithi064.stdout:9/245: dread d3/f18 [0,4194304] 0 2024-02-16T22:56:16.031 INFO:tasks.workunit.client.1.smithi192.stdout:0/315: getdents d1/d3/de/d22 0 2024-02-16T22:56:16.032 INFO:tasks.workunit.client.1.smithi192.stdout:4/350: rename d1/d15/d29/d3c/l80 to d1/d26/d66/l83 0 2024-02-16T22:56:16.033 INFO:tasks.workunit.client.1.smithi192.stdout:7/393: creat d1/d2/de/f8f x:0 0 0 2024-02-16T22:56:16.034 INFO:tasks.workunit.client.0.smithi064.stdout:8/233: mkdir df/d2a/d4d 0 2024-02-16T22:56:16.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:15 smithi192 ceph-mon[36254]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:15 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:15 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:15 smithi192 ceph-mon[36254]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:16.035 INFO:tasks.workunit.client.1.smithi192.stdout:8/320: getdents d5/d3a/d4d 0 2024-02-16T22:56:16.036 INFO:tasks.workunit.client.1.smithi192.stdout:8/321: write d5/d3e/d5f/f4e [4740849,118278] 0 2024-02-16T22:56:16.038 INFO:tasks.workunit.client.1.smithi192.stdout:4/351: dwrite d1/f35 [4194304,4194304] 0 2024-02-16T22:56:16.041 INFO:tasks.workunit.client.1.smithi192.stdout:4/352: read d1/d15/d48/f55 [2692582,32790] 0 2024-02-16T22:56:16.041 INFO:tasks.workunit.client.0.smithi064.stdout:3/298: unlink d4/d13/d1b/d27/d44/f5e 0 2024-02-16T22:56:16.042 INFO:tasks.workunit.client.1.smithi192.stdout:4/353: write d1/d26/f77 [468815,57345] 0 2024-02-16T22:56:16.044 INFO:tasks.workunit.client.0.smithi064.stdout:4/252: getdents dc/d1d 0 2024-02-16T22:56:16.044 INFO:tasks.workunit.client.1.smithi192.stdout:8/322: dwrite d5/db/de/f5d [4194304,4194304] 0 2024-02-16T22:56:16.046 INFO:tasks.workunit.client.1.smithi192.stdout:8/323: write d5/d9/f54 [2485370,77226] 0 2024-02-16T22:56:16.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/394: rename d1/d19 to d1/d2/d2d/d61/d88/d90 0 2024-02-16T22:56:16.055 INFO:tasks.workunit.client.0.smithi064.stdout:9/246: unlink d3/d11/d2b/d2a/d2e/d31/c34 0 2024-02-16T22:56:16.056 INFO:tasks.workunit.client.0.smithi064.stdout:0/206: creat da/dc/d3b/f3c x:0 0 0 2024-02-16T22:56:16.057 INFO:tasks.workunit.client.1.smithi192.stdout:5/337: link d6/d12/d2d/l3f d6/d12/d4e/d58/l6a 0 2024-02-16T22:56:16.058 INFO:tasks.workunit.client.0.smithi064.stdout:0/207: dread - da/dc/de/d12/d1d/f2f zero size 2024-02-16T22:56:16.062 INFO:tasks.workunit.client.1.smithi192.stdout:9/419: getdents d7/d19/d37 0 2024-02-16T22:56:16.063 INFO:tasks.workunit.client.1.smithi192.stdout:9/420: write d7/d19/d37/d5c/f87 [131061,63409] 0 2024-02-16T22:56:16.065 INFO:tasks.workunit.client.1.smithi192.stdout:2/368: truncate d22/d24/f49 4099059 0 2024-02-16T22:56:16.065 INFO:tasks.workunit.client.1.smithi192.stdout:2/369: chown d22/d47/f7d 5 1 2024-02-16T22:56:16.066 INFO:tasks.workunit.client.0.smithi064.stdout:4/253: mkdir dc/dd/d32/d4b 0 2024-02-16T22:56:16.069 INFO:tasks.workunit.client.0.smithi064.stdout:7/226: creat d5/d6/f43 x:0 0 0 2024-02-16T22:56:16.071 INFO:tasks.workunit.client.0.smithi064.stdout:9/247: unlink d3/d11/d2b/d2a/d2e/d31/d3f/l4f 0 2024-02-16T22:56:16.072 INFO:tasks.workunit.client.1.smithi192.stdout:4/354: fsync d1/d6b/f7f 0 2024-02-16T22:56:16.075 INFO:tasks.workunit.client.1.smithi192.stdout:4/355: dread d1/d26/d5e/f63 [0,4194304] 0 2024-02-16T22:56:16.077 INFO:tasks.workunit.client.0.smithi064.stdout:6/297: dwrite d0/d1d/d21/f2d [0,4194304] 0 2024-02-16T22:56:16.078 INFO:tasks.workunit.client.1.smithi192.stdout:4/356: dread d1/d15/d23/f5c [4194304,4194304] 0 2024-02-16T22:56:16.078 INFO:tasks.workunit.client.0.smithi064.stdout:0/208: creat da/dc/de/d12/d1d/f3d x:0 0 0 2024-02-16T22:56:16.078 INFO:tasks.workunit.client.0.smithi064.stdout:6/298: write d0/d6/d10/d33/f32 [103014,99921] 0 2024-02-16T22:56:16.079 INFO:tasks.workunit.client.0.smithi064.stdout:0/209: fsync da/f18 0 2024-02-16T22:56:16.081 INFO:tasks.workunit.client.1.smithi192.stdout:8/324: creat d5/f61 x:0 0 0 2024-02-16T22:56:16.082 INFO:tasks.workunit.client.0.smithi064.stdout:2/243: rmdir d7/d14/d1e/d38 0 2024-02-16T22:56:16.083 INFO:tasks.workunit.client.0.smithi064.stdout:2/244: chown d7/d14/c40 101 1 2024-02-16T22:56:16.084 INFO:tasks.workunit.client.0.smithi064.stdout:2/245: chown d7/d14/l26 872938 1 2024-02-16T22:56:16.086 INFO:tasks.workunit.client.0.smithi064.stdout:3/299: unlink d4/d13/c19 0 2024-02-16T22:56:16.088 INFO:tasks.workunit.client.0.smithi064.stdout:4/254: rmdir dc 39 2024-02-16T22:56:16.089 INFO:tasks.workunit.client.1.smithi192.stdout:1/309: getdents d3/de 0 2024-02-16T22:56:16.090 INFO:tasks.workunit.client.1.smithi192.stdout:3/345: sync 2024-02-16T22:56:16.091 INFO:tasks.workunit.client.1.smithi192.stdout:5/338: creat d6/d12/d4e/d46/f6b x:0 0 0 2024-02-16T22:56:16.091 INFO:tasks.workunit.client.0.smithi064.stdout:9/248: write d3/d11/f28 [1462414,10123] 0 2024-02-16T22:56:16.092 INFO:tasks.workunit.client.0.smithi064.stdout:7/227: dread d5/d6/fb [0,4194304] 0 2024-02-16T22:56:16.092 INFO:tasks.workunit.client.1.smithi192.stdout:9/421: creat d7/d19/f94 x:0 0 0 2024-02-16T22:56:16.095 INFO:tasks.workunit.client.1.smithi192.stdout:2/370: write d22/d24/d39/d42/d52/f71 [2280145,89732] 0 2024-02-16T22:56:16.097 INFO:tasks.workunit.client.1.smithi192.stdout:2/371: stat d22/d24/c2f 0 2024-02-16T22:56:16.098 INFO:tasks.workunit.client.0.smithi064.stdout:6/299: symlink d0/d5/df/d17/d18/l52 0 2024-02-16T22:56:16.099 INFO:tasks.workunit.client.1.smithi192.stdout:0/316: creat d1/f5d x:0 0 0 2024-02-16T22:56:16.100 INFO:tasks.workunit.client.1.smithi192.stdout:3/346: dwrite da/d11/f3f [0,4194304] 0 2024-02-16T22:56:16.100 INFO:tasks.workunit.client.1.smithi192.stdout:1/310: dwrite d3/de/d27/d34/d52/f54 [4194304,4194304] 0 2024-02-16T22:56:16.101 INFO:tasks.workunit.client.1.smithi192.stdout:4/357: symlink d1/d15/d23/l84 0 2024-02-16T22:56:16.102 INFO:tasks.workunit.client.0.smithi064.stdout:2/246: creat d7/d14/d25/f4a x:0 0 0 2024-02-16T22:56:16.103 INFO:tasks.workunit.client.1.smithi192.stdout:0/317: stat d1/d3/f4c 0 2024-02-16T22:56:16.104 INFO:tasks.workunit.client.1.smithi192.stdout:2/372: write d22/d24/d39/d42/d65/d69/f68 [1508573,114431] 0 2024-02-16T22:56:16.105 INFO:tasks.workunit.client.1.smithi192.stdout:8/325: creat d5/db/de/df/f62 x:0 0 0 2024-02-16T22:56:16.106 INFO:tasks.workunit.client.0.smithi064.stdout:3/300: symlink d4/d13/d4d/l65 0 2024-02-16T22:56:16.106 INFO:tasks.workunit.client.1.smithi192.stdout:8/326: fsync d5/d16/f4f 0 2024-02-16T22:56:16.106 INFO:tasks.workunit.client.1.smithi192.stdout:0/318: chown d1/d3/de/d22/f28 332938565 1 2024-02-16T22:56:16.107 INFO:tasks.workunit.client.1.smithi192.stdout:9/422: dwrite d7/d19/d5e/d6f/f7a [0,4194304] 0 2024-02-16T22:56:16.109 INFO:tasks.workunit.client.0.smithi064.stdout:4/255: fdatasync dc/dd/d16/f23 0 2024-02-16T22:56:16.111 INFO:tasks.workunit.client.0.smithi064.stdout:8/234: getdents df/d31 0 2024-02-16T22:56:16.113 INFO:tasks.workunit.client.1.smithi192.stdout:0/319: dwrite d1/d3/d29/f27 [0,4194304] 0 2024-02-16T22:56:16.114 INFO:tasks.workunit.client.0.smithi064.stdout:9/249: creat d3/d9/d1e/d42/f58 x:0 0 0 2024-02-16T22:56:16.116 INFO:tasks.workunit.client.0.smithi064.stdout:6/300: symlink d0/d12/l53 0 2024-02-16T22:56:16.117 INFO:tasks.workunit.client.0.smithi064.stdout:6/301: write d0/d1d/d21/f2d [1900736,1673] 0 2024-02-16T22:56:16.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:15 smithi064 ceph-mon[29226]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:16.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:15 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:16.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:15 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:16.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:15 smithi064 ceph-mon[29226]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:16.119 INFO:tasks.workunit.client.0.smithi064.stdout:0/210: link f9 da/dc/de/d12/d2c/d2e/f3e 0 2024-02-16T22:56:16.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/423: dread f0 [0,4194304] 0 2024-02-16T22:56:16.131 INFO:tasks.workunit.client.1.smithi192.stdout:1/311: creat d3/de/f65 x:0 0 0 2024-02-16T22:56:16.134 INFO:tasks.workunit.client.1.smithi192.stdout:2/373: read d22/d24/d2b/d2c/f43 [311764,119223] 0 2024-02-16T22:56:16.138 INFO:tasks.workunit.client.1.smithi192.stdout:5/339: rename d6/d12/d1d/d3c to d6/d12/d4e/d58/d6c 0 2024-02-16T22:56:16.139 INFO:tasks.workunit.client.1.smithi192.stdout:3/347: fdatasync da/d11/f54 0 2024-02-16T22:56:16.142 INFO:tasks.workunit.client.1.smithi192.stdout:8/327: mknod d5/db/c63 0 2024-02-16T22:56:16.143 INFO:tasks.workunit.client.1.smithi192.stdout:7/395: getdents d1/d1f 0 2024-02-16T22:56:16.143 INFO:tasks.workunit.client.1.smithi192.stdout:0/320: mkdir d1/d3/de/d22/d37/d47/d53/d5e 0 2024-02-16T22:56:16.144 INFO:tasks.workunit.client.1.smithi192.stdout:4/358: creat d1/f85 x:0 0 0 2024-02-16T22:56:16.145 INFO:tasks.workunit.client.1.smithi192.stdout:5/340: dwrite d6/d12/f1e [0,4194304] 0 2024-02-16T22:56:16.146 INFO:tasks.workunit.client.1.smithi192.stdout:9/424: creat d7/d19/d5e/d81/f95 x:0 0 0 2024-02-16T22:56:16.148 INFO:tasks.workunit.client.1.smithi192.stdout:2/374: link d22/d24/d39/d42/l77 d22/d24/d2b/d2c/l7e 0 2024-02-16T22:56:16.149 INFO:tasks.workunit.client.1.smithi192.stdout:4/359: truncate d1/d8/f1e 862274 0 2024-02-16T22:56:16.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/348: dwrite da/df/f79 [0,4194304] 0 2024-02-16T22:56:16.149 INFO:tasks.workunit.client.1.smithi192.stdout:5/341: symlink d6/d12/d4e/l6d 0 2024-02-16T22:56:16.149 INFO:tasks.workunit.client.1.smithi192.stdout:0/321: dread d1/f15 [0,4194304] 0 2024-02-16T22:56:16.150 INFO:tasks.workunit.client.1.smithi192.stdout:9/425: dread - d7/d19/d28/d43/d70/f8d zero size 2024-02-16T22:56:16.153 INFO:tasks.workunit.client.1.smithi192.stdout:0/322: fsync d1/d2/f42 0 2024-02-16T22:56:16.154 INFO:tasks.workunit.client.1.smithi192.stdout:2/375: symlink d22/d24/d2b/d2c/d3f/l7f 0 2024-02-16T22:56:16.155 INFO:tasks.workunit.client.1.smithi192.stdout:5/342: creat d6/d12/d4e/d46/d4f/f6e x:0 0 0 2024-02-16T22:56:16.156 INFO:tasks.workunit.client.0.smithi064.stdout:9/250: creat d3/d11/d2b/d2a/d2e/f59 x:0 0 0 2024-02-16T22:56:16.158 INFO:tasks.workunit.client.1.smithi192.stdout:9/426: creat d7/d15/d36/d54/f96 x:0 0 0 2024-02-16T22:56:16.159 INFO:tasks.workunit.client.0.smithi064.stdout:9/251: write d3/fc [610546,40028] 0 2024-02-16T22:56:16.159 INFO:tasks.workunit.client.1.smithi192.stdout:7/396: dwrite d1/d2/d2d/d61/d88/d90/f1d [0,4194304] 0 2024-02-16T22:56:16.162 INFO:tasks.workunit.client.0.smithi064.stdout:0/211: write f9 [507898,6651] 0 2024-02-16T22:56:16.164 INFO:tasks.workunit.client.0.smithi064.stdout:9/252: fdatasync d3/d11/f28 0 2024-02-16T22:56:16.164 INFO:tasks.workunit.client.0.smithi064.stdout:3/301: dread d4/d13/f26 [0,4194304] 0 2024-02-16T22:56:16.165 INFO:tasks.workunit.client.0.smithi064.stdout:6/302: dwrite d0/d6/d10/d33/f2a [0,4194304] 0 2024-02-16T22:56:16.167 INFO:tasks.workunit.client.1.smithi192.stdout:3/349: dwrite da/d11/d63/d6b/d66/f67 [0,4194304] 0 2024-02-16T22:56:16.167 INFO:tasks.workunit.client.0.smithi064.stdout:3/302: chown d4/l7 1472190011 1 2024-02-16T22:56:16.167 INFO:tasks.workunit.client.0.smithi064.stdout:9/253: chown d3/d11/d2b/f52 18360812 1 2024-02-16T22:56:16.168 INFO:tasks.workunit.client.0.smithi064.stdout:8/235: symlink df/d31/d47/l4e 0 2024-02-16T22:56:16.169 INFO:tasks.workunit.client.0.smithi064.stdout:2/247: mkdir d7/d14/d4b 0 2024-02-16T22:56:16.169 INFO:tasks.workunit.client.1.smithi192.stdout:2/376: dwrite f19 [4194304,4194304] 0 2024-02-16T22:56:16.170 INFO:tasks.workunit.client.0.smithi064.stdout:8/236: chown f1 1634 1 2024-02-16T22:56:16.172 INFO:tasks.workunit.client.0.smithi064.stdout:0/212: mkdir da/d26/d3f 0 2024-02-16T22:56:16.173 INFO:tasks.workunit.client.1.smithi192.stdout:0/323: creat d1/d3/d39/f5f x:0 0 0 2024-02-16T22:56:16.173 INFO:tasks.workunit.client.1.smithi192.stdout:0/324: write d1/d3/de/f4e [937430,28735] 0 2024-02-16T22:56:16.176 INFO:tasks.workunit.client.0.smithi064.stdout:8/237: chown df/f3d 407602892 1 2024-02-16T22:56:16.178 INFO:tasks.workunit.client.0.smithi064.stdout:6/303: dwrite d0/f41 [0,4194304] 0 2024-02-16T22:56:16.181 INFO:tasks.workunit.client.1.smithi192.stdout:9/427: mknod d7/d19/d28/d43/d70/c97 0 2024-02-16T22:56:16.181 INFO:tasks.workunit.client.1.smithi192.stdout:2/377: chown d22/d24/d39/d42/f5f 3453 1 2024-02-16T22:56:16.181 INFO:tasks.workunit.client.1.smithi192.stdout:9/428: stat d7/d19/d28/d43/d70/c97 0 2024-02-16T22:56:16.181 INFO:tasks.workunit.client.1.smithi192.stdout:0/325: chown d1/d3/d29/f3f 625887561 1 2024-02-16T22:56:16.181 INFO:tasks.workunit.client.1.smithi192.stdout:7/397: mknod d1/d2/d2d/d61/d88/d90/d4c/c91 0 2024-02-16T22:56:16.182 INFO:tasks.workunit.client.1.smithi192.stdout:3/350: unlink da/d11/d35/l45 0 2024-02-16T22:56:16.185 INFO:tasks.workunit.client.1.smithi192.stdout:3/351: write da/d11/f2f [1669354,87637] 0 2024-02-16T22:56:16.188 INFO:tasks.workunit.client.0.smithi064.stdout:0/213: dwrite da/dc/de/d12/d1d/f2f [0,4194304] 0 2024-02-16T22:56:16.188 INFO:tasks.workunit.client.1.smithi192.stdout:5/343: dread d6/d12/d1d/f36 [0,4194304] 0 2024-02-16T22:56:16.191 INFO:tasks.workunit.client.0.smithi064.stdout:0/214: write da/dc/d3b/f3c [975028,104604] 0 2024-02-16T22:56:16.192 INFO:tasks.workunit.client.0.smithi064.stdout:0/215: dread - da/dc/de/d12/d2c/f31 zero size 2024-02-16T22:56:16.195 INFO:tasks.workunit.client.1.smithi192.stdout:4/360: dread d1/f5 [0,4194304] 0 2024-02-16T22:56:16.196 INFO:tasks.workunit.client.1.smithi192.stdout:4/361: write d1/d26/d34/f60 [908786,108812] 0 2024-02-16T22:56:16.200 INFO:tasks.workunit.client.0.smithi064.stdout:0/216: write da/dc/de/d12/d1d/f2b [4473350,89013] 0 2024-02-16T22:56:16.208 INFO:tasks.workunit.client.1.smithi192.stdout:9/429: rename d7/df to d7/d15/d36/d38/d3f/d8a/d98 0 2024-02-16T22:56:16.210 INFO:tasks.workunit.client.1.smithi192.stdout:9/430: truncate d7/d19/d28/d43/d70/f8d 798389 0 2024-02-16T22:56:16.213 INFO:tasks.workunit.client.1.smithi192.stdout:3/352: symlink da/d11/d35/l7e 0 2024-02-16T22:56:16.213 INFO:tasks.workunit.client.1.smithi192.stdout:3/353: chown da/d11/d2b/c37 228847 1 2024-02-16T22:56:16.214 INFO:tasks.workunit.client.1.smithi192.stdout:3/354: chown da/d11/d63/d6b 0 1 2024-02-16T22:56:16.217 INFO:tasks.workunit.client.0.smithi064.stdout:9/254: rename d3/d11/d2b to d3/d11/d15/d5a 0 2024-02-16T22:56:16.222 INFO:tasks.workunit.client.1.smithi192.stdout:2/378: link d22/d24/d2b/f64 d22/d24/d39/d3d/f80 0 2024-02-16T22:56:16.223 INFO:tasks.workunit.client.1.smithi192.stdout:7/398: mkdir d1/d2/d2d/d92 0 2024-02-16T22:56:16.224 INFO:tasks.workunit.client.1.smithi192.stdout:6/320: sync 2024-02-16T22:56:16.225 INFO:tasks.workunit.client.1.smithi192.stdout:6/321: write da/df/f5e [455760,93389] 0 2024-02-16T22:56:16.226 INFO:tasks.workunit.client.0.smithi064.stdout:6/304: rename d0/d5/df/d17/d18/d4f to d0/d5/d54 0 2024-02-16T22:56:16.226 INFO:tasks.workunit.client.1.smithi192.stdout:6/322: fdatasync da/f5f 0 2024-02-16T22:56:16.228 INFO:tasks.workunit.client.0.smithi064.stdout:6/305: chown d0/d6/f15 13373 1 2024-02-16T22:56:16.228 INFO:tasks.workunit.client.1.smithi192.stdout:2/379: dwrite d22/d24/d39/d42/d65/d69/f7a [0,4194304] 0 2024-02-16T22:56:16.228 INFO:tasks.workunit.client.1.smithi192.stdout:0/326: rename d1/d3/f4c to d1/d3/de/d22/f60 0 2024-02-16T22:56:16.228 INFO:tasks.workunit.client.0.smithi064.stdout:0/217: dwrite da/f1b [4194304,4194304] 0 2024-02-16T22:56:16.229 INFO:tasks.workunit.client.1.smithi192.stdout:2/380: write d22/d24/d2b/d2c/f2e [73313,61599] 0 2024-02-16T22:56:16.230 INFO:tasks.workunit.client.1.smithi192.stdout:9/431: creat d7/d15/d36/f99 x:0 0 0 2024-02-16T22:56:16.231 INFO:tasks.workunit.client.1.smithi192.stdout:2/381: chown d22/l32 596 1 2024-02-16T22:56:16.233 INFO:tasks.workunit.client.1.smithi192.stdout:3/355: fdatasync da/d11/d63/d6b/d66/f7a 0 2024-02-16T22:56:16.237 INFO:tasks.workunit.client.1.smithi192.stdout:4/362: creat d1/d26/d6f/f86 x:0 0 0 2024-02-16T22:56:16.242 INFO:tasks.workunit.client.0.smithi064.stdout:0/218: rename l2 to da/dc/de/d12/d2c/d2e/l40 0 2024-02-16T22:56:16.247 INFO:tasks.workunit.client.1.smithi192.stdout:6/323: dread da/df/d12/d15/f5c [0,4194304] 0 2024-02-16T22:56:16.249 INFO:tasks.workunit.client.0.smithi064.stdout:0/219: symlink da/d26/d3f/l41 0 2024-02-16T22:56:16.257 INFO:tasks.workunit.client.1.smithi192.stdout:9/432: dread f1 [0,4194304] 0 2024-02-16T22:56:16.257 INFO:tasks.workunit.client.1.smithi192.stdout:3/356: fdatasync da/f12 0 2024-02-16T22:56:16.259 INFO:tasks.workunit.client.1.smithi192.stdout:9/433: write d7/d19/d28/f53 [560989,98865] 0 2024-02-16T22:56:16.259 INFO:tasks.workunit.client.1.smithi192.stdout:5/344: creat d6/f6f x:0 0 0 2024-02-16T22:56:16.260 INFO:tasks.workunit.client.0.smithi064.stdout:0/220: creat da/dc/de/d12/f42 x:0 0 0 2024-02-16T22:56:16.264 INFO:tasks.workunit.client.1.smithi192.stdout:6/324: rmdir da/df/d18/d46 39 2024-02-16T22:56:16.266 INFO:tasks.workunit.client.1.smithi192.stdout:1/312: mknod d3/de/d27/d34/d52/d5a/c66 0 2024-02-16T22:56:16.266 INFO:tasks.workunit.client.0.smithi064.stdout:0/221: dwrite da/dc/d3b/f3c [0,4194304] 0 2024-02-16T22:56:16.267 INFO:tasks.workunit.client.1.smithi192.stdout:2/382: mknod d22/d24/d39/d42/d52/d7c/c81 0 2024-02-16T22:56:16.267 INFO:tasks.workunit.client.0.smithi064.stdout:0/222: fsync da/d26/f28 0 2024-02-16T22:56:16.268 INFO:tasks.workunit.client.1.smithi192.stdout:2/383: chown d22/d24/d2b/d2c/f43 89064 1 2024-02-16T22:56:16.269 INFO:tasks.workunit.client.1.smithi192.stdout:0/327: rmdir d1/d3/d39 39 2024-02-16T22:56:16.274 INFO:tasks.workunit.client.1.smithi192.stdout:3/357: unlink da/d11/d2b/d43/c5c 0 2024-02-16T22:56:16.275 INFO:tasks.workunit.client.1.smithi192.stdout:1/313: dwrite d3/f2d [0,4194304] 0 2024-02-16T22:56:16.275 INFO:tasks.workunit.client.0.smithi064.stdout:0/223: mknod da/dc/de/d12/d2c/c43 0 2024-02-16T22:56:16.276 INFO:tasks.workunit.client.1.smithi192.stdout:3/358: chown da/df/f1f 7465 1 2024-02-16T22:56:16.276 INFO:tasks.workunit.client.1.smithi192.stdout:9/434: dread d7/d15/f58 [0,4194304] 0 2024-02-16T22:56:16.276 INFO:tasks.workunit.client.1.smithi192.stdout:8/328: sync 2024-02-16T22:56:16.278 INFO:tasks.workunit.client.1.smithi192.stdout:3/359: stat da/d11/d63/d6b/f4d 0 2024-02-16T22:56:16.279 INFO:tasks.workunit.client.1.smithi192.stdout:8/329: write d5/d9/f44 [4289145,126721] 0 2024-02-16T22:56:16.280 INFO:tasks.workunit.client.1.smithi192.stdout:3/360: fdatasync da/f51 0 2024-02-16T22:56:16.280 INFO:tasks.workunit.client.0.smithi064.stdout:4/256: dwrite dc/d10/f14 [0,4194304] 0 2024-02-16T22:56:16.281 INFO:tasks.workunit.client.1.smithi192.stdout:3/361: fdatasync da/d11/f38 0 2024-02-16T22:56:16.281 INFO:tasks.workunit.client.1.smithi192.stdout:8/330: truncate d5/f61 830941 0 2024-02-16T22:56:16.282 INFO:tasks.workunit.client.1.smithi192.stdout:6/325: dread da/df/d12/d57/f2f [0,4194304] 0 2024-02-16T22:56:16.283 INFO:tasks.workunit.client.1.smithi192.stdout:6/326: readlink da/df/d12/l49 0 2024-02-16T22:56:16.285 INFO:tasks.workunit.client.1.smithi192.stdout:6/327: read da/df/d18/d52/f4a [140985,47363] 0 2024-02-16T22:56:16.287 INFO:tasks.workunit.client.1.smithi192.stdout:9/435: dwrite d7/d19/d5e/d6f/f7d [0,4194304] 0 2024-02-16T22:56:16.288 INFO:tasks.workunit.client.0.smithi064.stdout:0/224: rename da/dc/de/d12/d2c/f2d to da/f44 0 2024-02-16T22:56:16.293 INFO:tasks.workunit.client.0.smithi064.stdout:4/257: creat dc/d1d/f4c x:0 0 0 2024-02-16T22:56:16.296 INFO:tasks.workunit.client.0.smithi064.stdout:3/303: truncate d4/d13/d1b/d4f/f5d 110558 0 2024-02-16T22:56:16.301 INFO:tasks.workunit.client.0.smithi064.stdout:8/238: truncate df/d31/f36 495758 0 2024-02-16T22:56:16.305 INFO:tasks.workunit.client.0.smithi064.stdout:3/304: creat d4/d13/d1b/d27/d61/f66 x:0 0 0 2024-02-16T22:56:16.308 INFO:tasks.workunit.client.0.smithi064.stdout:8/239: dwrite df/f4c [0,4194304] 0 2024-02-16T22:56:16.309 INFO:tasks.workunit.client.0.smithi064.stdout:8/240: chown f9 13909 1 2024-02-16T22:56:16.312 INFO:tasks.workunit.client.0.smithi064.stdout:4/258: creat dc/d4a/f4d x:0 0 0 2024-02-16T22:56:16.316 INFO:tasks.workunit.client.0.smithi064.stdout:3/305: creat d4/d1f/f67 x:0 0 0 2024-02-16T22:56:16.317 INFO:tasks.workunit.client.1.smithi192.stdout:7/399: write d1/d2/d2d/f40 [893650,33177] 0 2024-02-16T22:56:16.318 INFO:tasks.workunit.client.1.smithi192.stdout:7/400: chown d1/d2/d2d/d61/d88/d90/d2a/f87 6046245 1 2024-02-16T22:56:16.319 INFO:tasks.workunit.client.0.smithi064.stdout:9/255: creat d3/d11/d15/d5a/d2a/d2e/d31/d3f/f5b x:0 0 0 2024-02-16T22:56:16.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/401: dwrite d1/d1f/f85 [0,4194304] 0 2024-02-16T22:56:16.323 INFO:tasks.workunit.client.0.smithi064.stdout:3/306: creat d4/d13/d1b/f68 x:0 0 0 2024-02-16T22:56:16.326 INFO:tasks.workunit.client.0.smithi064.stdout:4/259: dwrite dc/dd/f42 [4194304,4194304] 0 2024-02-16T22:56:16.329 INFO:tasks.workunit.client.0.smithi064.stdout:9/256: dread d3/d9/f36 [0,4194304] 0 2024-02-16T22:56:16.340 INFO:tasks.workunit.client.0.smithi064.stdout:3/307: truncate d4/d13/f55 1150424 0 2024-02-16T22:56:16.343 INFO:tasks.workunit.client.1.smithi192.stdout:1/314: mknod d3/d5b/c67 0 2024-02-16T22:56:16.346 INFO:tasks.workunit.client.1.smithi192.stdout:4/363: link d1/d26/d66/l83 d1/d8/d11/d58/l87 0 2024-02-16T22:56:16.347 INFO:tasks.workunit.client.1.smithi192.stdout:5/345: dwrite d6/f10 [0,4194304] 0 2024-02-16T22:56:16.348 INFO:tasks.workunit.client.0.smithi064.stdout:4/260: symlink dc/l4e 0 2024-02-16T22:56:16.349 INFO:tasks.workunit.client.0.smithi064.stdout:4/261: readlink dc/d10/l1b 0 2024-02-16T22:56:16.351 INFO:tasks.workunit.client.0.smithi064.stdout:9/257: dwrite d3/d11/d15/d5a/d2a/f4b [0,4194304] 0 2024-02-16T22:56:16.355 INFO:tasks.workunit.client.1.smithi192.stdout:7/402: creat d1/d69/f93 x:0 0 0 2024-02-16T22:56:16.361 INFO:tasks.workunit.client.0.smithi064.stdout:9/258: read d3/fc [920474,123126] 0 2024-02-16T22:56:16.362 INFO:tasks.workunit.client.1.smithi192.stdout:6/328: mknod da/df/d12/c64 0 2024-02-16T22:56:16.366 INFO:tasks.workunit.client.1.smithi192.stdout:9/436: creat d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f9a x:0 0 0 2024-02-16T22:56:16.366 INFO:tasks.workunit.client.1.smithi192.stdout:2/384: link d22/d24/d2b/d2c/d3f/l55 d22/d24/d39/d42/d52/l82 0 2024-02-16T22:56:16.370 INFO:tasks.workunit.client.1.smithi192.stdout:2/385: dread f5 [4194304,4194304] 0 2024-02-16T22:56:16.371 INFO:tasks.workunit.client.1.smithi192.stdout:8/331: dwrite d5/f1e [0,4194304] 0 2024-02-16T22:56:16.375 INFO:tasks.workunit.client.1.smithi192.stdout:4/364: rmdir d1/d26/d66 39 2024-02-16T22:56:16.378 INFO:tasks.workunit.client.1.smithi192.stdout:5/346: stat d6/d8/l31 0 2024-02-16T22:56:16.382 INFO:tasks.workunit.client.0.smithi064.stdout:0/225: link da/c24 da/d26/d3f/c45 0 2024-02-16T22:56:16.383 INFO:tasks.workunit.client.1.smithi192.stdout:6/329: mknod da/df/d12/d15/d1b/c65 0 2024-02-16T22:56:16.386 INFO:tasks.workunit.client.0.smithi064.stdout:3/308: truncate d4/d1f/d34/f45 842141 0 2024-02-16T22:56:16.387 INFO:tasks.workunit.client.1.smithi192.stdout:0/328: getdents d1/d3/de/d22 0 2024-02-16T22:56:16.388 INFO:tasks.workunit.client.0.smithi064.stdout:9/259: getdents d3/d11/d15/d5a/d2a/d2e 0 2024-02-16T22:56:16.389 INFO:tasks.workunit.client.0.smithi064.stdout:9/260: fsync d3/d11/d15/d5a/d2a/d2e/f59 0 2024-02-16T22:56:16.390 INFO:tasks.workunit.client.1.smithi192.stdout:9/437: rename d7/d19/d5e/l93 to d7/d19/d28/d43/d70/d8e/l9b 0 2024-02-16T22:56:16.390 INFO:tasks.workunit.client.1.smithi192.stdout:9/438: dread - d7/d19/d5b/f89 zero size 2024-02-16T22:56:16.391 INFO:tasks.workunit.client.0.smithi064.stdout:3/309: mknod d4/d14/c69 0 2024-02-16T22:56:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.392+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b60005860 con 0x7f4b74104750 2024-02-16T22:56:16.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.393+0000 7f4b5b7fe700 1 -- 172.21.15.64:0/2704225952 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f4b640588a0 con 0x7f4b74104750 2024-02-16T22:56:16.394 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:56:16.394 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:56:16.395 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:56:16.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:56:16.395 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:56:16.395 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T22:56:16.395 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:56:16.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:56:16.396 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:56:16.396 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:56:16.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:56:16.396 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:56:16.396 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:56:16.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:56:16.397 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:56:16.397 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 13, 2024-02-16T22:56:16.397 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:56:16.397 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:56:16.397 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:56:16.399 INFO:tasks.workunit.client.0.smithi064.stdout:9/261: dwrite d3/d9/ff [0,4194304] 0 2024-02-16T22:56:16.399 INFO:tasks.workunit.client.1.smithi192.stdout:2/386: unlink d22/l32 0 2024-02-16T22:56:16.399 INFO:tasks.workunit.client.1.smithi192.stdout:2/387: dread d22/d24/d39/d42/f54 [0,4194304] 0 2024-02-16T22:56:16.399 INFO:tasks.workunit.client.1.smithi192.stdout:4/365: fdatasync d1/f35 0 2024-02-16T22:56:16.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f4b5c06b2e0 msgr2=0x7f4b5c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:16.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f4b5c06b2e0 0x7f4b5c06d790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009fd0 tx=0x7f4b6800b040 comp rx=0 tx=0).stop 2024-02-16T22:56:16.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 msgr2=0x7f4b74114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 0x7f4b74114430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4b640093b0 tx=0x7f4b64009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 shutdown_connections 2024-02-16T22:56:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f4b5c06b2e0 0x7f4b5c06d790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b74103540 0x7f4b74113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 --2- 172.21.15.64:0/2704225952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b74104750 0x7f4b74114430 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 >> 172.21.15.64:0/2704225952 conn(0x7f4b740727d0 msgr2=0x7f4b74107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:16.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 shutdown_connections 2024-02-16T22:56:16.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.397+0000 7f4b79372700 1 -- 172.21.15.64:0/2704225952 wait complete. 2024-02-16T22:56:16.402 INFO:tasks.workunit.client.0.smithi064.stdout:3/310: creat d4/d1f/d5b/d51/f6a x:0 0 0 2024-02-16T22:56:16.403 INFO:tasks.workunit.client.0.smithi064.stdout:0/226: link da/dc/de/d12/f2a da/dc/de/d37/f46 0 2024-02-16T22:56:16.403 INFO:tasks.workunit.client.1.smithi192.stdout:3/362: getdents da/d11/d35 0 2024-02-16T22:56:16.404 INFO:tasks.workunit.client.0.smithi064.stdout:9/262: dwrite d3/d11/d15/d5a/d2a/d2e/f59 [0,4194304] 0 2024-02-16T22:56:16.405 INFO:tasks.workunit.client.1.smithi192.stdout:2/388: dread f4 [0,4194304] 0 2024-02-16T22:56:16.406 INFO:tasks.workunit.client.1.smithi192.stdout:6/330: creat da/df/d18/d52/f66 x:0 0 0 2024-02-16T22:56:16.407 INFO:tasks.workunit.client.1.smithi192.stdout:2/389: chown d22/d24/d39/d3d/f40 27027414 1 2024-02-16T22:56:16.408 INFO:tasks.workunit.client.0.smithi064.stdout:0/227: truncate da/f44 334875 0 2024-02-16T22:56:16.408 INFO:tasks.workunit.client.1.smithi192.stdout:4/366: dwrite d1/d15/d29/d3c/f43 [0,4194304] 0 2024-02-16T22:56:16.409 INFO:tasks.workunit.client.1.smithi192.stdout:6/331: stat da/df/d12/d15/f21 0 2024-02-16T22:56:16.409 INFO:tasks.workunit.client.1.smithi192.stdout:2/390: chown d22/d24/d2b/f64 5107 1 2024-02-16T22:56:16.410 INFO:tasks.workunit.client.0.smithi064.stdout:9/263: mknod d3/d11/d15/d5a/d2a/c5c 0 2024-02-16T22:56:16.410 INFO:tasks.workunit.client.1.smithi192.stdout:6/332: write da/df/d18/f43 [3354088,122669] 0 2024-02-16T22:56:16.413 INFO:tasks.workunit.client.0.smithi064.stdout:3/311: link d4/d14/f22 d4/d13/d1b/d4f/f6b 0 2024-02-16T22:56:16.417 INFO:tasks.workunit.client.0.smithi064.stdout:0/228: creat da/d26/d3f/f47 x:0 0 0 2024-02-16T22:56:16.417 INFO:tasks.workunit.client.1.smithi192.stdout:1/315: link d3/l43 d3/de/d27/d34/d36/d47/d5d/l68 0 2024-02-16T22:56:16.421 INFO:tasks.workunit.client.0.smithi064.stdout:9/264: fdatasync d3/f18 0 2024-02-16T22:56:16.423 INFO:tasks.workunit.client.0.smithi064.stdout:9/265: write d3/d9/f4a [910703,80328] 0 2024-02-16T22:56:16.426 INFO:tasks.workunit.client.0.smithi064.stdout:0/229: symlink da/d26/l48 0 2024-02-16T22:56:16.429 INFO:tasks.workunit.client.0.smithi064.stdout:9/266: dwrite d3/d11/d15/d5a/f56 [0,4194304] 0 2024-02-16T22:56:16.431 INFO:tasks.workunit.client.0.smithi064.stdout:3/312: creat d4/d13/d4d/d5c/f6c x:0 0 0 2024-02-16T22:56:16.432 INFO:tasks.workunit.client.0.smithi064.stdout:3/313: read - d4/d1f/d5b/d51/f6a zero size 2024-02-16T22:56:16.433 INFO:tasks.workunit.client.1.smithi192.stdout:5/347: mknod d6/d12/d1d/d56/c70 0 2024-02-16T22:56:16.439 INFO:tasks.workunit.client.1.smithi192.stdout:3/363: symlink da/d11/d63/d6b/d4e/l7f 0 2024-02-16T22:56:16.446 INFO:tasks.workunit.client.1.smithi192.stdout:4/367: truncate d1/d15/d48/f55 6950053 0 2024-02-16T22:56:16.447 INFO:tasks.workunit.client.0.smithi064.stdout:3/314: rmdir d4/d13/d35 39 2024-02-16T22:56:16.448 INFO:tasks.workunit.client.0.smithi064.stdout:3/315: read - d4/d1f/f67 zero size 2024-02-16T22:56:16.448 INFO:tasks.workunit.client.1.smithi192.stdout:3/364: dwrite da/f51 [0,4194304] 0 2024-02-16T22:56:16.448 INFO:tasks.workunit.client.1.smithi192.stdout:4/368: read d1/d15/d29/d3c/f43 [3197981,11621] 0 2024-02-16T22:56:16.449 INFO:tasks.workunit.client.1.smithi192.stdout:3/365: write da/f1a [1564630,47858] 0 2024-02-16T22:56:16.450 INFO:tasks.workunit.client.0.smithi064.stdout:0/230: creat da/dc/de/d12/f49 x:0 0 0 2024-02-16T22:56:16.452 INFO:tasks.workunit.client.1.smithi192.stdout:1/316: rmdir d3/de/d27/d34/d36 39 2024-02-16T22:56:16.454 INFO:tasks.workunit.client.0.smithi064.stdout:9/267: write d3/d9/ff [4579627,25272] 0 2024-02-16T22:56:16.455 INFO:tasks.workunit.client.1.smithi192.stdout:8/332: creat d5/db/f64 x:0 0 0 2024-02-16T22:56:16.456 INFO:tasks.workunit.client.1.smithi192.stdout:8/333: write d5/d9/f5a [656001,62225] 0 2024-02-16T22:56:16.457 INFO:tasks.workunit.client.0.smithi064.stdout:3/316: symlink d4/d13/l6d 0 2024-02-16T22:56:16.457 INFO:tasks.workunit.client.1.smithi192.stdout:0/329: truncate d1/d3/f12 3666601 0 2024-02-16T22:56:16.458 INFO:tasks.workunit.client.0.smithi064.stdout:3/317: dread - d4/d1f/f67 zero size 2024-02-16T22:56:16.460 INFO:tasks.workunit.client.1.smithi192.stdout:5/348: mknod d6/d12/d4e/d46/d4f/c71 0 2024-02-16T22:56:16.462 INFO:tasks.workunit.client.1.smithi192.stdout:7/403: getdents d1/d2/de 0 2024-02-16T22:56:16.463 INFO:tasks.workunit.client.0.smithi064.stdout:9/268: dread d3/d11/d15/d23/f3d [0,4194304] 0 2024-02-16T22:56:16.464 INFO:tasks.workunit.client.1.smithi192.stdout:7/404: write d1/d69/f7f [294773,25135] 0 2024-02-16T22:56:16.465 INFO:tasks.workunit.client.1.smithi192.stdout:5/349: dread d6/fd [0,4194304] 0 2024-02-16T22:56:16.467 INFO:tasks.workunit.client.0.smithi064.stdout:0/231: getdents da/dc/de/d12/d2c 0 2024-02-16T22:56:16.470 INFO:tasks.workunit.client.1.smithi192.stdout:5/350: dread d6/d12/d4e/d46/f5d [0,4194304] 0 2024-02-16T22:56:16.471 INFO:tasks.workunit.client.0.smithi064.stdout:9/269: rename d3/d11/d15/d5a/d38/l4d to d3/d11/d15/d23/l5d 0 2024-02-16T22:56:16.472 INFO:tasks.workunit.client.0.smithi064.stdout:3/318: fsync d4/d1f/d34/f45 0 2024-02-16T22:56:16.479 INFO:tasks.workunit.client.0.smithi064.stdout:9/270: creat d3/d11/d15/d5a/d2a/d2e/d31/d3f/f5e x:0 0 0 2024-02-16T22:56:16.482 INFO:tasks.workunit.client.0.smithi064.stdout:0/232: dwrite da/dc/de/d12/f27 [0,4194304] 0 2024-02-16T22:56:16.488 INFO:tasks.workunit.client.0.smithi064.stdout:9/271: creat d3/d11/d15/d5a/d2a/d2e/f5f x:0 0 0 2024-02-16T22:56:16.493 INFO:tasks.workunit.client.1.smithi192.stdout:2/391: creat d22/d24/d39/d42/d65/d66/f83 x:0 0 0 2024-02-16T22:56:16.494 INFO:tasks.workunit.client.0.smithi064.stdout:9/272: creat d3/d9/f60 x:0 0 0 2024-02-16T22:56:16.494 INFO:tasks.workunit.client.1.smithi192.stdout:2/392: chown d22/d24/d2b/f38 2 1 2024-02-16T22:56:16.496 INFO:tasks.workunit.client.1.smithi192.stdout:6/333: symlink da/df/d12/d15/l67 0 2024-02-16T22:56:16.498 INFO:tasks.workunit.client.1.smithi192.stdout:2/393: dread d22/d24/d39/d42/d65/d69/f7a [0,4194304] 0 2024-02-16T22:56:16.498 INFO:tasks.workunit.client.0.smithi064.stdout:0/233: link da/dc/l23 da/d26/l4a 0 2024-02-16T22:56:16.503 INFO:tasks.workunit.client.0.smithi064.stdout:3/319: dread d4/d1f/f20 [0,4194304] 0 2024-02-16T22:56:16.504 INFO:tasks.workunit.client.1.smithi192.stdout:0/330: chown d1/d3/d39/f4f 25 1 2024-02-16T22:56:16.505 INFO:tasks.workunit.client.1.smithi192.stdout:0/331: write d1/d3/d39/f4f [390177,109187] 0 2024-02-16T22:56:16.507 INFO:tasks.workunit.client.0.smithi064.stdout:0/234: rename da/f18 to da/dc/f4b 0 2024-02-16T22:56:16.510 INFO:tasks.workunit.client.0.smithi064.stdout:3/320: mknod d4/d13/d41/c6e 0 2024-02-16T22:56:16.511 INFO:tasks.workunit.client.0.smithi064.stdout:3/321: write d4/d13/d1b/d27/f36 [380890,53845] 0 2024-02-16T22:56:16.512 INFO:tasks.workunit.client.1.smithi192.stdout:5/351: unlink d6/d12/d2d/c4a 0 2024-02-16T22:56:16.514 INFO:tasks.workunit.client.0.smithi064.stdout:9/273: link d3/d11/d15/d5a/d2a/d2e/d31/d3f/c55 d3/d9/c61 0 2024-02-16T22:56:16.520 INFO:tasks.workunit.client.0.smithi064.stdout:3/322: creat d4/d13/d1b/d4f/f6f x:0 0 0 2024-02-16T22:56:16.524 INFO:tasks.workunit.client.0.smithi064.stdout:9/274: unlink d3/d11/d15/d5a/d2a/d2e/d31/c53 0 2024-02-16T22:56:16.527 INFO:tasks.workunit.client.1.smithi192.stdout:6/334: rmdir da/df/d18/d52 39 2024-02-16T22:56:16.530 INFO:tasks.workunit.client.1.smithi192.stdout:3/366: mkdir da/d11/d63/d6b/d80 0 2024-02-16T22:56:16.535 INFO:tasks.workunit.client.1.smithi192.stdout:1/317: mkdir d3/de/d27/d34/d69 0 2024-02-16T22:56:16.536 INFO:tasks.workunit.client.0.smithi064.stdout:3/323: dread d4/d1f/d34/f3c [0,4194304] 0 2024-02-16T22:56:16.538 INFO:tasks.workunit.client.0.smithi064.stdout:0/235: symlink da/dc/de/d12/d1d/l4c 0 2024-02-16T22:56:16.540 INFO:tasks.workunit.client.0.smithi064.stdout:0/236: dread - da/dc/de/d12/f29 zero size 2024-02-16T22:56:16.540 INFO:tasks.workunit.client.1.smithi192.stdout:7/405: creat d1/d2/d2d/d61/d8a/f94 x:0 0 0 2024-02-16T22:56:16.544 INFO:tasks.workunit.client.0.smithi064.stdout:9/275: dwrite d3/fc [0,4194304] 0 2024-02-16T22:56:16.544 INFO:tasks.workunit.client.1.smithi192.stdout:5/352: mknod d6/d12/d4e/c72 0 2024-02-16T22:56:16.545 INFO:tasks.workunit.client.0.smithi064.stdout:3/324: dread d4/d13/d1b/f1c [0,4194304] 0 2024-02-16T22:56:16.545 INFO:tasks.workunit.client.1.smithi192.stdout:1/318: chown d3/de/d27/c51 19032 1 2024-02-16T22:56:16.546 INFO:tasks.workunit.client.0.smithi064.stdout:0/237: chown da/dc/l23 50 1 2024-02-16T22:56:16.548 INFO:tasks.workunit.client.1.smithi192.stdout:8/334: rename d5/db/l23 to d5/db/d47/l65 0 2024-02-16T22:56:16.553 INFO:tasks.workunit.client.1.smithi192.stdout:7/406: fdatasync d1/d2/d2d/d61/d88/d90/f64 0 2024-02-16T22:56:16.555 INFO:tasks.workunit.client.1.smithi192.stdout:5/353: fsync d6/d12/d1d/f36 0 2024-02-16T22:56:16.555 INFO:tasks.workunit.client.0.smithi064.stdout:9/276: dwrite d3/d9/f4a [0,4194304] 0 2024-02-16T22:56:16.557 INFO:tasks.workunit.client.1.smithi192.stdout:3/367: link da/f51 da/d6a/f81 0 2024-02-16T22:56:16.557 INFO:tasks.workunit.client.1.smithi192.stdout:3/368: dread - da/d11/d63/d6b/f4d zero size 2024-02-16T22:56:16.563 INFO:tasks.workunit.client.0.smithi064.stdout:9/277: unlink d3/c47 0 2024-02-16T22:56:16.565 INFO:tasks.workunit.client.1.smithi192.stdout:8/335: mknod d5/d26/c66 0 2024-02-16T22:56:16.567 INFO:tasks.workunit.client.0.smithi064.stdout:3/325: getdents d4/d1f/d34 0 2024-02-16T22:56:16.568 INFO:tasks.workunit.client.0.smithi064.stdout:3/326: write d4/f17 [2385528,21911] 0 2024-02-16T22:56:16.569 INFO:tasks.workunit.client.1.smithi192.stdout:0/332: rmdir d1/d3/de/d22/d37/d47/d53/d5e 0 2024-02-16T22:56:16.572 INFO:tasks.workunit.client.1.smithi192.stdout:7/407: creat d1/d2/de/d2c/d49/d54/d6a/f95 x:0 0 0 2024-02-16T22:56:16.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.571+0000 7f543d781700 1 -- 172.21.15.64:0/2586967365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54380fc7f0 msgr2=0x7f54380fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:16.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.571+0000 7f543d781700 1 --2- 172.21.15.64:0/2586967365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54380fc7f0 0x7f54380fcc40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f542c004a50 tx=0x7f542c00af40 comp rx=0 tx=0).stop 2024-02-16T22:56:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.571+0000 7f543d781700 1 -- 172.21.15.64:0/2586967365 shutdown_connections 2024-02-16T22:56:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.571+0000 7f543d781700 1 --2- 172.21.15.64:0/2586967365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54380fc7f0 0x7f54380fcc40 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f542c004a50 tx=0x7f542c00af40 comp rx=0 tx=0).stop 2024-02-16T22:56:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.571+0000 7f543d781700 1 --2- 172.21.15.64:0/2586967365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54380fb5e0 0x7f54380fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.571+0000 7f543d781700 1 -- 172.21.15.64:0/2586967365 >> 172.21.15.64:0/2586967365 conn(0x7f54380f6ca0 msgr2=0x7f54380f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.572+0000 7f543d781700 1 -- 172.21.15.64:0/2586967365 shutdown_connections 2024-02-16T22:56:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.572+0000 7f543d781700 1 -- 172.21.15.64:0/2586967365 wait complete. 2024-02-16T22:56:16.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.573+0000 7f543d781700 1 Processor -- start 2024-02-16T22:56:16.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.573+0000 7f543d781700 1 -- start start 2024-02-16T22:56:16.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.574+0000 7f543d781700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54380fb5e0 0x7f543810b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:16.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.574+0000 7f543d781700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f543810bee0 0x7f543810cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.574+0000 7f543d781700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543810c3a0 con 0x7f54380fb5e0 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.574+0000 7f543d781700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543810c4e0 con 0x7f543810bee0 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.574+0000 7f54367fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f543810bee0 0x7f543810cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.574+0000 7f54367fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f543810bee0 0x7f543810cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:39488/0 (socket says 172.21.15.64:39488) 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.574+0000 7f54367fc700 1 -- 172.21.15.64:0/1632917819 learned_addr learned my addr 172.21.15.64:0/1632917819 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f5436ffd700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54380fb5e0 0x7f543810b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f54367fc700 1 -- 172.21.15.64:0/1632917819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54380fb5e0 msgr2=0x7f543810b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f54367fc700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54380fb5e0 0x7f543810b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f54367fc700 1 -- 172.21.15.64:0/1632917819 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5428005040 con 0x7f543810bee0 2024-02-16T22:56:16.577 INFO:tasks.workunit.client.1.smithi192.stdout:3/369: dread f9 [0,4194304] 0 2024-02-16T22:56:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f54367fc700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f543810bee0 0x7f543810cd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f542c005010 tx=0x7f542c00af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f541f7fe700 1 -- 172.21.15.64:0/1632917819 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542c00f050 con 0x7f543810bee0 2024-02-16T22:56:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f541f7fe700 1 -- 172.21.15.64:0/1632917819 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f542c00a880 con 0x7f543810bee0 2024-02-16T22:56:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f541f7fe700 1 -- 172.21.15.64:0/1632917819 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542c014770 con 0x7f543810bee0 2024-02-16T22:56:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c005040 con 0x7f543810bee0 2024-02-16T22:56:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.575+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543810d5f0 con 0x7f543810bee0 2024-02-16T22:56:16.578 INFO:tasks.workunit.client.1.smithi192.stdout:2/394: getdents d22 0 2024-02-16T22:56:16.578 INFO:tasks.workunit.client.1.smithi192.stdout:2/395: dread - d22/f41 zero size 2024-02-16T22:56:16.578 INFO:tasks.workunit.client.0.smithi064.stdout:0/238: dwrite da/dc/de/d37/f46 [0,4194304] 0 2024-02-16T22:56:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.578+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5438105880 con 0x7f543810bee0 2024-02-16T22:56:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.578+0000 7f541f7fe700 1 -- 172.21.15.64:0/1632917819 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f542c01dae0 con 0x7f543810bee0 2024-02-16T22:56:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.579+0000 7f541f7fe700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f542006f7b0 0x7f5420071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.579+0000 7f541f7fe700 1 -- 172.21.15.64:0/1632917819 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f542c08b910 con 0x7f543810bee0 2024-02-16T22:56:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.579+0000 7f5436ffd700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f542006f7b0 0x7f5420071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.580+0000 7f5436ffd700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f542006f7b0 0x7f5420071c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f54380fc650 tx=0x7f5428009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:16.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:16.586+0000 7f541f7fe700 1 -- 172.21.15.64:0/1632917819 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f542c059480 con 0x7f543810bee0 2024-02-16T22:56:16.587 INFO:tasks.workunit.client.0.smithi064.stdout:0/239: dread da/dc/de/d12/d1d/f2b [0,4194304] 0 2024-02-16T22:56:16.588 INFO:tasks.workunit.client.0.smithi064.stdout:9/278: getdents d3/d11 0 2024-02-16T22:56:16.592 INFO:tasks.workunit.client.1.smithi192.stdout:1/319: truncate d3/f2d 1908121 0 2024-02-16T22:56:16.594 INFO:tasks.workunit.client.0.smithi064.stdout:3/327: dwrite d4/d14/f22 [0,4194304] 0 2024-02-16T22:56:16.595 INFO:tasks.workunit.client.0.smithi064.stdout:9/279: mkdir d3/d11/d15/d5a/d2a/d2e/d31/d62 0 2024-02-16T22:56:16.597 INFO:tasks.workunit.client.0.smithi064.stdout:3/328: read d4/d14/f46 [899109,102110] 0 2024-02-16T22:56:16.599 INFO:tasks.workunit.client.1.smithi192.stdout:2/396: mkdir d22/d24/d39/d42/d84 0 2024-02-16T22:56:16.599 INFO:tasks.workunit.client.0.smithi064.stdout:5/189: sync 2024-02-16T22:56:16.602 INFO:tasks.workunit.client.0.smithi064.stdout:9/280: dread d3/d9/d1e/d42/f54 [0,4194304] 0 2024-02-16T22:56:16.605 INFO:tasks.workunit.client.1.smithi192.stdout:6/335: rename da/df/d12/d57/f4b to da/df/d12/d15/f68 0 2024-02-16T22:56:16.607 INFO:tasks.workunit.client.0.smithi064.stdout:5/190: dwrite d6/d18/d19/d1a/f3b [0,4194304] 0 2024-02-16T22:56:16.612 INFO:tasks.workunit.client.0.smithi064.stdout:0/240: rename da/dc/de/f16 to da/f4d 0 2024-02-16T22:56:16.613 INFO:tasks.workunit.client.1.smithi192.stdout:1/320: creat d3/de/d27/f6a x:0 0 0 2024-02-16T22:56:16.617 INFO:tasks.workunit.client.1.smithi192.stdout:1/321: dread d3/de/d27/d34/d52/f54 [0,4194304] 0 2024-02-16T22:56:16.618 INFO:tasks.workunit.client.1.smithi192.stdout:7/408: link d1/d2/de/d2c/d49/f6f d1/d2/de/d2c/f96 0 2024-02-16T22:56:16.618 INFO:tasks.workunit.client.0.smithi064.stdout:5/191: dwrite d6/d18/f41 [0,4194304] 0 2024-02-16T22:56:16.619 INFO:tasks.workunit.client.1.smithi192.stdout:7/409: fsync d1/d2/d2d/d61/d8a/f94 0 2024-02-16T22:56:16.620 INFO:tasks.workunit.client.1.smithi192.stdout:7/410: write d1/d2/d2d/d61/d88/d90/d4c/f6b [635152,122437] 0 2024-02-16T22:56:16.623 INFO:tasks.workunit.client.1.smithi192.stdout:2/397: unlink d22/d24/d39/d42/d65/d69/c5a 0 2024-02-16T22:56:16.624 INFO:tasks.workunit.client.1.smithi192.stdout:2/398: chown d22/d24/d39/d42/c5c 44135896 1 2024-02-16T22:56:16.625 INFO:tasks.workunit.client.0.smithi064.stdout:5/192: dwrite d6/d25/f32 [0,4194304] 0 2024-02-16T22:56:16.631 INFO:tasks.workunit.client.0.smithi064.stdout:0/241: fsync da/dc/d3b/f3c 0 2024-02-16T22:56:16.632 INFO:tasks.workunit.client.1.smithi192.stdout:0/333: creat d1/f61 x:0 0 0 2024-02-16T22:56:16.632 INFO:tasks.workunit.client.1.smithi192.stdout:0/334: chown d1/d2 2517221 1 2024-02-16T22:56:16.632 INFO:tasks.workunit.client.0.smithi064.stdout:3/329: rename d4/d13/d1b/d27/l2d to d4/d13/d1b/d48/l70 0 2024-02-16T22:56:16.634 INFO:tasks.workunit.client.0.smithi064.stdout:3/330: write d4/d14/f22 [3130782,5] 0 2024-02-16T22:56:16.641 INFO:tasks.workunit.client.1.smithi192.stdout:2/399: mkdir d22/d24/d39/d42/d85 0 2024-02-16T22:56:16.643 INFO:tasks.workunit.client.1.smithi192.stdout:6/336: fdatasync da/df/d18/d52/f4a 0 2024-02-16T22:56:16.643 INFO:tasks.workunit.client.1.smithi192.stdout:6/337: chown da/f5f 84 1 2024-02-16T22:56:16.644 INFO:tasks.workunit.client.0.smithi064.stdout:0/242: chown da/fb 228472 1 2024-02-16T22:56:16.645 INFO:tasks.workunit.client.0.smithi064.stdout:6/306: sync 2024-02-16T22:56:16.645 INFO:tasks.workunit.client.0.smithi064.stdout:2/248: sync 2024-02-16T22:56:16.645 INFO:tasks.workunit.client.0.smithi064.stdout:8/241: sync 2024-02-16T22:56:16.645 INFO:tasks.workunit.client.0.smithi064.stdout:4/262: sync 2024-02-16T22:56:16.645 INFO:tasks.workunit.client.0.smithi064.stdout:1/231: sync 2024-02-16T22:56:16.645 INFO:tasks.workunit.client.0.smithi064.stdout:7/228: sync 2024-02-16T22:56:16.646 INFO:tasks.workunit.client.1.smithi192.stdout:0/335: mkdir d1/d2/d34/d62 0 2024-02-16T22:56:16.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/338: write da/df/d18/d52/f55 [984744,118388] 0 2024-02-16T22:56:16.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/339: chown da/df/d12/c5d 102917 1 2024-02-16T22:56:16.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/340: chown da/df/d12/c5d 486 1 2024-02-16T22:56:16.647 INFO:tasks.workunit.client.0.smithi064.stdout:2/249: read d7/f35 [1169314,32005] 0 2024-02-16T22:56:16.650 INFO:tasks.workunit.client.0.smithi064.stdout:7/229: truncate d5/d6/f22 537506 0 2024-02-16T22:56:16.650 INFO:tasks.workunit.client.0.smithi064.stdout:3/331: rename d4/d14/f30 to d4/d13/d1b/d27/d61/d63/f71 0 2024-02-16T22:56:16.651 INFO:tasks.workunit.client.1.smithi192.stdout:1/322: creat d3/de/d27/d34/d69/f6b x:0 0 0 2024-02-16T22:56:16.652 INFO:tasks.workunit.client.0.smithi064.stdout:1/232: chown d2/df/d14/d18/f28 0 1 2024-02-16T22:56:16.652 INFO:tasks.workunit.client.0.smithi064.stdout:0/243: write da/dc/de/d12/d1d/f1f [4787516,58977] 0 2024-02-16T22:56:16.653 INFO:tasks.workunit.client.0.smithi064.stdout:7/230: chown d5/d6 1301 1 2024-02-16T22:56:16.656 INFO:tasks.workunit.client.0.smithi064.stdout:3/332: fdatasync d4/d13/d1b/d27/d44/f4b 0 2024-02-16T22:56:16.657 INFO:tasks.workunit.client.1.smithi192.stdout:7/411: creat d1/d2/de/d2c/d49/d89/f97 x:0 0 0 2024-02-16T22:56:16.657 INFO:tasks.workunit.client.1.smithi192.stdout:7/412: stat d1/d69/f93 0 2024-02-16T22:56:16.658 INFO:tasks.workunit.client.1.smithi192.stdout:7/413: stat d1/d69/f93 0 2024-02-16T22:56:16.658 INFO:tasks.workunit.client.0.smithi064.stdout:1/233: stat d2/df/d14/d18/f26 0 2024-02-16T22:56:16.659 INFO:tasks.workunit.client.0.smithi064.stdout:4/263: dwrite dc/d1d/f4c [0,4194304] 0 2024-02-16T22:56:16.659 INFO:tasks.workunit.client.0.smithi064.stdout:8/242: mkdir df/d31/d2f/d4f 0 2024-02-16T22:56:16.660 INFO:tasks.workunit.client.1.smithi192.stdout:9/439: sync 2024-02-16T22:56:16.662 INFO:tasks.workunit.client.1.smithi192.stdout:5/354: rename d6/d20/c27 to d6/d12/d4e/d58/d6c/c73 0 2024-02-16T22:56:16.663 INFO:tasks.workunit.client.1.smithi192.stdout:7/414: dread d1/d2/d2d/f40 [0,4194304] 0 2024-02-16T22:56:16.665 INFO:tasks.workunit.client.1.smithi192.stdout:1/323: dread d3/de/d27/f30 [0,4194304] 0 2024-02-16T22:56:16.667 INFO:tasks.workunit.client.0.smithi064.stdout:9/281: truncate d3/d9/f21 3601393 0 2024-02-16T22:56:16.669 INFO:tasks.workunit.client.1.smithi192.stdout:0/336: mknod d1/d3/d39/c63 0 2024-02-16T22:56:16.672 INFO:tasks.workunit.client.1.smithi192.stdout:7/415: dwrite d1/d2/de/d2c/d49/f71 [4194304,4194304] 0 2024-02-16T22:56:16.674 INFO:tasks.workunit.client.1.smithi192.stdout:1/324: dwrite d3/de/d27/d34/d52/f57 [0,4194304] 0 2024-02-16T22:56:16.681 INFO:tasks.workunit.client.1.smithi192.stdout:1/325: dwrite d3/de/d27/d34/d69/f6b [0,4194304] 0 2024-02-16T22:56:16.687 INFO:tasks.workunit.client.1.smithi192.stdout:9/440: mknod d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/c9c 0 2024-02-16T22:56:16.690 INFO:tasks.workunit.client.0.smithi064.stdout:9/282: dread d3/d11/f1f [0,4194304] 0 2024-02-16T22:56:16.691 INFO:tasks.workunit.client.0.smithi064.stdout:9/283: fdatasync d3/d11/d15/d5a/d2a/d2e/d31/d3f/f43 0 2024-02-16T22:56:16.692 INFO:tasks.workunit.client.1.smithi192.stdout:8/336: rename d5/db/de/d21 to d5/db/d47/d43/d67 0 2024-02-16T22:56:16.696 INFO:tasks.workunit.client.0.smithi064.stdout:7/231: rename d5/d33/d35/f3a to d5/d33/d35/f44 0 2024-02-16T22:56:16.700 INFO:tasks.workunit.client.0.smithi064.stdout:3/333: mkdir d4/d14/d2b/d72 0 2024-02-16T22:56:16.701 INFO:tasks.workunit.client.1.smithi192.stdout:2/400: creat d22/d24/d39/d42/d85/f86 x:0 0 0 2024-02-16T22:56:16.704 INFO:tasks.workunit.client.1.smithi192.stdout:6/341: mknod da/df/d18/c69 0 2024-02-16T22:56:16.706 INFO:tasks.workunit.client.1.smithi192.stdout:2/401: dwrite d22/d24/d2b/d2c/f37 [0,4194304] 0 2024-02-16T22:56:16.709 INFO:tasks.workunit.client.1.smithi192.stdout:5/355: write d6/d12/d4e/f1c [712854,16207] 0 2024-02-16T22:56:16.712 INFO:tasks.workunit.client.0.smithi064.stdout:2/250: symlink d7/d14/d4b/l4c 0 2024-02-16T22:56:16.717 INFO:tasks.workunit.client.0.smithi064.stdout:5/193: getdents d6/d18/d19 0 2024-02-16T22:56:16.717 INFO:tasks.workunit.client.0.smithi064.stdout:5/194: fsync d6/d18/d19/d1a/f3b 0 2024-02-16T22:56:16.719 INFO:tasks.workunit.client.0.smithi064.stdout:0/244: fdatasync da/f10 0 2024-02-16T22:56:16.724 INFO:tasks.workunit.client.1.smithi192.stdout:0/337: rename d1/d2/l1b to d1/d3/d29/l64 0 2024-02-16T22:56:16.726 INFO:tasks.workunit.client.1.smithi192.stdout:9/441: dwrite f0 [4194304,4194304] 0 2024-02-16T22:56:16.727 INFO:tasks.workunit.client.0.smithi064.stdout:1/234: symlink d2/df/d14/d18/l51 0 2024-02-16T22:56:16.729 INFO:tasks.workunit.client.0.smithi064.stdout:8/243: mknod df/c50 0 2024-02-16T22:56:16.730 INFO:tasks.workunit.client.1.smithi192.stdout:5/356: mknod d6/d8/c74 0 2024-02-16T22:56:16.731 INFO:tasks.workunit.client.0.smithi064.stdout:4/264: mkdir dc/dd/d4f 0 2024-02-16T22:56:16.732 INFO:tasks.workunit.client.0.smithi064.stdout:3/334: dwrite d4/d13/f26 [0,4194304] 0 2024-02-16T22:56:16.736 INFO:tasks.workunit.client.0.smithi064.stdout:6/307: creat d0/d6/d10/d33/f55 x:0 0 0 2024-02-16T22:56:16.740 INFO:tasks.workunit.client.0.smithi064.stdout:3/335: truncate d4/d13/d4d/d5c/f6c 129093 0 2024-02-16T22:56:16.740 INFO:tasks.workunit.client.1.smithi192.stdout:0/338: creat d1/d3/de/d22/d37/d47/d53/f65 x:0 0 0 2024-02-16T22:56:16.747 INFO:tasks.workunit.client.1.smithi192.stdout:5/357: mkdir d6/d8/d75 0 2024-02-16T22:56:16.750 INFO:tasks.workunit.client.1.smithi192.stdout:5/358: dread d6/d12/d4e/d5b/f5e [0,4194304] 0 2024-02-16T22:56:16.754 INFO:tasks.workunit.client.1.smithi192.stdout:0/339: dwrite d1/fd [8388608,4194304] 0 2024-02-16T22:56:16.755 INFO:tasks.workunit.client.0.smithi064.stdout:4/265: symlink dc/de/d28/l50 0 2024-02-16T22:56:16.756 INFO:tasks.workunit.client.1.smithi192.stdout:0/340: chown d1/d3/de/d22/f28 31 1 2024-02-16T22:56:16.757 INFO:tasks.workunit.client.1.smithi192.stdout:7/416: getdents d1/d2/d2d/d61/d88/d90/d4c 0 2024-02-16T22:56:16.758 INFO:tasks.workunit.client.0.smithi064.stdout:6/308: mkdir d0/d6/da/d56 0 2024-02-16T22:56:16.759 INFO:tasks.workunit.client.1.smithi192.stdout:8/337: getdents d5/db/de/df 0 2024-02-16T22:56:16.764 INFO:tasks.workunit.client.0.smithi064.stdout:1/235: dread d2/df/d11/d1a/d30/f32 [0,4194304] 0 2024-02-16T22:56:16.765 INFO:tasks.workunit.client.1.smithi192.stdout:5/359: symlink d6/d12/l76 0 2024-02-16T22:56:16.765 INFO:tasks.workunit.client.1.smithi192.stdout:5/360: chown d6/d8/l21 2085687 1 2024-02-16T22:56:16.765 INFO:tasks.workunit.client.1.smithi192.stdout:5/361: chown d6/d12/d1d/f36 2 1 2024-02-16T22:56:16.767 INFO:tasks.workunit.client.1.smithi192.stdout:6/342: getdents da/df/d18/d52 0 2024-02-16T22:56:16.767 INFO:tasks.workunit.client.1.smithi192.stdout:1/326: truncate d3/de/d27/d34/d69/f6b 673138 0 2024-02-16T22:56:16.770 INFO:tasks.workunit.client.1.smithi192.stdout:9/442: write d7/d19/d37/d5a/f63 [5221973,65545] 0 2024-02-16T22:56:16.774 INFO:tasks.workunit.client.1.smithi192.stdout:6/343: dwrite da/df/d12/f13 [0,4194304] 0 2024-02-16T22:56:16.777 INFO:tasks.workunit.client.0.smithi064.stdout:8/244: mknod df/c51 0 2024-02-16T22:56:16.777 INFO:tasks.workunit.client.1.smithi192.stdout:0/341: dread d1/f1f [0,4194304] 0 2024-02-16T22:56:16.778 INFO:tasks.workunit.client.0.smithi064.stdout:9/284: truncate d3/d11/d15/d5a/d2a/f4b 1716030 0 2024-02-16T22:56:16.778 INFO:tasks.workunit.client.1.smithi192.stdout:4/369: sync 2024-02-16T22:56:16.780 INFO:tasks.workunit.client.1.smithi192.stdout:6/344: dwrite da/df/d18/f43 [0,4194304] 0 2024-02-16T22:56:16.783 INFO:tasks.workunit.client.1.smithi192.stdout:6/345: fsync da/df/d12/f39 0 2024-02-16T22:56:16.783 INFO:tasks.workunit.client.0.smithi064.stdout:8/245: dread df/f12 [0,4194304] 0 2024-02-16T22:56:16.784 INFO:tasks.workunit.client.1.smithi192.stdout:7/417: mknod d1/c98 0 2024-02-16T22:56:16.784 INFO:tasks.workunit.client.0.smithi064.stdout:8/246: chown df/d31/d2f/f2b 804177383 1 2024-02-16T22:56:16.785 INFO:tasks.workunit.client.1.smithi192.stdout:7/418: chown d1/d69/f93 1697 1 2024-02-16T22:56:16.787 INFO:tasks.workunit.client.1.smithi192.stdout:5/362: dread d6/d12/f18 [4194304,4194304] 0 2024-02-16T22:56:16.788 INFO:tasks.workunit.client.0.smithi064.stdout:2/251: truncate d7/d14/f2e 1005011 0 2024-02-16T22:56:16.793 INFO:tasks.workunit.client.1.smithi192.stdout:8/338: write d5/d9/f17 [3410590,22860] 0 2024-02-16T22:56:16.794 INFO:tasks.workunit.client.0.smithi064.stdout:8/247: dwrite df/d31/d2f/d38/d48/f49 [0,4194304] 0 2024-02-16T22:56:16.799 INFO:tasks.workunit.client.1.smithi192.stdout:0/342: creat d1/d2/f66 x:0 0 0 2024-02-16T22:56:16.799 INFO:tasks.workunit.client.0.smithi064.stdout:7/232: getdents d5/d6 0 2024-02-16T22:56:16.800 INFO:tasks.workunit.client.1.smithi192.stdout:0/343: chown d1/d3/c2b 66651 1 2024-02-16T22:56:16.801 INFO:tasks.workunit.client.1.smithi192.stdout:1/327: mknod d3/de/c6c 0 2024-02-16T22:56:16.802 INFO:tasks.workunit.client.1.smithi192.stdout:1/328: write d3/fb [6622841,667] 0 2024-02-16T22:56:16.803 INFO:tasks.workunit.client.1.smithi192.stdout:4/370: creat d1/d26/d6f/f88 x:0 0 0 2024-02-16T22:56:16.806 INFO:tasks.workunit.client.0.smithi064.stdout:4/266: truncate dc/f33 7056003 0 2024-02-16T22:56:16.807 INFO:tasks.workunit.client.0.smithi064.stdout:4/267: write dc/dd/d16/f23 [3703043,98268] 0 2024-02-16T22:56:16.807 INFO:tasks.workunit.client.0.smithi064.stdout:4/268: chown dc/dd 7412899 1 2024-02-16T22:56:16.811 INFO:tasks.workunit.client.1.smithi192.stdout:7/419: creat d1/d2/de/f99 x:0 0 0 2024-02-16T22:56:16.813 INFO:tasks.workunit.client.0.smithi064.stdout:6/309: symlink d0/d5/df/d17/d4a/l57 0 2024-02-16T22:56:16.814 INFO:tasks.workunit.client.1.smithi192.stdout:7/420: dread d1/d2/f1e [0,4194304] 0 2024-02-16T22:56:16.816 INFO:tasks.workunit.client.1.smithi192.stdout:9/443: link d7/f83 d7/f9d 0 2024-02-16T22:56:16.817 INFO:tasks.workunit.client.1.smithi192.stdout:9/444: chown d7/d15/d36/l48 1134 1 2024-02-16T22:56:16.820 INFO:tasks.workunit.client.1.smithi192.stdout:3/370: sync 2024-02-16T22:56:16.822 INFO:tasks.workunit.client.1.smithi192.stdout:7/421: dwrite d1/d2/de/f28 [0,4194304] 0 2024-02-16T22:56:16.822 INFO:tasks.workunit.client.1.smithi192.stdout:5/363: mkdir d6/d8/d77 0 2024-02-16T22:56:16.823 INFO:tasks.workunit.client.0.smithi064.stdout:8/248: dread f8 [0,4194304] 0 2024-02-16T22:56:16.823 INFO:tasks.workunit.client.0.smithi064.stdout:7/233: dread d5/d6/f30 [0,4194304] 0 2024-02-16T22:56:16.824 INFO:tasks.workunit.client.0.smithi064.stdout:3/336: truncate d4/d13/d1b/d4f/f6b 1536080 0 2024-02-16T22:56:16.831 INFO:tasks.workunit.client.0.smithi064.stdout:5/195: link d6/c7 d6/d1c/c44 0 2024-02-16T22:56:16.831 INFO:tasks.workunit.client.1.smithi192.stdout:8/339: mkdir d5/d68 0 2024-02-16T22:56:16.835 INFO:tasks.workunit.client.1.smithi192.stdout:0/344: rename d1/d2 to d1/d3/de/d22/d37/d67 0 2024-02-16T22:56:16.836 INFO:tasks.workunit.client.0.smithi064.stdout:0/245: getdents da/dc/d3b 0 2024-02-16T22:56:16.836 INFO:tasks.workunit.client.1.smithi192.stdout:8/340: dwrite d5/d26/f3f [0,4194304] 0 2024-02-16T22:56:16.842 INFO:tasks.workunit.client.1.smithi192.stdout:6/346: write da/df/f1c [340344,129177] 0 2024-02-16T22:56:16.842 INFO:tasks.workunit.client.1.smithi192.stdout:1/329: read d3/de/d27/d34/f45 [1201279,97648] 0 2024-02-16T22:56:16.844 INFO:tasks.workunit.client.1.smithi192.stdout:4/371: unlink d1/d15/d29/l7c 0 2024-02-16T22:56:16.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/347: dwrite da/df/d18/d52/f66 [0,4194304] 0 2024-02-16T22:56:16.848 INFO:tasks.workunit.client.0.smithi064.stdout:4/269: creat dc/dd/d16/f51 x:0 0 0 2024-02-16T22:56:16.853 INFO:tasks.workunit.client.0.smithi064.stdout:2/252: mknod d7/d14/d3e/c4d 0 2024-02-16T22:56:16.855 INFO:tasks.workunit.client.1.smithi192.stdout:9/445: creat d7/d19/d37/f9e x:0 0 0 2024-02-16T22:56:16.856 INFO:tasks.workunit.client.1.smithi192.stdout:9/446: stat d7/d19/d37/c4a 0 2024-02-16T22:56:16.857 INFO:tasks.workunit.client.1.smithi192.stdout:9/447: dread - d7/d15/d36/d54/f96 zero size 2024-02-16T22:56:16.858 INFO:tasks.workunit.client.1.smithi192.stdout:9/448: fsync d7/d19/d5b/f89 0 2024-02-16T22:56:16.858 INFO:tasks.workunit.client.0.smithi064.stdout:8/249: mknod df/d31/c52 0 2024-02-16T22:56:16.860 INFO:tasks.workunit.client.0.smithi064.stdout:7/234: mkdir d5/d33/d45 0 2024-02-16T22:56:16.862 INFO:tasks.workunit.client.0.smithi064.stdout:3/337: fsync d4/d1f/f20 0 2024-02-16T22:56:16.864 INFO:tasks.workunit.client.1.smithi192.stdout:7/422: symlink d1/d2/de/d2c/d49/d54/l9a 0 2024-02-16T22:56:16.865 INFO:tasks.workunit.client.1.smithi192.stdout:7/423: chown d1/d2/de/d2c/d49/c56 221913964 1 2024-02-16T22:56:16.869 INFO:tasks.workunit.client.0.smithi064.stdout:5/196: rename d6/ff to d6/d18/d19/d1a/f45 0 2024-02-16T22:56:16.870 INFO:tasks.workunit.client.1.smithi192.stdout:0/345: creat d1/d3/d29/d43/f68 x:0 0 0 2024-02-16T22:56:16.872 INFO:tasks.workunit.client.1.smithi192.stdout:8/341: creat d5/d9/f69 x:0 0 0 2024-02-16T22:56:16.873 INFO:tasks.workunit.client.1.smithi192.stdout:8/342: write d5/d3e/d5f/f4e [2855026,13469] 0 2024-02-16T22:56:16.876 INFO:tasks.workunit.client.0.smithi064.stdout:9/285: creat d3/f63 x:0 0 0 2024-02-16T22:56:16.878 INFO:tasks.workunit.client.1.smithi192.stdout:4/372: read d1/d8/d11/f59 [131456,96986] 0 2024-02-16T22:56:16.880 INFO:tasks.workunit.client.1.smithi192.stdout:3/371: write da/d11/f58 [67803,4430] 0 2024-02-16T22:56:16.881 INFO:tasks.workunit.client.1.smithi192.stdout:3/372: write da/d11/d20/f30 [4531798,56655] 0 2024-02-16T22:56:16.882 INFO:tasks.workunit.client.0.smithi064.stdout:4/270: creat dc/dd/d32/d22/f52 x:0 0 0 2024-02-16T22:56:16.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/373: readlink da/d11/l29 0 2024-02-16T22:56:16.883 INFO:tasks.workunit.client.0.smithi064.stdout:4/271: chown dc/dd/d16/f3a 2117948 1 2024-02-16T22:56:16.883 INFO:tasks.workunit.client.1.smithi192.stdout:6/348: rmdir da/df/d12 39 2024-02-16T22:56:16.884 INFO:tasks.workunit.client.1.smithi192.stdout:6/349: chown da/d40/f4c 1059460959 1 2024-02-16T22:56:16.886 INFO:tasks.workunit.client.0.smithi064.stdout:5/197: dread d6/d1c/f20 [0,4194304] 0 2024-02-16T22:56:16.890 INFO:tasks.workunit.client.0.smithi064.stdout:2/253: readlink d7/l30 0 2024-02-16T22:56:16.893 INFO:tasks.workunit.client.1.smithi192.stdout:5/364: mkdir d6/d12/d4e/d5b/d67/d78 0 2024-02-16T22:56:16.895 INFO:tasks.workunit.client.1.smithi192.stdout:2/402: sync 2024-02-16T22:56:16.896 INFO:tasks.workunit.client.1.smithi192.stdout:0/346: rename d1/d3/de/d22/d37/d67/d34/d62 to d1/d3/de/d22/d37/d69 0 2024-02-16T22:56:16.901 INFO:tasks.workunit.client.1.smithi192.stdout:8/343: unlink d5/db/d47/d43/f52 0 2024-02-16T22:56:16.902 INFO:tasks.workunit.client.1.smithi192.stdout:8/344: readlink d5/db/d47/l59 0 2024-02-16T22:56:16.902 INFO:tasks.workunit.client.0.smithi064.stdout:0/246: creat da/dc/de/d37/f4e x:0 0 0 2024-02-16T22:56:16.903 INFO:tasks.workunit.client.1.smithi192.stdout:9/449: dwrite d7/d15/d36/d38/f47 [0,4194304] 0 2024-02-16T22:56:16.904 INFO:tasks.workunit.client.1.smithi192.stdout:1/330: symlink d3/l6d 0 2024-02-16T22:56:16.905 INFO:tasks.workunit.client.1.smithi192.stdout:1/331: truncate d3/de/d27/f30 9171526 0 2024-02-16T22:56:16.906 INFO:tasks.workunit.client.1.smithi192.stdout:1/332: dread - d3/de/d27/d34/d52/f56 zero size 2024-02-16T22:56:16.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/374: symlink da/d11/d20/d39/d53/l82 0 2024-02-16T22:56:16.913 INFO:tasks.workunit.client.0.smithi064.stdout:4/272: rmdir dc/d1d 39 2024-02-16T22:56:16.918 INFO:tasks.workunit.client.0.smithi064.stdout:5/198: mknod d6/d18/d19/d1a/c46 0 2024-02-16T22:56:16.919 INFO:tasks.workunit.client.1.smithi192.stdout:4/373: dwrite d1/d8/d11/f4e [0,4194304] 0 2024-02-16T22:56:16.920 INFO:tasks.workunit.client.1.smithi192.stdout:7/424: mkdir d1/d2/d2d/d92/d9b 0 2024-02-16T22:56:16.921 INFO:tasks.workunit.client.1.smithi192.stdout:4/374: dread - d1/d26/d6f/f88 zero size 2024-02-16T22:56:16.921 INFO:tasks.workunit.client.1.smithi192.stdout:7/425: chown d1/d2/d2d/c4e 782171 1 2024-02-16T22:56:16.923 INFO:tasks.workunit.client.0.smithi064.stdout:7/235: creat d5/d33/d45/f46 x:0 0 0 2024-02-16T22:56:16.923 INFO:tasks.workunit.client.1.smithi192.stdout:9/450: dread d7/d19/d28/f2c [0,4194304] 0 2024-02-16T22:56:16.925 INFO:tasks.workunit.client.1.smithi192.stdout:2/403: mknod d22/c87 0 2024-02-16T22:56:16.925 INFO:tasks.workunit.client.0.smithi064.stdout:5/199: dwrite d6/f34 [0,4194304] 0 2024-02-16T22:56:16.927 INFO:tasks.workunit.client.1.smithi192.stdout:7/426: dwrite d1/d2/de/f28 [0,4194304] 0 2024-02-16T22:56:16.928 INFO:tasks.workunit.client.1.smithi192.stdout:7/427: fsync d1/d2/d37/d86/f8c 0 2024-02-16T22:56:16.930 INFO:tasks.workunit.client.1.smithi192.stdout:2/404: dread d22/d24/d2b/d2c/f37 [0,4194304] 0 2024-02-16T22:56:16.931 INFO:tasks.workunit.client.1.smithi192.stdout:0/347: write d1/d3/de/f10 [6443320,113006] 0 2024-02-16T22:56:16.935 INFO:tasks.workunit.client.0.smithi064.stdout:0/247: mknod da/dc/c4f 0 2024-02-16T22:56:16.937 INFO:tasks.workunit.client.1.smithi192.stdout:8/345: unlink d5/db/de/df/f2d 0 2024-02-16T22:56:16.940 INFO:tasks.workunit.client.0.smithi064.stdout:6/310: getdents d0/d5/df 0 2024-02-16T22:56:16.940 INFO:tasks.workunit.client.0.smithi064.stdout:6/311: fdatasync d0/d5/f42 0 2024-02-16T22:56:16.946 INFO:tasks.workunit.client.0.smithi064.stdout:6/312: dread d0/d5/f11 [0,4194304] 0 2024-02-16T22:56:16.949 INFO:tasks.workunit.client.1.smithi192.stdout:3/375: rmdir da 39 2024-02-16T22:56:16.951 INFO:tasks.workunit.client.0.smithi064.stdout:2/254: fdatasync d7/f1a 0 2024-02-16T22:56:16.952 INFO:tasks.workunit.client.0.smithi064.stdout:2/255: chown d7/d14/d4b/l4c 20496 1 2024-02-16T22:56:16.952 INFO:tasks.workunit.client.1.smithi192.stdout:4/375: rmdir d1/d26/d6f 39 2024-02-16T22:56:16.954 INFO:tasks.workunit.client.0.smithi064.stdout:7/236: chown d5/l14 906 1 2024-02-16T22:56:16.958 INFO:tasks.workunit.client.0.smithi064.stdout:5/200: creat d6/d18/d19/f47 x:0 0 0 2024-02-16T22:56:16.960 INFO:tasks.workunit.client.0.smithi064.stdout:3/338: link d4/cf d4/d14/d2b/d72/c73 0 2024-02-16T22:56:16.963 INFO:tasks.workunit.client.0.smithi064.stdout:0/248: symlink da/dc/de/d12/d2c/l50 0 2024-02-16T22:56:16.964 INFO:tasks.workunit.client.1.smithi192.stdout:2/405: fsync fe 0 2024-02-16T22:56:16.965 INFO:tasks.workunit.client.0.smithi064.stdout:5/201: dwrite d6/fa [0,4194304] 0 2024-02-16T22:56:16.969 INFO:tasks.workunit.client.1.smithi192.stdout:0/348: mknod d1/d3/de/d22/d37/d47/d53/c6a 0 2024-02-16T22:56:16.970 INFO:tasks.workunit.client.1.smithi192.stdout:0/349: fdatasync d1/d3/de/d22/d37/d47/d53/f65 0 2024-02-16T22:56:16.971 INFO:tasks.workunit.client.0.smithi064.stdout:4/273: dread dc/d1d/f41 [0,4194304] 0 2024-02-16T22:56:16.973 INFO:tasks.workunit.client.0.smithi064.stdout:4/274: write dc/dd/f42 [6447658,120803] 0 2024-02-16T22:56:16.975 INFO:tasks.workunit.client.0.smithi064.stdout:4/275: truncate dc/dd/d16/f51 1032486 0 2024-02-16T22:56:16.982 INFO:tasks.workunit.client.0.smithi064.stdout:6/313: creat d0/d1d/f58 x:0 0 0 2024-02-16T22:56:16.982 INFO:tasks.workunit.client.1.smithi192.stdout:9/451: write f1 [1562476,16857] 0 2024-02-16T22:56:16.982 INFO:tasks.workunit.client.1.smithi192.stdout:9/452: write d7/d15/d36/f7b [141264,54836] 0 2024-02-16T22:56:16.983 INFO:tasks.workunit.client.0.smithi064.stdout:6/314: write d0/d5/df/d17/d18/f51 [799972,37330] 0 2024-02-16T22:56:16.990 INFO:tasks.workunit.client.1.smithi192.stdout:5/365: link d6/d8/c63 d6/d12/c79 0 2024-02-16T22:56:16.991 INFO:tasks.workunit.client.1.smithi192.stdout:5/366: truncate d6/d12/d4e/d46/d4f/f6e 660600 0 2024-02-16T22:56:16.992 INFO:tasks.workunit.client.1.smithi192.stdout:5/367: readlink d6/d12/d4e/l6d 0 2024-02-16T22:56:16.996 INFO:tasks.workunit.client.0.smithi064.stdout:8/250: getdents df/d31/d2f/d4f 0 2024-02-16T22:56:16.998 INFO:tasks.workunit.client.1.smithi192.stdout:2/406: creat d22/d47/f88 x:0 0 0 2024-02-16T22:56:17.001 INFO:tasks.workunit.client.0.smithi064.stdout:7/237: rmdir d5/d33 39 2024-02-16T22:56:17.001 INFO:tasks.workunit.client.1.smithi192.stdout:0/350: dread d1/f24 [0,4194304] 0 2024-02-16T22:56:17.003 INFO:tasks.workunit.client.0.smithi064.stdout:8/251: dwrite df/d31/d2f/f3a [0,4194304] 0 2024-02-16T22:56:17.003 INFO:tasks.workunit.client.1.smithi192.stdout:1/333: rename d3/l6d to d3/de/l6e 0 2024-02-16T22:56:17.007 INFO:tasks.workunit.client.1.smithi192.stdout:6/350: getdents da/d40 0 2024-02-16T22:56:17.008 INFO:tasks.workunit.client.1.smithi192.stdout:6/351: stat da/df/c28 0 2024-02-16T22:56:17.008 INFO:tasks.workunit.client.0.smithi064.stdout:9/286: getdents d3 0 2024-02-16T22:56:17.009 INFO:tasks.workunit.client.0.smithi064.stdout:5/202: mkdir d6/d25/d48 0 2024-02-16T22:56:17.009 INFO:tasks.workunit.client.0.smithi064.stdout:5/203: readlink d6/d1c/l2b 0 2024-02-16T22:56:17.009 INFO:tasks.workunit.client.1.smithi192.stdout:8/346: dwrite d5/db/d47/f3b [0,4194304] 0 2024-02-16T22:56:17.010 INFO:tasks.workunit.client.1.smithi192.stdout:6/352: fdatasync da/df/d18/d52/f66 0 2024-02-16T22:56:17.011 INFO:tasks.workunit.client.1.smithi192.stdout:8/347: dread - d5/db/f64 zero size 2024-02-16T22:56:17.014 INFO:tasks.workunit.client.1.smithi192.stdout:5/368: write d6/fd [2037164,113629] 0 2024-02-16T22:56:17.016 INFO:tasks.workunit.client.0.smithi064.stdout:5/204: dwrite d6/f3c [0,4194304] 0 2024-02-16T22:56:17.018 INFO:tasks.workunit.client.1.smithi192.stdout:8/348: dwrite d5/d3a/d4d/f4b [0,4194304] 0 2024-02-16T22:56:17.019 INFO:tasks.workunit.client.1.smithi192.stdout:8/349: fsync d5/d3e/d5f/f5e 0 2024-02-16T22:56:17.028 INFO:tasks.workunit.client.1.smithi192.stdout:4/376: rename d1/d8 to d1/d26/d66/d89 0 2024-02-16T22:56:17.028 INFO:tasks.workunit.client.1.smithi192.stdout:4/377: readlink d1/d26/d5e/l79 0 2024-02-16T22:56:17.033 INFO:tasks.workunit.client.1.smithi192.stdout:1/334: unlink d3/d5b/c67 0 2024-02-16T22:56:17.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:16 smithi192 ceph-mon[36254]: pgmap v13: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 8.3 MiB/s rd, 115 MiB/s wr, 224 op/s 2024-02-16T22:56:17.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:16 smithi192 ceph-mon[36254]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:17.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:16 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2704225952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:56:17.039 INFO:tasks.workunit.client.1.smithi192.stdout:9/453: dwrite d7/d19/d28/f2c [0,4194304] 0 2024-02-16T22:56:17.040 INFO:tasks.workunit.client.1.smithi192.stdout:9/454: read - d7/d19/d37/f55 zero size 2024-02-16T22:56:17.041 INFO:tasks.workunit.client.1.smithi192.stdout:6/353: write da/df/d12/f51 [1022868,65075] 0 2024-02-16T22:56:17.042 INFO:tasks.workunit.client.1.smithi192.stdout:3/376: creat da/d11/d63/d6b/d80/f83 x:0 0 0 2024-02-16T22:56:17.046 INFO:tasks.workunit.client.1.smithi192.stdout:5/369: unlink d6/d12/d1d/l5f 0 2024-02-16T22:56:17.047 INFO:tasks.workunit.client.1.smithi192.stdout:9/455: dwrite d7/d15/d36/d38/f4b [0,4194304] 0 2024-02-16T22:56:17.047 INFO:tasks.workunit.client.0.smithi064.stdout:9/287: mknod d3/d9/c64 0 2024-02-16T22:56:17.048 INFO:tasks.workunit.client.0.smithi064.stdout:9/288: chown d3/d9/cd 241 1 2024-02-16T22:56:17.051 INFO:tasks.workunit.client.1.smithi192.stdout:2/407: mkdir d22/d24/d39/d42/d84/d89 0 2024-02-16T22:56:17.053 INFO:tasks.workunit.client.1.smithi192.stdout:8/350: mknod d5/db/d47/d43/c6a 0 2024-02-16T22:56:17.055 INFO:tasks.workunit.client.1.smithi192.stdout:4/378: mknod d1/d26/d66/d89/d11/d58/c8a 0 2024-02-16T22:56:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.056+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f543804bed0 con 0x7f543810bee0 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:56:17.058 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:56:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:56:17.060 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:56:17.061 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:56:17.061 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:56:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.057+0000 7f541f7fe700 1 -- 172.21.15.64:0/1632917819 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f542c059480 con 0x7f543810bee0 2024-02-16T22:56:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.059+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f542006f7b0 msgr2=0x7f5420071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.059+0000 7f543d781700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f542006f7b0 0x7f5420071c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f54380fc650 tx=0x7f5428009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.059+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f543810bee0 msgr2=0x7f543810cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.059+0000 7f543d781700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f543810bee0 0x7f543810cd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f542c005010 tx=0x7f542c00af40 comp rx=0 tx=0).stop 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.060+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 shutdown_connections 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.060+0000 7f543d781700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54380fb5e0 0x7f543810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.060+0000 7f543d781700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f542006f7b0 0x7f5420071c60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.060+0000 7f543d781700 1 --2- 172.21.15.64:0/1632917819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f543810bee0 0x7f543810cd80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.060+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 >> 172.21.15.64:0/1632917819 conn(0x7f54380f6ca0 msgr2=0x7f54380f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.060+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 shutdown_connections 2024-02-16T22:56:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.060+0000 7f543d781700 1 -- 172.21.15.64:0/1632917819 wait complete. 2024-02-16T22:56:17.063 INFO:tasks.workunit.client.0.smithi064.stdout:2/256: write d7/de/f15 [4006945,20115] 0 2024-02-16T22:56:17.063 INFO:tasks.workunit.client.0.smithi064.stdout:2/257: fdatasync d7/d14/d25/f32 0 2024-02-16T22:56:17.063 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:56:17.063 INFO:tasks.workunit.client.1.smithi192.stdout:8/351: dwrite d5/d9/f44 [0,4194304] 0 2024-02-16T22:56:17.064 INFO:tasks.workunit.client.0.smithi064.stdout:7/238: creat d5/d33/d45/f47 x:0 0 0 2024-02-16T22:56:17.066 INFO:tasks.workunit.client.0.smithi064.stdout:3/339: truncate d4/f11 1752414 0 2024-02-16T22:56:17.069 INFO:tasks.workunit.client.0.smithi064.stdout:0/249: write f3 [1042170,25255] 0 2024-02-16T22:56:17.072 INFO:tasks.workunit.client.0.smithi064.stdout:1/236: sync 2024-02-16T22:56:17.073 INFO:tasks.workunit.client.1.smithi192.stdout:3/377: rename da/d11/d2b/d43/f64 to da/d6a/f84 0 2024-02-16T22:56:17.075 INFO:tasks.workunit.client.1.smithi192.stdout:9/456: creat d7/d19/d28/f9f x:0 0 0 2024-02-16T22:56:17.076 INFO:tasks.workunit.client.1.smithi192.stdout:9/457: truncate d7/d19/d37/f7e 900024 0 2024-02-16T22:56:17.077 INFO:tasks.workunit.client.1.smithi192.stdout:9/458: stat d7/d15/d36/d38/d3f/d8a/d98/f16 0 2024-02-16T22:56:17.077 INFO:tasks.workunit.client.1.smithi192.stdout:9/459: stat d7/d15/d36/d54/f65 0 2024-02-16T22:56:17.078 INFO:tasks.workunit.client.0.smithi064.stdout:4/276: rename dc/dd/d16/l2d to dc/dd/l53 0 2024-02-16T22:56:17.078 INFO:tasks.workunit.client.0.smithi064.stdout:4/277: stat dc/de/d28 0 2024-02-16T22:56:17.082 INFO:tasks.workunit.client.0.smithi064.stdout:6/315: rmdir d0/d5/df/d17/d48 0 2024-02-16T22:56:17.083 INFO:tasks.workunit.client.0.smithi064.stdout:6/316: write d0/d6/d10/d33/f32 [2503581,92688] 0 2024-02-16T22:56:17.084 INFO:tasks.workunit.client.1.smithi192.stdout:4/379: creat d1/d15/d29/f8b x:0 0 0 2024-02-16T22:56:17.087 INFO:tasks.workunit.client.0.smithi064.stdout:2/258: creat d7/d14/d1e/d34/d3f/f4e x:0 0 0 2024-02-16T22:56:17.087 INFO:tasks.workunit.client.1.smithi192.stdout:1/335: symlink d3/de/d27/d34/d52/d63/l6f 0 2024-02-16T22:56:17.089 INFO:tasks.workunit.client.0.smithi064.stdout:8/252: write df/d31/f36 [136093,121463] 0 2024-02-16T22:56:17.091 INFO:tasks.workunit.client.1.smithi192.stdout:8/352: creat d5/d9/f6b x:0 0 0 2024-02-16T22:56:17.093 INFO:tasks.workunit.client.0.smithi064.stdout:3/340: creat d4/d1f/d34/f74 x:0 0 0 2024-02-16T22:56:17.094 INFO:tasks.workunit.client.0.smithi064.stdout:3/341: write d4/f18 [5010327,81755] 0 2024-02-16T22:56:17.096 INFO:tasks.workunit.client.0.smithi064.stdout:3/342: read d4/d13/f16 [2418575,51814] 0 2024-02-16T22:56:17.097 INFO:tasks.workunit.client.1.smithi192.stdout:6/354: truncate da/df/d12/f39 118722 0 2024-02-16T22:56:17.097 INFO:tasks.workunit.client.1.smithi192.stdout:8/353: dwrite d5/f61 [0,4194304] 0 2024-02-16T22:56:17.100 INFO:tasks.workunit.client.0.smithi064.stdout:3/343: dread d4/d13/d1b/d27/f36 [0,4194304] 0 2024-02-16T22:56:17.101 INFO:tasks.workunit.client.1.smithi192.stdout:2/408: dwrite d22/d24/d39/d42/f54 [0,4194304] 0 2024-02-16T22:56:17.101 INFO:tasks.workunit.client.0.smithi064.stdout:1/237: creat d2/df/d14/f52 x:0 0 0 2024-02-16T22:56:17.102 INFO:tasks.workunit.client.0.smithi064.stdout:1/238: write d2/f4f [841745,25305] 0 2024-02-16T22:56:17.104 INFO:tasks.workunit.client.1.smithi192.stdout:6/355: dwrite da/df/d12/d15/f3e [4194304,4194304] 0 2024-02-16T22:56:17.104 INFO:tasks.workunit.client.1.smithi192.stdout:7/428: sync 2024-02-16T22:56:17.105 INFO:tasks.workunit.client.1.smithi192.stdout:6/356: truncate da/f5f 30257 0 2024-02-16T22:56:17.107 INFO:tasks.workunit.client.0.smithi064.stdout:1/239: read d2/df/d14/d18/f28 [173302,10994] 0 2024-02-16T22:56:17.109 INFO:tasks.workunit.client.1.smithi192.stdout:5/370: rename d6/d12/l41 to d6/d12/d4e/d46/d4f/l7a 0 2024-02-16T22:56:17.111 INFO:tasks.workunit.client.0.smithi064.stdout:4/278: mknod dc/dd/d32/c54 0 2024-02-16T22:56:17.114 INFO:tasks.workunit.client.1.smithi192.stdout:3/378: unlink da/d11/d63/d6b/d66/l3b 0 2024-02-16T22:56:17.117 INFO:tasks.workunit.client.1.smithi192.stdout:9/460: symlink d7/d19/d5e/d6f/la0 0 2024-02-16T22:56:17.118 INFO:tasks.workunit.client.0.smithi064.stdout:7/239: dwrite d5/d6/f1d [0,4194304] 0 2024-02-16T22:56:17.119 INFO:tasks.workunit.client.0.smithi064.stdout:7/240: fdatasync d5/d33/d45/f47 0 2024-02-16T22:56:17.123 INFO:tasks.workunit.client.0.smithi064.stdout:6/317: mknod d0/d5/df/d17/c59 0 2024-02-16T22:56:17.124 INFO:tasks.workunit.client.1.smithi192.stdout:1/336: write d3/de/d27/f39 [5313913,8920] 0 2024-02-16T22:56:17.126 INFO:tasks.workunit.client.0.smithi064.stdout:8/253: creat df/d31/d2f/d38/f53 x:0 0 0 2024-02-16T22:56:17.129 INFO:tasks.workunit.client.0.smithi064.stdout:8/254: read df/f1a [2165894,46058] 0 2024-02-16T22:56:17.133 INFO:tasks.workunit.client.0.smithi064.stdout:3/344: mkdir d4/d1f/d75 0 2024-02-16T22:56:17.134 INFO:tasks.workunit.client.1.smithi192.stdout:2/409: unlink d22/l4b 0 2024-02-16T22:56:17.135 INFO:tasks.workunit.client.1.smithi192.stdout:2/410: chown d22/d24/d39 10 1 2024-02-16T22:56:17.135 INFO:tasks.workunit.client.1.smithi192.stdout:2/411: dread - d22/f41 zero size 2024-02-16T22:56:17.136 INFO:tasks.workunit.client.0.smithi064.stdout:9/289: truncate d3/d11/f1f 3075680 0 2024-02-16T22:56:17.139 INFO:tasks.workunit.client.0.smithi064.stdout:1/240: rename d2/df/d11/d1a/d30/c3f to d2/df/d11/d1a/d1d/c53 0 2024-02-16T22:56:17.142 INFO:tasks.workunit.client.1.smithi192.stdout:6/357: rename da/df/d12/f13 to da/df/d58/f6a 0 2024-02-16T22:56:17.142 INFO:tasks.workunit.client.1.smithi192.stdout:6/358: fdatasync da/df/d12/f47 0 2024-02-16T22:56:17.143 INFO:tasks.workunit.client.1.smithi192.stdout:3/379: rmdir da/d11/d63/d6b/d66 39 2024-02-16T22:56:17.144 INFO:tasks.workunit.client.1.smithi192.stdout:3/380: dread da/d11/d2b/f5f [0,4194304] 0 2024-02-16T22:56:17.144 INFO:tasks.workunit.client.0.smithi064.stdout:5/205: getdents d6/d18/d19/d1a 0 2024-02-16T22:56:17.145 INFO:tasks.workunit.client.1.smithi192.stdout:3/381: chown da/d11/d20/f30 10789349 1 2024-02-16T22:56:17.145 INFO:tasks.workunit.client.0.smithi064.stdout:5/206: write d6/d18/d19/f1f [445613,34921] 0 2024-02-16T22:56:17.146 INFO:tasks.workunit.client.0.smithi064.stdout:5/207: truncate f0 5186380 0 2024-02-16T22:56:17.149 INFO:tasks.workunit.client.0.smithi064.stdout:7/241: unlink d5/l1a 0 2024-02-16T22:56:17.150 INFO:tasks.workunit.client.0.smithi064.stdout:7/242: read - d5/d6/f43 zero size 2024-02-16T22:56:17.150 INFO:tasks.workunit.client.1.smithi192.stdout:1/337: creat d3/de/d27/d34/d52/f70 x:0 0 0 2024-02-16T22:56:17.152 INFO:tasks.workunit.client.0.smithi064.stdout:6/318: mkdir d0/d12/d5a 0 2024-02-16T22:56:17.154 INFO:tasks.workunit.client.0.smithi064.stdout:4/279: dread dc/f33 [4194304,4194304] 0 2024-02-16T22:56:17.159 INFO:tasks.workunit.client.1.smithi192.stdout:7/429: getdents d1/d2/d2d/d61/d88/d90/d4c/d60 0 2024-02-16T22:56:17.160 INFO:tasks.workunit.client.0.smithi064.stdout:8/255: creat df/d31/d2f/d38/d48/f54 x:0 0 0 2024-02-16T22:56:17.161 INFO:tasks.workunit.client.1.smithi192.stdout:7/430: dread d1/d2/d2d/d61/d88/d90/f1d [4194304,4194304] 0 2024-02-16T22:56:17.163 INFO:tasks.workunit.client.0.smithi064.stdout:3/345: dread d4/d13/d1b/d4f/f5d [0,4194304] 0 2024-02-16T22:56:17.168 INFO:tasks.workunit.client.0.smithi064.stdout:9/290: creat d3/d9/d1e/d3e/f65 x:0 0 0 2024-02-16T22:56:17.169 INFO:tasks.workunit.client.1.smithi192.stdout:3/382: creat da/d11/d20/f85 x:0 0 0 2024-02-16T22:56:17.170 INFO:tasks.workunit.client.0.smithi064.stdout:6/319: dread d0/d6/f15 [4194304,4194304] 0 2024-02-16T22:56:17.172 INFO:tasks.workunit.client.0.smithi064.stdout:6/320: write d0/d5/df/d17/d18/f3f [8357766,21830] 0 2024-02-16T22:56:17.172 INFO:tasks.workunit.client.0.smithi064.stdout:1/241: rename d2/df/d11/d1a/d30/f32 to d2/df/f54 0 2024-02-16T22:56:17.174 INFO:tasks.workunit.client.0.smithi064.stdout:1/242: truncate d2/df/d11/f3b 1008732 0 2024-02-16T22:56:17.174 INFO:tasks.workunit.client.1.smithi192.stdout:2/412: write f4 [2206228,37591] 0 2024-02-16T22:56:17.175 INFO:tasks.workunit.client.1.smithi192.stdout:4/380: link d1/d26/d66/d89/c10 d1/d15/d48/d4c/c8c 0 2024-02-16T22:56:17.176 INFO:tasks.workunit.client.1.smithi192.stdout:1/338: chown d3/de/d27/d34/d36/d47/f5e 8701 1 2024-02-16T22:56:17.177 INFO:tasks.workunit.client.0.smithi064.stdout:7/243: mknod d5/d6/c48 0 2024-02-16T22:56:17.177 INFO:tasks.workunit.client.1.smithi192.stdout:1/339: write d3/de/f65 [247741,39785] 0 2024-02-16T22:56:17.179 INFO:tasks.workunit.client.1.smithi192.stdout:8/354: link d5/d26/l42 d5/l6c 0 2024-02-16T22:56:17.182 INFO:tasks.workunit.client.0.smithi064.stdout:0/250: rename da/f44 to da/f51 0 2024-02-16T22:56:17.183 INFO:tasks.workunit.client.1.smithi192.stdout:7/431: creat d1/d2/d37/f9c x:0 0 0 2024-02-16T22:56:17.183 INFO:tasks.workunit.client.0.smithi064.stdout:8/256: creat df/d31/d2f/f55 x:0 0 0 2024-02-16T22:56:17.184 INFO:tasks.workunit.client.1.smithi192.stdout:6/359: mkdir da/df/d6b 0 2024-02-16T22:56:17.186 INFO:tasks.workunit.client.0.smithi064.stdout:3/346: write d4/d13/d41/f3b [941497,82687] 0 2024-02-16T22:56:17.191 INFO:tasks.workunit.client.1.smithi192.stdout:4/381: unlink d1/d26/d66/d89/d11/f2a 0 2024-02-16T22:56:17.192 INFO:tasks.workunit.client.0.smithi064.stdout:6/321: symlink d0/d3c/l5b 0 2024-02-16T22:56:17.193 INFO:tasks.workunit.client.1.smithi192.stdout:1/340: creat d3/d5b/f71 x:0 0 0 2024-02-16T22:56:17.194 INFO:tasks.workunit.client.0.smithi064.stdout:1/243: symlink d2/df/d11/d1a/d30/l55 0 2024-02-16T22:56:17.195 INFO:tasks.workunit.client.1.smithi192.stdout:8/355: mknod d5/d16/c6d 0 2024-02-16T22:56:17.196 INFO:tasks.workunit.client.1.smithi192.stdout:4/382: dwrite d1/d26/d66/d89/d11/f51 [0,4194304] 0 2024-02-16T22:56:17.198 INFO:tasks.workunit.client.1.smithi192.stdout:4/383: write d1/d15/d23/f4a [286252,124088] 0 2024-02-16T22:56:17.198 INFO:tasks.workunit.client.0.smithi064.stdout:5/208: mknod d6/d25/d48/c49 0 2024-02-16T22:56:17.199 INFO:tasks.workunit.client.1.smithi192.stdout:4/384: chown d1/d15/d23/l84 14 1 2024-02-16T22:56:17.200 INFO:tasks.workunit.client.1.smithi192.stdout:7/432: mkdir d1/d2/de/d2c/d49/d54/d9d 0 2024-02-16T22:56:17.200 INFO:tasks.workunit.client.0.smithi064.stdout:2/259: link l4 d7/de/l4f 0 2024-02-16T22:56:17.201 INFO:tasks.workunit.client.1.smithi192.stdout:8/356: dread d5/db/d47/f3b [0,4194304] 0 2024-02-16T22:56:17.206 INFO:tasks.workunit.client.1.smithi192.stdout:6/360: dwrite da/df/d18/f29 [0,4194304] 0 2024-02-16T22:56:17.214 INFO:tasks.workunit.client.1.smithi192.stdout:9/461: getdents d7/d15 0 2024-02-16T22:56:17.215 INFO:tasks.workunit.client.0.smithi064.stdout:4/280: creat dc/dd/d32/d46/f55 x:0 0 0 2024-02-16T22:56:17.216 INFO:tasks.workunit.client.0.smithi064.stdout:8/257: rename df/d31/d2f/d3e/l45 to df/d31/d2f/d38/d48/l56 0 2024-02-16T22:56:17.217 INFO:tasks.workunit.client.0.smithi064.stdout:9/291: fsync d3/d11/d15/d5a/d2a/f4b 0 2024-02-16T22:56:17.218 INFO:tasks.workunit.client.0.smithi064.stdout:0/251: read da/f10 [6447242,56046] 0 2024-02-16T22:56:17.218 INFO:tasks.workunit.client.0.smithi064.stdout:8/258: dread - df/d31/d2f/d38/f46 zero size 2024-02-16T22:56:17.219 INFO:tasks.workunit.client.0.smithi064.stdout:8/259: chown df/f24 327193605 1 2024-02-16T22:56:17.224 INFO:tasks.workunit.client.0.smithi064.stdout:3/347: symlink d4/d13/d1b/d27/d61/d63/l76 0 2024-02-16T22:56:17.224 INFO:tasks.workunit.client.1.smithi192.stdout:1/341: mkdir d3/de/d27/d34/d52/d72 0 2024-02-16T22:56:17.225 INFO:tasks.workunit.client.0.smithi064.stdout:3/348: chown d4/cf 21260265 1 2024-02-16T22:56:17.229 INFO:tasks.workunit.client.0.smithi064.stdout:4/281: dread dc/d1d/f41 [0,4194304] 0 2024-02-16T22:56:17.229 INFO:tasks.workunit.client.0.smithi064.stdout:1/244: mkdir d2/d4e/d56 0 2024-02-16T22:56:17.230 INFO:tasks.workunit.client.1.smithi192.stdout:7/433: rename d1/l79 to d1/d1f/l9e 0 2024-02-16T22:56:17.231 INFO:tasks.workunit.client.0.smithi064.stdout:2/260: creat d7/d14/d1e/d34/f50 x:0 0 0 2024-02-16T22:56:17.232 INFO:tasks.workunit.client.1.smithi192.stdout:3/383: link da/d11/d20/f69 da/d11/d63/d6b/d66/f86 0 2024-02-16T22:56:17.232 INFO:tasks.workunit.client.0.smithi064.stdout:8/260: dwrite df/d31/f36 [0,4194304] 0 2024-02-16T22:56:17.234 INFO:tasks.workunit.client.0.smithi064.stdout:9/292: dread d3/d9/d1e/d42/f54 [0,4194304] 0 2024-02-16T22:56:17.234 INFO:tasks.workunit.client.0.smithi064.stdout:0/252: dread - da/d26/f32 zero size 2024-02-16T22:56:17.235 INFO:tasks.workunit.client.1.smithi192.stdout:0/351: sync 2024-02-16T22:56:17.236 INFO:tasks.workunit.client.1.smithi192.stdout:2/413: creat d22/d24/d2b/f8a x:0 0 0 2024-02-16T22:56:17.242 INFO:tasks.workunit.client.0.smithi064.stdout:2/261: dwrite d7/d14/d1e/d34/d3f/f43 [0,4194304] 0 2024-02-16T22:56:17.245 INFO:tasks.workunit.client.1.smithi192.stdout:4/385: dread d1/d6b/f7f [0,4194304] 0 2024-02-16T22:56:17.246 INFO:tasks.workunit.client.0.smithi064.stdout:9/293: dwrite d3/d9/f60 [0,4194304] 0 2024-02-16T22:56:17.248 INFO:tasks.workunit.client.0.smithi064.stdout:3/349: dread - d4/d1f/f32 zero size 2024-02-16T22:56:17.249 INFO:tasks.workunit.client.1.smithi192.stdout:1/342: unlink d3/de/d27/l2e 0 2024-02-16T22:56:17.250 INFO:tasks.workunit.client.0.smithi064.stdout:4/282: readlink dc/dd/l53 0 2024-02-16T22:56:17.252 INFO:tasks.workunit.client.1.smithi192.stdout:7/434: mkdir d1/d2/de/d83/d9f 0 2024-02-16T22:56:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.253+0000 7f67c488b700 1 -- 172.21.15.64:0/4005235428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 msgr2=0x7f67c010b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.253+0000 7f67c488b700 1 --2- 172.21.15.64:0/4005235428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 0x7f67c010b1c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67b0004940 tx=0x7f67b0009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:17.255 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:16 smithi064 ceph-mon[29226]: pgmap v13: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 8.3 MiB/s rd, 115 MiB/s wr, 224 op/s 2024-02-16T22:56:17.255 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:16 smithi064 ceph-mon[29226]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:17.255 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:16 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2704225952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:56:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.254+0000 7f67c488b700 1 -- 172.21.15.64:0/4005235428 shutdown_connections 2024-02-16T22:56:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.254+0000 7f67c488b700 1 --2- 172.21.15.64:0/4005235428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 0x7f67c010b1c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.254+0000 7f67c488b700 1 --2- 172.21.15.64:0/4005235428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67c006e560 0x7f67c006e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.254+0000 7f67c488b700 1 -- 172.21.15.64:0/4005235428 >> 172.21.15.64:0/4005235428 conn(0x7f67c0069e70 msgr2=0x7f67c006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:17.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.254+0000 7f67c488b700 1 -- 172.21.15.64:0/4005235428 shutdown_connections 2024-02-16T22:56:17.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.254+0000 7f67c488b700 1 -- 172.21.15.64:0/4005235428 wait complete. 2024-02-16T22:56:17.256 INFO:tasks.workunit.client.0.smithi064.stdout:9/294: dwrite d3/d11/d15/d5a/f27 [0,4194304] 0 2024-02-16T22:56:17.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.255+0000 7f67c488b700 1 Processor -- start 2024-02-16T22:56:17.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.255+0000 7f67c488b700 1 -- start start 2024-02-16T22:56:17.257 INFO:tasks.workunit.client.1.smithi192.stdout:3/384: dread da/d11/d2b/f5f [0,4194304] 0 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.255+0000 7f67c488b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67c006e560 0x7f67c011c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.255+0000 7f67c488b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 0x7f67c011cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.255+0000 7f67c488b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c011d1c0 con 0x7f67c006eeb0 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.255+0000 7f67c488b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c011da60 con 0x7f67c006e560 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.256+0000 7f67bdd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 0x7f67c011cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.256+0000 7f67bdd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 0x7f67c011cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43318/0 (socket says 172.21.15.64:43318) 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.256+0000 7f67bdd9b700 1 -- 172.21.15.64:0/993055064 learned_addr learned my addr 172.21.15.64:0/993055064 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.256+0000 7f67be59c700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67c006e560 0x7f67c011c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.256+0000 7f67bdd9b700 1 -- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67c006e560 msgr2=0x7f67c011c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.256+0000 7f67bdd9b700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67c006e560 0x7f67c011c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.257+0000 7f67bdd9b700 1 -- 172.21.15.64:0/993055064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0005040 con 0x7f67c006eeb0 2024-02-16T22:56:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.257+0000 7f67bdd9b700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 0x7f67c011cbc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f67b00040d0 tx=0x7f67b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.257+0000 7f67aeffd700 1 -- 172.21.15.64:0/993055064 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b000e070 con 0x7f67c006eeb0 2024-02-16T22:56:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.257+0000 7f67aeffd700 1 -- 172.21.15.64:0/993055064 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67b000bbf0 con 0x7f67c006eeb0 2024-02-16T22:56:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.257+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c011dce0 con 0x7f67c006eeb0 2024-02-16T22:56:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.257+0000 7f67aeffd700 1 -- 172.21.15.64:0/993055064 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b001c640 con 0x7f67c006eeb0 2024-02-16T22:56:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.258+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c011e1c0 con 0x7f67c006eeb0 2024-02-16T22:56:17.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.259+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c010a8b0 con 0x7f67c006eeb0 2024-02-16T22:56:17.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.260+0000 7f67aeffd700 1 -- 172.21.15.64:0/993055064 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f67b0013420 con 0x7f67c006eeb0 2024-02-16T22:56:17.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.260+0000 7f67aeffd700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f67a806b3a0 0x7f67a806d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:17.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.261+0000 7f67aeffd700 1 -- 172.21.15.64:0/993055064 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f67b008b5a0 con 0x7f67c006eeb0 2024-02-16T22:56:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.261+0000 7f67be59c700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f67a806b3a0 0x7f67a806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:17.262 INFO:tasks.workunit.client.0.smithi064.stdout:0/253: rmdir da/dc/de/d37 39 2024-02-16T22:56:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.262+0000 7f67be59c700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f67a806b3a0 0x7f67a806d850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67b8004fa0 tx=0x7f67b800b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:17.264 INFO:tasks.workunit.client.1.smithi192.stdout:0/352: mknod d1/d3/de/d22/d37/d67/c6b 0 2024-02-16T22:56:17.267 INFO:tasks.workunit.client.1.smithi192.stdout:2/414: rename d22/d24/d39/d42/d65/c74 to d22/d24/c8b 0 2024-02-16T22:56:17.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.270+0000 7f67aeffd700 1 -- 172.21.15.64:0/993055064 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f67b00591d0 con 0x7f67c006eeb0 2024-02-16T22:56:17.272 INFO:tasks.workunit.client.0.smithi064.stdout:2/262: rmdir d7/d14/d25/d37/d44 39 2024-02-16T22:56:17.273 INFO:tasks.workunit.client.0.smithi064.stdout:2/263: write d7/de/f15 [3246153,92508] 0 2024-02-16T22:56:17.276 INFO:tasks.workunit.client.1.smithi192.stdout:4/386: rename d1/d15/d48/d4c to d1/d15/d48/d8d 0 2024-02-16T22:56:17.278 INFO:tasks.workunit.client.1.smithi192.stdout:4/387: dread d1/d15/d48/f55 [4194304,4194304] 0 2024-02-16T22:56:17.279 INFO:tasks.workunit.client.1.smithi192.stdout:9/462: write d7/fe [1760356,56679] 0 2024-02-16T22:56:17.280 INFO:tasks.workunit.client.0.smithi064.stdout:3/350: symlink d4/d1f/d5b/d51/l77 0 2024-02-16T22:56:17.281 INFO:tasks.workunit.client.0.smithi064.stdout:2/264: dwrite d7/d14/d1e/d34/f50 [0,4194304] 0 2024-02-16T22:56:17.281 INFO:tasks.workunit.client.0.smithi064.stdout:6/322: truncate d0/d5/df/d17/d18/f3f 5722545 0 2024-02-16T22:56:17.282 INFO:tasks.workunit.client.0.smithi064.stdout:6/323: write d0/d1d/f58 [327579,61802] 0 2024-02-16T22:56:17.286 INFO:tasks.workunit.client.0.smithi064.stdout:4/283: rename dc/d1d/f41 to dc/dd/d32/d22/f56 0 2024-02-16T22:56:17.290 INFO:tasks.workunit.client.0.smithi064.stdout:1/245: creat d2/d4e/d56/f57 x:0 0 0 2024-02-16T22:56:17.290 INFO:tasks.workunit.client.0.smithi064.stdout:1/246: chown d2/f36 141025 1 2024-02-16T22:56:17.291 INFO:tasks.workunit.client.1.smithi192.stdout:6/361: link da/l14 da/d40/l6c 0 2024-02-16T22:56:17.293 INFO:tasks.workunit.client.1.smithi192.stdout:0/353: symlink d1/d3/de/d22/d37/d69/l6c 0 2024-02-16T22:56:17.294 INFO:tasks.workunit.client.1.smithi192.stdout:0/354: chown d1/f1f 70 1 2024-02-16T22:56:17.296 INFO:tasks.workunit.client.0.smithi064.stdout:1/247: dwrite d2/df/d14/f52 [0,4194304] 0 2024-02-16T22:56:17.296 INFO:tasks.workunit.client.1.smithi192.stdout:2/415: creat d22/d24/d39/d42/d84/f8c x:0 0 0 2024-02-16T22:56:17.299 INFO:tasks.workunit.client.1.smithi192.stdout:4/388: unlink d1/d15/d29/c75 0 2024-02-16T22:56:17.300 INFO:tasks.workunit.client.0.smithi064.stdout:1/248: chown d2/df/d11/d1a/d1d/l2b 702 1 2024-02-16T22:56:17.302 INFO:tasks.workunit.client.0.smithi064.stdout:3/351: mkdir d4/d13/d4d/d5c/d78 0 2024-02-16T22:56:17.302 INFO:tasks.workunit.client.1.smithi192.stdout:4/389: dread d1/d15/d23/f5c [0,4194304] 0 2024-02-16T22:56:17.306 INFO:tasks.workunit.client.0.smithi064.stdout:4/284: mkdir dc/de/d28/d57 0 2024-02-16T22:56:17.311 INFO:tasks.workunit.client.1.smithi192.stdout:7/435: unlink d1/d2/d2d/d61/d88/d90/l5c 0 2024-02-16T22:56:17.314 INFO:tasks.workunit.client.0.smithi064.stdout:0/254: rename da/d26/l48 to da/dc/de/d12/d2c/d2e/d38/l52 0 2024-02-16T22:56:17.321 INFO:tasks.workunit.client.1.smithi192.stdout:2/416: mknod d22/d24/d39/d42/d85/c8d 0 2024-02-16T22:56:17.325 INFO:tasks.workunit.client.1.smithi192.stdout:4/390: symlink d1/d15/l8e 0 2024-02-16T22:56:17.327 INFO:tasks.workunit.client.0.smithi064.stdout:1/249: rename d2/df/d14/f39 to d2/df/d11/d1a/d1d/f58 0 2024-02-16T22:56:17.327 INFO:tasks.workunit.client.1.smithi192.stdout:2/417: dwrite d22/d24/d2b/d2c/f2e [0,4194304] 0 2024-02-16T22:56:17.327 INFO:tasks.workunit.client.1.smithi192.stdout:2/418: stat d22/d47/f7d 0 2024-02-16T22:56:17.328 INFO:tasks.workunit.client.1.smithi192.stdout:9/463: link d7/d19/d37/d39/f44 d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/fa1 0 2024-02-16T22:56:17.329 INFO:tasks.workunit.client.0.smithi064.stdout:0/255: symlink da/dc/de/d12/d2c/l53 0 2024-02-16T22:56:17.331 INFO:tasks.workunit.client.1.smithi192.stdout:3/385: creat da/d11/d63/d6b/f87 x:0 0 0 2024-02-16T22:56:17.338 INFO:tasks.workunit.client.0.smithi064.stdout:4/285: rename dc/d10 to dc/dd/d32/d58 0 2024-02-16T22:56:17.338 INFO:tasks.workunit.client.1.smithi192.stdout:5/371: sync 2024-02-16T22:56:17.338 INFO:tasks.workunit.client.1.smithi192.stdout:8/357: sync 2024-02-16T22:56:17.339 INFO:tasks.workunit.client.1.smithi192.stdout:5/372: fsync d6/d8/f3e 0 2024-02-16T22:56:17.340 INFO:tasks.workunit.client.1.smithi192.stdout:8/358: dread - d5/d3e/d5f/f5e zero size 2024-02-16T22:56:17.340 INFO:tasks.workunit.client.1.smithi192.stdout:1/343: link d3/de/c4c d3/de/d27/c73 0 2024-02-16T22:56:17.340 INFO:tasks.workunit.client.0.smithi064.stdout:4/286: read f3 [1442409,89182] 0 2024-02-16T22:56:17.342 INFO:tasks.workunit.client.0.smithi064.stdout:0/256: unlink da/f51 0 2024-02-16T22:56:17.346 INFO:tasks.workunit.client.0.smithi064.stdout:9/295: getdents d3/d11/d15/d5a/d38 0 2024-02-16T22:56:17.347 INFO:tasks.workunit.client.0.smithi064.stdout:1/250: dwrite d2/d4e/f45 [0,4194304] 0 2024-02-16T22:56:17.351 INFO:tasks.workunit.client.1.smithi192.stdout:9/464: mkdir d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/da2 0 2024-02-16T22:56:17.353 INFO:tasks.workunit.client.0.smithi064.stdout:0/257: dwrite da/d26/d3f/f47 [0,4194304] 0 2024-02-16T22:56:17.356 INFO:tasks.workunit.client.1.smithi192.stdout:6/362: truncate da/df/d12/d15/d1b/f1e 129893 0 2024-02-16T22:56:17.357 INFO:tasks.workunit.client.1.smithi192.stdout:2/419: dread d22/d24/d39/d42/d65/d69/f68 [0,4194304] 0 2024-02-16T22:56:17.358 INFO:tasks.workunit.client.1.smithi192.stdout:3/386: rmdir da/d11/d63/d6b/d4e 39 2024-02-16T22:56:17.359 INFO:tasks.workunit.client.1.smithi192.stdout:3/387: dread - da/d11/d20/f72 zero size 2024-02-16T22:56:17.363 INFO:tasks.workunit.client.0.smithi064.stdout:0/258: dwrite da/d26/d3f/f47 [0,4194304] 0 2024-02-16T22:56:17.363 INFO:tasks.workunit.client.1.smithi192.stdout:0/355: creat d1/d3/f6d x:0 0 0 2024-02-16T22:56:17.364 INFO:tasks.workunit.client.1.smithi192.stdout:0/356: chown d1/d3/de/c5b 51935729 1 2024-02-16T22:56:17.370 INFO:tasks.workunit.client.0.smithi064.stdout:9/296: creat d3/d11/d15/d5a/d38/f66 x:0 0 0 2024-02-16T22:56:17.372 INFO:tasks.workunit.client.0.smithi064.stdout:9/297: dread d3/d9/d1e/d42/f54 [0,4194304] 0 2024-02-16T22:56:17.374 INFO:tasks.workunit.client.0.smithi064.stdout:1/251: creat d2/df/d14/f59 x:0 0 0 2024-02-16T22:56:17.375 INFO:tasks.workunit.client.0.smithi064.stdout:3/352: getdents d4/d13/d1b/d27/d44 0 2024-02-16T22:56:17.377 INFO:tasks.workunit.client.1.smithi192.stdout:4/391: creat d1/d26/d6f/f8f x:0 0 0 2024-02-16T22:56:17.379 INFO:tasks.workunit.client.1.smithi192.stdout:9/465: symlink d7/d19/d5e/d6f/la3 0 2024-02-16T22:56:17.379 INFO:tasks.workunit.client.1.smithi192.stdout:9/466: write d7/f12 [961654,10645] 0 2024-02-16T22:56:17.380 INFO:tasks.workunit.client.1.smithi192.stdout:9/467: chown f2 918613659 1 2024-02-16T22:56:17.380 INFO:tasks.workunit.client.0.smithi064.stdout:0/259: rmdir da/d26/d3f 39 2024-02-16T22:56:17.382 INFO:tasks.workunit.client.0.smithi064.stdout:0/260: chown da/d26/f32 72234234 1 2024-02-16T22:56:17.383 INFO:tasks.workunit.client.0.smithi064.stdout:0/261: truncate da/d26/f32 747219 0 2024-02-16T22:56:17.384 INFO:tasks.workunit.client.0.smithi064.stdout:6/324: truncate d0/d5/df/d17/d18/f3f 5946965 0 2024-02-16T22:56:17.385 INFO:tasks.workunit.client.0.smithi064.stdout:6/325: chown d0/d5/d54 4 1 2024-02-16T22:56:17.388 INFO:tasks.workunit.client.0.smithi064.stdout:9/298: symlink d3/d9/d1e/l67 0 2024-02-16T22:56:17.389 INFO:tasks.workunit.client.1.smithi192.stdout:8/359: dwrite d5/db/d47/f3b [4194304,4194304] 0 2024-02-16T22:56:17.389 INFO:tasks.workunit.client.0.smithi064.stdout:9/299: readlink d3/d11/d15/d5a/d2a/d2e/l57 0 2024-02-16T22:56:17.389 INFO:tasks.workunit.client.1.smithi192.stdout:8/360: write d5/f38 [240276,1554] 0 2024-02-16T22:56:17.390 INFO:tasks.workunit.client.0.smithi064.stdout:1/252: rename d2/f4f to d2/d4e/f5a 0 2024-02-16T22:56:17.391 INFO:tasks.workunit.client.1.smithi192.stdout:5/373: creat d6/d8/d75/f7b x:0 0 0 2024-02-16T22:56:17.391 INFO:tasks.workunit.client.0.smithi064.stdout:1/253: write d2/df/d11/d1a/f34 [1158968,50343] 0 2024-02-16T22:56:17.392 INFO:tasks.workunit.client.0.smithi064.stdout:3/353: mknod d4/d13/d1b/d27/d44/c79 0 2024-02-16T22:56:17.394 INFO:tasks.workunit.client.1.smithi192.stdout:8/361: dread d5/d3e/f41 [4194304,4194304] 0 2024-02-16T22:56:17.394 INFO:tasks.workunit.client.1.smithi192.stdout:1/344: rename d3/c10 to d3/d5b/c74 0 2024-02-16T22:56:17.394 INFO:tasks.workunit.client.1.smithi192.stdout:8/362: dread - d5/db/f64 zero size 2024-02-16T22:56:17.396 INFO:tasks.workunit.client.1.smithi192.stdout:4/392: fsync d1/d15/d29/d3c/f43 0 2024-02-16T22:56:17.397 INFO:tasks.workunit.client.1.smithi192.stdout:4/393: write d1/d26/d66/d89/f1e [1441222,12029] 0 2024-02-16T22:56:17.397 INFO:tasks.workunit.client.0.smithi064.stdout:4/287: creat dc/dd/f59 x:0 0 0 2024-02-16T22:56:17.398 INFO:tasks.workunit.client.1.smithi192.stdout:4/394: fdatasync d1/d15/d48/d8d/f7b 0 2024-02-16T22:56:17.399 INFO:tasks.workunit.client.1.smithi192.stdout:4/395: truncate d1/d26/d66/d89/f68 205214 0 2024-02-16T22:56:17.400 INFO:tasks.workunit.client.0.smithi064.stdout:4/288: write dc/dd/d32/f1f [346183,117490] 0 2024-02-16T22:56:17.401 INFO:tasks.workunit.client.1.smithi192.stdout:9/468: symlink d7/d15/d36/d38/d3f/d8a/d98/la4 0 2024-02-16T22:56:17.403 INFO:tasks.workunit.client.1.smithi192.stdout:7/436: getdents d1 0 2024-02-16T22:56:17.406 INFO:tasks.workunit.client.1.smithi192.stdout:6/363: mknod da/df/d18/c6d 0 2024-02-16T22:56:17.409 INFO:tasks.workunit.client.0.smithi064.stdout:3/354: mknod d4/d13/d1b/d48/c7a 0 2024-02-16T22:56:17.410 INFO:tasks.workunit.client.0.smithi064.stdout:3/355: fdatasync d4/f42 0 2024-02-16T22:56:17.414 INFO:tasks.workunit.client.0.smithi064.stdout:0/262: mknod da/dc/de/d12/c54 0 2024-02-16T22:56:17.417 INFO:tasks.workunit.client.0.smithi064.stdout:6/326: symlink d0/d6/l5c 0 2024-02-16T22:56:17.420 INFO:tasks.workunit.client.0.smithi064.stdout:4/289: mkdir dc/dd/d32/d46/d5a 0 2024-02-16T22:56:17.426 INFO:tasks.workunit.client.0.smithi064.stdout:9/300: dwrite d3/f10 [0,4194304] 0 2024-02-16T22:56:17.426 INFO:tasks.workunit.client.1.smithi192.stdout:3/388: rename da/d11/d20/f72 to da/d11/d2b/d43/f88 0 2024-02-16T22:56:17.426 INFO:tasks.workunit.client.1.smithi192.stdout:2/420: dwrite d22/d24/f49 [0,4194304] 0 2024-02-16T22:56:17.427 INFO:tasks.workunit.client.0.smithi064.stdout:9/301: write d3/d11/d15/d5a/f56 [4038675,37726] 0 2024-02-16T22:56:17.431 INFO:tasks.workunit.client.0.smithi064.stdout:4/290: dread f3 [0,4194304] 0 2024-02-16T22:56:17.511 INFO:tasks.workunit.client.0.smithi064.stdout:3/356: mkdir d4/d1f/d34/d7b 0 2024-02-16T22:56:17.512 INFO:tasks.workunit.client.0.smithi064.stdout:3/357: write d4/d13/d41/f3b [262582,1298] 0 2024-02-16T22:56:17.517 INFO:tasks.workunit.client.1.smithi192.stdout:9/469: creat d7/d19/d5b/fa5 x:0 0 0 2024-02-16T22:56:17.517 INFO:tasks.workunit.client.1.smithi192.stdout:9/470: chown d7/cc 36373554 1 2024-02-16T22:56:17.517 INFO:tasks.workunit.client.1.smithi192.stdout:9/471: stat f0 0 2024-02-16T22:56:17.518 INFO:tasks.workunit.client.0.smithi064.stdout:0/263: mknod da/dc/de/d12/d1d/c55 0 2024-02-16T22:56:17.521 INFO:tasks.workunit.client.0.smithi064.stdout:6/327: mknod d0/c5d 0 2024-02-16T22:56:17.525 INFO:tasks.workunit.client.0.smithi064.stdout:1/254: truncate d2/df/d14/f52 3320994 0 2024-02-16T22:56:17.526 INFO:tasks.workunit.client.0.smithi064.stdout:1/255: chown d2/df/d14/f2d 150504 1 2024-02-16T22:56:17.546 INFO:tasks.workunit.client.1.smithi192.stdout:4/396: rename d1/d26/d66/d89/d11/f4e to d1/d26/d66/d89/d11/d58/f90 0 2024-02-16T22:56:17.548 INFO:tasks.workunit.client.0.smithi064.stdout:4/291: mkdir dc/dd/d5b 0 2024-02-16T22:56:17.549 INFO:tasks.workunit.client.0.smithi064.stdout:4/292: write dc/f17 [2731598,10294] 0 2024-02-16T22:56:17.552 INFO:tasks.workunit.client.1.smithi192.stdout:2/421: creat d22/d24/d39/d42/d65/f8e x:0 0 0 2024-02-16T22:56:17.561 INFO:tasks.workunit.client.0.smithi064.stdout:4/293: creat dc/dd/d4f/f5c x:0 0 0 2024-02-16T22:56:17.561 INFO:tasks.workunit.client.0.smithi064.stdout:4/294: stat dc/f33 0 2024-02-16T22:56:17.576 INFO:tasks.workunit.client.1.smithi192.stdout:8/363: link d5/d3e/d5f/f5e d5/d3e/f6e 0 2024-02-16T22:56:17.576 INFO:tasks.workunit.client.1.smithi192.stdout:8/364: chown d5/d3a 1284 1 2024-02-16T22:56:17.576 INFO:tasks.workunit.client.1.smithi192.stdout:9/472: mknod d7/d15/d36/d38/d3f/d8a/d98/d6e/ca6 0 2024-02-16T22:56:17.580 INFO:tasks.workunit.client.1.smithi192.stdout:0/357: getdents d1/d3/d39 0 2024-02-16T22:56:17.591 INFO:tasks.workunit.client.1.smithi192.stdout:2/422: mknod d22/d24/d39/d42/d65/d69/d4f/c8f 0 2024-02-16T22:56:17.593 INFO:tasks.workunit.client.1.smithi192.stdout:8/365: mknod d5/db/d47/d43/d67/c6f 0 2024-02-16T22:56:17.596 INFO:tasks.workunit.client.1.smithi192.stdout:9/473: unlink d7/d15/d36/d38/f4b 0 2024-02-16T22:56:17.597 INFO:tasks.workunit.client.1.smithi192.stdout:6/364: creat da/df/f6e x:0 0 0 2024-02-16T22:56:17.600 INFO:tasks.workunit.client.1.smithi192.stdout:0/358: rename d1/f15 to d1/d3/de/d22/d37/d67/d34/f6e 0 2024-02-16T22:56:17.600 INFO:tasks.workunit.client.1.smithi192.stdout:5/374: getdents d6/d12/d1d/d56 0 2024-02-16T22:56:17.601 INFO:tasks.workunit.client.1.smithi192.stdout:0/359: write d1/fd [9029083,92983] 0 2024-02-16T22:56:17.607 INFO:tasks.workunit.client.1.smithi192.stdout:2/423: dread d22/f31 [0,4194304] 0 2024-02-16T22:56:17.608 INFO:tasks.workunit.client.1.smithi192.stdout:8/366: unlink d5/db/de/f5d 0 2024-02-16T22:56:17.610 INFO:tasks.workunit.client.1.smithi192.stdout:9/474: creat d7/d19/d37/d5a/fa7 x:0 0 0 2024-02-16T22:56:17.610 INFO:tasks.workunit.client.1.smithi192.stdout:6/365: mknod da/df/d12/d15/d1b/c6f 0 2024-02-16T22:56:17.613 INFO:tasks.workunit.client.1.smithi192.stdout:5/375: symlink d6/d8/d75/l7c 0 2024-02-16T22:56:17.615 INFO:tasks.workunit.client.1.smithi192.stdout:0/360: creat d1/d3/d18/f6f x:0 0 0 2024-02-16T22:56:17.616 INFO:tasks.workunit.client.1.smithi192.stdout:3/389: getdents da/d11/d20/d39/d53 0 2024-02-16T22:56:17.622 INFO:tasks.workunit.client.1.smithi192.stdout:8/367: symlink d5/d3e/l70 0 2024-02-16T22:56:17.624 INFO:tasks.workunit.client.1.smithi192.stdout:8/368: truncate d5/d26/f3f 4862633 0 2024-02-16T22:56:17.624 INFO:tasks.workunit.client.1.smithi192.stdout:9/475: mknod d7/d19/d5e/d6f/ca8 0 2024-02-16T22:56:17.625 INFO:tasks.workunit.client.1.smithi192.stdout:2/424: dwrite d22/d24/d39/d42/d65/d69/f7a [0,4194304] 0 2024-02-16T22:56:17.625 INFO:tasks.workunit.client.1.smithi192.stdout:9/476: write d7/d15/f6a [4434710,125483] 0 2024-02-16T22:56:17.634 INFO:tasks.workunit.client.1.smithi192.stdout:9/477: mknod d7/d19/d37/d5c/ca9 0 2024-02-16T22:56:17.636 INFO:tasks.workunit.client.1.smithi192.stdout:8/369: dread d5/db/de/f37 [0,4194304] 0 2024-02-16T22:56:17.638 INFO:tasks.workunit.client.1.smithi192.stdout:9/478: dread d7/d15/d36/f7b [0,4194304] 0 2024-02-16T22:56:17.639 INFO:tasks.workunit.client.1.smithi192.stdout:9/479: chown d7/d19/d5b/c66 11 1 2024-02-16T22:56:17.640 INFO:tasks.workunit.client.1.smithi192.stdout:9/480: write d7/d19/d28/f53 [1582270,36062] 0 2024-02-16T22:56:17.641 INFO:tasks.workunit.client.1.smithi192.stdout:8/370: dwrite d5/f1e [4194304,4194304] 0 2024-02-16T22:56:17.642 INFO:tasks.workunit.client.1.smithi192.stdout:8/371: write d5/f61 [3498358,45363] 0 2024-02-16T22:56:17.646 INFO:tasks.workunit.client.0.smithi064.stdout:0/264: rename da/dc/de/cf to da/dc/de/d12/d2c/d2e/c56 0 2024-02-16T22:56:17.646 INFO:tasks.workunit.client.1.smithi192.stdout:0/361: link d1/d3/de/d22/d37/d67/f42 d1/d3/d29/f70 0 2024-02-16T22:56:17.653 INFO:tasks.workunit.client.1.smithi192.stdout:9/481: readlink d7/d19/l2b 0 2024-02-16T22:56:17.656 INFO:tasks.workunit.client.0.smithi064.stdout:0/265: symlink da/dc/de/d12/d1d/l57 0 2024-02-16T22:56:17.657 INFO:tasks.workunit.client.0.smithi064.stdout:0/266: stat da/dc/de/d12/d2c/d2e/f3e 0 2024-02-16T22:56:17.658 INFO:tasks.workunit.client.1.smithi192.stdout:7/437: write d1/d2/de/f20 [838165,70105] 0 2024-02-16T22:56:17.660 INFO:tasks.workunit.client.1.smithi192.stdout:8/372: mkdir d5/d3a/d4d/d71 0 2024-02-16T22:56:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.662+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67c002b340 con 0x7f67a806b3a0 2024-02-16T22:56:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.663+0000 7f67aeffd700 1 -- 172.21.15.64:0/993055064 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f67c002b340 con 0x7f67a806b3a0 2024-02-16T22:56:17.664 INFO:tasks.workunit.client.0.smithi064.stdout:0/267: symlink da/dc/de/d12/d2c/l58 0 2024-02-16T22:56:17.664 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:56:17.664 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:56:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:56:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:56:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [], 2024-02-16T22:56:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "1/21 daemons upgraded", 2024-02-16T22:56:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "", 2024-02-16T22:56:17.665 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:56:17.665 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:56:17.667 INFO:tasks.workunit.client.1.smithi192.stdout:7/438: truncate d1/d2/d2d/f40 1121764 0 2024-02-16T22:56:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f67a806b3a0 msgr2=0x7f67a806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f67a806b3a0 0x7f67a806d850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67b8004fa0 tx=0x7f67b800b040 comp rx=0 tx=0).stop 2024-02-16T22:56:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 msgr2=0x7f67c011cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 0x7f67c011cbc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f67b00040d0 tx=0x7f67b0009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 shutdown_connections 2024-02-16T22:56:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f67a806b3a0 0x7f67a806d850 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67c006e560 0x7f67c011c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 --2- 172.21.15.64:0/993055064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c006eeb0 0x7f67c011cbc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 >> 172.21.15.64:0/993055064 conn(0x7f67c0069e70 msgr2=0x7f67c0107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.666+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 shutdown_connections 2024-02-16T22:56:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.668+0000 7f67c488b700 1 -- 172.21.15.64:0/993055064 wait complete. 2024-02-16T22:56:17.669 INFO:tasks.workunit.client.0.smithi064.stdout:0/268: symlink da/dc/de/d12/d2c/l59 0 2024-02-16T22:56:17.671 INFO:tasks.workunit.client.1.smithi192.stdout:9/482: dwrite d7/d15/d36/d38/d3f/d8a/d98/f62 [0,4194304] 0 2024-02-16T22:56:17.671 INFO:tasks.workunit.client.1.smithi192.stdout:9/483: stat d7/d19/d5e/d6f/f7d 0 2024-02-16T22:56:17.674 INFO:tasks.workunit.client.0.smithi064.stdout:0/269: mkdir da/dc/d3b/d5a 0 2024-02-16T22:56:17.675 INFO:tasks.workunit.client.1.smithi192.stdout:7/439: write d1/d2/de/d2c/d49/d54/f67 [1427850,106285] 0 2024-02-16T22:56:17.679 INFO:tasks.workunit.client.1.smithi192.stdout:8/373: dread d5/f38 [0,4194304] 0 2024-02-16T22:56:17.681 INFO:tasks.workunit.client.0.smithi064.stdout:0/270: symlink da/dc/de/d12/d2c/d2e/l5b 0 2024-02-16T22:56:17.685 INFO:tasks.workunit.client.0.smithi064.stdout:3/358: write d4/d14/f46 [4658813,79852] 0 2024-02-16T22:56:17.687 INFO:tasks.workunit.client.1.smithi192.stdout:8/374: creat d5/d3a/f72 x:0 0 0 2024-02-16T22:56:17.688 INFO:tasks.workunit.client.0.smithi064.stdout:1/256: write d2/df/f13 [4633677,55081] 0 2024-02-16T22:56:17.688 INFO:tasks.workunit.client.1.smithi192.stdout:7/440: dread d1/d2/f1e [0,4194304] 0 2024-02-16T22:56:17.689 INFO:tasks.workunit.client.1.smithi192.stdout:7/441: write d1/d2/f25 [938193,90195] 0 2024-02-16T22:56:17.691 INFO:tasks.workunit.client.1.smithi192.stdout:7/442: write d1/d2/d2d/d61/d88/d90/d4c/f6b [3743611,119864] 0 2024-02-16T22:56:17.691 INFO:tasks.workunit.client.1.smithi192.stdout:0/362: link d1/d3/f12 d1/d3/de/d22/d37/f71 0 2024-02-16T22:56:17.692 INFO:tasks.workunit.client.1.smithi192.stdout:0/363: write d1/d3/de/f10 [2087371,24975] 0 2024-02-16T22:56:17.693 INFO:tasks.workunit.client.1.smithi192.stdout:0/364: stat d1/d3/de/l52 0 2024-02-16T22:56:17.694 INFO:tasks.workunit.client.0.smithi064.stdout:1/257: mkdir d2/d4e/d56/d5b 0 2024-02-16T22:56:17.695 INFO:tasks.workunit.client.1.smithi192.stdout:8/375: creat d5/d9/f73 x:0 0 0 2024-02-16T22:56:17.696 INFO:tasks.workunit.client.0.smithi064.stdout:3/359: truncate d4/d13/f55 212686 0 2024-02-16T22:56:17.702 INFO:tasks.workunit.client.1.smithi192.stdout:7/443: symlink d1/d2/d2d/d92/la0 0 2024-02-16T22:56:17.703 INFO:tasks.workunit.client.1.smithi192.stdout:7/444: write d1/d2/d2d/d61/d88/d90/d4c/f6b [5470958,110762] 0 2024-02-16T22:56:17.708 INFO:tasks.workunit.client.1.smithi192.stdout:0/365: creat d1/d3/de/d22/d37/d69/f72 x:0 0 0 2024-02-16T22:56:17.713 INFO:tasks.workunit.client.1.smithi192.stdout:7/445: mkdir d1/d2/de/d2c/d49/da1 0 2024-02-16T22:56:17.714 INFO:tasks.workunit.client.1.smithi192.stdout:5/376: fsync d6/d12/d4e/d58/d6c/f40 0 2024-02-16T22:56:17.716 INFO:tasks.workunit.client.1.smithi192.stdout:7/446: dread d1/d69/f7f [0,4194304] 0 2024-02-16T22:56:17.717 INFO:tasks.workunit.client.1.smithi192.stdout:7/447: fsync d1/d2/de/d2c/d49/d54/d6a/f95 0 2024-02-16T22:56:17.719 INFO:tasks.workunit.client.1.smithi192.stdout:0/366: link d1/d3/de/d22/d37/d47/d53/f65 d1/d3/d29/f73 0 2024-02-16T22:56:17.720 INFO:tasks.workunit.client.1.smithi192.stdout:0/367: read - d1/d3/de/d22/d37/d47/d53/f65 zero size 2024-02-16T22:56:17.724 INFO:tasks.workunit.client.1.smithi192.stdout:1/345: creat d3/f75 x:0 0 0 2024-02-16T22:56:17.724 INFO:tasks.workunit.client.1.smithi192.stdout:7/448: creat d1/d2/de/d83/d9f/fa2 x:0 0 0 2024-02-16T22:56:17.725 INFO:tasks.workunit.client.1.smithi192.stdout:6/366: sync 2024-02-16T22:56:17.725 INFO:tasks.workunit.client.1.smithi192.stdout:1/346: write d3/de/d27/f3b [4629341,107673] 0 2024-02-16T22:56:17.727 INFO:tasks.workunit.client.1.smithi192.stdout:7/449: creat d1/d1f/fa3 x:0 0 0 2024-02-16T22:56:17.728 INFO:tasks.workunit.client.1.smithi192.stdout:7/450: dread - d1/d2/de/d2c/d49/d54/d6a/f95 zero size 2024-02-16T22:56:17.729 INFO:tasks.workunit.client.0.smithi064.stdout:1/258: creat d2/f5c x:0 0 0 2024-02-16T22:56:17.730 INFO:tasks.workunit.client.1.smithi192.stdout:6/367: mkdir da/df/d12/d15/d1b/d70 0 2024-02-16T22:56:17.733 INFO:tasks.workunit.client.1.smithi192.stdout:4/397: dwrite d1/d26/d66/d89/fb [0,4194304] 0 2024-02-16T22:56:17.736 INFO:tasks.workunit.client.1.smithi192.stdout:4/398: symlink d1/d15/d29/l91 0 2024-02-16T22:56:17.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/399: fsync d1/d15/d48/f7d 0 2024-02-16T22:56:17.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/400: dread - d1/d15/d48/d8d/f7b zero size 2024-02-16T22:56:17.738 INFO:tasks.workunit.client.1.smithi192.stdout:0/368: link d1/d3/de/c5b d1/c74 0 2024-02-16T22:56:17.740 INFO:tasks.workunit.client.0.smithi064.stdout:1/259: dwrite d2/d4e/f5a [0,4194304] 0 2024-02-16T22:56:17.743 INFO:tasks.workunit.client.0.smithi064.stdout:6/328: rename d0/d5/f3d to d0/d6/d10/f5e 0 2024-02-16T22:56:17.745 INFO:tasks.workunit.client.1.smithi192.stdout:6/368: mkdir da/df/d71 0 2024-02-16T22:56:17.746 INFO:tasks.workunit.client.1.smithi192.stdout:4/401: write d1/d26/d66/d89/f64 [717347,89778] 0 2024-02-16T22:56:17.748 INFO:tasks.workunit.client.0.smithi064.stdout:7/244: sync 2024-02-16T22:56:17.749 INFO:tasks.workunit.client.0.smithi064.stdout:3/360: rename d4/d13/d1b/d43 to d4/d1f/d5b/d51/d7c 0 2024-02-16T22:56:17.751 INFO:tasks.workunit.client.0.smithi064.stdout:6/329: mkdir d0/d5/df/d17/d18/d5f 0 2024-02-16T22:56:17.753 INFO:tasks.workunit.client.1.smithi192.stdout:6/369: mkdir da/df/d58/d72 0 2024-02-16T22:56:17.760 INFO:tasks.workunit.client.1.smithi192.stdout:4/402: mknod d1/d6b/c92 0 2024-02-16T22:56:17.766 INFO:tasks.workunit.client.0.smithi064.stdout:3/361: mkdir d4/d14/d2b/d72/d7d 0 2024-02-16T22:56:17.769 INFO:tasks.workunit.client.0.smithi064.stdout:7/245: mkdir d5/d6/d49 0 2024-02-16T22:56:17.770 INFO:tasks.workunit.client.0.smithi064.stdout:3/362: rmdir d4/d1f 39 2024-02-16T22:56:17.771 INFO:tasks.workunit.client.0.smithi064.stdout:3/363: read d4/d13/d4d/d5c/f6c [14138,5420] 0 2024-02-16T22:56:17.771 INFO:tasks.workunit.client.0.smithi064.stdout:1/260: link d2/df/d11/d1a/d1d/l2b d2/df/d14/d18/l5d 0 2024-02-16T22:56:17.773 INFO:tasks.workunit.client.1.smithi192.stdout:6/370: dread da/d40/f54 [0,4194304] 0 2024-02-16T22:56:17.777 INFO:tasks.workunit.client.1.smithi192.stdout:2/425: write d22/d24/d2b/f64 [1010545,17274] 0 2024-02-16T22:56:17.778 INFO:tasks.workunit.client.1.smithi192.stdout:6/371: truncate da/df/d12/f39 774506 0 2024-02-16T22:56:17.779 INFO:tasks.workunit.client.0.smithi064.stdout:3/364: symlink d4/d1f/d34/l7e 0 2024-02-16T22:56:17.783 INFO:tasks.workunit.client.0.smithi064.stdout:1/261: stat d2/df/d11/d1a/d1d/l2b 0 2024-02-16T22:56:17.784 INFO:tasks.workunit.client.0.smithi064.stdout:3/365: chown d4/f11 1677 1 2024-02-16T22:56:17.784 INFO:tasks.workunit.client.1.smithi192.stdout:3/390: truncate da/df/f23 3429644 0 2024-02-16T22:56:17.784 INFO:tasks.workunit.client.1.smithi192.stdout:3/391: chown da/df/f2e 46 1 2024-02-16T22:56:17.785 INFO:tasks.workunit.client.1.smithi192.stdout:2/426: creat d22/d24/d2b/d67/f90 x:0 0 0 2024-02-16T22:56:17.785 INFO:tasks.workunit.client.1.smithi192.stdout:2/427: readlink d22/d24/d39/d42/l4d 0 2024-02-16T22:56:17.787 INFO:tasks.workunit.client.1.smithi192.stdout:3/392: symlink da/d11/d63/d6b/d66/l89 0 2024-02-16T22:56:17.790 INFO:tasks.workunit.client.1.smithi192.stdout:2/428: symlink d22/d24/d39/d42/d65/l91 0 2024-02-16T22:56:17.792 INFO:tasks.workunit.client.1.smithi192.stdout:8/376: rmdir d5/d3e 39 2024-02-16T22:56:17.792 INFO:tasks.workunit.client.1.smithi192.stdout:2/429: chown d22/d24/d2b/l36 16198017 1 2024-02-16T22:56:17.796 INFO:tasks.workunit.client.1.smithi192.stdout:8/377: getdents d5/d68 0 2024-02-16T22:56:17.799 INFO:tasks.workunit.client.1.smithi192.stdout:8/378: creat d5/d3e/f74 x:0 0 0 2024-02-16T22:56:17.799 INFO:tasks.workunit.client.1.smithi192.stdout:2/430: dwrite f10 [0,4194304] 0 2024-02-16T22:56:17.800 INFO:tasks.workunit.client.1.smithi192.stdout:2/431: chown d22/d24/d2b/d2c/f2d 0 1 2024-02-16T22:56:17.803 INFO:tasks.workunit.client.1.smithi192.stdout:9/484: write d7/f2e [628174,66620] 0 2024-02-16T22:56:17.806 INFO:tasks.workunit.client.1.smithi192.stdout:8/379: dwrite d5/d9/f54 [0,4194304] 0 2024-02-16T22:56:17.811 INFO:tasks.workunit.client.1.smithi192.stdout:0/369: fsync d1/d3/de/d22/d37/d69/f72 0 2024-02-16T22:56:17.813 INFO:tasks.workunit.client.1.smithi192.stdout:8/380: dwrite d5/d9/f33 [4194304,4194304] 0 2024-02-16T22:56:17.814 INFO:tasks.workunit.client.1.smithi192.stdout:8/381: chown d5/d9/f6b 1047927442 1 2024-02-16T22:56:17.814 INFO:tasks.workunit.client.1.smithi192.stdout:0/370: mkdir d1/d3/de/d22/d37/d67/d75 0 2024-02-16T22:56:17.815 INFO:tasks.workunit.client.1.smithi192.stdout:0/371: readlink d1/d3/de/l40 0 2024-02-16T22:56:17.820 INFO:tasks.workunit.client.1.smithi192.stdout:0/372: read d1/d3/d29/f70 [100194,114457] 0 2024-02-16T22:56:17.824 INFO:tasks.workunit.client.1.smithi192.stdout:0/373: creat d1/d3/d39/f76 x:0 0 0 2024-02-16T22:56:17.825 INFO:tasks.workunit.client.1.smithi192.stdout:8/382: rmdir d5/d3a/d4d/d71 0 2024-02-16T22:56:17.826 INFO:tasks.workunit.client.1.smithi192.stdout:8/383: chown d5/db/f64 2647 1 2024-02-16T22:56:17.826 INFO:tasks.workunit.client.1.smithi192.stdout:0/374: creat d1/d3/d39/f77 x:0 0 0 2024-02-16T22:56:17.827 INFO:tasks.workunit.client.1.smithi192.stdout:0/375: fdatasync d1/d3/de/d22/f45 0 2024-02-16T22:56:17.827 INFO:tasks.workunit.client.1.smithi192.stdout:7/451: rename d1/d1f to d1/d2/de/d2c/d49/da4 0 2024-02-16T22:56:17.828 INFO:tasks.workunit.client.1.smithi192.stdout:4/403: rename d1/d26/d66/d89 to d1/d26/d66/d89/d93 22 2024-02-16T22:56:17.829 INFO:tasks.workunit.client.1.smithi192.stdout:8/384: write d5/f38 [1355565,38491] 0 2024-02-16T22:56:17.831 INFO:tasks.workunit.client.1.smithi192.stdout:0/376: symlink d1/d3/de/l78 0 2024-02-16T22:56:17.836 INFO:tasks.workunit.client.1.smithi192.stdout:9/485: rename d7/d19/f45 to d7/d19/d28/faa 0 2024-02-16T22:56:17.838 INFO:tasks.workunit.client.1.smithi192.stdout:0/377: creat d1/d3/d39/f79 x:0 0 0 2024-02-16T22:56:17.840 INFO:tasks.workunit.client.1.smithi192.stdout:0/378: write d1/d3/de/d22/d37/d47/f48 [1800565,32993] 0 2024-02-16T22:56:17.840 INFO:tasks.workunit.client.1.smithi192.stdout:4/404: symlink d1/d26/l94 0 2024-02-16T22:56:17.842 INFO:tasks.workunit.client.1.smithi192.stdout:9/486: unlink d7/d15/d36/d38/d3f/d8a/d98/f23 0 2024-02-16T22:56:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 -- 172.21.15.64:0/3935257436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 msgr2=0x7f8ffc0fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 --2- 172.21.15.64:0/3935257436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 0x7f8ffc0fb9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00049f0 tx=0x7f8ff000c040 comp rx=0 tx=0).stop 2024-02-16T22:56:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 -- 172.21.15.64:0/3935257436 shutdown_connections 2024-02-16T22:56:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 --2- 172.21.15.64:0/3935257436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc0fc7f0 0x7f8ffc0fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 --2- 172.21.15.64:0/3935257436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 0x7f8ffc0fb9f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 -- 172.21.15.64:0/3935257436 >> 172.21.15.64:0/3935257436 conn(0x7f8ffc0f6ca0 msgr2=0x7f8ffc0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:17.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 -- 172.21.15.64:0/3935257436 shutdown_connections 2024-02-16T22:56:17.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 -- 172.21.15.64:0/3935257436 wait complete. 2024-02-16T22:56:17.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.842+0000 7f9002a93700 1 Processor -- start 2024-02-16T22:56:17.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f9002a93700 1 -- start start 2024-02-16T22:56:17.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f9002a93700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 0x7f8ffc114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f9002a93700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc0fc7f0 0x7f8ffc114580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f9002a93700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc114b80 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f9002a93700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc115420 con 0x7f8ffc0fb5e0 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f900082f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 0x7f8ffc114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f8ffbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc0fc7f0 0x7f8ffc114580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f8ffbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc0fc7f0 0x7f8ffc114580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43332/0 (socket says 172.21.15.64:43332) 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.843+0000 7f8ffbfff700 1 -- 172.21.15.64:0/3826636511 learned_addr learned my addr 172.21.15.64:0/3826636511 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.844+0000 7f8ffbfff700 1 -- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 msgr2=0x7f8ffc114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.844+0000 7f8ffbfff700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 0x7f8ffc114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.844+0000 7f8ffbfff700 1 -- 172.21.15.64:0/3826636511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe8005040 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.844+0000 7f900082f700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 0x7f8ffc114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.844+0000 7f8ffbfff700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc0fc7f0 0x7f8ffc114580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8004e40 tx=0x7f8fe800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.844+0000 7f8ff97fa700 1 -- 172.21.15.64:0/3826636511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe8007ea0 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.844+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff0005040 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.844+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc115ab0 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.847 INFO:tasks.workunit.client.1.smithi192.stdout:4/405: unlink d1/d26/d66/d89/d11/f59 0 2024-02-16T22:56:17.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.845+0000 7f8ff97fa700 1 -- 172.21.15.64:0/3826636511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fe8014070 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.845+0000 7f8ff97fa700 1 -- 172.21.15.64:0/3826636511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe800b0f0 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.846+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe00052e0 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.847+0000 7f8ff97fa700 1 -- 172.21.15.64:0/3826636511 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f8fe800da00 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.847+0000 7f8ff97fa700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f8fec06b2e0 0x7f8fec06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:17.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.848+0000 7f8ff97fa700 1 -- 172.21.15.64:0/3826636511 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f8fe8010070 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.848+0000 7f900082f700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f8fec06b2e0 0x7f8fec06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:17.849 INFO:tasks.workunit.client.1.smithi192.stdout:4/406: creat d1/d15/d48/f95 x:0 0 0 2024-02-16T22:56:17.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.849+0000 7f900082f700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f8fec06b2e0 0x7f8fec06d790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000b3a0 tx=0x7f8ff000c040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:17.850 INFO:tasks.workunit.client.0.smithi064.stdout:6/330: dwrite d0/d5/df/d17/d18/f3f [0,4194304] 0 2024-02-16T22:56:17.852 INFO:tasks.workunit.client.1.smithi192.stdout:4/407: dread d1/d26/d66/d89/f3a [0,4194304] 0 2024-02-16T22:56:17.856 INFO:tasks.workunit.client.0.smithi064.stdout:6/331: mkdir d0/d5/d54/d60 0 2024-02-16T22:56:17.857 INFO:tasks.workunit.client.1.smithi192.stdout:9/487: getdents d7/d19/d28/d43/d70 0 2024-02-16T22:56:17.858 INFO:tasks.workunit.client.1.smithi192.stdout:9/488: chown d7/l1c 335084 1 2024-02-16T22:56:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:17.857+0000 7f8ff97fa700 1 -- 172.21.15.64:0/3826636511 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f8fe8057ba0 con 0x7f8ffc0fc7f0 2024-02-16T22:56:17.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/408: creat d1/d26/d5e/f96 x:0 0 0 2024-02-16T22:56:17.863 INFO:tasks.workunit.client.1.smithi192.stdout:4/409: chown d1/d15/d29/c2c 67671 1 2024-02-16T22:56:17.864 INFO:tasks.workunit.client.0.smithi064.stdout:6/332: creat d0/d1d/f61 x:0 0 0 2024-02-16T22:56:17.865 INFO:tasks.workunit.client.1.smithi192.stdout:9/489: dwrite d7/d15/d36/d38/d3f/d8a/d98/f16 [0,4194304] 0 2024-02-16T22:56:17.865 INFO:tasks.workunit.client.0.smithi064.stdout:3/366: write d4/d1f/d34/f45 [1502266,103208] 0 2024-02-16T22:56:17.868 INFO:tasks.workunit.client.1.smithi192.stdout:4/410: creat d1/d26/d66/d89/f97 x:0 0 0 2024-02-16T22:56:17.868 INFO:tasks.workunit.client.0.smithi064.stdout:1/262: truncate d2/df/f54 1313678 0 2024-02-16T22:56:17.870 INFO:tasks.workunit.client.0.smithi064.stdout:6/333: creat d0/d5/df/d17/f62 x:0 0 0 2024-02-16T22:56:17.872 INFO:tasks.workunit.client.1.smithi192.stdout:3/393: dwrite da/d11/d2b/f5f [0,4194304] 0 2024-02-16T22:56:17.874 INFO:tasks.workunit.client.1.smithi192.stdout:3/394: write da/d11/d20/f30 [868225,127944] 0 2024-02-16T22:56:17.874 INFO:tasks.workunit.client.0.smithi064.stdout:1/263: creat d2/df/d14/d2c/f5e x:0 0 0 2024-02-16T22:56:17.875 INFO:tasks.workunit.client.0.smithi064.stdout:3/367: dwrite d4/d13/f26 [4194304,4194304] 0 2024-02-16T22:56:17.881 INFO:tasks.workunit.client.0.smithi064.stdout:6/334: unlink d0/d6/d10/d33/f32 0 2024-02-16T22:56:17.882 INFO:tasks.workunit.client.0.smithi064.stdout:6/335: write d0/f41 [1881448,107643] 0 2024-02-16T22:56:17.889 INFO:tasks.workunit.client.0.smithi064.stdout:6/336: dread d0/d6/da/f25 [0,4194304] 0 2024-02-16T22:56:17.890 INFO:tasks.workunit.client.0.smithi064.stdout:6/337: dread - d0/d1d/f61 zero size 2024-02-16T22:56:17.890 INFO:tasks.workunit.client.0.smithi064.stdout:1/264: rename d2/df/d11/f27 to d2/df/d14/d2c/f5f 0 2024-02-16T22:56:17.895 INFO:tasks.workunit.client.1.smithi192.stdout:4/411: unlink d1/d15/d29/d3c/l73 0 2024-02-16T22:56:17.898 INFO:tasks.workunit.client.0.smithi064.stdout:1/265: mknod d2/df/d14/c60 0 2024-02-16T22:56:17.900 INFO:tasks.workunit.client.1.smithi192.stdout:3/395: rename da/d11/d63/d6b/f4d to da/d11/d20/d39/d53/f8a 0 2024-02-16T22:56:17.902 INFO:tasks.workunit.client.1.smithi192.stdout:8/385: dwrite d5/d3e/f1c [4194304,4194304] 0 2024-02-16T22:56:17.903 INFO:tasks.workunit.client.1.smithi192.stdout:8/386: chown d5/db/d47/d43/l60 15 1 2024-02-16T22:56:17.905 INFO:tasks.workunit.client.1.smithi192.stdout:8/387: write d5/db/de/df/f2b [3285489,120094] 0 2024-02-16T22:56:17.906 INFO:tasks.workunit.client.1.smithi192.stdout:8/388: truncate d5/d16/f4f 1006549 0 2024-02-16T22:56:17.908 INFO:tasks.workunit.client.0.smithi064.stdout:3/368: getdents d4/d13/d1b/d27/d44 0 2024-02-16T22:56:17.912 INFO:tasks.workunit.client.1.smithi192.stdout:3/396: unlink da/d11/f58 0 2024-02-16T22:56:17.913 INFO:tasks.workunit.client.0.smithi064.stdout:5/209: sync 2024-02-16T22:56:17.913 INFO:tasks.workunit.client.0.smithi064.stdout:8/261: sync 2024-02-16T22:56:17.916 INFO:tasks.workunit.client.0.smithi064.stdout:1/266: unlink d2/df/d14/d18/d40/l44 0 2024-02-16T22:56:17.917 INFO:tasks.workunit.client.0.smithi064.stdout:6/338: dread d0/f22 [0,4194304] 0 2024-02-16T22:56:17.920 INFO:tasks.workunit.client.0.smithi064.stdout:6/339: stat d0/d1b 0 2024-02-16T22:56:17.921 INFO:tasks.workunit.client.1.smithi192.stdout:7/452: dread d1/d2/d2d/f40 [0,4194304] 0 2024-02-16T22:56:17.923 INFO:tasks.workunit.client.0.smithi064.stdout:8/262: write df/d31/d2f/f3a [3143852,35744] 0 2024-02-16T22:56:17.924 INFO:tasks.workunit.client.0.smithi064.stdout:7/246: dwrite d5/d6/fb [0,4194304] 0 2024-02-16T22:56:17.926 INFO:tasks.workunit.client.1.smithi192.stdout:4/412: fsync d1/d26/d66/d89/d11/d58/f7e 0 2024-02-16T22:56:17.927 INFO:tasks.workunit.client.0.smithi064.stdout:1/267: creat d2/df/d11/d1a/d1d/f61 x:0 0 0 2024-02-16T22:56:17.929 INFO:tasks.workunit.client.0.smithi064.stdout:5/210: dwrite f0 [4194304,4194304] 0 2024-02-16T22:56:17.937 INFO:tasks.workunit.client.0.smithi064.stdout:6/340: fsync d0/f22 0 2024-02-16T22:56:17.941 INFO:tasks.workunit.client.0.smithi064.stdout:8/263: creat df/d2a/f57 x:0 0 0 2024-02-16T22:56:17.941 INFO:tasks.workunit.client.0.smithi064.stdout:8/264: readlink df/d31/d2f/l3b 0 2024-02-16T22:56:17.943 INFO:tasks.workunit.client.1.smithi192.stdout:3/397: creat da/d11/d20/f8b x:0 0 0 2024-02-16T22:56:17.945 INFO:tasks.workunit.client.0.smithi064.stdout:7/247: rename d5/d33/d45/f47 to d5/d33/d35/d41/f4a 0 2024-02-16T22:56:17.947 INFO:tasks.workunit.client.0.smithi064.stdout:1/268: fdatasync d2/df/d14/d18/f28 0 2024-02-16T22:56:17.949 INFO:tasks.workunit.client.0.smithi064.stdout:3/369: creat d4/d13/d1b/d27/f7f x:0 0 0 2024-02-16T22:56:17.950 INFO:tasks.workunit.client.0.smithi064.stdout:3/370: fsync d4/d1f/f67 0 2024-02-16T22:56:17.951 INFO:tasks.workunit.client.1.smithi192.stdout:7/453: mknod d1/d2/d2d/d61/d88/d90/d2a/ca5 0 2024-02-16T22:56:17.952 INFO:tasks.workunit.client.1.smithi192.stdout:7/454: chown d1/d2/d2d/d92/d9b 14850 1 2024-02-16T22:56:17.954 INFO:tasks.workunit.client.0.smithi064.stdout:3/371: truncate d4/f1e 8786688 0 2024-02-16T22:56:17.957 INFO:tasks.workunit.client.1.smithi192.stdout:4/413: creat d1/d26/d66/d89/d11/f98 x:0 0 0 2024-02-16T22:56:17.957 INFO:tasks.workunit.client.1.smithi192.stdout:4/414: chown d1/c39 0 1 2024-02-16T22:56:17.958 INFO:tasks.workunit.client.1.smithi192.stdout:4/415: fsync d1/d26/d66/d89/d11/d58/f5d 0 2024-02-16T22:56:17.964 INFO:tasks.workunit.client.0.smithi064.stdout:5/211: rename f0 to d6/d25/d48/f4a 0 2024-02-16T22:56:17.966 INFO:tasks.workunit.client.1.smithi192.stdout:7/455: symlink d1/d2/d2d/d61/d8a/la6 0 2024-02-16T22:56:17.967 INFO:tasks.workunit.client.1.smithi192.stdout:8/389: dwrite d5/d3e/d5f/f5e [0,4194304] 0 2024-02-16T22:56:17.967 INFO:tasks.workunit.client.1.smithi192.stdout:7/456: stat d1/d2/d2d 0 2024-02-16T22:56:17.967 INFO:tasks.workunit.client.1.smithi192.stdout:8/390: chown d5/d3a/c56 1556938 1 2024-02-16T22:56:17.967 INFO:tasks.workunit.client.0.smithi064.stdout:7/248: creat d5/d33/d35/d41/f4b x:0 0 0 2024-02-16T22:56:17.968 INFO:tasks.workunit.client.1.smithi192.stdout:8/391: write d5/d9/f44 [1293649,22328] 0 2024-02-16T22:56:17.970 INFO:tasks.workunit.client.1.smithi192.stdout:8/392: chown d5/d3a/d4d/f4b 1682 1 2024-02-16T22:56:17.971 INFO:tasks.workunit.client.1.smithi192.stdout:8/393: truncate d5/d3e/f74 620365 0 2024-02-16T22:56:17.977 INFO:tasks.workunit.client.1.smithi192.stdout:3/398: rmdir da/d11/d63/d6b/d4e 39 2024-02-16T22:56:17.980 INFO:tasks.workunit.client.1.smithi192.stdout:7/457: mknod d1/d2/d2d/d92/ca7 0 2024-02-16T22:56:17.988 INFO:tasks.workunit.client.1.smithi192.stdout:3/399: unlink da/d11/c5a 0 2024-02-16T22:56:17.989 INFO:tasks.workunit.client.1.smithi192.stdout:3/400: read da/df/f79 [1166408,8412] 0 2024-02-16T22:56:17.991 INFO:tasks.workunit.client.1.smithi192.stdout:7/458: mknod d1/d2/d2d/d61/d88/d90/d4c/ca8 0 2024-02-16T22:56:17.993 INFO:tasks.workunit.client.1.smithi192.stdout:4/416: rename d1/d26/d6f/f88 to d1/d26/d66/d89/f99 0 2024-02-16T22:56:17.993 INFO:tasks.workunit.client.1.smithi192.stdout:8/394: dread d5/d16/f27 [0,4194304] 0 2024-02-16T22:56:17.998 INFO:tasks.workunit.client.0.smithi064.stdout:3/372: symlink d4/d1f/d5b/d51/l80 0 2024-02-16T22:56:17.999 INFO:tasks.workunit.client.1.smithi192.stdout:3/401: symlink da/df/l8c 0 2024-02-16T22:56:18.000 INFO:tasks.workunit.client.1.smithi192.stdout:3/402: readlink da/d11/d2b/l31 0 2024-02-16T22:56:18.004 INFO:tasks.workunit.client.0.smithi064.stdout:1/269: write d2/f3 [3544685,111901] 0 2024-02-16T22:56:18.005 INFO:tasks.workunit.client.0.smithi064.stdout:1/270: stat d2/lc 0 2024-02-16T22:56:18.005 INFO:tasks.workunit.client.1.smithi192.stdout:7/459: dread d1/f4b [0,4194304] 0 2024-02-16T22:56:18.005 INFO:tasks.workunit.client.1.smithi192.stdout:3/403: dwrite f4 [0,4194304] 0 2024-02-16T22:56:18.011 INFO:tasks.workunit.client.1.smithi192.stdout:4/417: creat d1/d15/d48/d8d/f9a x:0 0 0 2024-02-16T22:56:18.014 INFO:tasks.workunit.client.0.smithi064.stdout:5/212: mkdir d6/d1c/d4b 0 2024-02-16T22:56:18.017 INFO:tasks.workunit.client.0.smithi064.stdout:7/249: symlink d5/d33/d35/l4c 0 2024-02-16T22:56:18.018 INFO:tasks.workunit.client.0.smithi064.stdout:7/250: write d5/f25 [4152034,81099] 0 2024-02-16T22:56:18.025 INFO:tasks.workunit.client.0.smithi064.stdout:5/213: dread d6/d18/d19/f26 [0,4194304] 0 2024-02-16T22:56:18.026 INFO:tasks.workunit.client.0.smithi064.stdout:1/271: mkdir d2/df/d14/d2c/d62 0 2024-02-16T22:56:18.027 INFO:tasks.workunit.client.0.smithi064.stdout:1/272: chown d2/df/d11/d1a/d30/f41 252312 1 2024-02-16T22:56:18.028 INFO:tasks.workunit.client.1.smithi192.stdout:7/460: rename d1/d2/d2d/d61/d88/d90/f3a to d1/d2/de/d2c/d49/d54/d9d/fa9 0 2024-02-16T22:56:18.029 INFO:tasks.workunit.client.1.smithi192.stdout:7/461: chown d1/d2/d2d/f40 14267 1 2024-02-16T22:56:18.029 INFO:tasks.workunit.client.1.smithi192.stdout:7/462: dread - d1/d2/de/d2c/d49/f80 zero size 2024-02-16T22:56:18.033 INFO:tasks.workunit.client.1.smithi192.stdout:5/377: sync 2024-02-16T22:56:18.034 INFO:tasks.workunit.client.0.smithi064.stdout:3/373: write d4/f12 [1798537,75940] 0 2024-02-16T22:56:18.038 INFO:tasks.workunit.client.1.smithi192.stdout:3/404: dwrite da/f51 [0,4194304] 0 2024-02-16T22:56:18.038 INFO:tasks.workunit.client.0.smithi064.stdout:7/251: mknod d5/d6/c4d 0 2024-02-16T22:56:18.039 INFO:tasks.workunit.client.0.smithi064.stdout:5/214: creat d6/d1c/d43/f4c x:0 0 0 2024-02-16T22:56:18.041 INFO:tasks.workunit.client.1.smithi192.stdout:7/463: mkdir d1/d2/de/daa 0 2024-02-16T22:56:18.041 INFO:tasks.workunit.client.0.smithi064.stdout:5/215: fsync d6/f31 0 2024-02-16T22:56:18.041 INFO:tasks.workunit.client.1.smithi192.stdout:4/418: dread d1/f19 [0,4194304] 0 2024-02-16T22:56:18.047 INFO:tasks.workunit.client.0.smithi064.stdout:3/374: dwrite d4/d1f/d34/f74 [0,4194304] 0 2024-02-16T22:56:18.048 INFO:tasks.workunit.client.1.smithi192.stdout:5/378: rmdir d6/d12/d4e/d5b 39 2024-02-16T22:56:18.052 INFO:tasks.workunit.client.1.smithi192.stdout:3/405: unlink da/d11/d63/d6b/c6c 0 2024-02-16T22:56:18.054 INFO:tasks.workunit.client.0.smithi064.stdout:7/252: dwrite d5/d33/d35/f40 [0,4194304] 0 2024-02-16T22:56:18.058 INFO:tasks.workunit.client.1.smithi192.stdout:4/419: mkdir d1/d15/d48/d8d/d9b 0 2024-02-16T22:56:18.059 INFO:tasks.workunit.client.1.smithi192.stdout:4/420: write d1/d26/d6f/f86 [1022636,22923] 0 2024-02-16T22:56:18.059 INFO:tasks.workunit.client.1.smithi192.stdout:8/395: getdents d5/db/d47/d43 0 2024-02-16T22:56:18.061 INFO:tasks.workunit.client.0.smithi064.stdout:5/216: dwrite d6/d25/f32 [0,4194304] 0 2024-02-16T22:56:18.063 INFO:tasks.workunit.client.0.smithi064.stdout:1/273: dwrite d2/df/d14/f2d [0,4194304] 0 2024-02-16T22:56:18.065 INFO:tasks.workunit.client.1.smithi192.stdout:8/396: dwrite d5/d9/f54 [0,4194304] 0 2024-02-16T22:56:18.069 INFO:tasks.workunit.client.1.smithi192.stdout:5/379: rename d6/d12/d4e/c17 to d6/d8/d75/c7d 0 2024-02-16T22:56:18.071 INFO:tasks.workunit.client.0.smithi064.stdout:7/253: dread d5/d33/d35/f40 [0,4194304] 0 2024-02-16T22:56:18.073 INFO:tasks.workunit.client.0.smithi064.stdout:3/375: mkdir d4/d13/d1b/d27/d81 0 2024-02-16T22:56:18.075 INFO:tasks.workunit.client.0.smithi064.stdout:3/376: chown d4/d1f/f67 110191143 1 2024-02-16T22:56:18.081 INFO:tasks.workunit.client.0.smithi064.stdout:1/274: creat d2/df/d11/d1a/d30/f63 x:0 0 0 2024-02-16T22:56:18.082 INFO:tasks.workunit.client.0.smithi064.stdout:1/275: write d2/df/d11/d1a/d30/f63 [1036122,37666] 0 2024-02-16T22:56:18.083 INFO:tasks.workunit.client.1.smithi192.stdout:7/464: dwrite d1/d2/d2d/f40 [0,4194304] 0 2024-02-16T22:56:18.083 INFO:tasks.workunit.client.0.smithi064.stdout:5/217: read d6/d18/d19/f1f [660367,85867] 0 2024-02-16T22:56:18.084 INFO:tasks.workunit.client.1.smithi192.stdout:6/372: sync 2024-02-16T22:56:18.085 INFO:tasks.workunit.client.1.smithi192.stdout:9/490: sync 2024-02-16T22:56:18.085 INFO:tasks.workunit.client.1.smithi192.stdout:1/347: sync 2024-02-16T22:56:18.085 INFO:tasks.workunit.client.1.smithi192.stdout:0/379: sync 2024-02-16T22:56:18.085 INFO:tasks.workunit.client.1.smithi192.stdout:2/432: sync 2024-02-16T22:56:18.085 INFO:tasks.workunit.client.0.smithi064.stdout:3/377: mkdir d4/d13/d1b/d27/d82 0 2024-02-16T22:56:18.086 INFO:tasks.workunit.client.1.smithi192.stdout:8/397: read d5/db/de/df/f1b [1613615,111747] 0 2024-02-16T22:56:18.086 INFO:tasks.workunit.client.0.smithi064.stdout:0/271: sync 2024-02-16T22:56:18.086 INFO:tasks.workunit.client.0.smithi064.stdout:4/295: sync 2024-02-16T22:56:18.086 INFO:tasks.workunit.client.0.smithi064.stdout:9/302: sync 2024-02-16T22:56:18.086 INFO:tasks.workunit.client.0.smithi064.stdout:2/265: sync 2024-02-16T22:56:18.086 INFO:tasks.workunit.client.1.smithi192.stdout:7/465: stat d1/d2/d2d/d61/d88/d90/f1d 0 2024-02-16T22:56:18.087 INFO:tasks.workunit.client.1.smithi192.stdout:1/348: dread - d3/f75 zero size 2024-02-16T22:56:18.087 INFO:tasks.workunit.client.1.smithi192.stdout:4/421: dread d1/d15/f45 [0,4194304] 0 2024-02-16T22:56:18.088 INFO:tasks.workunit.client.1.smithi192.stdout:2/433: fdatasync d22/d24/d2b/d67/f90 0 2024-02-16T22:56:18.088 INFO:tasks.workunit.client.1.smithi192.stdout:5/380: symlink d6/d12/d4e/l7e 0 2024-02-16T22:56:18.088 INFO:tasks.workunit.client.1.smithi192.stdout:1/349: chown d3/f22 607 1 2024-02-16T22:56:18.090 INFO:tasks.workunit.client.1.smithi192.stdout:7/466: chown d1/d2/de/d2c/d49/c56 13540 1 2024-02-16T22:56:18.090 INFO:tasks.workunit.client.1.smithi192.stdout:1/350: stat d3/de/d27/d34/d52/d63 0 2024-02-16T22:56:18.093 INFO:tasks.workunit.client.0.smithi064.stdout:0/272: readlink da/dc/de/d12/d1d/l4c 0 2024-02-16T22:56:18.093 INFO:tasks.workunit.client.1.smithi192.stdout:9/491: dwrite d7/d19/d37/f7e [0,4194304] 0 2024-02-16T22:56:18.094 INFO:tasks.workunit.client.1.smithi192.stdout:6/373: mknod da/df/d58/c73 0 2024-02-16T22:56:18.095 INFO:tasks.workunit.client.1.smithi192.stdout:2/434: creat d22/f92 x:0 0 0 2024-02-16T22:56:18.097 INFO:tasks.workunit.client.0.smithi064.stdout:9/303: mknod d3/d11/d15/d5a/d2a/d2e/d31/d3f/c68 0 2024-02-16T22:56:18.098 INFO:tasks.workunit.client.1.smithi192.stdout:4/422: truncate d1/d26/d66/d89/d11/d58/f5d 642678 0 2024-02-16T22:56:18.100 INFO:tasks.workunit.client.1.smithi192.stdout:4/423: fdatasync d1/f5 0 2024-02-16T22:56:18.100 INFO:tasks.workunit.client.1.smithi192.stdout:5/381: rename d6/d12/l1b to d6/d12/d4e/d46/l7f 0 2024-02-16T22:56:18.101 INFO:tasks.workunit.client.0.smithi064.stdout:3/378: symlink d4/d13/d1b/d27/d61/d63/l83 0 2024-02-16T22:56:18.101 INFO:tasks.workunit.client.0.smithi064.stdout:2/266: dwrite d7/d14/d25/f4a [0,4194304] 0 2024-02-16T22:56:18.103 INFO:tasks.workunit.client.1.smithi192.stdout:8/398: dwrite d5/d3a/d4d/f4b [0,4194304] 0 2024-02-16T22:56:18.103 INFO:tasks.workunit.client.0.smithi064.stdout:2/267: fdatasync d7/de/f24 0 2024-02-16T22:56:18.106 INFO:tasks.workunit.client.0.smithi064.stdout:9/304: write d3/d11/d15/d5a/f29 [510737,71001] 0 2024-02-16T22:56:18.110 INFO:tasks.workunit.client.0.smithi064.stdout:2/268: write d7/de/f1f [4981390,29169] 0 2024-02-16T22:56:18.113 INFO:tasks.workunit.client.1.smithi192.stdout:9/492: readlink d7/d19/d28/d43/d70/d8e/l9b 0 2024-02-16T22:56:18.113 INFO:tasks.workunit.client.0.smithi064.stdout:0/273: dwrite da/dc/de/d12/d2c/d2e/f36 [4194304,4194304] 0 2024-02-16T22:56:18.115 INFO:tasks.workunit.client.0.smithi064.stdout:0/274: chown da/dc/de/c1a 834344600 1 2024-02-16T22:56:18.116 INFO:tasks.workunit.client.1.smithi192.stdout:3/406: dwrite da/f22 [0,4194304] 0 2024-02-16T22:56:18.116 INFO:tasks.workunit.client.0.smithi064.stdout:0/275: stat da/f10 0 2024-02-16T22:56:18.118 INFO:tasks.workunit.client.1.smithi192.stdout:3/407: chown da/f22 61343811 1 2024-02-16T22:56:18.118 INFO:tasks.workunit.client.1.smithi192.stdout:4/424: mknod d1/d15/d48/c9c 0 2024-02-16T22:56:18.119 INFO:tasks.workunit.client.1.smithi192.stdout:3/408: chown da/df/c44 6618515 1 2024-02-16T22:56:18.123 INFO:tasks.workunit.client.1.smithi192.stdout:7/467: rename d1/d2/d2d/c4e to d1/d2/de/cab 0 2024-02-16T22:56:18.123 INFO:tasks.workunit.client.1.smithi192.stdout:5/382: dread - d6/d12/d2d/f42 zero size 2024-02-16T22:56:18.125 INFO:tasks.workunit.client.0.smithi064.stdout:2/269: dread d7/d14/d1e/d34/f50 [0,4194304] 0 2024-02-16T22:56:18.126 INFO:tasks.workunit.client.1.smithi192.stdout:7/468: truncate d1/d2/de/d2c/d49/da4/f85 4877538 0 2024-02-16T22:56:18.126 INFO:tasks.workunit.client.0.smithi064.stdout:2/270: chown d7/d14/d3e 10 1 2024-02-16T22:56:18.127 INFO:tasks.workunit.client.0.smithi064.stdout:3/379: creat d4/f84 x:0 0 0 2024-02-16T22:56:18.130 INFO:tasks.workunit.client.0.smithi064.stdout:9/305: dwrite d3/d11/d15/d23/f3d [0,4194304] 0 2024-02-16T22:56:18.131 INFO:tasks.workunit.client.1.smithi192.stdout:1/351: mknod d3/de/d27/d34/d36/d47/d5d/c76 0 2024-02-16T22:56:18.131 INFO:tasks.workunit.client.0.smithi064.stdout:9/306: stat f1 0 2024-02-16T22:56:18.131 INFO:tasks.workunit.client.1.smithi192.stdout:1/352: stat d3/de/d27/d34/f38 0 2024-02-16T22:56:18.132 INFO:tasks.workunit.client.1.smithi192.stdout:1/353: fsync d3/de/d27/f30 0 2024-02-16T22:56:18.132 INFO:tasks.workunit.client.0.smithi064.stdout:1/276: truncate d2/df/d14/d2c/f5f 3856192 0 2024-02-16T22:56:18.136 INFO:tasks.workunit.client.0.smithi064.stdout:1/277: write d2/df/d11/d1a/d30/f41 [325619,115234] 0 2024-02-16T22:56:18.136 INFO:tasks.workunit.client.1.smithi192.stdout:6/374: truncate f9 211467 0 2024-02-16T22:56:18.137 INFO:tasks.workunit.client.1.smithi192.stdout:6/375: chown da/df/d12/d15/f68 101979482 1 2024-02-16T22:56:18.138 INFO:tasks.workunit.client.1.smithi192.stdout:6/376: chown da/df/d18/c6d 64 1 2024-02-16T22:56:18.140 INFO:tasks.workunit.client.1.smithi192.stdout:9/493: fdatasync d7/d19/d37/f3d 0 2024-02-16T22:56:18.146 INFO:tasks.workunit.client.1.smithi192.stdout:6/377: dwrite da/df/d12/f51 [0,4194304] 0 2024-02-16T22:56:18.146 INFO:tasks.workunit.client.1.smithi192.stdout:4/425: mknod d1/d15/d48/d8d/c9d 0 2024-02-16T22:56:18.146 INFO:tasks.workunit.client.1.smithi192.stdout:6/378: stat da/df/d12 0 2024-02-16T22:56:18.146 INFO:tasks.workunit.client.0.smithi064.stdout:3/380: rmdir d4/d14/d2b 39 2024-02-16T22:56:18.146 INFO:tasks.workunit.client.0.smithi064.stdout:2/271: dread d7/d14/d1e/d34/f50 [0,4194304] 0 2024-02-16T22:56:18.148 INFO:tasks.workunit.client.0.smithi064.stdout:4/296: dread dc/d1d/f2b [0,4194304] 0 2024-02-16T22:56:18.148 INFO:tasks.workunit.client.0.smithi064.stdout:1/278: creat d2/d4e/f64 x:0 0 0 2024-02-16T22:56:18.151 INFO:tasks.workunit.client.1.smithi192.stdout:7/469: rmdir d1/d2/de/d2c/d49/d54/d6a 39 2024-02-16T22:56:18.152 INFO:tasks.workunit.client.0.smithi064.stdout:9/307: dwrite d3/d11/d15/d5a/d2a/d2e/d31/d3f/f5e [0,4194304] 0 2024-02-16T22:56:18.154 INFO:tasks.workunit.client.0.smithi064.stdout:0/276: link da/f1b da/f5c 0 2024-02-16T22:56:18.168 INFO:tasks.workunit.client.1.smithi192.stdout:9/494: creat d7/d15/fab x:0 0 0 2024-02-16T22:56:18.169 INFO:tasks.workunit.client.1.smithi192.stdout:9/495: write d7/d15/f33 [4609550,16250] 0 2024-02-16T22:56:18.170 INFO:tasks.workunit.client.1.smithi192.stdout:9/496: fsync d7/d15/d36/d38/d3f/d8a/d98/f62 0 2024-02-16T22:56:18.174 INFO:tasks.workunit.client.1.smithi192.stdout:3/409: symlink da/d11/d63/d6b/l8d 0 2024-02-16T22:56:18.175 INFO:tasks.workunit.client.0.smithi064.stdout:2/272: getdents d7/d14/d25/d46 0 2024-02-16T22:56:18.177 INFO:tasks.workunit.client.0.smithi064.stdout:1/279: dread d2/f36 [0,4194304] 0 2024-02-16T22:56:18.177 INFO:tasks.workunit.client.1.smithi192.stdout:8/399: dread d5/f1e [0,4194304] 0 2024-02-16T22:56:18.178 INFO:tasks.workunit.client.1.smithi192.stdout:4/426: dwrite d1/d15/f45 [0,4194304] 0 2024-02-16T22:56:18.178 INFO:tasks.workunit.client.0.smithi064.stdout:1/280: dread - d2/df/d11/d1a/d1d/f61 zero size 2024-02-16T22:56:18.180 INFO:tasks.workunit.client.1.smithi192.stdout:9/497: dwrite d7/d19/d5e/d6f/f7a [0,4194304] 0 2024-02-16T22:56:18.180 INFO:tasks.workunit.client.0.smithi064.stdout:1/281: truncate d2/d4e/f64 584176 0 2024-02-16T22:56:18.182 INFO:tasks.workunit.client.1.smithi192.stdout:2/435: getdents d22/d24/d2b/d2c 0 2024-02-16T22:56:18.189 INFO:tasks.workunit.client.0.smithi064.stdout:0/277: write da/fb [4892829,56739] 0 2024-02-16T22:56:18.195 INFO:tasks.workunit.client.0.smithi064.stdout:4/297: truncate dc/dd/f42 6995367 0 2024-02-16T22:56:18.196 INFO:tasks.workunit.client.1.smithi192.stdout:7/470: dread d1/d2/de/d2c/d49/f71 [0,4194304] 0 2024-02-16T22:56:18.196 INFO:tasks.workunit.client.1.smithi192.stdout:7/471: read d1/d2/de/d2c/d49/f71 [2820724,63441] 0 2024-02-16T22:56:18.197 INFO:tasks.workunit.client.0.smithi064.stdout:3/381: dwrite d4/d1f/d5b/d51/d7c/f49 [0,4194304] 0 2024-02-16T22:56:18.197 INFO:tasks.workunit.client.1.smithi192.stdout:6/379: mkdir da/df/d6b/d74 0 2024-02-16T22:56:18.198 INFO:tasks.workunit.client.0.smithi064.stdout:1/282: creat d2/d4e/f65 x:0 0 0 2024-02-16T22:56:18.200 INFO:tasks.workunit.client.1.smithi192.stdout:0/380: rename d1/d3/d29/f3f to d1/f7a 0 2024-02-16T22:56:18.202 INFO:tasks.workunit.client.0.smithi064.stdout:2/273: symlink d7/de/l51 0 2024-02-16T22:56:18.202 INFO:tasks.workunit.client.0.smithi064.stdout:3/382: write d4/d13/d1b/d27/f7f [302311,46753] 0 2024-02-16T22:56:18.205 INFO:tasks.workunit.client.1.smithi192.stdout:6/380: dwrite da/d40/f4e [0,4194304] 0 2024-02-16T22:56:18.207 INFO:tasks.workunit.client.1.smithi192.stdout:8/400: readlink d5/db/de/l1d 0 2024-02-16T22:56:18.207 INFO:tasks.workunit.client.1.smithi192.stdout:6/381: truncate da/df/d18/f63 53143 0 2024-02-16T22:56:18.208 INFO:tasks.workunit.client.0.smithi064.stdout:9/308: getdents d3/d11/d15/d5a/d2a/d2e/d31 0 2024-02-16T22:56:18.211 INFO:tasks.workunit.client.0.smithi064.stdout:1/283: dwrite d2/df/d14/f59 [0,4194304] 0 2024-02-16T22:56:18.214 INFO:tasks.workunit.client.1.smithi192.stdout:8/401: dwrite d5/d16/f45 [0,4194304] 0 2024-02-16T22:56:18.214 INFO:tasks.workunit.client.0.smithi064.stdout:0/278: dwrite f3 [0,4194304] 0 2024-02-16T22:56:18.216 INFO:tasks.workunit.client.0.smithi064.stdout:0/279: truncate da/d26/f28 4223128 0 2024-02-16T22:56:18.218 INFO:tasks.workunit.client.1.smithi192.stdout:9/498: dread d7/d15/d36/f7b [0,4194304] 0 2024-02-16T22:56:18.221 INFO:tasks.workunit.client.1.smithi192.stdout:5/383: rename d6/d12/d4e/d58/d6c to d6/d8/d80 0 2024-02-16T22:56:18.224 INFO:tasks.workunit.client.1.smithi192.stdout:9/499: dwrite d7/d19/d37/d5a/fa7 [0,4194304] 0 2024-02-16T22:56:18.224 INFO:tasks.workunit.client.1.smithi192.stdout:0/381: chown d1/d3/de/d22/d37/d67/l2d 4188 1 2024-02-16T22:56:18.225 INFO:tasks.workunit.client.1.smithi192.stdout:9/500: truncate d7/d15/d36/f99 662833 0 2024-02-16T22:56:18.226 INFO:tasks.workunit.client.0.smithi064.stdout:2/274: creat d7/d14/d25/f52 x:0 0 0 2024-02-16T22:56:18.227 INFO:tasks.workunit.client.0.smithi064.stdout:4/298: write dc/dd/d16/f2e [3538722,106862] 0 2024-02-16T22:56:18.228 INFO:tasks.workunit.client.0.smithi064.stdout:4/299: read dc/dd/d32/d22/f56 [573404,78411] 0 2024-02-16T22:56:18.229 INFO:tasks.workunit.client.0.smithi064.stdout:9/309: rmdir d3/d9/d1e 39 2024-02-16T22:56:18.230 INFO:tasks.workunit.client.0.smithi064.stdout:2/275: write d7/d14/d25/f4a [3056926,21912] 0 2024-02-16T22:56:18.233 INFO:tasks.workunit.client.0.smithi064.stdout:1/284: mknod d2/df/d14/d18/d40/c66 0 2024-02-16T22:56:18.234 INFO:tasks.workunit.client.0.smithi064.stdout:9/310: write d3/d11/d15/d5a/d2a/d2e/f39 [4175112,110859] 0 2024-02-16T22:56:18.236 INFO:tasks.workunit.client.0.smithi064.stdout:1/285: chown d2/df/d11/f3b 0 1 2024-02-16T22:56:18.237 INFO:tasks.workunit.client.1.smithi192.stdout:1/354: getdents d3/de/d27/d34/d52/d63 0 2024-02-16T22:56:18.240 INFO:tasks.workunit.client.1.smithi192.stdout:4/427: link d1/d15/d23/f4a d1/d26/d66/f9e 0 2024-02-16T22:56:18.242 INFO:tasks.workunit.client.0.smithi064.stdout:2/276: dwrite d7/d14/f33 [0,4194304] 0 2024-02-16T22:56:18.247 INFO:tasks.workunit.client.0.smithi064.stdout:0/280: creat da/dc/d3b/d5a/f5d x:0 0 0 2024-02-16T22:56:18.248 INFO:tasks.workunit.client.0.smithi064.stdout:9/311: creat d3/f69 x:0 0 0 2024-02-16T22:56:18.250 INFO:tasks.workunit.client.0.smithi064.stdout:3/383: link d4/d13/d1b/d27/d61/d63/l83 d4/d13/d1b/l85 0 2024-02-16T22:56:18.251 INFO:tasks.workunit.client.0.smithi064.stdout:1/286: symlink d2/df/l67 0 2024-02-16T22:56:18.252 INFO:tasks.workunit.client.1.smithi192.stdout:0/382: symlink d1/d3/d39/l7b 0 2024-02-16T22:56:18.253 INFO:tasks.workunit.client.1.smithi192.stdout:9/501: creat d7/d19/d28/d43/d70/d8e/fac x:0 0 0 2024-02-16T22:56:18.253 INFO:tasks.workunit.client.0.smithi064.stdout:2/277: read d7/f8 [2172712,66736] 0 2024-02-16T22:56:18.255 INFO:tasks.workunit.client.1.smithi192.stdout:3/410: symlink da/d11/d63/d6b/d66/d75/l8e 0 2024-02-16T22:56:18.256 INFO:tasks.workunit.client.1.smithi192.stdout:3/411: chown da/d11/l13 15089 1 2024-02-16T22:56:18.259 INFO:tasks.workunit.client.1.smithi192.stdout:1/355: mkdir d3/d5b/d77 0 2024-02-16T22:56:18.262 INFO:tasks.workunit.client.1.smithi192.stdout:4/428: write d1/d26/d66/d89/f68 [1016569,108384] 0 2024-02-16T22:56:18.270 INFO:tasks.workunit.client.1.smithi192.stdout:7/472: creat d1/fac x:0 0 0 2024-02-16T22:56:18.271 INFO:tasks.workunit.client.1.smithi192.stdout:6/382: dread da/df/d18/d52/f55 [0,4194304] 0 2024-02-16T22:56:18.272 INFO:tasks.workunit.client.0.smithi064.stdout:3/384: fsync d4/d14/f22 0 2024-02-16T22:56:18.275 INFO:tasks.workunit.client.0.smithi064.stdout:6/341: sync 2024-02-16T22:56:18.276 INFO:tasks.workunit.client.0.smithi064.stdout:2/278: mknod d7/de/c53 0 2024-02-16T22:56:18.278 INFO:tasks.workunit.client.0.smithi064.stdout:0/281: dread da/dc/de/f17 [0,4194304] 0 2024-02-16T22:56:18.283 INFO:tasks.workunit.client.0.smithi064.stdout:3/385: chown d4/d14/d2b/d72 58159 1 2024-02-16T22:56:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:17 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:17 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:18.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:17 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1632917819' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:56:18.285 INFO:tasks.workunit.client.1.smithi192.stdout:1/356: creat d3/de/d27/d34/d52/f78 x:0 0 0 2024-02-16T22:56:18.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/502: rmdir d7 39 2024-02-16T22:56:18.288 INFO:tasks.workunit.client.0.smithi064.stdout:3/386: dread d4/d13/d41/f3b [0,4194304] 0 2024-02-16T22:56:18.291 INFO:tasks.workunit.client.0.smithi064.stdout:6/342: creat d0/d5/df/f63 x:0 0 0 2024-02-16T22:56:18.292 INFO:tasks.workunit.client.1.smithi192.stdout:4/429: mknod d1/d15/c9f 0 2024-02-16T22:56:18.293 INFO:tasks.workunit.client.1.smithi192.stdout:6/383: dread da/df/d18/f43 [0,4194304] 0 2024-02-16T22:56:18.293 INFO:tasks.workunit.client.0.smithi064.stdout:4/300: truncate dc/d1d/f2b 650302 0 2024-02-16T22:56:18.295 INFO:tasks.workunit.client.0.smithi064.stdout:4/301: write dc/dd/d16/f23 [2632946,99021] 0 2024-02-16T22:56:18.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/402: link d5/d26/l42 d5/d3a/d4d/l75 0 2024-02-16T22:56:18.297 INFO:tasks.workunit.client.0.smithi064.stdout:0/282: fdatasync da/dc/de/d12/d2c/d2e/f3e 0 2024-02-16T22:56:18.300 INFO:tasks.workunit.client.0.smithi064.stdout:9/312: write d3/d11/d15/d5a/f52 [843122,123085] 0 2024-02-16T22:56:18.301 INFO:tasks.workunit.client.1.smithi192.stdout:2/436: rename d22/d24/d2b/d2c/d3f/l7f to d22/d24/d39/l93 0 2024-02-16T22:56:18.302 INFO:tasks.workunit.client.1.smithi192.stdout:2/437: truncate d22/d24/d39/d42/f5f 37669 0 2024-02-16T22:56:18.303 INFO:tasks.workunit.client.1.smithi192.stdout:2/438: dread - d22/d24/d2b/f8a zero size 2024-02-16T22:56:18.304 INFO:tasks.workunit.client.0.smithi064.stdout:1/287: getdents d2/df/d11/d1a/d1d 0 2024-02-16T22:56:18.305 INFO:tasks.workunit.client.1.smithi192.stdout:5/384: getdents d6/d12/d4e/d5b/d67/d78 0 2024-02-16T22:56:18.307 INFO:tasks.workunit.client.0.smithi064.stdout:3/387: mknod d4/d1f/c86 0 2024-02-16T22:56:18.310 INFO:tasks.workunit.client.0.smithi064.stdout:6/343: fsync d0/d1d/d21/f2d 0 2024-02-16T22:56:18.312 INFO:tasks.workunit.client.1.smithi192.stdout:3/412: link da/d11/d20/f24 da/df/f8f 0 2024-02-16T22:56:18.315 INFO:tasks.workunit.client.1.smithi192.stdout:1/357: creat d3/de/d27/d34/d36/d47/f79 x:0 0 0 2024-02-16T22:56:18.315 INFO:tasks.workunit.client.0.smithi064.stdout:2/279: dwrite d7/f8 [0,4194304] 0 2024-02-16T22:56:18.316 INFO:tasks.workunit.client.1.smithi192.stdout:1/358: chown d3/d40 1918311940 1 2024-02-16T22:56:18.318 INFO:tasks.workunit.client.0.smithi064.stdout:4/302: truncate dc/f33 4346091 0 2024-02-16T22:56:18.321 INFO:tasks.workunit.client.1.smithi192.stdout:4/430: fdatasync d1/d26/d66/d89/f1e 0 2024-02-16T22:56:18.322 INFO:tasks.workunit.client.1.smithi192.stdout:9/503: dwrite d7/d15/d36/d54/f96 [0,4194304] 0 2024-02-16T22:56:18.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/431: chown d1/d6b/f7f 377 1 2024-02-16T22:56:18.324 INFO:tasks.workunit.client.0.smithi064.stdout:2/280: dwrite d7/d14/d25/f52 [0,4194304] 0 2024-02-16T22:56:18.327 INFO:tasks.workunit.client.1.smithi192.stdout:7/473: creat d1/d2/de/d83/d9f/fad x:0 0 0 2024-02-16T22:56:18.333 INFO:tasks.workunit.client.0.smithi064.stdout:1/288: write d2/df/d11/d1a/d1d/f58 [1003451,95866] 0 2024-02-16T22:56:18.337 INFO:tasks.workunit.client.1.smithi192.stdout:0/383: link d1/d3/d39/f79 d1/d3/f7c 0 2024-02-16T22:56:18.341 INFO:tasks.workunit.client.1.smithi192.stdout:3/413: rename da/d11/d63/d6b/l48 to da/d11/d20/d39/l90 0 2024-02-16T22:56:18.342 INFO:tasks.workunit.client.0.smithi064.stdout:4/303: creat dc/dd/d16/f5d x:0 0 0 2024-02-16T22:56:18.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/359: dwrite d3/de/d27/d34/f45 [0,4194304] 0 2024-02-16T22:56:18.348 INFO:tasks.workunit.client.0.smithi064.stdout:0/283: write da/dc/de/d12/d1d/f2b [3607794,100763] 0 2024-02-16T22:56:18.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/360: stat d3/l33 0 2024-02-16T22:56:18.349 INFO:tasks.workunit.client.0.smithi064.stdout:0/284: write da/dc/d3b/f3c [2527528,49667] 0 2024-02-16T22:56:18.351 INFO:tasks.workunit.client.1.smithi192.stdout:9/504: symlink d7/d19/d37/d39/lad 0 2024-02-16T22:56:18.351 INFO:tasks.workunit.client.1.smithi192.stdout:2/439: write f5 [5255217,56756] 0 2024-02-16T22:56:18.353 INFO:tasks.workunit.client.1.smithi192.stdout:4/432: mknod d1/d6b/ca0 0 2024-02-16T22:56:18.355 INFO:tasks.workunit.client.1.smithi192.stdout:4/433: chown d1/d26/d66/d89/f97 234 1 2024-02-16T22:56:18.356 INFO:tasks.workunit.client.1.smithi192.stdout:4/434: stat d1/d26/c32 0 2024-02-16T22:56:18.358 INFO:tasks.workunit.client.1.smithi192.stdout:5/385: dwrite d6/d12/d2d/f42 [0,4194304] 0 2024-02-16T22:56:18.360 INFO:tasks.workunit.client.0.smithi064.stdout:1/289: rename d2/df/d11/d1a/d30/f41 to d2/df/d14/d18/d40/f68 0 2024-02-16T22:56:18.364 INFO:tasks.workunit.client.1.smithi192.stdout:3/414: mknod da/d11/d20/d39/d53/c91 0 2024-02-16T22:56:18.365 INFO:tasks.workunit.client.1.smithi192.stdout:3/415: write da/d11/d20/f8b [157725,29499] 0 2024-02-16T22:56:18.366 INFO:tasks.workunit.client.0.smithi064.stdout:0/285: mknod da/dc/de/d12/d1d/c5e 0 2024-02-16T22:56:18.367 INFO:tasks.workunit.client.1.smithi192.stdout:1/361: mkdir d3/de/d27/d34/d52/d63/d7a 0 2024-02-16T22:56:18.367 INFO:tasks.workunit.client.0.smithi064.stdout:0/286: write da/dc/de/d12/d1d/f2b [5101467,16437] 0 2024-02-16T22:56:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:17 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:17 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:17 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1632917819' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:56:18.369 INFO:tasks.workunit.client.0.smithi064.stdout:9/313: mkdir d3/d9/d6a 0 2024-02-16T22:56:18.370 INFO:tasks.workunit.client.1.smithi192.stdout:9/505: unlink d7/d15/d36/l48 0 2024-02-16T22:56:18.372 INFO:tasks.workunit.client.0.smithi064.stdout:6/344: rmdir d0/d12/d5a 0 2024-02-16T22:56:18.375 INFO:tasks.workunit.client.1.smithi192.stdout:6/384: creat da/df/d12/f75 x:0 0 0 2024-02-16T22:56:18.376 INFO:tasks.workunit.client.1.smithi192.stdout:6/385: write da/df/d12/d15/f31 [5238462,33949] 0 2024-02-16T22:56:18.378 INFO:tasks.workunit.client.0.smithi064.stdout:1/290: creat d2/df/f69 x:0 0 0 2024-02-16T22:56:18.381 INFO:tasks.workunit.client.1.smithi192.stdout:4/435: stat d1/d26/d66/d89/d11/c62 0 2024-02-16T22:56:18.381 INFO:tasks.workunit.client.1.smithi192.stdout:4/436: stat d1/d26/l94 0 2024-02-16T22:56:18.381 INFO:tasks.workunit.client.1.smithi192.stdout:6/386: dread da/df/d12/d57/f2f [0,4194304] 0 2024-02-16T22:56:18.382 INFO:tasks.workunit.client.0.smithi064.stdout:2/281: creat d7/f54 x:0 0 0 2024-02-16T22:56:18.383 INFO:tasks.workunit.client.1.smithi192.stdout:5/386: mknod d6/d12/d4e/d5b/c81 0 2024-02-16T22:56:18.387 INFO:tasks.workunit.client.1.smithi192.stdout:1/362: symlink d3/de/d27/d34/d36/d47/d5d/l7b 0 2024-02-16T22:56:18.388 INFO:tasks.workunit.client.1.smithi192.stdout:1/363: fdatasync d3/fb 0 2024-02-16T22:56:18.390 INFO:tasks.workunit.client.1.smithi192.stdout:9/506: creat d7/d15/d36/d38/d3f/d8a/d98/fae x:0 0 0 2024-02-16T22:56:18.391 INFO:tasks.workunit.client.0.smithi064.stdout:1/291: symlink d2/d4e/l6a 0 2024-02-16T22:56:18.393 INFO:tasks.workunit.client.1.smithi192.stdout:2/440: mknod d22/d24/d2b/d2c/d78/c94 0 2024-02-16T22:56:18.393 INFO:tasks.workunit.client.1.smithi192.stdout:2/441: fsync d22/d24/d39/d42/d65/d66/f83 0 2024-02-16T22:56:18.394 INFO:tasks.workunit.client.1.smithi192.stdout:2/442: dread - d22/d24/d39/d42/d85/f86 zero size 2024-02-16T22:56:18.399 INFO:tasks.workunit.client.1.smithi192.stdout:7/474: truncate d1/d2/d2d/d61/d88/d90/f78 2164054 0 2024-02-16T22:56:18.400 INFO:tasks.workunit.client.1.smithi192.stdout:7/475: stat d1/d2/de/d2c/f8b 0 2024-02-16T22:56:18.404 INFO:tasks.workunit.client.1.smithi192.stdout:6/387: fsync da/df/d12/d57/f2f 0 2024-02-16T22:56:18.404 INFO:tasks.workunit.client.0.smithi064.stdout:9/314: mknod d3/d9/d1e/d3e/c6b 0 2024-02-16T22:56:18.404 INFO:tasks.workunit.client.0.smithi064.stdout:9/315: dread - d3/d9/d1e/d42/f58 zero size 2024-02-16T22:56:18.405 INFO:tasks.workunit.client.1.smithi192.stdout:1/364: dread d3/de/d27/d34/d52/f57 [0,4194304] 0 2024-02-16T22:56:18.405 INFO:tasks.workunit.client.1.smithi192.stdout:1/365: fsync d3/d40/f59 0 2024-02-16T22:56:18.406 INFO:tasks.workunit.client.0.smithi064.stdout:4/304: getdents dc/dd/d4f 0 2024-02-16T22:56:18.406 INFO:tasks.workunit.client.0.smithi064.stdout:4/305: chown ca 229 1 2024-02-16T22:56:18.407 INFO:tasks.workunit.client.0.smithi064.stdout:4/306: chown dc/dd/d16/f5d 11491 1 2024-02-16T22:56:18.408 INFO:tasks.workunit.client.1.smithi192.stdout:0/384: dwrite d1/d3/de/d22/d37/d67/d34/f6e [0,4194304] 0 2024-02-16T22:56:18.409 INFO:tasks.workunit.client.0.smithi064.stdout:2/282: mkdir d7/d14/d25/d46/d55 0 2024-02-16T22:56:18.411 INFO:tasks.workunit.client.1.smithi192.stdout:9/507: creat d7/d15/d36/d77/faf x:0 0 0 2024-02-16T22:56:18.412 INFO:tasks.workunit.client.1.smithi192.stdout:4/437: write d1/d26/d66/f9e [1406533,37344] 0 2024-02-16T22:56:18.413 INFO:tasks.workunit.client.0.smithi064.stdout:9/316: chown d3/d11/f1f 269556322 1 2024-02-16T22:56:18.416 INFO:tasks.workunit.client.0.smithi064.stdout:6/345: unlink d0/d6/d10/d33/c47 0 2024-02-16T22:56:18.417 INFO:tasks.workunit.client.1.smithi192.stdout:9/508: dwrite d7/d15/f20 [8388608,4194304] 0 2024-02-16T22:56:18.418 INFO:tasks.workunit.client.1.smithi192.stdout:2/443: rename d22/d24/d2b/d2c/d3f/f6c to d22/d47/f95 0 2024-02-16T22:56:18.422 INFO:tasks.workunit.client.0.smithi064.stdout:1/292: unlink d2/df/f69 0 2024-02-16T22:56:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.424+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8fe0005590 con 0x7f8ffc0fc7f0 2024-02-16T22:56:18.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.425+0000 7f8ff97fa700 1 -- 172.21.15.64:0/3826636511 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8fe8057ba0 con 0x7f8ffc0fc7f0 2024-02-16T22:56:18.426 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:56:18.426 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:56:18.426 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:56:18.428 INFO:tasks.workunit.client.0.smithi064.stdout:2/283: write d7/d14/d1e/d34/f50 [2362406,95474] 0 2024-02-16T22:56:18.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.429+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f8fec06b2e0 msgr2=0x7f8fec06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:18.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.429+0000 7f9002a93700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f8fec06b2e0 0x7f8fec06d790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000b3a0 tx=0x7f8ff000c040 comp rx=0 tx=0).stop 2024-02-16T22:56:18.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.429+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc0fc7f0 msgr2=0x7f8ffc114580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:18.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.429+0000 7f9002a93700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc0fc7f0 0x7f8ffc114580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8004e40 tx=0x7f8fe800d040 comp rx=0 tx=0).stop 2024-02-16T22:56:18.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.429+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 shutdown_connections 2024-02-16T22:56:18.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.429+0000 7f9002a93700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f8fec06b2e0 0x7f8fec06d790 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000b3a0 tx=0x7f8ff000c040 comp rx=0 tx=0).stop 2024-02-16T22:56:18.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.429+0000 7f9002a93700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ffc0fb5e0 0x7f8ffc114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:18.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.429+0000 7f9002a93700 1 --2- 172.21.15.64:0/3826636511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc0fc7f0 0x7f8ffc114580 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:18.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.430+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 >> 172.21.15.64:0/3826636511 conn(0x7f8ffc0f6ca0 msgr2=0x7f8ffc0f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:18.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.430+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 shutdown_connections 2024-02-16T22:56:18.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:18.430+0000 7f9002a93700 1 -- 172.21.15.64:0/3826636511 wait complete. 2024-02-16T22:56:18.432 INFO:tasks.workunit.client.1.smithi192.stdout:0/385: symlink d1/d3/d29/d43/l7d 0 2024-02-16T22:56:18.433 INFO:tasks.workunit.client.1.smithi192.stdout:0/386: chown d1/d3/d29/d43/f68 112 1 2024-02-16T22:56:18.434 INFO:tasks.workunit.client.0.smithi064.stdout:9/317: symlink d3/d11/d15/d5a/l6c 0 2024-02-16T22:56:18.436 INFO:tasks.workunit.client.1.smithi192.stdout:9/509: creat d7/d19/d5e/d81/fb0 x:0 0 0 2024-02-16T22:56:18.436 INFO:tasks.workunit.client.0.smithi064.stdout:6/346: dread d0/d6/d10/d33/f2a [0,4194304] 0 2024-02-16T22:56:18.439 INFO:tasks.workunit.client.1.smithi192.stdout:1/366: rename d3/de/d27/c28 to d3/d5b/c7c 0 2024-02-16T22:56:18.444 INFO:tasks.workunit.client.0.smithi064.stdout:2/284: creat d7/de/f56 x:0 0 0 2024-02-16T22:56:18.444 INFO:tasks.workunit.client.1.smithi192.stdout:6/388: mkdir da/d76 0 2024-02-16T22:56:18.444 INFO:tasks.workunit.client.1.smithi192.stdout:6/389: chown da/d40/f54 65623377 1 2024-02-16T22:56:18.446 INFO:tasks.workunit.client.0.smithi064.stdout:9/318: dwrite d3/d9/f60 [4194304,4194304] 0 2024-02-16T22:56:18.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/387: creat d1/d3/d29/d43/f7e x:0 0 0 2024-02-16T22:56:18.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/388: chown d1/d3/de/d22/f28 1452 1 2024-02-16T22:56:18.447 INFO:tasks.workunit.client.1.smithi192.stdout:1/367: mknod d3/de/d27/d34/d52/c7d 0 2024-02-16T22:56:18.451 INFO:tasks.workunit.client.0.smithi064.stdout:2/285: mkdir d7/d14/d25/d46/d55/d57 0 2024-02-16T22:56:18.453 INFO:tasks.workunit.client.0.smithi064.stdout:6/347: dwrite d0/d6/d10/d33/f2a [0,4194304] 0 2024-02-16T22:56:18.454 INFO:tasks.workunit.client.1.smithi192.stdout:0/389: creat d1/d3/de/d22/d37/d67/d34/f7f x:0 0 0 2024-02-16T22:56:18.457 INFO:tasks.workunit.client.0.smithi064.stdout:2/286: mknod d7/d14/d1e/c58 0 2024-02-16T22:56:18.458 INFO:tasks.workunit.client.0.smithi064.stdout:2/287: dread - d7/d14/d25/f32 zero size 2024-02-16T22:56:18.467 INFO:tasks.workunit.client.0.smithi064.stdout:2/288: mknod d7/d14/d1e/d34/c59 0 2024-02-16T22:56:18.471 INFO:tasks.workunit.client.0.smithi064.stdout:4/307: dread dc/dd/d16/f2e [0,4194304] 0 2024-02-16T22:56:18.472 INFO:tasks.workunit.client.0.smithi064.stdout:4/308: fsync dc/f17 0 2024-02-16T22:56:18.473 INFO:tasks.workunit.client.0.smithi064.stdout:6/348: mkdir d0/d6/d10/d33/d64 0 2024-02-16T22:56:18.474 INFO:tasks.workunit.client.0.smithi064.stdout:5/218: sync 2024-02-16T22:56:18.474 INFO:tasks.workunit.client.0.smithi064.stdout:7/254: sync 2024-02-16T22:56:18.474 INFO:tasks.workunit.client.0.smithi064.stdout:8/265: sync 2024-02-16T22:56:18.475 INFO:tasks.workunit.client.0.smithi064.stdout:8/266: chown df/d2a/f2d 743496872 1 2024-02-16T22:56:18.477 INFO:tasks.workunit.client.0.smithi064.stdout:2/289: rename d7/f12 to d7/d14/d25/d46/d55/f5a 0 2024-02-16T22:56:18.477 INFO:tasks.workunit.client.0.smithi064.stdout:2/290: chown d7/d14/d1e/f28 0 1 2024-02-16T22:56:18.478 INFO:tasks.workunit.client.1.smithi192.stdout:7/476: dwrite d1/d2/f1e [0,4194304] 0 2024-02-16T22:56:18.478 INFO:tasks.workunit.client.0.smithi064.stdout:2/291: chown d7/de/c53 515670 1 2024-02-16T22:56:18.479 INFO:tasks.workunit.client.1.smithi192.stdout:7/477: dread - d1/d2/d2d/d61/d8a/f94 zero size 2024-02-16T22:56:18.479 INFO:tasks.workunit.client.1.smithi192.stdout:0/390: dread d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:18.484 INFO:tasks.workunit.client.0.smithi064.stdout:5/219: creat d6/d1c/d43/f4d x:0 0 0 2024-02-16T22:56:18.486 INFO:tasks.workunit.client.1.smithi192.stdout:4/438: write d1/d15/d29/f61 [1117835,37126] 0 2024-02-16T22:56:18.488 INFO:tasks.workunit.client.0.smithi064.stdout:8/267: creat df/d31/d2f/f58 x:0 0 0 2024-02-16T22:56:18.489 INFO:tasks.workunit.client.0.smithi064.stdout:5/220: dread d6/d25/f32 [0,4194304] 0 2024-02-16T22:56:18.493 INFO:tasks.workunit.client.1.smithi192.stdout:6/390: symlink da/df/d18/d46/l77 0 2024-02-16T22:56:18.499 INFO:tasks.workunit.client.1.smithi192.stdout:2/444: getdents d22/d24/d39/d42/d85 0 2024-02-16T22:56:18.500 INFO:tasks.workunit.client.0.smithi064.stdout:8/268: creat df/d31/d2f/f59 x:0 0 0 2024-02-16T22:56:18.501 INFO:tasks.workunit.client.0.smithi064.stdout:8/269: stat df/d2a/d4d 0 2024-02-16T22:56:18.504 INFO:tasks.workunit.client.1.smithi192.stdout:0/391: stat d1/f5 0 2024-02-16T22:56:18.516 INFO:tasks.workunit.client.1.smithi192.stdout:1/368: creat d3/f7e x:0 0 0 2024-02-16T22:56:18.518 INFO:tasks.workunit.client.1.smithi192.stdout:7/478: symlink d1/lae 0 2024-02-16T22:56:18.519 INFO:tasks.workunit.client.1.smithi192.stdout:7/479: readlink d1/d2/de/l16 0 2024-02-16T22:56:18.521 INFO:tasks.workunit.client.1.smithi192.stdout:1/369: dwrite d3/de/d27/f4a [4194304,4194304] 0 2024-02-16T22:56:18.522 INFO:tasks.workunit.client.1.smithi192.stdout:0/392: creat d1/d3/de/d22/d37/d67/d75/f80 x:0 0 0 2024-02-16T22:56:18.526 INFO:tasks.workunit.client.1.smithi192.stdout:6/391: rename da/df/d18/c6d to da/df/d12/d15/c78 0 2024-02-16T22:56:18.529 INFO:tasks.workunit.client.1.smithi192.stdout:2/445: dread fe [0,4194304] 0 2024-02-16T22:56:18.530 INFO:tasks.workunit.client.1.smithi192.stdout:4/439: getdents d1/d26/d6f 0 2024-02-16T22:56:18.532 INFO:tasks.workunit.client.1.smithi192.stdout:0/393: write d1/f8 [62414,52789] 0 2024-02-16T22:56:18.532 INFO:tasks.workunit.client.1.smithi192.stdout:6/392: mknod da/df/d18/d52/c79 0 2024-02-16T22:56:18.535 INFO:tasks.workunit.client.1.smithi192.stdout:1/370: unlink d3/d5b/c74 0 2024-02-16T22:56:18.535 INFO:tasks.workunit.client.1.smithi192.stdout:1/371: readlink d3/l3d 0 2024-02-16T22:56:18.536 INFO:tasks.workunit.client.1.smithi192.stdout:1/372: truncate d3/de/d27/f6a 882375 0 2024-02-16T22:56:18.537 INFO:tasks.workunit.client.1.smithi192.stdout:4/440: rmdir d1/d15/d29/d3c 39 2024-02-16T22:56:18.537 INFO:tasks.workunit.client.1.smithi192.stdout:1/373: chown d3/l60 0 1 2024-02-16T22:56:18.540 INFO:tasks.workunit.client.1.smithi192.stdout:0/394: creat d1/d3/de/d22/d37/d47/f81 x:0 0 0 2024-02-16T22:56:18.541 INFO:tasks.workunit.client.1.smithi192.stdout:0/395: write d1/d3/de/d22/d37/d47/f48 [1299266,71234] 0 2024-02-16T22:56:18.543 INFO:tasks.workunit.client.1.smithi192.stdout:2/446: link d22/d24/d39/d3d/f80 d22/d24/f96 0 2024-02-16T22:56:18.544 INFO:tasks.workunit.client.1.smithi192.stdout:7/480: write d1/d2/de/d2c/d49/da4/f4f [690789,28874] 0 2024-02-16T22:56:18.546 INFO:tasks.workunit.client.1.smithi192.stdout:6/393: mknod da/df/d12/d15/c7a 0 2024-02-16T22:56:18.550 INFO:tasks.workunit.client.1.smithi192.stdout:4/441: link d1/d15/d23/f1a d1/d6b/fa1 0 2024-02-16T22:56:18.551 INFO:tasks.workunit.client.1.smithi192.stdout:4/442: write d1/d26/d6f/f8f [7375,56431] 0 2024-02-16T22:56:18.552 INFO:tasks.workunit.client.1.smithi192.stdout:6/394: mknod da/df/d6b/c7b 0 2024-02-16T22:56:18.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/396: dwrite d1/d3/de/d22/f60 [0,4194304] 0 2024-02-16T22:56:18.554 INFO:tasks.workunit.client.1.smithi192.stdout:0/397: chown d1/d3/de/d22/l46 7772 1 2024-02-16T22:56:18.555 INFO:tasks.workunit.client.1.smithi192.stdout:2/447: link d22/d24/d2b/d2c/f2d d22/d24/d39/d42/d65/d69/d4f/f97 0 2024-02-16T22:56:18.558 INFO:tasks.workunit.client.1.smithi192.stdout:6/395: creat da/df/d18/d52/f7c x:0 0 0 2024-02-16T22:56:18.558 INFO:tasks.workunit.client.1.smithi192.stdout:0/398: creat d1/d3/d39/f82 x:0 0 0 2024-02-16T22:56:18.559 INFO:tasks.workunit.client.1.smithi192.stdout:1/374: getdents d3/de/d27/d34/d52/d5a 0 2024-02-16T22:56:18.560 INFO:tasks.workunit.client.1.smithi192.stdout:1/375: fsync d3/de/d27/d34/d36/d47/f5e 0 2024-02-16T22:56:18.560 INFO:tasks.workunit.client.1.smithi192.stdout:2/448: mkdir d22/d24/d39/d42/d85/d98 0 2024-02-16T22:56:18.561 INFO:tasks.workunit.client.1.smithi192.stdout:4/443: symlink d1/d26/d66/d89/la2 0 2024-02-16T22:56:18.562 INFO:tasks.workunit.client.1.smithi192.stdout:6/396: mkdir da/df/d6b/d7d 0 2024-02-16T22:56:18.564 INFO:tasks.workunit.client.1.smithi192.stdout:4/444: mkdir d1/d26/d6f/da3 0 2024-02-16T22:56:18.566 INFO:tasks.workunit.client.1.smithi192.stdout:7/481: dwrite d1/d2/de/d2c/d49/f6f [0,4194304] 0 2024-02-16T22:56:18.568 INFO:tasks.workunit.client.1.smithi192.stdout:0/399: creat d1/d3/f83 x:0 0 0 2024-02-16T22:56:18.569 INFO:tasks.workunit.client.1.smithi192.stdout:0/400: dread - d1/f61 zero size 2024-02-16T22:56:18.572 INFO:tasks.workunit.client.1.smithi192.stdout:2/449: getdents d22/d24/d39/d42/d65 0 2024-02-16T22:56:18.572 INFO:tasks.workunit.client.1.smithi192.stdout:6/397: dwrite da/d40/f54 [0,4194304] 0 2024-02-16T22:56:18.575 INFO:tasks.workunit.client.0.smithi064.stdout:6/349: creat d0/d6/d10/d33/f65 x:0 0 0 2024-02-16T22:56:18.577 INFO:tasks.workunit.client.1.smithi192.stdout:1/376: getdents d3/de/d27/d34/d52 0 2024-02-16T22:56:18.578 INFO:tasks.workunit.client.0.smithi064.stdout:4/309: truncate dc/dd/d32/d22/f56 2423850 0 2024-02-16T22:56:18.579 INFO:tasks.workunit.client.1.smithi192.stdout:6/398: dwrite da/df/d12/f47 [0,4194304] 0 2024-02-16T22:56:18.579 INFO:tasks.workunit.client.0.smithi064.stdout:8/270: write df/d31/f29 [1020180,120283] 0 2024-02-16T22:56:18.580 INFO:tasks.workunit.client.0.smithi064.stdout:6/350: truncate d0/d12/f49 783172 0 2024-02-16T22:56:18.581 INFO:tasks.workunit.client.1.smithi192.stdout:0/401: symlink d1/d3/de/l84 0 2024-02-16T22:56:18.581 INFO:tasks.workunit.client.0.smithi064.stdout:6/351: stat d0/d5/df/d17/d18 0 2024-02-16T22:56:18.582 INFO:tasks.workunit.client.1.smithi192.stdout:0/402: fdatasync d1/d3/d29/d43/f68 0 2024-02-16T22:56:18.585 INFO:tasks.workunit.client.1.smithi192.stdout:1/377: dwrite d3/de/d27/f3b [8388608,4194304] 0 2024-02-16T22:56:18.587 INFO:tasks.workunit.client.1.smithi192.stdout:4/445: link d1/d26/d66/d89/la2 d1/d6b/la4 0 2024-02-16T22:56:18.590 INFO:tasks.workunit.client.0.smithi064.stdout:4/310: creat dc/dd/d16/f5e x:0 0 0 2024-02-16T22:56:18.593 INFO:tasks.workunit.client.1.smithi192.stdout:0/403: creat d1/d3/de/d22/f85 x:0 0 0 2024-02-16T22:56:18.595 INFO:tasks.workunit.client.0.smithi064.stdout:2/292: rename d7/de/l4f to d7/d14/l5b 0 2024-02-16T22:56:18.596 INFO:tasks.workunit.client.1.smithi192.stdout:2/450: mkdir d22/d24/d2b/d99 0 2024-02-16T22:56:18.597 INFO:tasks.workunit.client.0.smithi064.stdout:6/352: creat d0/d1d/d21/f66 x:0 0 0 2024-02-16T22:56:18.598 INFO:tasks.workunit.client.1.smithi192.stdout:1/378: mknod d3/de/d27/d34/d36/d47/c7f 0 2024-02-16T22:56:18.599 INFO:tasks.workunit.client.0.smithi064.stdout:7/255: link d5/d6/l16 d5/d6/l4e 0 2024-02-16T22:56:18.601 INFO:tasks.workunit.client.0.smithi064.stdout:5/221: getdents d6/d18/d19 0 2024-02-16T22:56:18.602 INFO:tasks.workunit.client.0.smithi064.stdout:6/353: dread d0/f22 [0,4194304] 0 2024-02-16T22:56:18.608 INFO:tasks.workunit.client.0.smithi064.stdout:4/311: creat dc/d1d/d43/f5f x:0 0 0 2024-02-16T22:56:18.609 INFO:tasks.workunit.client.0.smithi064.stdout:6/354: dread d0/d5/df/d17/d18/f3f [0,4194304] 0 2024-02-16T22:56:18.609 INFO:tasks.workunit.client.0.smithi064.stdout:4/312: write dc/dd/d32/d22/f24 [3339237,54382] 0 2024-02-16T22:56:18.609 INFO:tasks.workunit.client.0.smithi064.stdout:6/355: fdatasync d0/d1b/f36 0 2024-02-16T22:56:18.609 INFO:tasks.workunit.client.1.smithi192.stdout:2/451: readlink d22/d24/d39/l93 0 2024-02-16T22:56:18.611 INFO:tasks.workunit.client.0.smithi064.stdout:5/222: creat d6/d18/f4e x:0 0 0 2024-02-16T22:56:18.615 INFO:tasks.workunit.client.1.smithi192.stdout:4/446: creat d1/d26/fa5 x:0 0 0 2024-02-16T22:56:18.616 INFO:tasks.workunit.client.0.smithi064.stdout:4/313: rmdir dc/d4a 39 2024-02-16T22:56:18.617 INFO:tasks.workunit.client.1.smithi192.stdout:7/482: dread d1/f11 [0,4194304] 0 2024-02-16T22:56:18.617 INFO:tasks.workunit.client.0.smithi064.stdout:4/314: dread - dc/dd/d32/d58/f38 zero size 2024-02-16T22:56:18.619 INFO:tasks.workunit.client.0.smithi064.stdout:4/315: write dc/dd/d32/d58/f12 [5161836,33296] 0 2024-02-16T22:56:18.619 INFO:tasks.workunit.client.0.smithi064.stdout:2/293: creat d7/d14/d25/d37/d44/f5c x:0 0 0 2024-02-16T22:56:18.624 INFO:tasks.workunit.client.0.smithi064.stdout:7/256: creat d5/d33/d42/f4f x:0 0 0 2024-02-16T22:56:18.624 INFO:tasks.workunit.client.0.smithi064.stdout:2/294: dread d7/d14/d25/f4a [0,4194304] 0 2024-02-16T22:56:18.624 INFO:tasks.workunit.client.1.smithi192.stdout:6/399: write da/df/d12/d15/f33 [1329396,88236] 0 2024-02-16T22:56:18.627 INFO:tasks.workunit.client.1.smithi192.stdout:1/379: write d3/de/d27/d34/f38 [831843,98506] 0 2024-02-16T22:56:18.627 INFO:tasks.workunit.client.1.smithi192.stdout:4/447: truncate d1/f5 1633864 0 2024-02-16T22:56:18.632 INFO:tasks.workunit.client.0.smithi064.stdout:4/316: dread dc/d1d/f2b [0,4194304] 0 2024-02-16T22:56:18.632 INFO:tasks.workunit.client.1.smithi192.stdout:6/400: write da/df/d12/d15/f2d [1521694,54083] 0 2024-02-16T22:56:18.633 INFO:tasks.workunit.client.1.smithi192.stdout:2/452: dwrite fe [4194304,4194304] 0 2024-02-16T22:56:18.637 INFO:tasks.workunit.client.1.smithi192.stdout:1/380: chown d3/l43 6 1 2024-02-16T22:56:18.639 INFO:tasks.workunit.client.0.smithi064.stdout:5/223: write d6/d18/d19/f2e [32045,18203] 0 2024-02-16T22:56:18.639 INFO:tasks.workunit.client.0.smithi064.stdout:5/224: chown d6/c15 55 1 2024-02-16T22:56:18.640 INFO:tasks.workunit.client.0.smithi064.stdout:2/295: creat d7/d14/d3e/f5d x:0 0 0 2024-02-16T22:56:18.642 INFO:tasks.workunit.client.1.smithi192.stdout:7/483: rmdir d1/d2/de/daa 0 2024-02-16T22:56:18.643 INFO:tasks.workunit.client.1.smithi192.stdout:6/401: truncate da/df/d18/f43 4962226 0 2024-02-16T22:56:18.644 INFO:tasks.workunit.client.1.smithi192.stdout:6/402: fdatasync da/df/d12/d15/f33 0 2024-02-16T22:56:18.646 INFO:tasks.workunit.client.1.smithi192.stdout:1/381: mknod d3/de/d27/d34/d52/d63/c80 0 2024-02-16T22:56:18.648 INFO:tasks.workunit.client.0.smithi064.stdout:5/225: mknod d6/d25/c4f 0 2024-02-16T22:56:18.648 INFO:tasks.workunit.client.1.smithi192.stdout:7/484: chown d1/d2/lc 72 1 2024-02-16T22:56:18.650 INFO:tasks.workunit.client.0.smithi064.stdout:4/317: getdents dc/dd/d5b 0 2024-02-16T22:56:18.651 INFO:tasks.workunit.client.1.smithi192.stdout:2/453: truncate d22/d24/f96 150721 0 2024-02-16T22:56:18.651 INFO:tasks.workunit.client.0.smithi064.stdout:6/356: getdents d0/d6/da 0 2024-02-16T22:56:18.654 INFO:tasks.workunit.client.1.smithi192.stdout:1/382: mkdir d3/de/d27/d34/d52/d63/d81 0 2024-02-16T22:56:18.659 INFO:tasks.workunit.client.1.smithi192.stdout:1/383: dwrite d3/de/d27/d34/f45 [0,4194304] 0 2024-02-16T22:56:18.660 INFO:tasks.workunit.client.0.smithi064.stdout:5/226: dwrite d6/d18/d19/f1f [0,4194304] 0 2024-02-16T22:56:18.661 INFO:tasks.workunit.client.0.smithi064.stdout:6/357: rename d0/d5/df/d17/d18/f51 to d0/d5/df/d17/d18/d5f/f67 0 2024-02-16T22:56:18.662 INFO:tasks.workunit.client.1.smithi192.stdout:1/384: read d3/de/d27/f39 [3957912,33222] 0 2024-02-16T22:56:18.663 INFO:tasks.workunit.client.0.smithi064.stdout:4/318: mknod dc/dd/d32/d46/d5a/c60 0 2024-02-16T22:56:18.663 INFO:tasks.workunit.client.0.smithi064.stdout:7/257: write d5/f2f [200086,83286] 0 2024-02-16T22:56:18.664 INFO:tasks.workunit.client.1.smithi192.stdout:6/403: symlink da/df/d18/l7e 0 2024-02-16T22:56:18.667 INFO:tasks.workunit.client.0.smithi064.stdout:2/296: write d7/f35 [1839926,17385] 0 2024-02-16T22:56:18.670 INFO:tasks.workunit.client.1.smithi192.stdout:8/403: sync 2024-02-16T22:56:18.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/404: chown d5/db/d47/d43 6834602 1 2024-02-16T22:56:18.672 INFO:tasks.workunit.client.1.smithi192.stdout:4/448: getdents d1/d26/d5e 0 2024-02-16T22:56:18.672 INFO:tasks.workunit.client.0.smithi064.stdout:4/319: dwrite dc/dd/d32/d46/f55 [0,4194304] 0 2024-02-16T22:56:18.673 INFO:tasks.workunit.client.1.smithi192.stdout:4/449: truncate d1/d26/d66/d89/d11/d58/f5d 829190 0 2024-02-16T22:56:18.674 INFO:tasks.workunit.client.0.smithi064.stdout:5/227: symlink d6/d25/l50 0 2024-02-16T22:56:18.674 INFO:tasks.workunit.client.1.smithi192.stdout:4/450: chown d1/d15/c5a 32 1 2024-02-16T22:56:18.679 INFO:tasks.workunit.client.1.smithi192.stdout:4/451: dwrite d1/d15/d48/d8d/f7b [0,4194304] 0 2024-02-16T22:56:18.680 INFO:tasks.workunit.client.1.smithi192.stdout:2/454: symlink d22/d24/d39/l9a 0 2024-02-16T22:56:18.682 INFO:tasks.workunit.client.1.smithi192.stdout:1/385: write d3/de/f35 [4098527,12134] 0 2024-02-16T22:56:18.682 INFO:tasks.workunit.client.0.smithi064.stdout:2/297: dwrite d7/d14/d25/d37/d44/f5c [0,4194304] 0 2024-02-16T22:56:18.685 INFO:tasks.workunit.client.1.smithi192.stdout:8/405: symlink d5/db/d47/l76 0 2024-02-16T22:56:18.685 INFO:tasks.workunit.client.0.smithi064.stdout:2/298: write d7/d14/d1e/d34/f50 [3086047,128576] 0 2024-02-16T22:56:18.686 INFO:tasks.workunit.client.0.smithi064.stdout:2/299: chown d7/d14/d25/d37/f47 2741397 1 2024-02-16T22:56:18.694 INFO:tasks.workunit.client.1.smithi192.stdout:7/485: dwrite d1/d2/d2d/d61/d88/d90/d2a/f3c [0,4194304] 0 2024-02-16T22:56:18.695 INFO:tasks.workunit.client.0.smithi064.stdout:2/300: dwrite d7/f8 [0,4194304] 0 2024-02-16T22:56:18.695 INFO:tasks.workunit.client.1.smithi192.stdout:6/404: dread da/f34 [0,4194304] 0 2024-02-16T22:56:18.695 INFO:tasks.workunit.client.0.smithi064.stdout:4/320: fdatasync dc/f33 0 2024-02-16T22:56:18.696 INFO:tasks.workunit.client.1.smithi192.stdout:6/405: fdatasync da/df/d12/f47 0 2024-02-16T22:56:18.696 INFO:tasks.workunit.client.1.smithi192.stdout:1/386: chown d3/l1c 1901347 1 2024-02-16T22:56:18.697 INFO:tasks.workunit.client.0.smithi064.stdout:2/301: chown d7/d14/d25/d46/d55 12682 1 2024-02-16T22:56:18.698 INFO:tasks.workunit.client.1.smithi192.stdout:6/406: fsync da/df/f1c 0 2024-02-16T22:56:18.698 INFO:tasks.workunit.client.1.smithi192.stdout:2/455: rmdir d22/d24/d2b/d2c/d78 39 2024-02-16T22:56:18.698 INFO:tasks.workunit.client.1.smithi192.stdout:4/452: dread d1/d15/f4f [0,4194304] 0 2024-02-16T22:56:18.699 INFO:tasks.workunit.client.1.smithi192.stdout:4/453: dread - d1/d26/d34/f50 zero size 2024-02-16T22:56:18.700 INFO:tasks.workunit.client.0.smithi064.stdout:6/358: rmdir d0/d6/d10/d33/d64 0 2024-02-16T22:56:18.701 INFO:tasks.workunit.client.0.smithi064.stdout:6/359: rename d0 to d0/d5/df/d17/d18/d5f/d68 22 2024-02-16T22:56:18.705 INFO:tasks.workunit.client.0.smithi064.stdout:2/302: mknod d7/d14/d25/d37/d44/c5e 0 2024-02-16T22:56:18.708 INFO:tasks.workunit.client.0.smithi064.stdout:4/321: unlink dc/dd/d16/f23 0 2024-02-16T22:56:18.709 INFO:tasks.workunit.client.0.smithi064.stdout:4/322: chown dc/d1d/f36 1250535 1 2024-02-16T22:56:18.711 INFO:tasks.workunit.client.0.smithi064.stdout:6/360: symlink d0/d5/df/d17/d4a/l69 0 2024-02-16T22:56:18.714 INFO:tasks.workunit.client.0.smithi064.stdout:2/303: rmdir d7/d14/d1e/d34/d3f 39 2024-02-16T22:56:18.718 INFO:tasks.workunit.client.1.smithi192.stdout:2/456: dread d22/d24/d39/d42/d65/d69/f68 [0,4194304] 0 2024-02-16T22:56:18.719 INFO:tasks.workunit.client.0.smithi064.stdout:5/228: dread f4 [0,4194304] 0 2024-02-16T22:56:18.721 INFO:tasks.workunit.client.0.smithi064.stdout:5/229: read d6/f23 [3103315,5745] 0 2024-02-16T22:56:18.722 INFO:tasks.workunit.client.0.smithi064.stdout:6/361: creat d0/d6/da/f6a x:0 0 0 2024-02-16T22:56:18.726 INFO:tasks.workunit.client.0.smithi064.stdout:4/323: mknod dc/dd/d32/d4b/c61 0 2024-02-16T22:56:18.728 INFO:tasks.workunit.client.1.smithi192.stdout:6/407: dread da/df/f1c [0,4194304] 0 2024-02-16T22:56:18.730 INFO:tasks.workunit.client.1.smithi192.stdout:1/387: link d3/de/f32 d3/de/f82 0 2024-02-16T22:56:18.732 INFO:tasks.workunit.client.1.smithi192.stdout:4/454: dread d1/d26/d66/d89/fc [0,4194304] 0 2024-02-16T22:56:18.733 INFO:tasks.workunit.client.0.smithi064.stdout:6/362: creat d0/d5/df/d17/d4a/f6b x:0 0 0 2024-02-16T22:56:18.733 INFO:tasks.workunit.client.0.smithi064.stdout:6/363: chown d0/d5/l38 1579689 1 2024-02-16T22:56:18.734 INFO:tasks.workunit.client.0.smithi064.stdout:6/364: write d0/f41 [228684,19514] 0 2024-02-16T22:56:18.736 INFO:tasks.workunit.client.1.smithi192.stdout:1/388: rename d3/c2a to d3/de/d27/d34/d69/c83 0 2024-02-16T22:56:18.737 INFO:tasks.workunit.client.0.smithi064.stdout:2/304: symlink d7/de/l5f 0 2024-02-16T22:56:18.738 INFO:tasks.workunit.client.1.smithi192.stdout:6/408: symlink da/df/d12/d15/d1b/d70/l7f 0 2024-02-16T22:56:18.739 INFO:tasks.workunit.client.1.smithi192.stdout:6/409: fdatasync da/df/d12/d15/f33 0 2024-02-16T22:56:18.739 INFO:tasks.workunit.client.0.smithi064.stdout:4/324: symlink dc/dd/d32/d46/l62 0 2024-02-16T22:56:18.739 INFO:tasks.workunit.client.0.smithi064.stdout:4/325: chown f3 0 1 2024-02-16T22:56:18.740 INFO:tasks.workunit.client.1.smithi192.stdout:4/455: creat d1/d26/d66/d89/d11/fa6 x:0 0 0 2024-02-16T22:56:18.742 INFO:tasks.workunit.client.0.smithi064.stdout:5/230: symlink d6/d1c/d4b/l51 0 2024-02-16T22:56:18.744 INFO:tasks.workunit.client.1.smithi192.stdout:4/456: mknod d1/d26/d6f/ca7 0 2024-02-16T22:56:18.745 INFO:tasks.workunit.client.1.smithi192.stdout:4/457: write d1/d15/d48/f7d [241327,7942] 0 2024-02-16T22:56:18.747 INFO:tasks.workunit.client.1.smithi192.stdout:4/458: write d1/f4 [3598262,82185] 0 2024-02-16T22:56:18.747 INFO:tasks.workunit.client.1.smithi192.stdout:4/459: stat d1/d26 0 2024-02-16T22:56:18.753 INFO:tasks.workunit.client.0.smithi064.stdout:6/365: symlink d0/d6/da/d56/l6c 0 2024-02-16T22:56:18.755 INFO:tasks.workunit.client.1.smithi192.stdout:2/457: write d22/d24/d39/d3d/f80 [969249,83552] 0 2024-02-16T22:56:18.756 INFO:tasks.workunit.client.0.smithi064.stdout:2/305: mkdir d7/de/d60 0 2024-02-16T22:56:18.757 INFO:tasks.workunit.client.0.smithi064.stdout:5/231: dwrite d6/d18/d19/f26 [0,4194304] 0 2024-02-16T22:56:18.757 INFO:tasks.workunit.client.1.smithi192.stdout:4/460: mkdir d1/d26/d66/d89/da8 0 2024-02-16T22:56:18.757 INFO:tasks.workunit.client.0.smithi064.stdout:5/232: readlink d6/l11 0 2024-02-16T22:56:18.758 INFO:tasks.workunit.client.1.smithi192.stdout:4/461: dread - d1/d26/d66/d89/f99 zero size 2024-02-16T22:56:18.758 INFO:tasks.workunit.client.0.smithi064.stdout:2/306: chown d7/d14/d25/f52 5 1 2024-02-16T22:56:18.759 INFO:tasks.workunit.client.0.smithi064.stdout:2/307: chown d7/d14/c48 46078 1 2024-02-16T22:56:18.759 INFO:tasks.workunit.client.0.smithi064.stdout:5/233: chown d6/d18/d19/f26 13572878 1 2024-02-16T22:56:18.761 INFO:tasks.workunit.client.0.smithi064.stdout:2/308: read d7/d14/f33 [2331089,73176] 0 2024-02-16T22:56:18.761 INFO:tasks.workunit.client.1.smithi192.stdout:6/410: getdents da/df 0 2024-02-16T22:56:18.762 INFO:tasks.workunit.client.0.smithi064.stdout:4/326: mknod dc/de/d28/d57/c63 0 2024-02-16T22:56:18.762 INFO:tasks.workunit.client.0.smithi064.stdout:5/234: dread d6/f31 [0,4194304] 0 2024-02-16T22:56:18.763 INFO:tasks.workunit.client.0.smithi064.stdout:2/309: write d7/f54 [1036420,108691] 0 2024-02-16T22:56:18.763 INFO:tasks.workunit.client.0.smithi064.stdout:6/366: unlink d0/d5/df/d17/c30 0 2024-02-16T22:56:18.764 INFO:tasks.workunit.client.0.smithi064.stdout:4/327: stat dc/d1d/d43/l48 0 2024-02-16T22:56:18.765 INFO:tasks.workunit.client.0.smithi064.stdout:6/367: dread - d0/d6/d10/d33/f65 zero size 2024-02-16T22:56:18.766 INFO:tasks.workunit.client.0.smithi064.stdout:5/235: mknod d6/d1c/d43/c52 0 2024-02-16T22:56:18.767 INFO:tasks.workunit.client.0.smithi064.stdout:4/328: fsync dc/f19 0 2024-02-16T22:56:18.770 INFO:tasks.workunit.client.0.smithi064.stdout:6/368: creat d0/d5/d54/d60/f6d x:0 0 0 2024-02-16T22:56:18.771 INFO:tasks.workunit.client.0.smithi064.stdout:2/310: creat d7/d14/d1e/d34/f61 x:0 0 0 2024-02-16T22:56:18.774 INFO:tasks.workunit.client.0.smithi064.stdout:6/369: rmdir d0/d6/da 39 2024-02-16T22:56:18.774 INFO:tasks.workunit.client.0.smithi064.stdout:5/236: dwrite d6/d18/d19/d1a/f3b [0,4194304] 0 2024-02-16T22:56:18.776 INFO:tasks.workunit.client.0.smithi064.stdout:5/237: write d6/d1c/d43/f4d [70731,66412] 0 2024-02-16T22:56:18.779 INFO:tasks.workunit.client.0.smithi064.stdout:4/329: dread dc/ff [0,4194304] 0 2024-02-16T22:56:18.780 INFO:tasks.workunit.client.0.smithi064.stdout:2/311: dread d7/f54 [0,4194304] 0 2024-02-16T22:56:18.781 INFO:tasks.workunit.client.1.smithi192.stdout:3/416: sync 2024-02-16T22:56:18.782 INFO:tasks.workunit.client.0.smithi064.stdout:5/238: creat d6/d1c/f53 x:0 0 0 2024-02-16T22:56:18.782 INFO:tasks.workunit.client.0.smithi064.stdout:5/239: stat d6/d1c/c44 0 2024-02-16T22:56:18.784 INFO:tasks.workunit.client.1.smithi192.stdout:4/462: dwrite d1/d26/d34/f50 [0,4194304] 0 2024-02-16T22:56:18.786 INFO:tasks.workunit.client.1.smithi192.stdout:4/463: truncate d1/d26/f77 1471695 0 2024-02-16T22:56:18.787 INFO:tasks.workunit.client.0.smithi064.stdout:5/240: creat d6/d1c/f54 x:0 0 0 2024-02-16T22:56:18.788 INFO:tasks.workunit.client.0.smithi064.stdout:4/330: dwrite dc/dd/d32/d22/f52 [0,4194304] 0 2024-02-16T22:56:18.789 INFO:tasks.workunit.client.1.smithi192.stdout:2/458: dread d22/f31 [0,4194304] 0 2024-02-16T22:56:18.789 INFO:tasks.workunit.client.1.smithi192.stdout:2/459: chown l1e 0 1 2024-02-16T22:56:18.791 INFO:tasks.workunit.client.1.smithi192.stdout:2/460: fsync d22/d24/d39/d42/d52/f60 0 2024-02-16T22:56:18.792 INFO:tasks.workunit.client.1.smithi192.stdout:4/464: rmdir d1/d26 39 2024-02-16T22:56:18.793 INFO:tasks.workunit.client.0.smithi064.stdout:6/370: link d0/d3c/l5b d0/d6/d10/d33/l6e 0 2024-02-16T22:56:18.793 INFO:tasks.workunit.client.1.smithi192.stdout:4/465: write d1/d15/d48/f95 [159697,31818] 0 2024-02-16T22:56:18.794 INFO:tasks.workunit.client.1.smithi192.stdout:6/411: unlink da/df/d18/l62 0 2024-02-16T22:56:18.795 INFO:tasks.workunit.client.1.smithi192.stdout:6/412: write da/df/d18/f63 [155722,1136] 0 2024-02-16T22:56:18.799 INFO:tasks.workunit.client.1.smithi192.stdout:3/417: write da/d11/d20/d39/d53/f62 [168102,44658] 0 2024-02-16T22:56:18.800 INFO:tasks.workunit.client.0.smithi064.stdout:2/312: creat d7/d14/d1e/d34/f62 x:0 0 0 2024-02-16T22:56:18.801 INFO:tasks.workunit.client.1.smithi192.stdout:4/466: dread - d1/d26/fa5 zero size 2024-02-16T22:56:18.801 INFO:tasks.workunit.client.0.smithi064.stdout:6/371: creat d0/d5/df/f6f x:0 0 0 2024-02-16T22:56:18.802 INFO:tasks.workunit.client.1.smithi192.stdout:4/467: dread - d1/d15/d29/f8b zero size 2024-02-16T22:56:18.802 INFO:tasks.workunit.client.0.smithi064.stdout:4/331: symlink dc/d4a/l64 0 2024-02-16T22:56:18.804 INFO:tasks.workunit.client.1.smithi192.stdout:6/413: creat da/df/d18/d46/f80 x:0 0 0 2024-02-16T22:56:18.804 INFO:tasks.workunit.client.0.smithi064.stdout:2/313: mkdir d7/d14/d4b/d63 0 2024-02-16T22:56:18.805 INFO:tasks.workunit.client.1.smithi192.stdout:2/461: mkdir d22/d24/d2b/d99/d9b 0 2024-02-16T22:56:18.805 INFO:tasks.workunit.client.0.smithi064.stdout:4/332: read dc/d1d/f4c [1158263,99160] 0 2024-02-16T22:56:18.805 INFO:tasks.workunit.client.1.smithi192.stdout:4/468: read d1/d26/d66/d89/d11/f51 [2199801,15556] 0 2024-02-16T22:56:18.811 INFO:tasks.workunit.client.0.smithi064.stdout:2/314: creat d7/d14/d4b/f64 x:0 0 0 2024-02-16T22:56:18.812 INFO:tasks.workunit.client.1.smithi192.stdout:2/462: dwrite d22/d24/d39/d3d/f80 [0,4194304] 0 2024-02-16T22:56:18.812 INFO:tasks.workunit.client.0.smithi064.stdout:6/372: truncate d0/d5/f42 129844 0 2024-02-16T22:56:18.812 INFO:tasks.workunit.client.0.smithi064.stdout:4/333: creat dc/dd/d32/d46/d5a/f65 x:0 0 0 2024-02-16T22:56:18.812 INFO:tasks.workunit.client.0.smithi064.stdout:4/334: write dc/f19 [1654721,87420] 0 2024-02-16T22:56:18.816 INFO:tasks.workunit.client.0.smithi064.stdout:2/315: unlink d7/de/c18 0 2024-02-16T22:56:18.820 INFO:tasks.workunit.client.1.smithi192.stdout:6/414: unlink da/df/d12/f39 0 2024-02-16T22:56:18.824 INFO:tasks.workunit.client.1.smithi192.stdout:3/418: rename da/d11/f54 to da/d11/d63/d6b/d66/f92 0 2024-02-16T22:56:18.824 INFO:tasks.workunit.client.0.smithi064.stdout:9/319: sync 2024-02-16T22:56:18.824 INFO:tasks.workunit.client.0.smithi064.stdout:1/293: sync 2024-02-16T22:56:18.824 INFO:tasks.workunit.client.0.smithi064.stdout:0/287: sync 2024-02-16T22:56:18.824 INFO:tasks.workunit.client.0.smithi064.stdout:3/388: sync 2024-02-16T22:56:18.826 INFO:tasks.workunit.client.0.smithi064.stdout:0/288: stat da/d26/f33 0 2024-02-16T22:56:18.826 INFO:tasks.workunit.client.1.smithi192.stdout:2/463: rmdir d22/d24/d2b/d99 39 2024-02-16T22:56:18.826 INFO:tasks.workunit.client.0.smithi064.stdout:1/294: write d2/f5c [630221,39102] 0 2024-02-16T22:56:18.831 INFO:tasks.workunit.client.0.smithi064.stdout:1/295: dread d2/df/d14/d2c/f3c [0,4194304] 0 2024-02-16T22:56:18.833 INFO:tasks.workunit.client.0.smithi064.stdout:4/335: rmdir dc/dd/d32/d46 39 2024-02-16T22:56:18.833 INFO:tasks.workunit.client.0.smithi064.stdout:2/316: dwrite d7/d14/d1e/d34/d3f/f43 [0,4194304] 0 2024-02-16T22:56:18.837 INFO:tasks.workunit.client.1.smithi192.stdout:3/419: creat da/d11/d20/d39/d53/f93 x:0 0 0 2024-02-16T22:56:18.838 INFO:tasks.workunit.client.1.smithi192.stdout:3/420: stat da/d11/d2b/c37 0 2024-02-16T22:56:18.838 INFO:tasks.workunit.client.0.smithi064.stdout:2/317: dread d7/d14/d1e/f21 [0,4194304] 0 2024-02-16T22:56:18.839 INFO:tasks.workunit.client.0.smithi064.stdout:3/389: creat d4/d14/d2b/d72/f87 x:0 0 0 2024-02-16T22:56:18.840 INFO:tasks.workunit.client.0.smithi064.stdout:2/318: dread - d7/d14/d1e/d34/f61 zero size 2024-02-16T22:56:18.841 INFO:tasks.workunit.client.0.smithi064.stdout:0/289: dread da/dc/f1e [0,4194304] 0 2024-02-16T22:56:18.848 INFO:tasks.workunit.client.1.smithi192.stdout:4/469: dwrite d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:18.849 INFO:tasks.workunit.client.0.smithi064.stdout:2/319: dwrite d7/d14/d4b/f64 [0,4194304] 0 2024-02-16T22:56:18.850 INFO:tasks.workunit.client.1.smithi192.stdout:9/510: sync 2024-02-16T22:56:18.850 INFO:tasks.workunit.client.1.smithi192.stdout:5/387: sync 2024-02-16T22:56:18.851 INFO:tasks.workunit.client.0.smithi064.stdout:9/320: dread d3/f18 [0,4194304] 0 2024-02-16T22:56:18.854 INFO:tasks.workunit.client.0.smithi064.stdout:0/290: unlink da/dc/de/c1a 0 2024-02-16T22:56:18.856 INFO:tasks.workunit.client.0.smithi064.stdout:0/291: readlink da/dc/de/d12/d1d/l4c 0 2024-02-16T22:56:18.857 INFO:tasks.workunit.client.0.smithi064.stdout:6/373: link d0/d6/d10/d33/l6e d0/d5/d54/l70 0 2024-02-16T22:56:18.857 INFO:tasks.workunit.client.0.smithi064.stdout:1/296: dwrite d2/f36 [4194304,4194304] 0 2024-02-16T22:56:18.859 INFO:tasks.workunit.client.1.smithi192.stdout:5/388: symlink d6/d12/d4e/d5b/l82 0 2024-02-16T22:56:18.862 INFO:tasks.workunit.client.1.smithi192.stdout:2/464: dread fa [0,4194304] 0 2024-02-16T22:56:18.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/465: chown d22/d24/d39/d42/d52/c76 1 1 2024-02-16T22:56:18.863 INFO:tasks.workunit.client.1.smithi192.stdout:9/511: mkdir d7/d15/d36/d77/db1 0 2024-02-16T22:56:18.864 INFO:tasks.workunit.client.1.smithi192.stdout:6/415: truncate da/df/d12/d15/f2d 673763 0 2024-02-16T22:56:18.865 INFO:tasks.workunit.client.1.smithi192.stdout:3/421: creat da/f94 x:0 0 0 2024-02-16T22:56:18.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/422: readlink da/df/l8c 0 2024-02-16T22:56:18.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/423: fsync da/f51 0 2024-02-16T22:56:18.869 INFO:tasks.workunit.client.0.smithi064.stdout:4/336: creat dc/dd/d32/d46/d5a/f66 x:0 0 0 2024-02-16T22:56:18.871 INFO:tasks.workunit.client.1.smithi192.stdout:5/389: creat d6/d12/f83 x:0 0 0 2024-02-16T22:56:18.871 INFO:tasks.workunit.client.1.smithi192.stdout:0/404: sync 2024-02-16T22:56:18.872 INFO:tasks.workunit.client.1.smithi192.stdout:1/389: sync 2024-02-16T22:56:18.872 INFO:tasks.workunit.client.1.smithi192.stdout:8/406: sync 2024-02-16T22:56:18.872 INFO:tasks.workunit.client.1.smithi192.stdout:7/486: sync 2024-02-16T22:56:18.873 INFO:tasks.workunit.client.1.smithi192.stdout:0/405: fsync d1/d3/de/f4e 0 2024-02-16T22:56:18.873 INFO:tasks.workunit.client.1.smithi192.stdout:8/407: stat d5/db/d47/d43/c6a 0 2024-02-16T22:56:18.873 INFO:tasks.workunit.client.1.smithi192.stdout:7/487: chown d1/d2/d2d/d61/d88/d90 59 1 2024-02-16T22:56:18.874 INFO:tasks.workunit.client.1.smithi192.stdout:0/406: write d1/f2a [4195477,45008] 0 2024-02-16T22:56:18.874 INFO:tasks.workunit.client.1.smithi192.stdout:8/408: write d5/d16/f27 [2393526,102136] 0 2024-02-16T22:56:18.875 INFO:tasks.workunit.client.0.smithi064.stdout:2/320: truncate d7/f1a 1704634 0 2024-02-16T22:56:18.876 INFO:tasks.workunit.client.0.smithi064.stdout:2/321: fdatasync d7/d14/d1e/d34/d3f/f4e 0 2024-02-16T22:56:18.876 INFO:tasks.workunit.client.0.smithi064.stdout:4/337: dwrite dc/f17 [0,4194304] 0 2024-02-16T22:56:18.878 INFO:tasks.workunit.client.1.smithi192.stdout:6/416: mkdir da/df/d12/d57/d81 0 2024-02-16T22:56:18.882 INFO:tasks.workunit.client.1.smithi192.stdout:0/407: dwrite d1/d3/d39/f76 [0,4194304] 0 2024-02-16T22:56:18.885 INFO:tasks.workunit.client.0.smithi064.stdout:4/338: dwrite dc/dd/d32/f1f [0,4194304] 0 2024-02-16T22:56:18.886 INFO:tasks.workunit.client.1.smithi192.stdout:0/408: read - d1/d3/de/d22/d37/d67/d34/f7f zero size 2024-02-16T22:56:18.887 INFO:tasks.workunit.client.0.smithi064.stdout:6/374: mknod d0/d5/df/d17/d18/d5f/c71 0 2024-02-16T22:56:18.888 INFO:tasks.workunit.client.1.smithi192.stdout:4/470: write d1/d26/d66/d89/f9 [3946210,107721] 0 2024-02-16T22:56:18.895 INFO:tasks.workunit.client.1.smithi192.stdout:6/417: dwrite da/df/d12/d57/f50 [0,4194304] 0 2024-02-16T22:56:18.896 INFO:tasks.workunit.client.0.smithi064.stdout:0/292: dread da/dc/de/d37/f46 [0,4194304] 0 2024-02-16T22:56:18.897 INFO:tasks.workunit.client.0.smithi064.stdout:1/297: mkdir d2/df/d14/d18/d48/d6b 0 2024-02-16T22:56:18.898 INFO:tasks.workunit.client.0.smithi064.stdout:2/322: dread d7/d14/d1e/f21 [0,4194304] 0 2024-02-16T22:56:18.899 INFO:tasks.workunit.client.1.smithi192.stdout:5/390: write d6/d12/d4e/d46/f5d [901674,120959] 0 2024-02-16T22:56:18.900 INFO:tasks.workunit.client.1.smithi192.stdout:1/390: truncate d3/de/d27/f42 1003971 0 2024-02-16T22:56:18.901 INFO:tasks.workunit.client.1.smithi192.stdout:8/409: mknod d5/db/d47/d43/c77 0 2024-02-16T22:56:18.902 INFO:tasks.workunit.client.0.smithi064.stdout:3/390: getdents d4/d13/d1b/d27 0 2024-02-16T22:56:18.902 INFO:tasks.workunit.client.0.smithi064.stdout:3/391: stat d4/d13/d1b/d48/c7a 0 2024-02-16T22:56:18.904 INFO:tasks.workunit.client.1.smithi192.stdout:0/409: dwrite d1/d3/de/d22/f60 [0,4194304] 0 2024-02-16T22:56:18.905 INFO:tasks.workunit.client.0.smithi064.stdout:6/375: fsync d0/d1b/f23 0 2024-02-16T22:56:18.905 INFO:tasks.workunit.client.0.smithi064.stdout:9/321: creat d3/d11/d15/d5a/f6d x:0 0 0 2024-02-16T22:56:18.906 INFO:tasks.workunit.client.1.smithi192.stdout:2/466: dwrite d22/f27 [0,4194304] 0 2024-02-16T22:56:18.907 INFO:tasks.workunit.client.0.smithi064.stdout:9/322: write d3/d11/d15/d5a/d38/f66 [325989,2097] 0 2024-02-16T22:56:18.907 INFO:tasks.workunit.client.1.smithi192.stdout:0/410: chown d1/d3/de/d22/d37/d67/d34 1333 1 2024-02-16T22:56:18.908 INFO:tasks.workunit.client.1.smithi192.stdout:3/424: mknod da/d11/d63/d6b/d4e/c95 0 2024-02-16T22:56:18.908 INFO:tasks.workunit.client.0.smithi064.stdout:9/323: chown d3/d11/d15/d23 3 1 2024-02-16T22:56:18.908 INFO:tasks.workunit.client.0.smithi064.stdout:0/293: chown da/c24 74183 1 2024-02-16T22:56:18.910 INFO:tasks.workunit.client.0.smithi064.stdout:1/298: write d2/df/d14/d18/d40/f68 [146828,25846] 0 2024-02-16T22:56:18.910 INFO:tasks.workunit.client.0.smithi064.stdout:9/324: chown d3/d11/d15/d5a/d2a/d2e/f5f 119453 1 2024-02-16T22:56:18.913 INFO:tasks.workunit.client.1.smithi192.stdout:0/411: dread d1/d3/d29/f1a [0,4194304] 0 2024-02-16T22:56:18.916 INFO:tasks.workunit.client.1.smithi192.stdout:1/391: creat d3/d5b/f84 x:0 0 0 2024-02-16T22:56:18.920 INFO:tasks.workunit.client.0.smithi064.stdout:2/323: dwrite d7/f54 [0,4194304] 0 2024-02-16T22:56:18.923 INFO:tasks.workunit.client.1.smithi192.stdout:8/410: truncate d5/db/de/df/f1b 3440079 0 2024-02-16T22:56:18.924 INFO:tasks.workunit.client.0.smithi064.stdout:1/299: dread d2/df/d11/d1a/f34 [0,4194304] 0 2024-02-16T22:56:18.925 INFO:tasks.workunit.client.0.smithi064.stdout:1/300: stat d2/f42 0 2024-02-16T22:56:18.925 INFO:tasks.workunit.client.0.smithi064.stdout:9/325: creat d3/d11/d15/d23/f6e x:0 0 0 2024-02-16T22:56:18.926 INFO:tasks.workunit.client.1.smithi192.stdout:2/467: write d22/f31 [1341914,44848] 0 2024-02-16T22:56:18.927 INFO:tasks.workunit.client.0.smithi064.stdout:1/301: write d2/d4e/d56/f57 [504123,78550] 0 2024-02-16T22:56:18.928 INFO:tasks.workunit.client.1.smithi192.stdout:4/471: mkdir d1/d26/d66/d89/da8/da9 0 2024-02-16T22:56:18.929 INFO:tasks.workunit.client.1.smithi192.stdout:4/472: dread - d1/d15/d29/f8b zero size 2024-02-16T22:56:18.929 INFO:tasks.workunit.client.1.smithi192.stdout:3/425: unlink da/d11/d20/c34 0 2024-02-16T22:56:18.931 INFO:tasks.workunit.client.1.smithi192.stdout:7/488: creat d1/faf x:0 0 0 2024-02-16T22:56:18.932 INFO:tasks.workunit.client.1.smithi192.stdout:0/412: dread d1/f51 [0,4194304] 0 2024-02-16T22:56:18.935 INFO:tasks.workunit.client.1.smithi192.stdout:5/391: dread d6/fd [0,4194304] 0 2024-02-16T22:56:18.936 INFO:tasks.workunit.client.0.smithi064.stdout:9/326: creat d3/d11/d15/d5a/d38/f6f x:0 0 0 2024-02-16T22:56:18.936 INFO:tasks.workunit.client.1.smithi192.stdout:5/392: readlink d6/d12/d4e/l59 0 2024-02-16T22:56:18.937 INFO:tasks.workunit.client.0.smithi064.stdout:9/327: chown d3/d11/d15/d5a 4 1 2024-02-16T22:56:18.937 INFO:tasks.workunit.client.1.smithi192.stdout:8/411: rmdir d5/db 39 2024-02-16T22:56:18.937 INFO:tasks.workunit.client.0.smithi064.stdout:9/328: readlink d3/d11/d15/l46 0 2024-02-16T22:56:18.938 INFO:tasks.workunit.client.1.smithi192.stdout:8/412: write d5/d3a/f72 [795234,84580] 0 2024-02-16T22:56:18.941 INFO:tasks.workunit.client.0.smithi064.stdout:1/302: dread d2/df/d11/d1a/d1d/f58 [0,4194304] 0 2024-02-16T22:56:18.941 INFO:tasks.workunit.client.0.smithi064.stdout:2/324: mkdir d7/d14/d1e/d34/d65 0 2024-02-16T22:56:18.941 INFO:tasks.workunit.client.0.smithi064.stdout:4/339: write dc/d1d/f2b [1195538,15066] 0 2024-02-16T22:56:18.942 INFO:tasks.workunit.client.1.smithi192.stdout:2/468: symlink d22/d24/d39/d42/d52/l9c 0 2024-02-16T22:56:18.943 INFO:tasks.workunit.client.0.smithi064.stdout:1/303: chown d2/df/d14/d18/f26 1037781939 1 2024-02-16T22:56:18.944 INFO:tasks.workunit.client.1.smithi192.stdout:9/512: truncate f0 3837231 0 2024-02-16T22:56:18.945 INFO:tasks.workunit.client.1.smithi192.stdout:9/513: chown d7/fe 11912 1 2024-02-16T22:56:18.946 INFO:tasks.workunit.client.1.smithi192.stdout:9/514: read d7/d15/f5d [451225,123450] 0 2024-02-16T22:56:18.947 INFO:tasks.workunit.client.1.smithi192.stdout:4/473: rename d1/d15/d23/f1a to d1/d26/d6f/faa 0 2024-02-16T22:56:18.947 INFO:tasks.workunit.client.1.smithi192.stdout:2/469: dread d22/d24/d39/d3d/f80 [0,4194304] 0 2024-02-16T22:56:18.953 INFO:tasks.workunit.client.1.smithi192.stdout:2/470: dread d22/d24/d39/d42/d65/d69/d4f/f97 [0,4194304] 0 2024-02-16T22:56:18.957 INFO:tasks.workunit.client.1.smithi192.stdout:4/474: dwrite d1/d26/d34/f50 [0,4194304] 0 2024-02-16T22:56:18.957 INFO:tasks.workunit.client.1.smithi192.stdout:6/418: dread da/df/d12/d15/f2d [0,4194304] 0 2024-02-16T22:56:18.959 INFO:tasks.workunit.client.1.smithi192.stdout:9/515: dwrite d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f9a [0,4194304] 0 2024-02-16T22:56:18.960 INFO:tasks.workunit.client.0.smithi064.stdout:0/294: write da/f1b [419398,99182] 0 2024-02-16T22:56:18.964 INFO:tasks.workunit.client.1.smithi192.stdout:2/471: dread d22/f27 [0,4194304] 0 2024-02-16T22:56:18.964 INFO:tasks.workunit.client.1.smithi192.stdout:8/413: dread d5/d9/f33 [4194304,4194304] 0 2024-02-16T22:56:18.965 INFO:tasks.workunit.client.0.smithi064.stdout:3/392: dwrite d4/d13/d1b/d27/f36 [0,4194304] 0 2024-02-16T22:56:18.966 INFO:tasks.workunit.client.0.smithi064.stdout:7/258: sync 2024-02-16T22:56:18.966 INFO:tasks.workunit.client.0.smithi064.stdout:8/271: sync 2024-02-16T22:56:18.967 INFO:tasks.workunit.client.0.smithi064.stdout:6/376: dwrite d0/d6/f15 [0,4194304] 0 2024-02-16T22:56:18.967 INFO:tasks.workunit.client.0.smithi064.stdout:0/295: readlink da/dc/l11 0 2024-02-16T22:56:18.971 INFO:tasks.workunit.client.0.smithi064.stdout:1/304: dwrite d2/df/d14/d2c/f3c [0,4194304] 0 2024-02-16T22:56:18.979 INFO:tasks.workunit.client.1.smithi192.stdout:1/392: write d3/de/f82 [821411,6432] 0 2024-02-16T22:56:18.981 INFO:tasks.workunit.client.0.smithi064.stdout:0/296: truncate da/dc/de/d12/d1d/f3d 714530 0 2024-02-16T22:56:18.982 INFO:tasks.workunit.client.1.smithi192.stdout:1/393: readlink d3/de/d27/d34/d52/d63/l6f 0 2024-02-16T22:56:18.983 INFO:tasks.workunit.client.0.smithi064.stdout:1/305: dwrite d2/df/d14/d18/f26 [0,4194304] 0 2024-02-16T22:56:18.983 INFO:tasks.workunit.client.1.smithi192.stdout:0/413: mknod d1/d3/c86 0 2024-02-16T22:56:18.984 INFO:tasks.workunit.client.1.smithi192.stdout:0/414: read - d1/d3/d39/f82 zero size 2024-02-16T22:56:18.993 INFO:tasks.workunit.client.1.smithi192.stdout:5/393: mknod d6/d12/d1d/c84 0 2024-02-16T22:56:18.994 INFO:tasks.workunit.client.1.smithi192.stdout:5/394: chown d6/d8/d75/l7c 13500 1 2024-02-16T22:56:18.996 INFO:tasks.workunit.client.0.smithi064.stdout:2/325: creat d7/d14/d25/d37/f66 x:0 0 0 2024-02-16T22:56:18.998 INFO:tasks.workunit.client.1.smithi192.stdout:3/426: dwrite da/df/f49 [0,4194304] 0 2024-02-16T22:56:18.999 INFO:tasks.workunit.client.1.smithi192.stdout:3/427: chown da/d11/d35/l7e 1 1 2024-02-16T22:56:18.999 INFO:tasks.workunit.client.1.smithi192.stdout:6/419: creat da/d40/f82 x:0 0 0 2024-02-16T22:56:19.000 INFO:tasks.workunit.client.1.smithi192.stdout:3/428: chown da/d11/d2b/l74 0 1 2024-02-16T22:56:19.001 INFO:tasks.workunit.client.1.smithi192.stdout:6/420: fsync da/df/d12/f47 0 2024-02-16T22:56:19.002 INFO:tasks.workunit.client.0.smithi064.stdout:7/259: rename d5/f32 to d5/d33/d35/d41/f50 0 2024-02-16T22:56:19.004 INFO:tasks.workunit.client.0.smithi064.stdout:9/329: truncate d3/d11/d15/d5a/d38/f66 33972 0 2024-02-16T22:56:19.005 INFO:tasks.workunit.client.0.smithi064.stdout:9/330: stat d3/d9/f36 0 2024-02-16T22:56:19.007 INFO:tasks.workunit.client.0.smithi064.stdout:9/331: dread d3/d9/f48 [0,4194304] 0 2024-02-16T22:56:19.007 INFO:tasks.workunit.client.0.smithi064.stdout:9/332: readlink d3/d9/d1e/l67 0 2024-02-16T22:56:19.008 INFO:tasks.workunit.client.1.smithi192.stdout:4/475: rename d1/d26/d66/d89/d11/f51 to d1/d26/d6f/fab 0 2024-02-16T22:56:19.009 INFO:tasks.workunit.client.0.smithi064.stdout:8/272: symlink df/d31/d2f/d38/d48/l5a 0 2024-02-16T22:56:19.011 INFO:tasks.workunit.client.0.smithi064.stdout:5/241: sync 2024-02-16T22:56:19.013 INFO:tasks.workunit.client.0.smithi064.stdout:0/297: creat da/dc/d3b/f5f x:0 0 0 2024-02-16T22:56:19.019 INFO:tasks.workunit.client.0.smithi064.stdout:6/377: mkdir d0/d6/d10/d33/d72 0 2024-02-16T22:56:19.019 INFO:tasks.workunit.client.0.smithi064.stdout:6/378: stat d0/d5/df/d17/d4a/f6b 0 2024-02-16T22:56:19.020 INFO:tasks.workunit.client.1.smithi192.stdout:1/394: read d3/de/d27/f30 [4296681,60945] 0 2024-02-16T22:56:19.021 INFO:tasks.workunit.client.0.smithi064.stdout:4/340: link dc/de/c2f dc/dd/d32/c67 0 2024-02-16T22:56:19.021 INFO:tasks.workunit.client.1.smithi192.stdout:6/421: fsync da/f34 0 2024-02-16T22:56:19.025 INFO:tasks.workunit.client.1.smithi192.stdout:9/516: rename d7/d15/d36/d38/d3f/d8a/d98/f16 to d7/d15/d36/d77/fb2 0 2024-02-16T22:56:19.026 INFO:tasks.workunit.client.1.smithi192.stdout:9/517: chown d7/d15/d36/l7c 24040216 1 2024-02-16T22:56:19.026 INFO:tasks.workunit.client.1.smithi192.stdout:9/518: readlink d7/d15/l5f 0 2024-02-16T22:56:19.027 INFO:tasks.workunit.client.0.smithi064.stdout:4/341: dwrite dc/d4a/f4d [0,4194304] 0 2024-02-16T22:56:19.028 INFO:tasks.workunit.client.1.smithi192.stdout:4/476: creat d1/d26/d66/fac x:0 0 0 2024-02-16T22:56:19.036 INFO:tasks.workunit.client.0.smithi064.stdout:5/242: unlink d6/f23 0 2024-02-16T22:56:19.037 INFO:tasks.workunit.client.1.smithi192.stdout:6/422: mkdir da/df/d6b/d83 0 2024-02-16T22:56:19.038 INFO:tasks.workunit.client.0.smithi064.stdout:0/298: mknod da/dc/c60 0 2024-02-16T22:56:19.043 INFO:tasks.workunit.client.1.smithi192.stdout:5/395: rename d6/f61 to d6/d8/d77/f85 0 2024-02-16T22:56:19.049 INFO:tasks.workunit.client.0.smithi064.stdout:6/379: mknod d0/d5/df/d17/c73 0 2024-02-16T22:56:19.050 INFO:tasks.workunit.client.1.smithi192.stdout:4/477: write d1/d26/d66/d89/f64 [142774,22038] 0 2024-02-16T22:56:19.054 INFO:tasks.workunit.client.0.smithi064.stdout:7/260: rename d5/d6/l4e to d5/d33/d35/l51 0 2024-02-16T22:56:19.054 INFO:tasks.workunit.client.0.smithi064.stdout:5/243: dread d6/fa [0,4194304] 0 2024-02-16T22:56:19.056 INFO:tasks.workunit.client.1.smithi192.stdout:8/414: link d5/d3e/f74 d5/db/d47/f78 0 2024-02-16T22:56:19.056 INFO:tasks.workunit.client.1.smithi192.stdout:3/429: dwrite da/df/f79 [0,4194304] 0 2024-02-16T22:56:19.060 INFO:tasks.workunit.client.0.smithi064.stdout:5/244: dwrite d6/d18/d19/f2e [0,4194304] 0 2024-02-16T22:56:19.065 INFO:tasks.workunit.client.1.smithi192.stdout:1/395: rename d3/d40/d55 to d3/de/d27/d34/d69/d85 0 2024-02-16T22:56:19.066 INFO:tasks.workunit.client.0.smithi064.stdout:8/273: symlink df/d31/d2f/d4f/l5b 0 2024-02-16T22:56:19.070 INFO:tasks.workunit.client.0.smithi064.stdout:1/306: truncate d2/df/d14/d18/f26 954033 0 2024-02-16T22:56:19.072 INFO:tasks.workunit.client.1.smithi192.stdout:2/472: link l17 d22/d24/d39/d3d/l9d 0 2024-02-16T22:56:19.074 INFO:tasks.workunit.client.1.smithi192.stdout:9/519: write f2 [5128835,98142] 0 2024-02-16T22:56:19.075 INFO:tasks.workunit.client.0.smithi064.stdout:6/380: creat d0/d12/f74 x:0 0 0 2024-02-16T22:56:19.077 INFO:tasks.workunit.client.0.smithi064.stdout:2/326: dwrite d7/d14/d25/d46/d55/f5a [0,4194304] 0 2024-02-16T22:56:19.078 INFO:tasks.workunit.client.1.smithi192.stdout:4/478: rename d1/d15/d48/f7d to d1/d15/d48/d8d/d9b/fad 0 2024-02-16T22:56:19.081 INFO:tasks.workunit.client.1.smithi192.stdout:1/396: chown d3/de/l13 95 1 2024-02-16T22:56:19.082 INFO:tasks.workunit.client.0.smithi064.stdout:7/261: creat d5/d33/d45/f52 x:0 0 0 2024-02-16T22:56:19.087 INFO:tasks.workunit.client.1.smithi192.stdout:9/520: mknod d7/d15/d36/d38/d3f/d8a/d98/cb3 0 2024-02-16T22:56:19.089 INFO:tasks.workunit.client.1.smithi192.stdout:9/521: write d7/d19/d5b/fa5 [703252,124333] 0 2024-02-16T22:56:19.094 INFO:tasks.workunit.client.0.smithi064.stdout:8/274: mkdir df/d2a/d5c 0 2024-02-16T22:56:19.095 INFO:tasks.workunit.client.1.smithi192.stdout:8/415: truncate d5/d3e/d5f/f5e 444745 0 2024-02-16T22:56:19.096 INFO:tasks.workunit.client.0.smithi064.stdout:3/393: getdents d4/d1f/d5b/d51 0 2024-02-16T22:56:19.097 INFO:tasks.workunit.client.0.smithi064.stdout:3/394: chown d4/d13/l25 108 1 2024-02-16T22:56:19.097 INFO:tasks.workunit.client.0.smithi064.stdout:4/342: truncate dc/d4a/f4d 2413095 0 2024-02-16T22:56:19.097 INFO:tasks.workunit.client.0.smithi064.stdout:3/395: fsync d4/f12 0 2024-02-16T22:56:19.097 INFO:tasks.workunit.client.0.smithi064.stdout:4/343: chown ca 9135 1 2024-02-16T22:56:19.099 INFO:tasks.workunit.client.0.smithi064.stdout:3/396: truncate d4/d14/f2a 1664768 0 2024-02-16T22:56:19.100 INFO:tasks.workunit.client.0.smithi064.stdout:1/307: mknod d2/d4e/c6c 0 2024-02-16T22:56:19.100 INFO:tasks.workunit.client.1.smithi192.stdout:9/522: dread - d7/d15/d36/d54/f65 zero size 2024-02-16T22:56:19.102 INFO:tasks.workunit.client.0.smithi064.stdout:1/308: dread d2/df/d14/d2c/f49 [0,4194304] 0 2024-02-16T22:56:19.105 INFO:tasks.workunit.client.0.smithi064.stdout:2/327: dread d7/d14/d25/d37/f47 [0,4194304] 0 2024-02-16T22:56:19.109 INFO:tasks.workunit.client.0.smithi064.stdout:7/262: mkdir d5/d33/d53 0 2024-02-16T22:56:19.111 INFO:tasks.workunit.client.1.smithi192.stdout:3/430: rename f3 to da/d11/f96 0 2024-02-16T22:56:19.114 INFO:tasks.workunit.client.0.smithi064.stdout:9/333: getdents d3/d11/d15/d5a/d2a/d2e/d31/d3f 0 2024-02-16T22:56:19.117 INFO:tasks.workunit.client.0.smithi064.stdout:8/275: creat df/d31/d2f/d38/f5d x:0 0 0 2024-02-16T22:56:19.118 INFO:tasks.workunit.client.1.smithi192.stdout:9/523: write d7/d15/f5d [155901,55059] 0 2024-02-16T22:56:19.121 INFO:tasks.workunit.client.0.smithi064.stdout:4/344: mknod dc/d1d/c68 0 2024-02-16T22:56:19.122 INFO:tasks.workunit.client.0.smithi064.stdout:5/245: truncate d6/fa 2806317 0 2024-02-16T22:56:19.122 INFO:tasks.workunit.client.0.smithi064.stdout:4/345: chown dc/d1d/f36 54 1 2024-02-16T22:56:19.123 INFO:tasks.workunit.client.1.smithi192.stdout:8/416: write d5/db/de/df/f1b [1330604,846] 0 2024-02-16T22:56:19.124 INFO:tasks.workunit.client.0.smithi064.stdout:3/397: unlink d4/l9 0 2024-02-16T22:56:19.129 INFO:tasks.workunit.client.0.smithi064.stdout:0/299: link da/dc/de/d12/d1d/f2b da/dc/de/d12/f61 0 2024-02-16T22:56:19.132 INFO:tasks.workunit.client.0.smithi064.stdout:1/309: rename d2/d4e/f45 to d2/df/d11/f6d 0 2024-02-16T22:56:19.133 INFO:tasks.workunit.client.1.smithi192.stdout:1/397: rename d3/de/c1d to d3/de/d27/d34/d36/d47/c86 0 2024-02-16T22:56:19.135 INFO:tasks.workunit.client.1.smithi192.stdout:8/417: creat d5/d9/f79 x:0 0 0 2024-02-16T22:56:19.135 INFO:tasks.workunit.client.1.smithi192.stdout:1/398: write d3/de/d27/d34/d36/d47/f79 [469863,68794] 0 2024-02-16T22:56:19.135 INFO:tasks.workunit.client.0.smithi064.stdout:0/300: dwrite da/dc/de/d12/d1d/f3d [0,4194304] 0 2024-02-16T22:56:19.136 INFO:tasks.workunit.client.1.smithi192.stdout:3/431: mknod da/d11/d63/d6b/c97 0 2024-02-16T22:56:19.137 INFO:tasks.workunit.client.1.smithi192.stdout:3/432: fdatasync da/d11/d20/d39/d53/f93 0 2024-02-16T22:56:19.137 INFO:tasks.workunit.client.1.smithi192.stdout:4/479: link d1/d26/d66/d89/c10 d1/cae 0 2024-02-16T22:56:19.140 INFO:tasks.workunit.client.0.smithi064.stdout:7/263: truncate d5/d6/f22 1477379 0 2024-02-16T22:56:19.140 INFO:tasks.workunit.client.0.smithi064.stdout:9/334: symlink d3/d11/d15/d5a/d2a/d2e/l70 0 2024-02-16T22:56:19.141 INFO:tasks.workunit.client.1.smithi192.stdout:8/418: mknod d5/db/d47/d43/d67/c7a 0 2024-02-16T22:56:19.144 INFO:tasks.workunit.client.0.smithi064.stdout:0/301: dwrite da/dc/de/d12/d2c/f31 [0,4194304] 0 2024-02-16T22:56:19.144 INFO:tasks.workunit.client.1.smithi192.stdout:3/433: dwrite da/d11/d63/d6b/d66/d75/f76 [0,4194304] 0 2024-02-16T22:56:19.146 INFO:tasks.workunit.client.0.smithi064.stdout:4/346: creat dc/d1d/f69 x:0 0 0 2024-02-16T22:56:19.147 INFO:tasks.workunit.client.1.smithi192.stdout:4/480: unlink d1/d15/d48/c7a 0 2024-02-16T22:56:19.147 INFO:tasks.workunit.client.1.smithi192.stdout:9/524: dwrite d7/d19/d28/f53 [0,4194304] 0 2024-02-16T22:56:19.147 INFO:tasks.workunit.client.0.smithi064.stdout:5/246: truncate f4 5006349 0 2024-02-16T22:56:19.147 INFO:tasks.workunit.client.1.smithi192.stdout:3/434: write da/d11/d63/d6b/d80/f83 [229392,107848] 0 2024-02-16T22:56:19.155 INFO:tasks.workunit.client.0.smithi064.stdout:5/247: dwrite d6/d18/d19/f47 [0,4194304] 0 2024-02-16T22:56:19.158 INFO:tasks.workunit.client.1.smithi192.stdout:9/525: dwrite d7/d19/d37/f9e [0,4194304] 0 2024-02-16T22:56:19.161 INFO:tasks.workunit.client.1.smithi192.stdout:1/399: dread d3/fb [4194304,4194304] 0 2024-02-16T22:56:19.163 INFO:tasks.workunit.client.1.smithi192.stdout:9/526: dread - d7/d19/f86 zero size 2024-02-16T22:56:19.166 INFO:tasks.workunit.client.1.smithi192.stdout:4/481: dwrite d1/d15/f4f [0,4194304] 0 2024-02-16T22:56:19.176 INFO:tasks.workunit.client.0.smithi064.stdout:1/310: rename d2/df/d14/c60 to d2/d4e/c6e 0 2024-02-16T22:56:19.177 INFO:tasks.workunit.client.0.smithi064.stdout:1/311: fdatasync d2/df/d14/d2c/f3c 0 2024-02-16T22:56:19.179 INFO:tasks.workunit.client.0.smithi064.stdout:2/328: creat d7/d14/d1e/d34/d65/f67 x:0 0 0 2024-02-16T22:56:19.180 INFO:tasks.workunit.client.1.smithi192.stdout:7/489: sync 2024-02-16T22:56:19.180 INFO:tasks.workunit.client.1.smithi192.stdout:0/415: sync 2024-02-16T22:56:19.185 INFO:tasks.workunit.client.0.smithi064.stdout:1/312: dwrite d2/f36 [4194304,4194304] 0 2024-02-16T22:56:19.186 INFO:tasks.workunit.client.1.smithi192.stdout:0/416: dwrite d1/d3/de/d22/d37/d47/f48 [0,4194304] 0 2024-02-16T22:56:19.186 INFO:tasks.workunit.client.0.smithi064.stdout:1/313: chown d2/df/d14/d18/d40 251 1 2024-02-16T22:56:19.187 INFO:tasks.workunit.client.0.smithi064.stdout:8/276: dwrite f1 [0,4194304] 0 2024-02-16T22:56:19.187 INFO:tasks.workunit.client.1.smithi192.stdout:0/417: chown d1/d3/de/c38 1 1 2024-02-16T22:56:19.188 INFO:tasks.workunit.client.0.smithi064.stdout:4/347: unlink dc/f17 0 2024-02-16T22:56:19.192 INFO:tasks.workunit.client.1.smithi192.stdout:3/435: link da/d11/d35/f70 da/d11/d63/d6b/d71/f98 0 2024-02-16T22:56:19.196 INFO:tasks.workunit.client.1.smithi192.stdout:1/400: mkdir d3/de/d87 0 2024-02-16T22:56:19.199 INFO:tasks.workunit.client.1.smithi192.stdout:8/419: truncate d5/d9/f17 3279088 0 2024-02-16T22:56:19.202 INFO:tasks.workunit.client.1.smithi192.stdout:8/420: dread d5/d26/f3f [4194304,4194304] 0 2024-02-16T22:56:19.203 INFO:tasks.workunit.client.0.smithi064.stdout:6/381: getdents d0/d1d/d21 0 2024-02-16T22:56:19.205 INFO:tasks.workunit.client.1.smithi192.stdout:4/482: unlink d1/cae 0 2024-02-16T22:56:19.206 INFO:tasks.workunit.client.1.smithi192.stdout:0/418: creat d1/d3/de/d22/d37/f87 x:0 0 0 2024-02-16T22:56:19.208 INFO:tasks.workunit.client.0.smithi064.stdout:2/329: mknod d7/d14/d25/c68 0 2024-02-16T22:56:19.210 INFO:tasks.workunit.client.1.smithi192.stdout:7/490: mknod d1/d69/cb0 0 2024-02-16T22:56:19.211 INFO:tasks.workunit.client.1.smithi192.stdout:7/491: write d1/d2/f7e [1187712,11011] 0 2024-02-16T22:56:19.211 INFO:tasks.workunit.client.1.smithi192.stdout:9/527: write d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/fa1 [1069264,62256] 0 2024-02-16T22:56:19.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/401: dread d3/de/d27/d34/f4e [0,4194304] 0 2024-02-16T22:56:19.213 INFO:tasks.workunit.client.1.smithi192.stdout:8/421: symlink d5/d3a/d4d/l7b 0 2024-02-16T22:56:19.214 INFO:tasks.workunit.client.1.smithi192.stdout:1/402: stat d3/de/d27/d34/d52/d63/l6f 0 2024-02-16T22:56:19.217 INFO:tasks.workunit.client.1.smithi192.stdout:8/422: dread d5/d16/f45 [0,4194304] 0 2024-02-16T22:56:19.218 INFO:tasks.workunit.client.0.smithi064.stdout:3/398: link d4/l7 d4/d13/d1b/d27/d61/l88 0 2024-02-16T22:56:19.220 INFO:tasks.workunit.client.1.smithi192.stdout:0/419: creat d1/d3/de/d22/d37/d67/d34/f88 x:0 0 0 2024-02-16T22:56:19.224 INFO:tasks.workunit.client.1.smithi192.stdout:8/423: dwrite d5/d9/f44 [0,4194304] 0 2024-02-16T22:56:19.224 INFO:tasks.workunit.client.1.smithi192.stdout:0/420: truncate d1/d3/de/d22/d37/d67/f66 906107 0 2024-02-16T22:56:19.227 INFO:tasks.workunit.client.1.smithi192.stdout:9/528: creat d7/d15/d36/fb4 x:0 0 0 2024-02-16T22:56:19.227 INFO:tasks.workunit.client.0.smithi064.stdout:8/277: mkdir df/d31/d5e 0 2024-02-16T22:56:19.230 INFO:tasks.workunit.client.1.smithi192.stdout:0/421: dwrite d1/d3/de/f4e [0,4194304] 0 2024-02-16T22:56:19.240 INFO:tasks.workunit.client.0.smithi064.stdout:7/264: rename d5/d33/d35 to d5/d6/d54 0 2024-02-16T22:56:19.243 INFO:tasks.workunit.client.0.smithi064.stdout:2/330: dread d7/f35 [0,4194304] 0 2024-02-16T22:56:19.243 INFO:tasks.workunit.client.1.smithi192.stdout:3/436: creat da/f99 x:0 0 0 2024-02-16T22:56:19.244 INFO:tasks.workunit.client.1.smithi192.stdout:3/437: chown da/d11/d20/f41 1008255492 1 2024-02-16T22:56:19.244 INFO:tasks.workunit.client.0.smithi064.stdout:4/348: creat dc/dd/d32/f6a x:0 0 0 2024-02-16T22:56:19.245 INFO:tasks.workunit.client.1.smithi192.stdout:3/438: write da/d11/f2f [1011754,101927] 0 2024-02-16T22:56:19.245 INFO:tasks.workunit.client.0.smithi064.stdout:4/349: truncate dc/d1d/f2b 1661677 0 2024-02-16T22:56:19.247 INFO:tasks.workunit.client.1.smithi192.stdout:3/439: read da/d11/d2b/f5e [77165,91300] 0 2024-02-16T22:56:19.247 INFO:tasks.workunit.client.1.smithi192.stdout:8/424: symlink d5/db/d47/l7c 0 2024-02-16T22:56:19.249 INFO:tasks.workunit.client.1.smithi192.stdout:9/529: fdatasync d7/f9d 0 2024-02-16T22:56:19.252 INFO:tasks.workunit.client.0.smithi064.stdout:9/335: rename c2 to d3/d11/d15/d5a/d2a/c71 0 2024-02-16T22:56:19.254 INFO:tasks.workunit.client.1.smithi192.stdout:7/492: symlink d1/d2/de/d2c/d49/da1/lb1 0 2024-02-16T22:56:19.255 INFO:tasks.workunit.client.0.smithi064.stdout:3/399: dwrite d4/d13/d1b/d4f/f5d [0,4194304] 0 2024-02-16T22:56:19.255 INFO:tasks.workunit.client.1.smithi192.stdout:4/483: write d1/d26/d6f/fab [2909331,59019] 0 2024-02-16T22:56:19.255 INFO:tasks.workunit.client.1.smithi192.stdout:4/484: fsync d1/f5 0 2024-02-16T22:56:19.259 INFO:tasks.workunit.client.0.smithi064.stdout:2/331: rename d7/d14/d25/f32 to d7/d14/d25/f69 0 2024-02-16T22:56:19.260 INFO:tasks.workunit.client.1.smithi192.stdout:0/422: symlink d1/d3/de/d22/d37/d67/d75/l89 0 2024-02-16T22:56:19.261 INFO:tasks.workunit.client.0.smithi064.stdout:2/332: chown d7/d14/d3e 56831 1 2024-02-16T22:56:19.261 INFO:tasks.workunit.client.1.smithi192.stdout:0/423: chown d1/f2a 4002751 1 2024-02-16T22:56:19.261 INFO:tasks.workunit.client.1.smithi192.stdout:0/424: stat d1/d3/de/d22/l46 0 2024-02-16T22:56:19.262 INFO:tasks.workunit.client.0.smithi064.stdout:5/248: dwrite d6/fa [0,4194304] 0 2024-02-16T22:56:19.262 INFO:tasks.workunit.client.1.smithi192.stdout:0/425: fdatasync d1/d3/de/d22/d37/d47/f81 0 2024-02-16T22:56:19.264 INFO:tasks.workunit.client.1.smithi192.stdout:1/403: creat d3/d5b/d77/f88 x:0 0 0 2024-02-16T22:56:19.266 INFO:tasks.workunit.client.1.smithi192.stdout:1/404: truncate d3/de/d27/d34/d52/f70 511189 0 2024-02-16T22:56:19.270 INFO:tasks.workunit.client.0.smithi064.stdout:4/350: dread dc/dd/d32/f3d [0,4194304] 0 2024-02-16T22:56:19.274 INFO:tasks.workunit.client.1.smithi192.stdout:3/440: mknod da/d11/d2b/c9a 0 2024-02-16T22:56:19.275 INFO:tasks.workunit.client.0.smithi064.stdout:3/400: creat d4/d14/d2b/d72/f89 x:0 0 0 2024-02-16T22:56:19.275 INFO:tasks.workunit.client.1.smithi192.stdout:8/425: creat d5/d16/f7d x:0 0 0 2024-02-16T22:56:19.276 INFO:tasks.workunit.client.0.smithi064.stdout:8/278: truncate df/d31/d2f/d38/d48/f49 181025 0 2024-02-16T22:56:19.278 INFO:tasks.workunit.client.0.smithi064.stdout:2/333: creat d7/d14/d25/d37/d44/f6a x:0 0 0 2024-02-16T22:56:19.279 INFO:tasks.workunit.client.0.smithi064.stdout:2/334: dread - d7/d14/d1e/d34/f61 zero size 2024-02-16T22:56:19.279 INFO:tasks.workunit.client.1.smithi192.stdout:9/530: creat d7/d19/d37/fb5 x:0 0 0 2024-02-16T22:56:19.280 INFO:tasks.workunit.client.0.smithi064.stdout:2/335: chown d7/de/f2f 60022062 1 2024-02-16T22:56:19.284 INFO:tasks.workunit.client.0.smithi064.stdout:6/382: rename d0/d5/d54/l70 to d0/d1b/l75 0 2024-02-16T22:56:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: pgmap v14: 65 pgs: 65 active+clean; 1.5 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 232 MiB/s wr, 411 op/s 2024-02-16T22:56:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3826636511' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:56:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:19.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:19 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:19.289 INFO:tasks.workunit.client.1.smithi192.stdout:1/405: mkdir d3/de/d27/d34/d69/d85/d89 0 2024-02-16T22:56:19.292 INFO:tasks.workunit.client.0.smithi064.stdout:9/336: link d3/d11/d15/d5a/d2a/d2e/d31/f4e d3/d11/d15/f72 0 2024-02-16T22:56:19.294 INFO:tasks.workunit.client.1.smithi192.stdout:1/406: dwrite d3/de/d27/f4a [0,4194304] 0 2024-02-16T22:56:19.294 INFO:tasks.workunit.client.0.smithi064.stdout:8/279: mkdir df/d31/d2f/d3e/d5f 0 2024-02-16T22:56:19.296 INFO:tasks.workunit.client.1.smithi192.stdout:3/441: symlink da/d11/d63/d6b/d80/l9b 0 2024-02-16T22:56:19.297 INFO:tasks.workunit.client.0.smithi064.stdout:2/336: dread d7/de/f15 [0,4194304] 0 2024-02-16T22:56:19.298 INFO:tasks.workunit.client.1.smithi192.stdout:2/473: sync 2024-02-16T22:56:19.298 INFO:tasks.workunit.client.1.smithi192.stdout:5/396: sync 2024-02-16T22:56:19.298 INFO:tasks.workunit.client.1.smithi192.stdout:6/423: sync 2024-02-16T22:56:19.299 INFO:tasks.workunit.client.0.smithi064.stdout:2/337: chown d7/d14/d1e/d34/d3f 725067653 1 2024-02-16T22:56:19.299 INFO:tasks.workunit.client.0.smithi064.stdout:9/337: dwrite d3/d9/f41 [0,4194304] 0 2024-02-16T22:56:19.300 INFO:tasks.workunit.client.1.smithi192.stdout:5/397: readlink d6/d12/d4e/l6d 0 2024-02-16T22:56:19.301 INFO:tasks.workunit.client.0.smithi064.stdout:9/338: readlink d3/d11/d15/d5a/d2a/d2e/l70 0 2024-02-16T22:56:19.302 INFO:tasks.workunit.client.0.smithi064.stdout:4/351: mknod dc/dd/c6b 0 2024-02-16T22:56:19.303 INFO:tasks.workunit.client.1.smithi192.stdout:8/426: dread d5/d16/f18 [8388608,4194304] 0 2024-02-16T22:56:19.303 INFO:tasks.workunit.client.0.smithi064.stdout:4/352: fdatasync dc/dd/d32/d22/f24 0 2024-02-16T22:56:19.304 INFO:tasks.workunit.client.1.smithi192.stdout:8/427: write d5/d3e/f1c [4993539,55970] 0 2024-02-16T22:56:19.304 INFO:tasks.workunit.client.1.smithi192.stdout:8/428: dread - d5/db/f64 zero size 2024-02-16T22:56:19.305 INFO:tasks.workunit.client.0.smithi064.stdout:6/383: mkdir d0/d5/df/d17/d18/d5f/d76 0 2024-02-16T22:56:19.308 INFO:tasks.workunit.client.0.smithi064.stdout:7/265: link d5/d6/d54/c3c d5/d33/d42/c55 0 2024-02-16T22:56:19.308 INFO:tasks.workunit.client.1.smithi192.stdout:9/531: rename d7/d19/d37/f7e to d7/d15/d36/d54/fb6 0 2024-02-16T22:56:19.309 INFO:tasks.workunit.client.1.smithi192.stdout:9/532: readlink d7/lb 0 2024-02-16T22:56:19.309 INFO:tasks.workunit.client.0.smithi064.stdout:2/338: dwrite d7/de/f24 [0,4194304] 0 2024-02-16T22:56:19.314 INFO:tasks.workunit.client.1.smithi192.stdout:6/424: creat da/df/f84 x:0 0 0 2024-02-16T22:56:19.315 INFO:tasks.workunit.client.0.smithi064.stdout:9/339: creat d3/d11/d15/d23/f73 x:0 0 0 2024-02-16T22:56:19.317 INFO:tasks.workunit.client.1.smithi192.stdout:1/407: dread d3/f50 [0,4194304] 0 2024-02-16T22:56:19.318 INFO:tasks.workunit.client.1.smithi192.stdout:5/398: chown d6/cb 50642 1 2024-02-16T22:56:19.329 INFO:tasks.workunit.client.0.smithi064.stdout:6/384: mknod d0/d12/c77 0 2024-02-16T22:56:19.331 INFO:tasks.workunit.client.0.smithi064.stdout:3/401: mknod d4/d13/d35/c8a 0 2024-02-16T22:56:19.331 INFO:tasks.workunit.client.1.smithi192.stdout:0/426: write d1/f7a [3257549,105284] 0 2024-02-16T22:56:19.332 INFO:tasks.workunit.client.0.smithi064.stdout:3/402: readlink d4/d1f/d5b/d51/d7c/l4a 0 2024-02-16T22:56:19.334 INFO:tasks.workunit.client.0.smithi064.stdout:6/385: dwrite d0/d5/f27 [0,4194304] 0 2024-02-16T22:56:19.340 INFO:tasks.workunit.client.0.smithi064.stdout:6/386: dwrite d0/d1d/f58 [0,4194304] 0 2024-02-16T22:56:19.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: pgmap v14: 65 pgs: 65 active+clean; 1.5 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 232 MiB/s wr, 411 op/s 2024-02-16T22:56:19.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:19.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3826636511' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:56:19.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:19.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:19.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:19.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:19 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:19.369 INFO:tasks.workunit.client.1.smithi192.stdout:4/485: creat d1/d26/faf x:0 0 0 2024-02-16T22:56:19.370 INFO:tasks.workunit.client.0.smithi064.stdout:2/339: rename d7 to d7/d14/d1e/d34/d6b 22 2024-02-16T22:56:19.372 INFO:tasks.workunit.client.1.smithi192.stdout:6/425: mkdir da/d40/d85 0 2024-02-16T22:56:19.375 INFO:tasks.workunit.client.1.smithi192.stdout:1/408: creat d3/d5b/f8a x:0 0 0 2024-02-16T22:56:19.376 INFO:tasks.workunit.client.1.smithi192.stdout:1/409: write d3/d5b/d77/f88 [610561,100756] 0 2024-02-16T22:56:19.376 INFO:tasks.workunit.client.1.smithi192.stdout:2/474: mknod d22/d24/d39/d42/d85/d98/c9e 0 2024-02-16T22:56:19.378 INFO:tasks.workunit.client.0.smithi064.stdout:7/266: mkdir d5/d6/d49/d56 0 2024-02-16T22:56:19.378 INFO:tasks.workunit.client.1.smithi192.stdout:2/475: truncate d22/d24/d39/f4c 638922 0 2024-02-16T22:56:19.378 INFO:tasks.workunit.client.0.smithi064.stdout:8/280: truncate df/f19 349732 0 2024-02-16T22:56:19.379 INFO:tasks.workunit.client.0.smithi064.stdout:7/267: truncate d5/f2f 1152703 0 2024-02-16T22:56:19.380 INFO:tasks.workunit.client.0.smithi064.stdout:3/403: mkdir d4/d13/d4d/d5c/d8b 0 2024-02-16T22:56:19.381 INFO:tasks.workunit.client.1.smithi192.stdout:3/442: truncate da/d11/d63/d6b/d80/f83 92294 0 2024-02-16T22:56:19.382 INFO:tasks.workunit.client.1.smithi192.stdout:3/443: write da/d11/d20/f4a [110514,80892] 0 2024-02-16T22:56:19.383 INFO:tasks.workunit.client.1.smithi192.stdout:7/493: getdents d1/d2/de/d2c 0 2024-02-16T22:56:19.384 INFO:tasks.workunit.client.1.smithi192.stdout:5/399: dread d6/d12/d4e/f2e [0,4194304] 0 2024-02-16T22:56:19.385 INFO:tasks.workunit.client.0.smithi064.stdout:6/387: rename d0/d1d/f58 to d0/d12/f78 0 2024-02-16T22:56:19.386 INFO:tasks.workunit.client.1.smithi192.stdout:7/494: truncate d1/d2/d2d/d61/d88/d90/d2a/f87 248334 0 2024-02-16T22:56:19.386 INFO:tasks.workunit.client.0.smithi064.stdout:9/340: symlink d3/d9/d6a/l74 0 2024-02-16T22:56:19.387 INFO:tasks.workunit.client.1.smithi192.stdout:4/486: truncate d1/d26/d66/d89/fc 1466166 0 2024-02-16T22:56:19.388 INFO:tasks.workunit.client.0.smithi064.stdout:4/353: creat dc/f6c x:0 0 0 2024-02-16T22:56:19.389 INFO:tasks.workunit.client.1.smithi192.stdout:0/427: dread d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:19.389 INFO:tasks.workunit.client.0.smithi064.stdout:4/354: read - dc/dd/d16/f5e zero size 2024-02-16T22:56:19.390 INFO:tasks.workunit.client.0.smithi064.stdout:6/388: dread d0/d6/d10/d33/f2a [0,4194304] 0 2024-02-16T22:56:19.390 INFO:tasks.workunit.client.0.smithi064.stdout:8/281: creat df/d2a/f60 x:0 0 0 2024-02-16T22:56:19.391 INFO:tasks.workunit.client.0.smithi064.stdout:8/282: fsync df/d31/d2f/f3a 0 2024-02-16T22:56:19.392 INFO:tasks.workunit.client.1.smithi192.stdout:1/410: dread d3/f22 [0,4194304] 0 2024-02-16T22:56:19.392 INFO:tasks.workunit.client.0.smithi064.stdout:7/268: chown d5/d6/l16 49124 1 2024-02-16T22:56:19.396 INFO:tasks.workunit.client.1.smithi192.stdout:1/411: dwrite d3/de/f32 [0,4194304] 0 2024-02-16T22:56:19.402 INFO:tasks.workunit.client.0.smithi064.stdout:2/340: rename d7/d14/d25/d37/d44/c5e to d7/d14/d4b/c6c 0 2024-02-16T22:56:19.403 INFO:tasks.workunit.client.0.smithi064.stdout:2/341: fsync d7/d14/d25/d37/f66 0 2024-02-16T22:56:19.404 INFO:tasks.workunit.client.0.smithi064.stdout:2/342: read d7/d14/d1e/d34/d3f/f43 [3735434,11597] 0 2024-02-16T22:56:19.406 INFO:tasks.workunit.client.1.smithi192.stdout:5/400: mkdir d6/d8/d86 0 2024-02-16T22:56:19.414 INFO:tasks.workunit.client.1.smithi192.stdout:0/428: creat d1/d3/de/d22/d37/d67/d34/f8a x:0 0 0 2024-02-16T22:56:19.415 INFO:tasks.workunit.client.1.smithi192.stdout:4/487: dwrite d1/d26/d66/d89/fb [0,4194304] 0 2024-02-16T22:56:19.416 INFO:tasks.workunit.client.1.smithi192.stdout:4/488: truncate d1/d15/d29/f8b 595593 0 2024-02-16T22:56:19.417 INFO:tasks.workunit.client.0.smithi064.stdout:3/404: dwrite d4/fd [4194304,4194304] 0 2024-02-16T22:56:19.417 INFO:tasks.workunit.client.1.smithi192.stdout:4/489: chown d1/d26/f36 2115771565 1 2024-02-16T22:56:19.418 INFO:tasks.workunit.client.0.smithi064.stdout:7/269: creat d5/d33/f57 x:0 0 0 2024-02-16T22:56:19.418 INFO:tasks.workunit.client.0.smithi064.stdout:6/389: dread d0/d5/df/d17/d18/f3f [0,4194304] 0 2024-02-16T22:56:19.420 INFO:tasks.workunit.client.0.smithi064.stdout:6/390: truncate d0/f41 5115458 0 2024-02-16T22:56:19.421 INFO:tasks.workunit.client.0.smithi064.stdout:6/391: chown d0/d5/d54/d60 224146 1 2024-02-16T22:56:19.424 INFO:tasks.workunit.client.1.smithi192.stdout:8/429: getdents d5/db/de/df 0 2024-02-16T22:56:19.426 INFO:tasks.workunit.client.0.smithi064.stdout:4/355: write dc/f33 [207439,63350] 0 2024-02-16T22:56:19.429 INFO:tasks.workunit.client.0.smithi064.stdout:2/343: creat d7/d14/d25/d37/d44/f6d x:0 0 0 2024-02-16T22:56:19.430 INFO:tasks.workunit.client.0.smithi064.stdout:9/341: symlink d3/d11/d15/d5a/d2a/d2e/d31/d62/l75 0 2024-02-16T22:56:19.430 INFO:tasks.workunit.client.1.smithi192.stdout:7/495: mkdir d1/d2/d2d/d61/d88/d90/db2 0 2024-02-16T22:56:19.432 INFO:tasks.workunit.client.0.smithi064.stdout:8/283: mkdir df/d31/d61 0 2024-02-16T22:56:19.432 INFO:tasks.workunit.client.1.smithi192.stdout:2/476: mknod d22/d24/d2b/d99/d9b/c9f 0 2024-02-16T22:56:19.433 INFO:tasks.workunit.client.1.smithi192.stdout:2/477: fsync d22/d24/d2b/f8a 0 2024-02-16T22:56:19.435 INFO:tasks.workunit.client.0.smithi064.stdout:3/405: mknod d4/d1f/d5b/d51/c8c 0 2024-02-16T22:56:19.436 INFO:tasks.workunit.client.0.smithi064.stdout:7/270: fdatasync d5/d6/f34 0 2024-02-16T22:56:19.437 INFO:tasks.workunit.client.1.smithi192.stdout:4/490: write d1/d15/d29/d3c/f43 [4454342,31590] 0 2024-02-16T22:56:19.438 INFO:tasks.workunit.client.0.smithi064.stdout:6/392: creat d0/d5/df/f79 x:0 0 0 2024-02-16T22:56:19.440 INFO:tasks.workunit.client.1.smithi192.stdout:1/412: mknod d3/de/d27/d34/d52/d63/d81/c8b 0 2024-02-16T22:56:19.440 INFO:tasks.workunit.client.1.smithi192.stdout:1/413: write d3/f50 [3621090,124944] 0 2024-02-16T22:56:19.441 INFO:tasks.workunit.client.1.smithi192.stdout:1/414: write d3/de/d27/f4a [1389782,39564] 0 2024-02-16T22:56:19.442 INFO:tasks.workunit.client.1.smithi192.stdout:1/415: stat d3/de/d27/d34/d52/d5a 0 2024-02-16T22:56:19.445 INFO:tasks.workunit.client.1.smithi192.stdout:7/496: mknod d1/d2/d2d/d61/d88/d90/d4c/cb3 0 2024-02-16T22:56:19.446 INFO:tasks.workunit.client.1.smithi192.stdout:6/426: symlink da/df/d12/d15/l86 0 2024-02-16T22:56:19.446 INFO:tasks.workunit.client.1.smithi192.stdout:3/444: write da/d11/d63/d6b/d66/f60 [348333,3738] 0 2024-02-16T22:56:19.448 INFO:tasks.workunit.client.1.smithi192.stdout:2/478: creat d22/d24/fa0 x:0 0 0 2024-02-16T22:56:19.458 INFO:tasks.workunit.client.1.smithi192.stdout:4/491: rename d1/d6b/f7f to d1/d15/d48/d8d/d9b/fb0 0 2024-02-16T22:56:19.458 INFO:tasks.workunit.client.1.smithi192.stdout:4/492: chown d1/l27 1934864 1 2024-02-16T22:56:19.458 INFO:tasks.workunit.client.1.smithi192.stdout:5/401: link d6/d8/c74 d6/d12/d4e/d46/c87 0 2024-02-16T22:56:19.458 INFO:tasks.workunit.client.1.smithi192.stdout:4/493: chown d1/d15/d23/f4a 99 1 2024-02-16T22:56:19.459 INFO:tasks.workunit.client.1.smithi192.stdout:4/494: fdatasync d1/d15/d48/d8d/f7b 0 2024-02-16T22:56:19.459 INFO:tasks.workunit.client.1.smithi192.stdout:1/416: dwrite d3/de/f1a [0,4194304] 0 2024-02-16T22:56:19.466 INFO:tasks.workunit.client.1.smithi192.stdout:0/429: truncate d1/d3/de/d22/d37/d67/f66 299434 0 2024-02-16T22:56:19.467 INFO:tasks.workunit.client.1.smithi192.stdout:1/417: dread d3/de/f65 [0,4194304] 0 2024-02-16T22:56:19.471 INFO:tasks.workunit.client.1.smithi192.stdout:6/427: creat da/d40/d85/f87 x:0 0 0 2024-02-16T22:56:19.472 INFO:tasks.workunit.client.1.smithi192.stdout:0/430: dwrite d1/d3/de/f10 [0,4194304] 0 2024-02-16T22:56:19.474 INFO:tasks.workunit.client.1.smithi192.stdout:4/495: mknod d1/d26/d66/d89/d11/d58/cb1 0 2024-02-16T22:56:19.475 INFO:tasks.workunit.client.1.smithi192.stdout:4/496: fsync d1/d15/d48/f95 0 2024-02-16T22:56:19.480 INFO:tasks.workunit.client.1.smithi192.stdout:8/430: getdents d5/db/de 0 2024-02-16T22:56:19.480 INFO:tasks.workunit.client.1.smithi192.stdout:8/431: dread - d5/d9/f6b zero size 2024-02-16T22:56:19.481 INFO:tasks.workunit.client.1.smithi192.stdout:8/432: chown d5/d16/c6d 444837733 1 2024-02-16T22:56:19.483 INFO:tasks.workunit.client.1.smithi192.stdout:7/497: creat d1/d2/fb4 x:0 0 0 2024-02-16T22:56:19.486 INFO:tasks.workunit.client.1.smithi192.stdout:4/497: creat d1/d15/d48/d8d/fb2 x:0 0 0 2024-02-16T22:56:19.490 INFO:tasks.workunit.client.0.smithi064.stdout:8/284: creat df/d31/d2f/d38/f62 x:0 0 0 2024-02-16T22:56:19.490 INFO:tasks.workunit.client.0.smithi064.stdout:7/271: mknod d5/d6/d49/c58 0 2024-02-16T22:56:19.492 INFO:tasks.workunit.client.0.smithi064.stdout:2/344: symlink d7/l6e 0 2024-02-16T22:56:19.493 INFO:tasks.workunit.client.0.smithi064.stdout:3/406: dwrite d4/d14/d2b/d72/f87 [0,4194304] 0 2024-02-16T22:56:19.495 INFO:tasks.workunit.client.0.smithi064.stdout:7/272: mknod d5/d6/d49/c59 0 2024-02-16T22:56:19.498 INFO:tasks.workunit.client.0.smithi064.stdout:2/345: dread d7/f35 [0,4194304] 0 2024-02-16T22:56:19.499 INFO:tasks.workunit.client.0.smithi064.stdout:2/346: chown d7/d14/d25/c68 232413 1 2024-02-16T22:56:19.500 INFO:tasks.workunit.client.0.smithi064.stdout:8/285: rename c5 to df/d31/c63 0 2024-02-16T22:56:19.501 INFO:tasks.workunit.client.0.smithi064.stdout:2/347: write d7/d14/d25/d37/d44/f6a [495737,120325] 0 2024-02-16T22:56:19.506 INFO:tasks.workunit.client.0.smithi064.stdout:2/348: mknod d7/c6f 0 2024-02-16T22:56:19.507 INFO:tasks.workunit.client.0.smithi064.stdout:8/286: creat df/d31/d5e/f64 x:0 0 0 2024-02-16T22:56:19.511 INFO:tasks.workunit.client.0.smithi064.stdout:7/273: truncate f0 3595350 0 2024-02-16T22:56:19.512 INFO:tasks.workunit.client.0.smithi064.stdout:3/407: dread d4/d1f/f20 [0,4194304] 0 2024-02-16T22:56:19.512 INFO:tasks.workunit.client.0.smithi064.stdout:2/349: mknod d7/c70 0 2024-02-16T22:56:19.514 INFO:tasks.workunit.client.0.smithi064.stdout:3/408: creat d4/d13/d41/f8d x:0 0 0 2024-02-16T22:56:19.516 INFO:tasks.workunit.client.0.smithi064.stdout:8/287: write df/d2a/f34 [4091402,102415] 0 2024-02-16T22:56:19.516 INFO:tasks.workunit.client.0.smithi064.stdout:7/274: write d5/d6/fb [964584,103607] 0 2024-02-16T22:56:19.520 INFO:tasks.workunit.client.0.smithi064.stdout:6/393: dread d0/d5/f11 [0,4194304] 0 2024-02-16T22:56:19.522 INFO:tasks.workunit.client.0.smithi064.stdout:7/275: mknod d5/d6/d49/d56/c5a 0 2024-02-16T22:56:19.522 INFO:tasks.workunit.client.0.smithi064.stdout:2/350: dwrite d7/d14/d4b/f64 [4194304,4194304] 0 2024-02-16T22:56:19.526 INFO:tasks.workunit.client.0.smithi064.stdout:7/276: dread d5/d6/f22 [0,4194304] 0 2024-02-16T22:56:19.527 INFO:tasks.workunit.client.0.smithi064.stdout:6/394: dread d0/d1d/d21/f2d [0,4194304] 0 2024-02-16T22:56:19.527 INFO:tasks.workunit.client.0.smithi064.stdout:8/288: rename df/d31/d2f/f3a to df/d31/d61/f65 0 2024-02-16T22:56:19.528 INFO:tasks.workunit.client.0.smithi064.stdout:7/277: write d5/d6/fb [2437539,40947] 0 2024-02-16T22:56:19.529 INFO:tasks.workunit.client.0.smithi064.stdout:2/351: readlink d7/de/lf 0 2024-02-16T22:56:19.530 INFO:tasks.workunit.client.0.smithi064.stdout:2/352: stat d7/d14/d3e 0 2024-02-16T22:56:19.531 INFO:tasks.workunit.client.0.smithi064.stdout:3/409: write d4/d13/d1b/d4f/f6b [688488,55838] 0 2024-02-16T22:56:19.535 INFO:tasks.workunit.client.0.smithi064.stdout:8/289: dread f1 [0,4194304] 0 2024-02-16T22:56:19.535 INFO:tasks.workunit.client.0.smithi064.stdout:6/395: creat d0/d1b/f7a x:0 0 0 2024-02-16T22:56:19.539 INFO:tasks.workunit.client.0.smithi064.stdout:7/278: dwrite d5/d6/d54/f3f [0,4194304] 0 2024-02-16T22:56:19.541 INFO:tasks.workunit.client.0.smithi064.stdout:8/290: unlink df/d31/f36 0 2024-02-16T22:56:19.545 INFO:tasks.workunit.client.0.smithi064.stdout:2/353: truncate d7/d14/f2e 1592930 0 2024-02-16T22:56:19.546 INFO:tasks.workunit.client.0.smithi064.stdout:2/354: stat d7/de/f56 0 2024-02-16T22:56:19.548 INFO:tasks.workunit.client.0.smithi064.stdout:8/291: readlink df/d31/d47/l4e 0 2024-02-16T22:56:19.552 INFO:tasks.workunit.client.0.smithi064.stdout:2/355: creat d7/d14/d25/d46/d55/f71 x:0 0 0 2024-02-16T22:56:19.553 INFO:tasks.workunit.client.1.smithi192.stdout:3/445: write da/d11/d63/d6b/d66/f7a [1036566,80387] 0 2024-02-16T22:56:19.554 INFO:tasks.workunit.client.1.smithi192.stdout:1/418: write d3/de/d27/f30 [9063060,58717] 0 2024-02-16T22:56:19.554 INFO:tasks.workunit.client.1.smithi192.stdout:5/402: write d6/fc [3365751,77536] 0 2024-02-16T22:56:19.554 INFO:tasks.workunit.client.0.smithi064.stdout:7/279: creat d5/d6/f5b x:0 0 0 2024-02-16T22:56:19.556 INFO:tasks.workunit.client.0.smithi064.stdout:8/292: dwrite df/d31/d5e/f64 [0,4194304] 0 2024-02-16T22:56:19.559 INFO:tasks.workunit.client.0.smithi064.stdout:2/356: dread d7/d14/d1e/f21 [0,4194304] 0 2024-02-16T22:56:19.561 INFO:tasks.workunit.client.0.smithi064.stdout:7/280: mknod d5/d6/d49/c5c 0 2024-02-16T22:56:19.563 INFO:tasks.workunit.client.0.smithi064.stdout:8/293: creat df/d31/d2f/d4f/f66 x:0 0 0 2024-02-16T22:56:19.566 INFO:tasks.workunit.client.0.smithi064.stdout:2/357: creat d7/d14/d1e/d34/d65/f72 x:0 0 0 2024-02-16T22:56:19.568 INFO:tasks.workunit.client.0.smithi064.stdout:7/281: unlink d5/d6/d54/f44 0 2024-02-16T22:56:19.570 INFO:tasks.workunit.client.0.smithi064.stdout:8/294: creat df/d31/d47/f67 x:0 0 0 2024-02-16T22:56:19.573 INFO:tasks.workunit.client.0.smithi064.stdout:6/396: dwrite d0/d1b/f23 [0,4194304] 0 2024-02-16T22:56:19.576 INFO:tasks.workunit.client.0.smithi064.stdout:3/410: dwrite d4/d13/f55 [0,4194304] 0 2024-02-16T22:56:19.584 INFO:tasks.workunit.client.0.smithi064.stdout:3/411: symlink d4/d13/d1b/d48/l8e 0 2024-02-16T22:56:19.586 INFO:tasks.workunit.client.0.smithi064.stdout:6/397: unlink d0/d3c/l5b 0 2024-02-16T22:56:19.587 INFO:tasks.workunit.client.0.smithi064.stdout:7/282: getdents d5/d6/d49 0 2024-02-16T22:56:19.588 INFO:tasks.workunit.client.0.smithi064.stdout:2/358: dread d7/f13 [0,4194304] 0 2024-02-16T22:56:19.588 INFO:tasks.workunit.client.0.smithi064.stdout:3/412: fsync d4/d1f/d34/f3c 0 2024-02-16T22:56:19.589 INFO:tasks.workunit.client.0.smithi064.stdout:6/398: mkdir d0/d12/d7b 0 2024-02-16T22:56:19.590 INFO:tasks.workunit.client.0.smithi064.stdout:6/399: read - d0/d5/df/d17/f62 zero size 2024-02-16T22:56:19.591 INFO:tasks.workunit.client.0.smithi064.stdout:7/283: mkdir d5/d6/d49/d56/d5d 0 2024-02-16T22:56:19.591 INFO:tasks.workunit.client.0.smithi064.stdout:6/400: dread - d0/d1d/f61 zero size 2024-02-16T22:56:19.592 INFO:tasks.workunit.client.0.smithi064.stdout:6/401: chown d0/d1b/f7a 0 1 2024-02-16T22:56:19.609 INFO:tasks.workunit.client.1.smithi192.stdout:9/533: sync 2024-02-16T22:56:19.611 INFO:tasks.workunit.client.1.smithi192.stdout:7/498: rename d1/l6e to d1/d2/de/d83/lb5 0 2024-02-16T22:56:19.612 INFO:tasks.workunit.client.1.smithi192.stdout:1/419: dread d3/de/d27/d34/d52/f70 [0,4194304] 0 2024-02-16T22:56:19.612 INFO:tasks.workunit.client.1.smithi192.stdout:5/403: truncate d6/d20/f4d 571665 0 2024-02-16T22:56:19.614 INFO:tasks.workunit.client.1.smithi192.stdout:9/534: mknod d7/d15/d36/d77/cb7 0 2024-02-16T22:56:19.615 INFO:tasks.workunit.client.1.smithi192.stdout:4/498: dwrite d1/d6b/fa1 [4194304,4194304] 0 2024-02-16T22:56:19.615 INFO:tasks.workunit.client.1.smithi192.stdout:3/446: dread da/d11/d20/f24 [0,4194304] 0 2024-02-16T22:56:19.616 INFO:tasks.workunit.client.1.smithi192.stdout:6/428: rename da/df/f5e to da/df/d6b/f88 0 2024-02-16T22:56:19.617 INFO:tasks.workunit.client.1.smithi192.stdout:6/429: readlink da/ld 0 2024-02-16T22:56:19.620 INFO:tasks.workunit.client.1.smithi192.stdout:4/499: dread d1/d26/d34/f60 [0,4194304] 0 2024-02-16T22:56:19.621 INFO:tasks.workunit.client.1.smithi192.stdout:4/500: stat d1/d15/d48/d8d/f9a 0 2024-02-16T22:56:19.635 INFO:tasks.workunit.client.1.smithi192.stdout:3/447: dread da/d11/f96 [0,4194304] 0 2024-02-16T22:56:19.640 INFO:tasks.workunit.client.0.smithi064.stdout:0/302: sync 2024-02-16T22:56:19.641 INFO:tasks.workunit.client.1.smithi192.stdout:1/420: creat d3/de/d27/d34/d69/f8c x:0 0 0 2024-02-16T22:56:19.645 INFO:tasks.workunit.client.1.smithi192.stdout:7/499: truncate d1/d2/d37/f68 865514 0 2024-02-16T22:56:19.649 INFO:tasks.workunit.client.1.smithi192.stdout:9/535: symlink d7/d19/d5e/d6f/lb8 0 2024-02-16T22:56:19.654 INFO:tasks.workunit.client.1.smithi192.stdout:1/421: creat d3/de/d27/d34/d52/d63/d81/f8d x:0 0 0 2024-02-16T22:56:19.658 INFO:tasks.workunit.client.0.smithi064.stdout:2/359: dwrite d7/d14/d25/d37/f47 [0,4194304] 0 2024-02-16T22:56:19.659 INFO:tasks.workunit.client.0.smithi064.stdout:8/295: dwrite df/d2a/f2d [0,4194304] 0 2024-02-16T22:56:19.661 INFO:tasks.workunit.client.1.smithi192.stdout:4/501: fsync d1/d15/d48/f55 0 2024-02-16T22:56:19.662 INFO:tasks.workunit.client.0.smithi064.stdout:7/284: dwrite d5/d6/f1d [0,4194304] 0 2024-02-16T22:56:19.662 INFO:tasks.workunit.client.0.smithi064.stdout:6/402: dwrite d0/d5/df/f6f [0,4194304] 0 2024-02-16T22:56:19.662 INFO:tasks.workunit.client.0.smithi064.stdout:2/360: chown d7/de/lf 843383942 1 2024-02-16T22:56:19.663 INFO:tasks.workunit.client.1.smithi192.stdout:4/502: truncate d1/d15/d48/d8d/fb2 726358 0 2024-02-16T22:56:19.672 INFO:tasks.workunit.client.0.smithi064.stdout:2/361: chown d7/d14/d25/l41 6698 1 2024-02-16T22:56:19.674 INFO:tasks.workunit.client.0.smithi064.stdout:7/285: write d5/d6/f43 [772990,117158] 0 2024-02-16T22:56:19.674 INFO:tasks.workunit.client.0.smithi064.stdout:0/303: mkdir da/dc/de/d12/d2c/d2e/d62 0 2024-02-16T22:56:19.676 INFO:tasks.workunit.client.0.smithi064.stdout:6/403: creat d0/d5/d54/d60/f7c x:0 0 0 2024-02-16T22:56:19.678 INFO:tasks.workunit.client.1.smithi192.stdout:4/503: mknod d1/d26/d5e/cb3 0 2024-02-16T22:56:19.681 INFO:tasks.workunit.client.0.smithi064.stdout:7/286: creat d5/d6/d54/f5e x:0 0 0 2024-02-16T22:56:19.681 INFO:tasks.workunit.client.1.smithi192.stdout:6/430: symlink da/df/d58/l89 0 2024-02-16T22:56:19.684 INFO:tasks.workunit.client.0.smithi064.stdout:6/404: creat d0/d5/df/d17/f7d x:0 0 0 2024-02-16T22:56:19.685 INFO:tasks.workunit.client.1.smithi192.stdout:1/422: symlink d3/de/d27/d34/l8e 0 2024-02-16T22:56:19.685 INFO:tasks.workunit.client.1.smithi192.stdout:2/479: sync 2024-02-16T22:56:19.688 INFO:tasks.workunit.client.1.smithi192.stdout:9/536: creat d7/d19/d28/fb9 x:0 0 0 2024-02-16T22:56:19.688 INFO:tasks.workunit.client.0.smithi064.stdout:2/362: mkdir d7/d14/d1e/d34/d73 0 2024-02-16T22:56:19.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/537: readlink d7/d15/d36/d38/d3f/d8a/d98/la4 0 2024-02-16T22:56:19.690 INFO:tasks.workunit.client.0.smithi064.stdout:8/296: dread df/d31/f3f [0,4194304] 0 2024-02-16T22:56:19.691 INFO:tasks.workunit.client.1.smithi192.stdout:9/538: write d7/d15/f6a [1102877,129413] 0 2024-02-16T22:56:19.691 INFO:tasks.workunit.client.1.smithi192.stdout:3/448: link f9 da/f9c 0 2024-02-16T22:56:19.691 INFO:tasks.workunit.client.1.smithi192.stdout:2/480: dwrite d22/f41 [0,4194304] 0 2024-02-16T22:56:19.698 INFO:tasks.workunit.client.0.smithi064.stdout:6/405: rmdir d0 39 2024-02-16T22:56:19.699 INFO:tasks.workunit.client.1.smithi192.stdout:7/500: getdents d1/d2/de/d2c/d49/d54 0 2024-02-16T22:56:19.701 INFO:tasks.workunit.client.1.smithi192.stdout:1/423: truncate d3/de/d27/d34/d52/f54 5349043 0 2024-02-16T22:56:19.702 INFO:tasks.workunit.client.1.smithi192.stdout:1/424: chown d3/de/d27/d34/l48 413 1 2024-02-16T22:56:19.703 INFO:tasks.workunit.client.0.smithi064.stdout:8/297: mkdir df/d31/d2f/d68 0 2024-02-16T22:56:19.704 INFO:tasks.workunit.client.0.smithi064.stdout:2/363: write d7/d14/d1e/f28 [1190981,29956] 0 2024-02-16T22:56:19.704 INFO:tasks.workunit.client.1.smithi192.stdout:7/501: dwrite d1/d2/de/d2c/d49/d54/d6a/f95 [0,4194304] 0 2024-02-16T22:56:19.710 INFO:tasks.workunit.client.1.smithi192.stdout:7/502: dwrite d1/d2/de/d2c/d49/d54/f67 [0,4194304] 0 2024-02-16T22:56:19.710 INFO:tasks.workunit.client.1.smithi192.stdout:4/504: truncate d1/f30 3891963 0 2024-02-16T22:56:19.711 INFO:tasks.workunit.client.1.smithi192.stdout:4/505: fsync d1/d26/d34/f50 0 2024-02-16T22:56:19.711 INFO:tasks.workunit.client.0.smithi064.stdout:2/364: dwrite d7/d14/f2c [0,4194304] 0 2024-02-16T22:56:19.715 INFO:tasks.workunit.client.0.smithi064.stdout:2/365: truncate d7/d14/f36 885105 0 2024-02-16T22:56:19.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/506: dread d1/f35 [0,4194304] 0 2024-02-16T22:56:19.718 INFO:tasks.workunit.client.1.smithi192.stdout:4/507: write d1/d26/d66/d89/f64 [1129935,12944] 0 2024-02-16T22:56:19.718 INFO:tasks.workunit.client.1.smithi192.stdout:9/539: mknod d7/d15/d36/d38/d3f/cba 0 2024-02-16T22:56:19.719 INFO:tasks.workunit.client.1.smithi192.stdout:4/508: stat d1/d15/c9f 0 2024-02-16T22:56:19.719 INFO:tasks.workunit.client.0.smithi064.stdout:8/298: mkdir df/d31/d2f/d38/d48/d69 0 2024-02-16T22:56:19.723 INFO:tasks.workunit.client.1.smithi192.stdout:6/431: creat da/df/d12/d15/d1b/f8a x:0 0 0 2024-02-16T22:56:19.726 INFO:tasks.workunit.client.0.smithi064.stdout:0/304: link da/dc/de/d12/d2c/l50 da/d26/l63 0 2024-02-16T22:56:19.727 INFO:tasks.workunit.client.0.smithi064.stdout:0/305: chown da/dc/de/d12/d1d/f2b 62 1 2024-02-16T22:56:19.731 INFO:tasks.workunit.client.1.smithi192.stdout:2/481: write fa [2476536,74003] 0 2024-02-16T22:56:19.734 INFO:tasks.workunit.client.0.smithi064.stdout:8/299: mkdir df/d31/d2f/d4f/d6a 0 2024-02-16T22:56:19.735 INFO:tasks.workunit.client.0.smithi064.stdout:8/300: read - df/d31/d2f/d3e/f4a zero size 2024-02-16T22:56:19.736 INFO:tasks.workunit.client.0.smithi064.stdout:7/287: link d5/d6/l17 d5/d6/d49/l5f 0 2024-02-16T22:56:19.737 INFO:tasks.workunit.client.1.smithi192.stdout:3/449: symlink da/l9d 0 2024-02-16T22:56:19.741 INFO:tasks.workunit.client.1.smithi192.stdout:0/431: sync 2024-02-16T22:56:19.741 INFO:tasks.workunit.client.1.smithi192.stdout:8/433: sync 2024-02-16T22:56:19.743 INFO:tasks.workunit.client.0.smithi064.stdout:2/366: symlink d7/de/l74 0 2024-02-16T22:56:19.745 INFO:tasks.workunit.client.1.smithi192.stdout:1/425: dwrite d3/de/d27/d34/d52/f57 [0,4194304] 0 2024-02-16T22:56:19.745 INFO:tasks.workunit.client.1.smithi192.stdout:1/426: chown d3/f46 22978 1 2024-02-16T22:56:19.751 INFO:tasks.workunit.client.1.smithi192.stdout:2/482: creat d22/d24/d39/d42/d65/d69/fa1 x:0 0 0 2024-02-16T22:56:19.755 INFO:tasks.workunit.client.0.smithi064.stdout:0/306: dwrite da/d26/f33 [0,4194304] 0 2024-02-16T22:56:19.755 INFO:tasks.workunit.client.0.smithi064.stdout:6/406: dwrite d0/d6/d10/f40 [0,4194304] 0 2024-02-16T22:56:19.755 INFO:tasks.workunit.client.1.smithi192.stdout:7/503: mknod d1/d2/d2d/d61/d88/d90/d4c/d60/cb6 0 2024-02-16T22:56:19.756 INFO:tasks.workunit.client.1.smithi192.stdout:3/450: rmdir da/d6a 39 2024-02-16T22:56:19.757 INFO:tasks.workunit.client.0.smithi064.stdout:2/367: creat d7/d14/d3e/f75 x:0 0 0 2024-02-16T22:56:19.757 INFO:tasks.workunit.client.0.smithi064.stdout:8/301: getdents df/d2a/d4d 0 2024-02-16T22:56:19.757 INFO:tasks.workunit.client.0.smithi064.stdout:6/407: write d0/d5/d54/d60/f6d [546237,92424] 0 2024-02-16T22:56:19.758 INFO:tasks.workunit.client.1.smithi192.stdout:3/451: dread - da/f99 zero size 2024-02-16T22:56:19.759 INFO:tasks.workunit.client.1.smithi192.stdout:2/483: dwrite d22/f31 [0,4194304] 0 2024-02-16T22:56:19.760 INFO:tasks.workunit.client.1.smithi192.stdout:8/434: rmdir d5/db/de/df 39 2024-02-16T22:56:19.760 INFO:tasks.workunit.client.1.smithi192.stdout:7/504: write d1/d2/de/d2c/d49/d89/f97 [482830,29725] 0 2024-02-16T22:56:19.760 INFO:tasks.workunit.client.0.smithi064.stdout:8/302: chown df/d31/d2f/d3e/d5f 17901 1 2024-02-16T22:56:19.762 INFO:tasks.workunit.client.0.smithi064.stdout:8/303: chown df/d31/d2f/c35 4 1 2024-02-16T22:56:19.762 INFO:tasks.workunit.client.1.smithi192.stdout:6/432: dread da/df/d18/d52/f66 [0,4194304] 0 2024-02-16T22:56:19.763 INFO:tasks.workunit.client.1.smithi192.stdout:0/432: unlink d1/d3/de/d22/d37/d67/d75/l89 0 2024-02-16T22:56:19.764 INFO:tasks.workunit.client.1.smithi192.stdout:4/509: mknod d1/cb4 0 2024-02-16T22:56:19.765 INFO:tasks.workunit.client.0.smithi064.stdout:1/314: sync 2024-02-16T22:56:19.765 INFO:tasks.workunit.client.0.smithi064.stdout:5/249: sync 2024-02-16T22:56:19.766 INFO:tasks.workunit.client.0.smithi064.stdout:2/368: rename d7/de/lf to d7/d14/d1e/l76 0 2024-02-16T22:56:19.767 INFO:tasks.workunit.client.1.smithi192.stdout:0/433: truncate d1/d3/d39/f4f 1162335 0 2024-02-16T22:56:19.767 INFO:tasks.workunit.client.0.smithi064.stdout:7/288: dwrite d5/d6/f22 [0,4194304] 0 2024-02-16T22:56:19.770 INFO:tasks.workunit.client.0.smithi064.stdout:1/315: chown d2/df/d14/f59 3878879 1 2024-02-16T22:56:19.770 INFO:tasks.workunit.client.1.smithi192.stdout:9/540: symlink d7/d15/d36/d77/db1/lbb 0 2024-02-16T22:56:19.771 INFO:tasks.workunit.client.0.smithi064.stdout:7/289: fsync d5/d33/d45/f46 0 2024-02-16T22:56:19.772 INFO:tasks.workunit.client.0.smithi064.stdout:2/369: write d7/d14/d4b/f64 [2572373,39951] 0 2024-02-16T22:56:19.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/541: write d7/d19/d5b/fa5 [886849,54046] 0 2024-02-16T22:56:19.773 INFO:tasks.workunit.client.1.smithi192.stdout:1/427: symlink d3/de/d27/l8f 0 2024-02-16T22:56:19.774 INFO:tasks.workunit.client.0.smithi064.stdout:2/370: stat d7/de/l2d 0 2024-02-16T22:56:19.776 INFO:tasks.workunit.client.0.smithi064.stdout:2/371: chown d7/de/f1f 25230642 1 2024-02-16T22:56:19.777 INFO:tasks.workunit.client.1.smithi192.stdout:0/434: dwrite d1/d3/de/d22/d37/d67/d34/f8a [0,4194304] 0 2024-02-16T22:56:19.778 INFO:tasks.workunit.client.0.smithi064.stdout:8/304: creat df/d31/d47/f6b x:0 0 0 2024-02-16T22:56:19.779 INFO:tasks.workunit.client.0.smithi064.stdout:7/290: dread d5/d6/f22 [0,4194304] 0 2024-02-16T22:56:19.779 INFO:tasks.workunit.client.1.smithi192.stdout:3/452: rmdir da/d11/d63 39 2024-02-16T22:56:19.783 INFO:tasks.workunit.client.0.smithi064.stdout:5/250: creat d6/d25/d48/f55 x:0 0 0 2024-02-16T22:56:19.787 INFO:tasks.workunit.client.1.smithi192.stdout:7/505: mknod d1/d2/de/d2c/d49/d89/cb7 0 2024-02-16T22:56:19.788 INFO:tasks.workunit.client.1.smithi192.stdout:4/510: mkdir d1/d15/d48/db5 0 2024-02-16T22:56:19.791 INFO:tasks.workunit.client.0.smithi064.stdout:6/408: symlink d0/d1b/l7e 0 2024-02-16T22:56:19.792 INFO:tasks.workunit.client.1.smithi192.stdout:4/511: dread d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:19.792 INFO:tasks.workunit.client.0.smithi064.stdout:2/372: readlink d7/d14/l5b 0 2024-02-16T22:56:19.793 INFO:tasks.workunit.client.1.smithi192.stdout:9/542: mknod d7/d19/d37/cbc 0 2024-02-16T22:56:19.795 INFO:tasks.workunit.client.1.smithi192.stdout:1/428: creat d3/de/d27/d34/d52/d5a/f90 x:0 0 0 2024-02-16T22:56:19.795 INFO:tasks.workunit.client.1.smithi192.stdout:6/433: dread da/df/d6b/f88 [0,4194304] 0 2024-02-16T22:56:19.796 INFO:tasks.workunit.client.0.smithi064.stdout:7/291: creat d5/d33/f60 x:0 0 0 2024-02-16T22:56:19.796 INFO:tasks.workunit.client.1.smithi192.stdout:1/429: readlink d3/de/d27/d34/d36/l4f 0 2024-02-16T22:56:19.796 INFO:tasks.workunit.client.0.smithi064.stdout:7/292: chown d5/d6/d49/d56 1003 1 2024-02-16T22:56:19.797 INFO:tasks.workunit.client.0.smithi064.stdout:7/293: chown d5/l14 5772023 1 2024-02-16T22:56:19.797 INFO:tasks.workunit.client.0.smithi064.stdout:5/251: stat f4 0 2024-02-16T22:56:19.797 INFO:tasks.workunit.client.1.smithi192.stdout:0/435: creat d1/d3/d29/d43/f8b x:0 0 0 2024-02-16T22:56:19.797 INFO:tasks.workunit.client.0.smithi064.stdout:5/252: chown d6/d25 53 1 2024-02-16T22:56:19.798 INFO:tasks.workunit.client.1.smithi192.stdout:0/436: write d1/d3/de/f3c [143475,63632] 0 2024-02-16T22:56:19.798 INFO:tasks.workunit.client.0.smithi064.stdout:0/307: link c7 da/c64 0 2024-02-16T22:56:19.799 INFO:tasks.workunit.client.1.smithi192.stdout:0/437: readlink d1/d3/d29/d43/l7d 0 2024-02-16T22:56:19.800 INFO:tasks.workunit.client.1.smithi192.stdout:6/434: dread da/df/d18/d52/f66 [0,4194304] 0 2024-02-16T22:56:19.801 INFO:tasks.workunit.client.1.smithi192.stdout:8/435: rename d5/db/d47/d43/l60 to d5/d3a/l7e 0 2024-02-16T22:56:19.802 INFO:tasks.workunit.client.0.smithi064.stdout:0/308: dread da/dc/de/d12/d1d/f3d [0,4194304] 0 2024-02-16T22:56:19.802 INFO:tasks.workunit.client.1.smithi192.stdout:0/438: write d1/d3/de/d22/d37/d47/f81 [506823,92052] 0 2024-02-16T22:56:19.803 INFO:tasks.workunit.client.0.smithi064.stdout:0/309: write da/f5c [5499111,99695] 0 2024-02-16T22:56:19.804 INFO:tasks.workunit.client.1.smithi192.stdout:2/484: unlink d22/d24/d39/d42/d52/l82 0 2024-02-16T22:56:19.805 INFO:tasks.workunit.client.1.smithi192.stdout:0/439: fsync d1/d3/de/d22/d37/d67/d34/f8a 0 2024-02-16T22:56:19.806 INFO:tasks.workunit.client.1.smithi192.stdout:4/512: symlink d1/d15/d29/d3c/lb6 0 2024-02-16T22:56:19.806 INFO:tasks.workunit.client.1.smithi192.stdout:4/513: readlink d1/l71 0 2024-02-16T22:56:19.808 INFO:tasks.workunit.client.1.smithi192.stdout:9/543: mkdir d7/d15/d36/d77/dbd 0 2024-02-16T22:56:19.809 INFO:tasks.workunit.client.0.smithi064.stdout:8/305: symlink df/l6c 0 2024-02-16T22:56:19.810 INFO:tasks.workunit.client.0.smithi064.stdout:8/306: chown df/d2a/c43 21 1 2024-02-16T22:56:19.810 INFO:tasks.workunit.client.1.smithi192.stdout:1/430: dread d3/f16 [0,4194304] 0 2024-02-16T22:56:19.812 INFO:tasks.workunit.client.0.smithi064.stdout:7/294: mknod d5/d6/d49/c61 0 2024-02-16T22:56:19.813 INFO:tasks.workunit.client.0.smithi064.stdout:7/295: dread - d5/d33/d45/f52 zero size 2024-02-16T22:56:19.814 INFO:tasks.workunit.client.1.smithi192.stdout:3/453: creat da/d11/d63/f9e x:0 0 0 2024-02-16T22:56:19.814 INFO:tasks.workunit.client.1.smithi192.stdout:3/454: stat da/df/f2e 0 2024-02-16T22:56:19.815 INFO:tasks.workunit.client.0.smithi064.stdout:5/253: creat d6/d18/f56 x:0 0 0 2024-02-16T22:56:19.815 INFO:tasks.workunit.client.0.smithi064.stdout:5/254: fsync d6/d1c/f54 0 2024-02-16T22:56:19.819 INFO:tasks.workunit.client.1.smithi192.stdout:3/455: dwrite da/d11/f38 [0,4194304] 0 2024-02-16T22:56:19.821 INFO:tasks.workunit.client.1.smithi192.stdout:7/506: link d1/d2/de/d2c/d49/f6f d1/d2/d2d/d61/d88/d90/d2a/fb8 0 2024-02-16T22:56:19.822 INFO:tasks.workunit.client.0.smithi064.stdout:0/310: symlink da/dc/d3b/d5a/l65 0 2024-02-16T22:56:19.822 INFO:tasks.workunit.client.1.smithi192.stdout:7/507: truncate d1/d2/de/f8f 160031 0 2024-02-16T22:56:19.824 INFO:tasks.workunit.client.0.smithi064.stdout:2/373: creat d7/d14/d1e/d34/d73/f77 x:0 0 0 2024-02-16T22:56:19.825 INFO:tasks.workunit.client.1.smithi192.stdout:5/404: sync 2024-02-16T22:56:19.827 INFO:tasks.workunit.client.0.smithi064.stdout:2/374: fsync d7/d14/d1e/d34/d3f/f4e 0 2024-02-16T22:56:19.828 INFO:tasks.workunit.client.0.smithi064.stdout:8/307: unlink df/d2a/c43 0 2024-02-16T22:56:19.828 INFO:tasks.workunit.client.0.smithi064.stdout:2/375: dread - d7/d14/d1e/d34/f61 zero size 2024-02-16T22:56:19.830 INFO:tasks.workunit.client.0.smithi064.stdout:2/376: readlink l3 0 2024-02-16T22:56:19.830 INFO:tasks.workunit.client.0.smithi064.stdout:0/311: dwrite da/dc/de/d12/f29 [0,4194304] 0 2024-02-16T22:56:19.836 INFO:tasks.workunit.client.1.smithi192.stdout:6/435: creat da/df/d6b/d74/f8b x:0 0 0 2024-02-16T22:56:19.838 INFO:tasks.workunit.client.0.smithi064.stdout:0/312: dread da/dc/de/d12/f29 [0,4194304] 0 2024-02-16T22:56:19.838 INFO:tasks.workunit.client.0.smithi064.stdout:8/308: dwrite df/d31/d2f/d4f/f66 [0,4194304] 0 2024-02-16T22:56:19.839 INFO:tasks.workunit.client.0.smithi064.stdout:1/316: truncate d2/f42 1014800 0 2024-02-16T22:56:19.845 INFO:tasks.workunit.client.1.smithi192.stdout:7/508: write d1/d2/de/d83/f8d [2510337,124222] 0 2024-02-16T22:56:19.851 INFO:tasks.workunit.client.1.smithi192.stdout:3/456: dread da/d11/d2b/f5e [0,4194304] 0 2024-02-16T22:56:19.853 INFO:tasks.workunit.client.1.smithi192.stdout:5/405: symlink d6/d20/l88 0 2024-02-16T22:56:19.853 INFO:tasks.workunit.client.1.smithi192.stdout:5/406: chown d6/d12/d2d/f43 29 1 2024-02-16T22:56:19.854 INFO:tasks.workunit.client.1.smithi192.stdout:5/407: chown d6/d12/d4e/f2e 740501 1 2024-02-16T22:56:19.857 INFO:tasks.workunit.client.0.smithi064.stdout:8/309: symlink df/d31/d5e/l6d 0 2024-02-16T22:56:19.857 INFO:tasks.workunit.client.0.smithi064.stdout:8/310: chown fa 438481 1 2024-02-16T22:56:19.859 INFO:tasks.workunit.client.1.smithi192.stdout:6/436: rename da/df/d18/d46/f61 to da/df/d6b/d83/f8c 0 2024-02-16T22:56:19.861 INFO:tasks.workunit.client.0.smithi064.stdout:0/313: dread da/fb [0,4194304] 0 2024-02-16T22:56:19.864 INFO:tasks.workunit.client.0.smithi064.stdout:5/255: link d6/l11 d6/d18/d19/l57 0 2024-02-16T22:56:19.864 INFO:tasks.workunit.client.1.smithi192.stdout:6/437: dwrite da/df/d18/f63 [0,4194304] 0 2024-02-16T22:56:19.865 INFO:tasks.workunit.client.0.smithi064.stdout:5/256: chown d6/d25 26445 1 2024-02-16T22:56:19.870 INFO:tasks.workunit.client.0.smithi064.stdout:5/257: dwrite d6/fa [0,4194304] 0 2024-02-16T22:56:19.872 INFO:tasks.workunit.client.1.smithi192.stdout:5/408: rmdir d6/d8 39 2024-02-16T22:56:19.875 INFO:tasks.workunit.client.1.smithi192.stdout:0/440: dwrite d1/f19 [4194304,4194304] 0 2024-02-16T22:56:19.876 INFO:tasks.workunit.client.1.smithi192.stdout:0/441: dread - d1/d3/d29/d43/f68 zero size 2024-02-16T22:56:19.877 INFO:tasks.workunit.client.1.smithi192.stdout:0/442: stat d1/d3/d39/c63 0 2024-02-16T22:56:19.880 INFO:tasks.workunit.client.1.smithi192.stdout:0/443: dread - d1/d3/de/d22/d37/d67/d34/f7f zero size 2024-02-16T22:56:19.883 INFO:tasks.workunit.client.1.smithi192.stdout:2/485: getdents d22/d24/d39/d42/d52 0 2024-02-16T22:56:19.883 INFO:tasks.workunit.client.0.smithi064.stdout:8/311: symlink df/d31/d2f/d38/d48/d69/l6e 0 2024-02-16T22:56:19.884 INFO:tasks.workunit.client.1.smithi192.stdout:9/544: dwrite d7/d19/d28/faa [0,4194304] 0 2024-02-16T22:56:19.884 INFO:tasks.workunit.client.0.smithi064.stdout:8/312: write df/d31/d2f/f59 [445082,3702] 0 2024-02-16T22:56:19.885 INFO:tasks.workunit.client.1.smithi192.stdout:9/545: chown d7/d19/d5e/d6f 3 1 2024-02-16T22:56:19.885 INFO:tasks.workunit.client.0.smithi064.stdout:8/313: chown df/d2a/d5c 4735999 1 2024-02-16T22:56:19.886 INFO:tasks.workunit.client.1.smithi192.stdout:7/509: unlink d1/d2/de/d83/lb5 0 2024-02-16T22:56:19.888 INFO:tasks.workunit.client.1.smithi192.stdout:6/438: dread - da/df/d12/d57/f35 zero size 2024-02-16T22:56:19.888 INFO:tasks.workunit.client.1.smithi192.stdout:9/546: read d7/d19/d5b/fa5 [260260,117945] 0 2024-02-16T22:56:19.888 INFO:tasks.workunit.client.1.smithi192.stdout:3/457: fdatasync da/d11/f4b 0 2024-02-16T22:56:19.888 INFO:tasks.workunit.client.0.smithi064.stdout:2/377: getdents d7/d14/d25/d37 0 2024-02-16T22:56:19.889 INFO:tasks.workunit.client.1.smithi192.stdout:7/510: write d1/d2/de/d2c/d49/d54/f67 [5185691,116557] 0 2024-02-16T22:56:19.891 INFO:tasks.workunit.client.1.smithi192.stdout:6/439: truncate da/df/d12/d57/f2f 3620859 0 2024-02-16T22:56:19.891 INFO:tasks.workunit.client.1.smithi192.stdout:9/547: chown d7/d19/d37/f34 247356 1 2024-02-16T22:56:19.891 INFO:tasks.workunit.client.0.smithi064.stdout:1/317: truncate d2/df/d14/f59 3900789 0 2024-02-16T22:56:19.892 INFO:tasks.workunit.client.1.smithi192.stdout:7/511: chown d1/d69/cb0 100838 1 2024-02-16T22:56:19.892 INFO:tasks.workunit.client.1.smithi192.stdout:6/440: chown da/ld 348805099 1 2024-02-16T22:56:19.895 INFO:tasks.workunit.client.0.smithi064.stdout:2/378: dwrite d7/d14/f2c [0,4194304] 0 2024-02-16T22:56:19.897 INFO:tasks.workunit.client.0.smithi064.stdout:2/379: truncate d7/de/f56 426459 0 2024-02-16T22:56:19.898 INFO:tasks.workunit.client.0.smithi064.stdout:2/380: write d7/d14/d25/d46/d55/f71 [908142,59410] 0 2024-02-16T22:56:19.900 INFO:tasks.workunit.client.0.smithi064.stdout:8/314: unlink df/d31/d2f/f2b 0 2024-02-16T22:56:19.905 INFO:tasks.workunit.client.1.smithi192.stdout:2/486: symlink d22/d24/d39/d42/d52/d7c/la2 0 2024-02-16T22:56:19.906 INFO:tasks.workunit.client.0.smithi064.stdout:0/314: creat da/d26/f66 x:0 0 0 2024-02-16T22:56:19.907 INFO:tasks.workunit.client.0.smithi064.stdout:0/315: write da/d26/f33 [3326636,54768] 0 2024-02-16T22:56:19.907 INFO:tasks.workunit.client.0.smithi064.stdout:5/258: truncate d6/d18/d19/d1a/f3b 2201055 0 2024-02-16T22:56:19.911 INFO:tasks.workunit.client.0.smithi064.stdout:1/318: symlink d2/d4e/d56/l6f 0 2024-02-16T22:56:19.913 INFO:tasks.workunit.client.0.smithi064.stdout:2/381: rmdir d7/d14/d25/d37/d44 39 2024-02-16T22:56:19.916 INFO:tasks.workunit.client.1.smithi192.stdout:3/458: read da/f4c [1777574,51836] 0 2024-02-16T22:56:19.916 INFO:tasks.workunit.client.1.smithi192.stdout:3/459: stat da/f1a 0 2024-02-16T22:56:19.916 INFO:tasks.workunit.client.1.smithi192.stdout:3/460: read - da/f94 zero size 2024-02-16T22:56:19.917 INFO:tasks.workunit.client.0.smithi064.stdout:8/315: dread df/f1a [0,4194304] 0 2024-02-16T22:56:19.917 INFO:tasks.workunit.client.1.smithi192.stdout:5/409: rename d6/d8/f3e to d6/d8/d86/f89 0 2024-02-16T22:56:19.917 INFO:tasks.workunit.client.1.smithi192.stdout:3/461: write da/d11/d20/d39/d53/f62 [29582,22708] 0 2024-02-16T22:56:19.918 INFO:tasks.workunit.client.0.smithi064.stdout:5/259: fdatasync d6/d25/f32 0 2024-02-16T22:56:19.919 INFO:tasks.workunit.client.1.smithi192.stdout:9/548: readlink d7/d19/d28/d43/d70/l71 0 2024-02-16T22:56:19.921 INFO:tasks.workunit.client.1.smithi192.stdout:9/549: read d7/d19/d37/f34 [35984,24798] 0 2024-02-16T22:56:19.922 INFO:tasks.workunit.client.1.smithi192.stdout:9/550: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/d92 468319 1 2024-02-16T22:56:19.927 INFO:tasks.workunit.client.1.smithi192.stdout:2/487: rmdir d22/d24/d39/d42/d84 39 2024-02-16T22:56:19.928 INFO:tasks.workunit.client.1.smithi192.stdout:2/488: fsync f2 0 2024-02-16T22:56:19.935 INFO:tasks.workunit.client.0.smithi064.stdout:5/260: creat d6/d1c/d43/f58 x:0 0 0 2024-02-16T22:56:19.939 INFO:tasks.workunit.client.0.smithi064.stdout:1/319: creat d2/df/d14/d2c/d62/f70 x:0 0 0 2024-02-16T22:56:19.940 INFO:tasks.workunit.client.1.smithi192.stdout:3/462: fdatasync da/d11/d63/d6b/f47 0 2024-02-16T22:56:19.941 INFO:tasks.workunit.client.0.smithi064.stdout:2/382: symlink d7/d14/d4b/d63/l78 0 2024-02-16T22:56:19.943 INFO:tasks.workunit.client.0.smithi064.stdout:0/316: truncate da/d26/f28 297845 0 2024-02-16T22:56:19.943 INFO:tasks.workunit.client.1.smithi192.stdout:5/410: dwrite d6/d12/d1d/f3a [0,4194304] 0 2024-02-16T22:56:19.944 INFO:tasks.workunit.client.1.smithi192.stdout:7/512: fsync d1/f4b 0 2024-02-16T22:56:19.949 INFO:tasks.workunit.client.0.smithi064.stdout:9/342: sync 2024-02-16T22:56:19.949 INFO:tasks.workunit.client.0.smithi064.stdout:4/356: sync 2024-02-16T22:56:19.949 INFO:tasks.workunit.client.0.smithi064.stdout:8/316: mknod df/d31/d2f/d4f/d6a/c6f 0 2024-02-16T22:56:19.949 INFO:tasks.workunit.client.0.smithi064.stdout:3/413: sync 2024-02-16T22:56:19.951 INFO:tasks.workunit.client.1.smithi192.stdout:9/551: mkdir d7/d15/d36/dbe 0 2024-02-16T22:56:19.952 INFO:tasks.workunit.client.1.smithi192.stdout:0/444: creat d1/d3/f8c x:0 0 0 2024-02-16T22:56:19.954 INFO:tasks.workunit.client.0.smithi064.stdout:1/320: write d2/df/f1e [1990379,113883] 0 2024-02-16T22:56:19.955 INFO:tasks.workunit.client.0.smithi064.stdout:2/383: creat d7/d14/d1e/d34/d3f/f79 x:0 0 0 2024-02-16T22:56:19.956 INFO:tasks.workunit.client.1.smithi192.stdout:2/489: unlink d22/f72 0 2024-02-16T22:56:19.957 INFO:tasks.workunit.client.1.smithi192.stdout:9/552: dwrite d7/d15/d36/d38/f47 [4194304,4194304] 0 2024-02-16T22:56:19.958 INFO:tasks.workunit.client.1.smithi192.stdout:9/553: dread - d7/d19/f86 zero size 2024-02-16T22:56:19.958 INFO:tasks.workunit.client.0.smithi064.stdout:2/384: chown l3 77190 1 2024-02-16T22:56:19.959 INFO:tasks.workunit.client.0.smithi064.stdout:3/414: write d4/f1e [9602244,28978] 0 2024-02-16T22:56:19.959 INFO:tasks.workunit.client.0.smithi064.stdout:4/357: fdatasync dc/d1d/f4c 0 2024-02-16T22:56:19.962 INFO:tasks.workunit.client.1.smithi192.stdout:3/463: fsync da/d11/d63/d6b/d4e/f56 0 2024-02-16T22:56:19.965 INFO:tasks.workunit.client.1.smithi192.stdout:7/513: symlink d1/d2/de/lb9 0 2024-02-16T22:56:19.966 INFO:tasks.workunit.client.1.smithi192.stdout:6/441: creat da/df/f8d x:0 0 0 2024-02-16T22:56:19.969 INFO:tasks.workunit.client.1.smithi192.stdout:0/445: creat d1/d3/d29/f8d x:0 0 0 2024-02-16T22:56:19.969 INFO:tasks.workunit.client.1.smithi192.stdout:2/490: symlink d22/d24/d39/d3d/la3 0 2024-02-16T22:56:19.970 INFO:tasks.workunit.client.1.smithi192.stdout:2/491: write d22/f92 [137085,112442] 0 2024-02-16T22:56:19.971 INFO:tasks.workunit.client.1.smithi192.stdout:9/554: mkdir d7/d19/d37/dbf 0 2024-02-16T22:56:19.973 INFO:tasks.workunit.client.1.smithi192.stdout:3/464: creat da/d11/d63/d6b/d66/d75/f9f x:0 0 0 2024-02-16T22:56:19.976 INFO:tasks.workunit.client.1.smithi192.stdout:7/514: fsync d1/d2/de/f34 0 2024-02-16T22:56:19.977 INFO:tasks.workunit.client.1.smithi192.stdout:9/555: dwrite d7/d15/d36/d54/f96 [0,4194304] 0 2024-02-16T22:56:19.978 INFO:tasks.workunit.client.1.smithi192.stdout:7/515: fsync d1/d2/de/d2c/d49/f80 0 2024-02-16T22:56:19.982 INFO:tasks.workunit.client.1.smithi192.stdout:5/411: rename d6/d12/d2d/f42 to d6/d8/d80/f8a 0 2024-02-16T22:56:19.982 INFO:tasks.workunit.client.1.smithi192.stdout:3/465: creat da/d11/d63/d6b/d66/d75/fa0 x:0 0 0 2024-02-16T22:56:19.985 INFO:tasks.workunit.client.1.smithi192.stdout:6/442: mknod da/df/d71/c8e 0 2024-02-16T22:56:19.986 INFO:tasks.workunit.client.1.smithi192.stdout:5/412: rmdir d6/d12/d4e/d46 39 2024-02-16T22:56:19.987 INFO:tasks.workunit.client.1.smithi192.stdout:7/516: dwrite d1/d2/d37/f9c [0,4194304] 0 2024-02-16T22:56:19.987 INFO:tasks.workunit.client.1.smithi192.stdout:2/492: dread f21 [0,4194304] 0 2024-02-16T22:56:19.988 INFO:tasks.workunit.client.1.smithi192.stdout:6/443: mknod da/c8f 0 2024-02-16T22:56:19.990 INFO:tasks.workunit.client.1.smithi192.stdout:2/493: dread - d22/d24/d2b/f38 zero size 2024-02-16T22:56:19.990 INFO:tasks.workunit.client.1.smithi192.stdout:7/517: write d1/d2/d37/d86/f8c [418855,18163] 0 2024-02-16T22:56:19.991 INFO:tasks.workunit.client.1.smithi192.stdout:3/466: getdents da/d11 0 2024-02-16T22:56:19.993 INFO:tasks.workunit.client.1.smithi192.stdout:3/467: chown da/d11/d20/d39/d53/f62 88609 1 2024-02-16T22:56:19.994 INFO:tasks.workunit.client.0.smithi064.stdout:8/317: truncate f9 3059930 0 2024-02-16T22:56:19.994 INFO:tasks.workunit.client.0.smithi064.stdout:9/343: mkdir d3/d9/d1e/d3e/d76 0 2024-02-16T22:56:19.995 INFO:tasks.workunit.client.1.smithi192.stdout:7/518: mkdir d1/d2/de/d2c/d49/dba 0 2024-02-16T22:56:19.995 INFO:tasks.workunit.client.0.smithi064.stdout:1/321: mkdir d2/df/d11/d71 0 2024-02-16T22:56:19.995 INFO:tasks.workunit.client.0.smithi064.stdout:2/385: rmdir d7/d14/d1e 39 2024-02-16T22:56:19.996 INFO:tasks.workunit.client.0.smithi064.stdout:3/415: rmdir d4/d13 39 2024-02-16T22:56:19.998 INFO:tasks.workunit.client.1.smithi192.stdout:2/494: creat d22/d24/d39/d42/d65/d66/fa4 x:0 0 0 2024-02-16T22:56:19.998 INFO:tasks.workunit.client.0.smithi064.stdout:3/416: stat d4/d14/d2b/l60 0 2024-02-16T22:56:20.000 INFO:tasks.workunit.client.0.smithi064.stdout:1/322: dread - d2/df/d11/d1a/d1d/f61 zero size 2024-02-16T22:56:20.001 INFO:tasks.workunit.client.1.smithi192.stdout:6/444: rmdir da/df/d6b/d74 39 2024-02-16T22:56:20.002 INFO:tasks.workunit.client.0.smithi064.stdout:4/358: dwrite dc/dd/d32/d22/f24 [0,4194304] 0 2024-02-16T22:56:20.003 INFO:tasks.workunit.client.1.smithi192.stdout:3/468: mkdir da/d11/d63/d6b/d71/da1 0 2024-02-16T22:56:20.008 INFO:tasks.workunit.client.0.smithi064.stdout:5/261: link d6/f29 d6/d1c/f59 0 2024-02-16T22:56:20.009 INFO:tasks.workunit.client.0.smithi064.stdout:5/262: dread - d6/d1c/f53 zero size 2024-02-16T22:56:20.010 INFO:tasks.workunit.client.1.smithi192.stdout:3/469: read da/d11/d20/f8b [94269,73100] 0 2024-02-16T22:56:20.014 INFO:tasks.workunit.client.0.smithi064.stdout:5/263: dread d6/fc [0,4194304] 0 2024-02-16T22:56:20.016 INFO:tasks.workunit.client.0.smithi064.stdout:2/386: truncate d7/d14/f33 1229736 0 2024-02-16T22:56:20.019 INFO:tasks.workunit.client.0.smithi064.stdout:4/359: fdatasync dc/dd/d16/f2e 0 2024-02-16T22:56:20.020 INFO:tasks.workunit.client.1.smithi192.stdout:6/445: readlink da/df/d12/l3b 0 2024-02-16T22:56:20.025 INFO:tasks.workunit.client.0.smithi064.stdout:3/417: rename d4/d13/f26 to d4/d13/d4d/f8f 0 2024-02-16T22:56:20.029 INFO:tasks.workunit.client.0.smithi064.stdout:5/264: rename d6/d18/f4e to d6/d18/f5a 0 2024-02-16T22:56:20.031 INFO:tasks.workunit.client.1.smithi192.stdout:2/495: getdents d22/d24/d39/d42/d52/d7c 0 2024-02-16T22:56:20.034 INFO:tasks.workunit.client.1.smithi192.stdout:2/496: rmdir d22/d24/d39/d42/d52/d7c 39 2024-02-16T22:56:20.036 INFO:tasks.workunit.client.0.smithi064.stdout:5/265: dwrite d6/d18/d19/f26 [0,4194304] 0 2024-02-16T22:56:20.037 INFO:tasks.workunit.client.0.smithi064.stdout:2/387: dread d7/d14/d1e/f21 [0,4194304] 0 2024-02-16T22:56:20.043 INFO:tasks.workunit.client.0.smithi064.stdout:1/323: dread d2/df/f1e [0,4194304] 0 2024-02-16T22:56:20.049 INFO:tasks.workunit.client.1.smithi192.stdout:6/446: dread da/d40/f54 [0,4194304] 0 2024-02-16T22:56:20.050 INFO:tasks.workunit.client.0.smithi064.stdout:4/360: mkdir dc/de/d6d 0 2024-02-16T22:56:20.050 INFO:tasks.workunit.client.0.smithi064.stdout:5/266: mknod d6/d25/d48/c5b 0 2024-02-16T22:56:20.050 INFO:tasks.workunit.client.0.smithi064.stdout:5/267: readlink d6/l12 0 2024-02-16T22:56:20.051 INFO:tasks.workunit.client.0.smithi064.stdout:8/318: creat df/d2a/f70 x:0 0 0 2024-02-16T22:56:20.051 INFO:tasks.workunit.client.0.smithi064.stdout:2/388: truncate d7/de/f2f 1751634 0 2024-02-16T22:56:20.051 INFO:tasks.workunit.client.0.smithi064.stdout:1/324: dwrite d2/df/d14/f16 [0,4194304] 0 2024-02-16T22:56:20.052 INFO:tasks.workunit.client.0.smithi064.stdout:3/418: symlink d4/d13/d1b/d27/d81/l90 0 2024-02-16T22:56:20.060 INFO:tasks.workunit.client.0.smithi064.stdout:2/389: unlink d7/d14/d1e/f28 0 2024-02-16T22:56:20.061 INFO:tasks.workunit.client.0.smithi064.stdout:1/325: rmdir d2/df/d14/d18 39 2024-02-16T22:56:20.062 INFO:tasks.workunit.client.1.smithi192.stdout:6/447: dread da/df/d18/d52/f55 [0,4194304] 0 2024-02-16T22:56:20.063 INFO:tasks.workunit.client.0.smithi064.stdout:3/419: mknod d4/d13/d1b/c91 0 2024-02-16T22:56:20.063 INFO:tasks.workunit.client.0.smithi064.stdout:1/326: write d2/df/d14/d2c/f5e [582628,21089] 0 2024-02-16T22:56:20.063 INFO:tasks.workunit.client.0.smithi064.stdout:4/361: getdents dc/dd/d5b 0 2024-02-16T22:56:20.064 INFO:tasks.workunit.client.0.smithi064.stdout:3/420: chown d4/d13/d1b/d48/l8e 57174213 1 2024-02-16T22:56:20.065 INFO:tasks.workunit.client.0.smithi064.stdout:5/268: dread d6/d18/d19/d1a/f45 [0,4194304] 0 2024-02-16T22:56:20.066 INFO:tasks.workunit.client.0.smithi064.stdout:3/421: stat d4/d13/d1b/d27/d81 0 2024-02-16T22:56:20.066 INFO:tasks.workunit.client.0.smithi064.stdout:5/269: readlink d6/d1c/d4b/l51 0 2024-02-16T22:56:20.068 INFO:tasks.workunit.client.0.smithi064.stdout:2/390: mknod d7/d14/d1e/d34/d73/c7a 0 2024-02-16T22:56:20.070 INFO:tasks.workunit.client.0.smithi064.stdout:4/362: mkdir dc/dd/d16/d6e 0 2024-02-16T22:56:20.071 INFO:tasks.workunit.client.0.smithi064.stdout:4/363: dread - dc/d1d/d43/f5f zero size 2024-02-16T22:56:20.071 INFO:tasks.workunit.client.0.smithi064.stdout:1/327: creat d2/df/d14/d2c/d62/f72 x:0 0 0 2024-02-16T22:56:20.072 INFO:tasks.workunit.client.0.smithi064.stdout:3/422: fdatasync d4/d1f/d34/f56 0 2024-02-16T22:56:20.072 INFO:tasks.workunit.client.0.smithi064.stdout:1/328: stat d2/d4e/l6a 0 2024-02-16T22:56:20.072 INFO:tasks.workunit.client.0.smithi064.stdout:4/364: write dc/dd/d32/d58/f38 [544172,58551] 0 2024-02-16T22:56:20.073 INFO:tasks.workunit.client.0.smithi064.stdout:1/329: fdatasync d2/d4e/d56/f57 0 2024-02-16T22:56:20.074 INFO:tasks.workunit.client.0.smithi064.stdout:5/270: write d6/d25/d48/f4a [5137075,46989] 0 2024-02-16T22:56:20.075 INFO:tasks.workunit.client.0.smithi064.stdout:5/271: chown d6/d18/f56 853 1 2024-02-16T22:56:20.077 INFO:tasks.workunit.client.0.smithi064.stdout:2/391: creat d7/d14/d25/d46/d55/d57/f7b x:0 0 0 2024-02-16T22:56:20.078 INFO:tasks.workunit.client.0.smithi064.stdout:3/423: chown d4/d13/d1b/d27/d61/d63/l64 18831 1 2024-02-16T22:56:20.083 INFO:tasks.workunit.client.0.smithi064.stdout:5/272: mknod d6/d18/d19/c5c 0 2024-02-16T22:56:20.088 INFO:tasks.workunit.client.0.smithi064.stdout:5/273: mkdir d6/d18/d5d 0 2024-02-16T22:56:20.090 INFO:tasks.workunit.client.0.smithi064.stdout:3/424: rename d4/d13/d1b/d27/d61/l88 to d4/d14/d2b/d72/d7d/l92 0 2024-02-16T22:56:20.092 INFO:tasks.workunit.client.1.smithi192.stdout:8/436: sync 2024-02-16T22:56:20.096 INFO:tasks.workunit.client.0.smithi064.stdout:3/425: symlink d4/d14/d2b/d72/l93 0 2024-02-16T22:56:20.096 INFO:tasks.workunit.client.0.smithi064.stdout:4/365: dread dc/f33 [0,4194304] 0 2024-02-16T22:56:20.097 INFO:tasks.workunit.client.0.smithi064.stdout:2/392: getdents d7/d14/d1e/d34/d3f 0 2024-02-16T22:56:20.097 INFO:tasks.workunit.client.0.smithi064.stdout:3/426: write d4/d13/d1b/d27/f7f [759295,106635] 0 2024-02-16T22:56:20.100 INFO:tasks.workunit.client.1.smithi192.stdout:8/437: rmdir d5/d3e 39 2024-02-16T22:56:20.105 INFO:tasks.workunit.client.0.smithi064.stdout:2/393: creat d7/d14/d25/f7c x:0 0 0 2024-02-16T22:56:20.105 INFO:tasks.workunit.client.0.smithi064.stdout:2/394: stat d7/l6e 0 2024-02-16T22:56:20.106 INFO:tasks.workunit.client.0.smithi064.stdout:5/274: fdatasync d6/d25/d48/f4a 0 2024-02-16T22:56:20.108 INFO:tasks.workunit.client.1.smithi192.stdout:0/446: dwrite d1/d3/d39/f79 [0,4194304] 0 2024-02-16T22:56:20.108 INFO:tasks.workunit.client.0.smithi064.stdout:5/275: write d6/f31 [273721,14598] 0 2024-02-16T22:56:20.109 INFO:tasks.workunit.client.0.smithi064.stdout:3/427: symlink d4/d1f/d5b/d51/d7c/l94 0 2024-02-16T22:56:20.110 INFO:tasks.workunit.client.1.smithi192.stdout:9/556: dwrite d7/d15/d36/d54/f65 [0,4194304] 0 2024-02-16T22:56:20.111 INFO:tasks.workunit.client.1.smithi192.stdout:5/413: dwrite d6/d8/d80/f40 [0,4194304] 0 2024-02-16T22:56:20.111 INFO:tasks.workunit.client.1.smithi192.stdout:7/519: dwrite d1/d2/d2d/d61/d88/d90/d2a/f63 [0,4194304] 0 2024-02-16T22:56:20.118 INFO:tasks.workunit.client.0.smithi064.stdout:8/319: dwrite df/d31/f41 [0,4194304] 0 2024-02-16T22:56:20.119 INFO:tasks.workunit.client.0.smithi064.stdout:8/320: readlink df/l21 0 2024-02-16T22:56:20.120 INFO:tasks.workunit.client.0.smithi064.stdout:2/395: creat d7/d14/d25/d46/d55/f7d x:0 0 0 2024-02-16T22:56:20.121 INFO:tasks.workunit.client.0.smithi064.stdout:8/321: chown df/d31/d2f/f55 10 1 2024-02-16T22:56:20.122 INFO:tasks.workunit.client.0.smithi064.stdout:2/396: chown d7/d14/d4b/d63/l78 5116 1 2024-02-16T22:56:20.127 INFO:tasks.workunit.client.1.smithi192.stdout:0/447: creat d1/d3/de/d22/d37/d67/d75/f8e x:0 0 0 2024-02-16T22:56:20.128 INFO:tasks.workunit.client.0.smithi064.stdout:5/276: creat d6/d25/d48/f5e x:0 0 0 2024-02-16T22:56:20.128 INFO:tasks.workunit.client.1.smithi192.stdout:0/448: dread - d1/d3/de/d22/d37/d47/d53/f65 zero size 2024-02-16T22:56:20.134 INFO:tasks.workunit.client.1.smithi192.stdout:7/520: mkdir d1/d2/d2d/d61/d8a/dbb 0 2024-02-16T22:56:20.135 INFO:tasks.workunit.client.1.smithi192.stdout:7/521: chown d1/d2/d37/l42 15641 1 2024-02-16T22:56:20.135 INFO:tasks.workunit.client.0.smithi064.stdout:1/330: dwrite d2/df/f1c [0,4194304] 0 2024-02-16T22:56:20.140 INFO:tasks.workunit.client.1.smithi192.stdout:5/414: truncate d6/d12/d1d/f36 539279 0 2024-02-16T22:56:20.141 INFO:tasks.workunit.client.1.smithi192.stdout:5/415: write d6/d12/f83 [701280,3926] 0 2024-02-16T22:56:20.141 INFO:tasks.workunit.client.0.smithi064.stdout:2/397: creat d7/d14/d25/d46/d55/f7e x:0 0 0 2024-02-16T22:56:20.143 INFO:tasks.workunit.client.0.smithi064.stdout:2/398: readlink d7/de/l51 0 2024-02-16T22:56:20.144 INFO:tasks.workunit.client.0.smithi064.stdout:2/399: chown d7/de/l42 661276 1 2024-02-16T22:56:20.145 INFO:tasks.workunit.client.0.smithi064.stdout:8/322: dwrite df/d2a/f70 [0,4194304] 0 2024-02-16T22:56:20.145 INFO:tasks.workunit.client.0.smithi064.stdout:8/323: chown df/f1b 0 1 2024-02-16T22:56:20.146 INFO:tasks.workunit.client.0.smithi064.stdout:5/277: mknod d6/d18/d39/c5f 0 2024-02-16T22:56:20.148 INFO:tasks.workunit.client.1.smithi192.stdout:1/431: sync 2024-02-16T22:56:20.148 INFO:tasks.workunit.client.1.smithi192.stdout:4/514: sync 2024-02-16T22:56:20.148 INFO:tasks.workunit.client.1.smithi192.stdout:9/557: mknod d7/d15/d36/dbe/cc0 0 2024-02-16T22:56:20.153 INFO:tasks.workunit.client.0.smithi064.stdout:6/409: sync 2024-02-16T22:56:20.153 INFO:tasks.workunit.client.1.smithi192.stdout:7/522: creat d1/d2/de/d83/fbc x:0 0 0 2024-02-16T22:56:20.155 INFO:tasks.workunit.client.0.smithi064.stdout:6/410: write d0/d5/d54/d60/f7c [722193,25987] 0 2024-02-16T22:56:20.155 INFO:tasks.workunit.client.1.smithi192.stdout:5/416: mkdir d6/d12/d4e/d5b/d67/d8b 0 2024-02-16T22:56:20.157 INFO:tasks.workunit.client.0.smithi064.stdout:3/428: mknod d4/d13/d35/c95 0 2024-02-16T22:56:20.157 INFO:tasks.workunit.client.1.smithi192.stdout:8/438: link d5/d9/c5b d5/d3a/c7f 0 2024-02-16T22:56:20.158 INFO:tasks.workunit.client.0.smithi064.stdout:3/429: stat d4/d13/d1b/d4f/f6f 0 2024-02-16T22:56:20.160 INFO:tasks.workunit.client.1.smithi192.stdout:0/449: symlink d1/l8f 0 2024-02-16T22:56:20.161 INFO:tasks.workunit.client.0.smithi064.stdout:4/366: dwrite dc/dd/f42 [0,4194304] 0 2024-02-16T22:56:20.164 INFO:tasks.workunit.client.0.smithi064.stdout:5/278: creat d6/d18/d19/d1a/f60 x:0 0 0 2024-02-16T22:56:20.165 INFO:tasks.workunit.client.0.smithi064.stdout:5/279: chown d6/d18/d19 143662 1 2024-02-16T22:56:20.167 INFO:tasks.workunit.client.1.smithi192.stdout:4/515: creat d1/d26/d66/d89/d11/fb7 x:0 0 0 2024-02-16T22:56:20.168 INFO:tasks.workunit.client.1.smithi192.stdout:4/516: chown d1/d26/d6f 0 1 2024-02-16T22:56:20.168 INFO:tasks.workunit.client.0.smithi064.stdout:1/331: fsync d2/df/d14/f52 0 2024-02-16T22:56:20.170 INFO:tasks.workunit.client.1.smithi192.stdout:6/448: link f9 da/df/d12/f90 0 2024-02-16T22:56:20.172 INFO:tasks.workunit.client.0.smithi064.stdout:6/411: write d0/d6/da/f6a [21341,13912] 0 2024-02-16T22:56:20.175 INFO:tasks.workunit.client.0.smithi064.stdout:2/400: write d7/d14/d25/d37/d44/f5c [2586535,3046] 0 2024-02-16T22:56:20.176 INFO:tasks.workunit.client.1.smithi192.stdout:7/523: mkdir d1/d2/de/d2c/d49/d89/dbd 0 2024-02-16T22:56:20.178 INFO:tasks.workunit.client.1.smithi192.stdout:5/417: mkdir d6/d8/d75/d8c 0 2024-02-16T22:56:20.178 INFO:tasks.workunit.client.1.smithi192.stdout:9/558: dread d7/d15/d36/d38/f47 [0,4194304] 0 2024-02-16T22:56:20.179 INFO:tasks.workunit.client.0.smithi064.stdout:8/324: creat df/d31/d2f/d68/f71 x:0 0 0 2024-02-16T22:56:20.182 INFO:tasks.workunit.client.1.smithi192.stdout:0/450: mknod d1/d3/de/d22/c90 0 2024-02-16T22:56:20.183 INFO:tasks.workunit.client.0.smithi064.stdout:5/280: mknod d6/d1c/d4b/c61 0 2024-02-16T22:56:20.184 INFO:tasks.workunit.client.0.smithi064.stdout:5/281: write d6/f31 [1620184,69539] 0 2024-02-16T22:56:20.185 INFO:tasks.workunit.client.1.smithi192.stdout:9/559: dwrite d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f9a [4194304,4194304] 0 2024-02-16T22:56:20.186 INFO:tasks.workunit.client.1.smithi192.stdout:4/517: creat d1/d26/d66/d89/da8/fb8 x:0 0 0 2024-02-16T22:56:20.188 INFO:tasks.workunit.client.0.smithi064.stdout:6/412: truncate d0/d5/df/d17/d18/f3f 6516275 0 2024-02-16T22:56:20.188 INFO:tasks.workunit.client.1.smithi192.stdout:6/449: chown da/df/d58/l89 214941005 1 2024-02-16T22:56:20.190 INFO:tasks.workunit.client.1.smithi192.stdout:6/450: read da/df/d12/f47 [1321599,35374] 0 2024-02-16T22:56:20.191 INFO:tasks.workunit.client.1.smithi192.stdout:7/524: write d1/f11 [850385,102554] 0 2024-02-16T22:56:20.193 INFO:tasks.workunit.client.0.smithi064.stdout:3/430: mkdir d4/d13/d4d/d5c/d78/d96 0 2024-02-16T22:56:20.194 INFO:tasks.workunit.client.0.smithi064.stdout:3/431: chown d4/d13/d4d/d5c/d8b 2017 1 2024-02-16T22:56:20.195 INFO:tasks.workunit.client.0.smithi064.stdout:3/432: chown d4/d1f/f32 371322 1 2024-02-16T22:56:20.196 INFO:tasks.workunit.client.1.smithi192.stdout:5/418: dwrite d6/d12/d2d/d5c/f65 [0,4194304] 0 2024-02-16T22:56:20.199 INFO:tasks.workunit.client.0.smithi064.stdout:4/367: symlink dc/l6f 0 2024-02-16T22:56:20.200 INFO:tasks.workunit.client.1.smithi192.stdout:0/451: fdatasync d1/d3/d29/f1a 0 2024-02-16T22:56:20.202 INFO:tasks.workunit.client.0.smithi064.stdout:3/433: dwrite d4/d1f/d34/f74 [0,4194304] 0 2024-02-16T22:56:20.205 INFO:tasks.workunit.client.1.smithi192.stdout:9/560: unlink d7/d15/d36/d38/d3f/d8a/d98/fae 0 2024-02-16T22:56:20.207 INFO:tasks.workunit.client.0.smithi064.stdout:4/368: dwrite dc/dd/f42 [0,4194304] 0 2024-02-16T22:56:20.216 INFO:tasks.workunit.client.1.smithi192.stdout:1/432: link d3/de/d27/d34/d36/d47/d5d/l7b d3/de/d27/d34/d36/d47/l91 0 2024-02-16T22:56:20.217 INFO:tasks.workunit.client.0.smithi064.stdout:2/401: truncate d7/de/f2f 2652011 0 2024-02-16T22:56:20.217 INFO:tasks.workunit.client.1.smithi192.stdout:6/451: fdatasync da/d40/f54 0 2024-02-16T22:56:20.220 INFO:tasks.workunit.client.0.smithi064.stdout:8/325: mknod df/d31/d2f/d3e/d5f/c72 0 2024-02-16T22:56:20.221 INFO:tasks.workunit.client.0.smithi064.stdout:1/332: write d2/df/d14/d18/f28 [601610,66307] 0 2024-02-16T22:56:20.221 INFO:tasks.workunit.client.0.smithi064.stdout:1/333: stat d2/lb 0 2024-02-16T22:56:20.221 INFO:tasks.workunit.client.0.smithi064.stdout:8/326: read - df/d31/d2f/d38/f62 zero size 2024-02-16T22:56:20.222 INFO:tasks.workunit.client.0.smithi064.stdout:1/334: rename d2 to d2/df/d14/d2a/d73 22 2024-02-16T22:56:20.223 INFO:tasks.workunit.client.1.smithi192.stdout:7/525: dread d1/d69/f7f [0,4194304] 0 2024-02-16T22:56:20.224 INFO:tasks.workunit.client.1.smithi192.stdout:8/439: creat d5/db/de/f80 x:0 0 0 2024-02-16T22:56:20.225 INFO:tasks.workunit.client.0.smithi064.stdout:3/434: mkdir d4/d13/d1b/d27/d44/d97 0 2024-02-16T22:56:20.225 INFO:tasks.workunit.client.1.smithi192.stdout:7/526: write d1/d2/f1e [1005657,127730] 0 2024-02-16T22:56:20.225 INFO:tasks.workunit.client.0.smithi064.stdout:3/435: fdatasync d4/d13/d1b/d27/d61/d63/f71 0 2024-02-16T22:56:20.226 INFO:tasks.workunit.client.1.smithi192.stdout:0/452: mknod d1/d3/de/d22/d37/d67/c91 0 2024-02-16T22:56:20.226 INFO:tasks.workunit.client.0.smithi064.stdout:3/436: dread - d4/d13/d1b/d27/d44/f4b zero size 2024-02-16T22:56:20.227 INFO:tasks.workunit.client.1.smithi192.stdout:4/518: dwrite d1/d26/d5e/f63 [0,4194304] 0 2024-02-16T22:56:20.227 INFO:tasks.workunit.client.1.smithi192.stdout:7/527: chown d1/d2/de/f24 3 1 2024-02-16T22:56:20.228 INFO:tasks.workunit.client.0.smithi064.stdout:6/413: getdents d0/d6/d10/d33/d72 0 2024-02-16T22:56:20.229 INFO:tasks.workunit.client.1.smithi192.stdout:1/433: symlink d3/de/d27/d34/d52/d63/l92 0 2024-02-16T22:56:20.229 INFO:tasks.workunit.client.0.smithi064.stdout:2/402: mkdir d7/d14/d25/d46/d55/d7f 0 2024-02-16T22:56:20.230 INFO:tasks.workunit.client.1.smithi192.stdout:6/452: fsync da/df/d6b/f88 0 2024-02-16T22:56:20.232 INFO:tasks.workunit.client.1.smithi192.stdout:6/453: truncate da/df/d12/d15/d1b/f8a 589123 0 2024-02-16T22:56:20.232 INFO:tasks.workunit.client.1.smithi192.stdout:5/419: symlink d6/l8d 0 2024-02-16T22:56:20.232 INFO:tasks.workunit.client.1.smithi192.stdout:5/420: fsync d6/d8/d75/f7b 0 2024-02-16T22:56:20.233 INFO:tasks.workunit.client.1.smithi192.stdout:5/421: stat d6/d12/d4e/d58/l6a 0 2024-02-16T22:56:20.233 INFO:tasks.workunit.client.0.smithi064.stdout:8/327: mkdir df/d31/d5e/d73 0 2024-02-16T22:56:20.234 INFO:tasks.workunit.client.1.smithi192.stdout:5/422: readlink d6/d8/d80/l69 0 2024-02-16T22:56:20.234 INFO:tasks.workunit.client.0.smithi064.stdout:8/328: fsync df/d31/f29 0 2024-02-16T22:56:20.234 INFO:tasks.workunit.client.1.smithi192.stdout:5/423: chown d6/d20/l35 229 1 2024-02-16T22:56:20.235 INFO:tasks.workunit.client.0.smithi064.stdout:8/329: stat df/c20 0 2024-02-16T22:56:20.235 INFO:tasks.workunit.client.1.smithi192.stdout:9/561: dread d7/d19/d37/d39/f4e [0,4194304] 0 2024-02-16T22:56:20.235 INFO:tasks.workunit.client.0.smithi064.stdout:6/414: dwrite d0/d5/d54/d60/f6d [0,4194304] 0 2024-02-16T22:56:20.237 INFO:tasks.workunit.client.0.smithi064.stdout:7/296: sync 2024-02-16T22:56:20.244 INFO:tasks.workunit.client.1.smithi192.stdout:4/519: mkdir d1/d26/d34/db9 0 2024-02-16T22:56:20.245 INFO:tasks.workunit.client.0.smithi064.stdout:1/335: mknod d2/d4e/d56/d5b/c74 0 2024-02-16T22:56:20.246 INFO:tasks.workunit.client.1.smithi192.stdout:4/520: write d1/d26/d66/d89/d11/f98 [304971,109885] 0 2024-02-16T22:56:20.247 INFO:tasks.workunit.client.1.smithi192.stdout:4/521: chown d1/d15/d48/d8d/d9b 1143833 1 2024-02-16T22:56:20.247 INFO:tasks.workunit.client.1.smithi192.stdout:7/528: symlink d1/d2/de/lbe 0 2024-02-16T22:56:20.251 INFO:tasks.workunit.client.1.smithi192.stdout:4/522: dread d1/d15/d48/d8d/d9b/fad [0,4194304] 0 2024-02-16T22:56:20.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/454: creat da/df/d58/f91 x:0 0 0 2024-02-16T22:56:20.253 INFO:tasks.workunit.client.1.smithi192.stdout:6/455: write da/df/d18/d52/f7c [661386,129851] 0 2024-02-16T22:56:20.253 INFO:tasks.workunit.client.0.smithi064.stdout:8/330: dread - df/d2a/f60 zero size 2024-02-16T22:56:20.253 INFO:tasks.workunit.client.1.smithi192.stdout:5/424: mknod d6/d12/d1d/d56/c8e 0 2024-02-16T22:56:20.255 INFO:tasks.workunit.client.0.smithi064.stdout:6/415: rename d0/d6/d10/d33/f65 to d0/d1d/d21/f7f 0 2024-02-16T22:56:20.257 INFO:tasks.workunit.client.0.smithi064.stdout:7/297: mknod d5/d6/d49/c62 0 2024-02-16T22:56:20.257 INFO:tasks.workunit.client.0.smithi064.stdout:7/298: chown d5/f2f 0 1 2024-02-16T22:56:20.259 INFO:tasks.workunit.client.1.smithi192.stdout:9/562: rename d7/d15/l42 to d7/d19/d28/d43/lc1 0 2024-02-16T22:56:20.259 INFO:tasks.workunit.client.0.smithi064.stdout:5/282: link d6/f1b d6/d1c/f62 0 2024-02-16T22:56:20.259 INFO:tasks.workunit.client.0.smithi064.stdout:5/283: stat d6/f34 0 2024-02-16T22:56:20.263 INFO:tasks.workunit.client.1.smithi192.stdout:7/529: symlink d1/d2/d2d/d61/d88/lbf 0 2024-02-16T22:56:20.263 INFO:tasks.workunit.client.1.smithi192.stdout:0/453: dwrite d1/d3/de/d22/f30 [0,4194304] 0 2024-02-16T22:56:20.263 INFO:tasks.workunit.client.0.smithi064.stdout:4/369: getdents dc/d4a 0 2024-02-16T22:56:20.267 INFO:tasks.workunit.client.1.smithi192.stdout:9/563: dwrite d7/d15/f5d [0,4194304] 0 2024-02-16T22:56:20.269 INFO:tasks.workunit.client.1.smithi192.stdout:5/425: mkdir d6/d12/d4e/d58/d8f 0 2024-02-16T22:56:20.269 INFO:tasks.workunit.client.0.smithi064.stdout:3/437: dread d4/d13/f28 [0,4194304] 0 2024-02-16T22:56:20.279 INFO:tasks.workunit.client.1.smithi192.stdout:1/434: dread d3/de/ff [0,4194304] 0 2024-02-16T22:56:20.280 INFO:tasks.workunit.client.1.smithi192.stdout:7/530: mknod d1/d69/cc0 0 2024-02-16T22:56:20.281 INFO:tasks.workunit.client.1.smithi192.stdout:4/523: dwrite d1/d15/d23/f52 [0,4194304] 0 2024-02-16T22:56:20.283 INFO:tasks.workunit.client.1.smithi192.stdout:6/456: mknod da/df/d12/c92 0 2024-02-16T22:56:20.283 INFO:tasks.workunit.client.1.smithi192.stdout:9/564: truncate d7/d19/d37/f3d 8507759 0 2024-02-16T22:56:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:20 smithi192 ceph-mon[36254]: Adjusting osd_memory_target on smithi064 to 1557M 2024-02-16T22:56:20.285 INFO:tasks.workunit.client.1.smithi192.stdout:2/497: sync 2024-02-16T22:56:20.293 INFO:tasks.workunit.client.1.smithi192.stdout:3/470: sync 2024-02-16T22:56:20.294 INFO:tasks.workunit.client.1.smithi192.stdout:5/426: rename d6/d12/d1d/l64 to d6/d12/d4e/d5b/l90 0 2024-02-16T22:56:20.294 INFO:tasks.workunit.client.1.smithi192.stdout:8/440: getdents d5/d3a/d4d 0 2024-02-16T22:56:20.294 INFO:tasks.workunit.client.1.smithi192.stdout:0/454: unlink d1/d3/de/c5b 0 2024-02-16T22:56:20.294 INFO:tasks.workunit.client.1.smithi192.stdout:0/455: stat d1/d3/de/d22/d37/d47/d53/f65 0 2024-02-16T22:56:20.294 INFO:tasks.workunit.client.1.smithi192.stdout:1/435: mknod d3/de/d27/d34/d52/d5a/c93 0 2024-02-16T22:56:20.295 INFO:tasks.workunit.client.1.smithi192.stdout:9/565: fsync d7/d19/d37/d39/f4e 0 2024-02-16T22:56:20.295 INFO:tasks.workunit.client.1.smithi192.stdout:2/498: rmdir d22/d24/d2b 39 2024-02-16T22:56:20.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/441: mkdir d5/d16/d81 0 2024-02-16T22:56:20.296 INFO:tasks.workunit.client.1.smithi192.stdout:2/499: dread d22/d24/d39/d42/d65/d69/d4f/f97 [0,4194304] 0 2024-02-16T22:56:20.298 INFO:tasks.workunit.client.1.smithi192.stdout:9/566: creat d7/d15/d36/fc2 x:0 0 0 2024-02-16T22:56:20.299 INFO:tasks.workunit.client.1.smithi192.stdout:9/567: stat d7/d15/d36/d77/c8f 0 2024-02-16T22:56:20.304 INFO:tasks.workunit.client.1.smithi192.stdout:2/500: mknod d22/d24/d39/d3d/ca5 0 2024-02-16T22:56:20.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/501: write f4 [473541,57548] 0 2024-02-16T22:56:20.306 INFO:tasks.workunit.client.1.smithi192.stdout:2/502: fsync d22/f31 0 2024-02-16T22:56:20.307 INFO:tasks.workunit.client.1.smithi192.stdout:4/524: write d1/d15/d48/f55 [1492334,6292] 0 2024-02-16T22:56:20.308 INFO:tasks.workunit.client.1.smithi192.stdout:2/503: dread - d22/d24/fa0 zero size 2024-02-16T22:56:20.308 INFO:tasks.workunit.client.1.smithi192.stdout:4/525: stat d1/d15/d48/d8d/fb2 0 2024-02-16T22:56:20.308 INFO:tasks.workunit.client.1.smithi192.stdout:8/442: symlink d5/db/l82 0 2024-02-16T22:56:20.309 INFO:tasks.workunit.client.1.smithi192.stdout:0/456: link d1/d3/de/d22/d37/d67/d34/f6e d1/d3/de/d22/d37/d67/d34/f92 0 2024-02-16T22:56:20.310 INFO:tasks.workunit.client.1.smithi192.stdout:8/443: dread - d5/d16/f7d zero size 2024-02-16T22:56:20.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/436: write d3/f16 [2998947,49507] 0 2024-02-16T22:56:20.311 INFO:tasks.workunit.client.0.smithi064.stdout:4/370: rmdir dc/d4a 39 2024-02-16T22:56:20.312 INFO:tasks.workunit.client.0.smithi064.stdout:6/416: dread - d0/d1b/f50 zero size 2024-02-16T22:56:20.313 INFO:tasks.workunit.client.0.smithi064.stdout:2/403: dwrite d7/d14/f2e [0,4194304] 0 2024-02-16T22:56:20.313 INFO:tasks.workunit.client.1.smithi192.stdout:3/471: dread da/d11/f3f [0,4194304] 0 2024-02-16T22:56:20.314 INFO:tasks.workunit.client.1.smithi192.stdout:4/526: dread d1/d26/d5e/f63 [0,4194304] 0 2024-02-16T22:56:20.314 INFO:tasks.workunit.client.1.smithi192.stdout:7/531: dread d1/d2/f58 [4194304,4194304] 0 2024-02-16T22:56:20.315 INFO:tasks.workunit.client.1.smithi192.stdout:7/532: fsync d1/d2/de/d83/d9f/fa2 0 2024-02-16T22:56:20.319 INFO:tasks.workunit.client.0.smithi064.stdout:1/336: dwrite d2/df/d14/d2c/f5f [0,4194304] 0 2024-02-16T22:56:20.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/527: dwrite d1/d15/d48/d8d/f7b [4194304,4194304] 0 2024-02-16T22:56:20.324 INFO:tasks.workunit.client.1.smithi192.stdout:9/568: write d7/f83 [493406,29461] 0 2024-02-16T22:56:20.327 INFO:tasks.workunit.client.1.smithi192.stdout:7/533: dwrite d1/d2/de/d2c/d49/da4/f85 [0,4194304] 0 2024-02-16T22:56:20.328 INFO:tasks.workunit.client.1.smithi192.stdout:5/427: dwrite d6/d12/d1d/f36 [0,4194304] 0 2024-02-16T22:56:20.329 INFO:tasks.workunit.client.1.smithi192.stdout:9/569: fdatasync d7/d19/d37/f9e 0 2024-02-16T22:56:20.329 INFO:tasks.workunit.client.1.smithi192.stdout:5/428: chown d6/d20 43 1 2024-02-16T22:56:20.330 INFO:tasks.workunit.client.1.smithi192.stdout:7/534: readlink d1/d2/l50 0 2024-02-16T22:56:20.330 INFO:tasks.workunit.client.1.smithi192.stdout:5/429: chown d6/d12/d1d 0 1 2024-02-16T22:56:20.332 INFO:tasks.workunit.client.0.smithi064.stdout:4/371: rename dc/dd/f42 to dc/dd/d32/d58/f70 0 2024-02-16T22:56:20.337 INFO:tasks.workunit.client.1.smithi192.stdout:3/472: mknod da/d11/d63/d6b/d4e/ca2 0 2024-02-16T22:56:20.338 INFO:tasks.workunit.client.0.smithi064.stdout:8/331: rmdir df/d2a/d4d 0 2024-02-16T22:56:20.339 INFO:tasks.workunit.client.0.smithi064.stdout:8/332: dread - df/d31/d2f/d68/f71 zero size 2024-02-16T22:56:20.340 INFO:tasks.workunit.client.0.smithi064.stdout:8/333: fdatasync df/d31/d2f/d4f/f66 0 2024-02-16T22:56:20.341 INFO:tasks.workunit.client.0.smithi064.stdout:3/438: link d4/l6 d4/d1f/d34/l98 0 2024-02-16T22:56:20.342 INFO:tasks.workunit.client.0.smithi064.stdout:8/334: chown df/d31/d2f/c26 34145547 1 2024-02-16T22:56:20.349 INFO:tasks.workunit.client.0.smithi064.stdout:4/372: fsync f3 0 2024-02-16T22:56:20.351 INFO:tasks.workunit.client.1.smithi192.stdout:8/444: dwrite d5/f25 [0,4194304] 0 2024-02-16T22:56:20.351 INFO:tasks.workunit.client.1.smithi192.stdout:1/437: dwrite d3/de/f65 [0,4194304] 0 2024-02-16T22:56:20.352 INFO:tasks.workunit.client.0.smithi064.stdout:1/337: mkdir d2/df/d14/d18/d48/d6b/d75 0 2024-02-16T22:56:20.352 INFO:tasks.workunit.client.0.smithi064.stdout:1/338: chown d2/d4e 19870907 1 2024-02-16T22:56:20.353 INFO:tasks.workunit.client.1.smithi192.stdout:0/457: chown d1/d3/de/l3d 7 1 2024-02-16T22:56:20.358 INFO:tasks.workunit.client.1.smithi192.stdout:5/430: dread d6/d20/f4d [0,4194304] 0 2024-02-16T22:56:20.360 INFO:tasks.workunit.client.0.smithi064.stdout:1/339: dwrite d2/df/d11/d1a/d30/f63 [0,4194304] 0 2024-02-16T22:56:20.361 INFO:tasks.workunit.client.0.smithi064.stdout:9/344: sync 2024-02-16T22:56:20.361 INFO:tasks.workunit.client.0.smithi064.stdout:0/317: sync 2024-02-16T22:56:20.361 INFO:tasks.workunit.client.1.smithi192.stdout:3/473: rename da/d11/l13 to da/d11/d2b/d43/la3 0 2024-02-16T22:56:20.363 INFO:tasks.workunit.client.0.smithi064.stdout:9/345: readlink d3/d11/d15/d23/l2d 0 2024-02-16T22:56:20.364 INFO:tasks.workunit.client.0.smithi064.stdout:1/340: write d2/df/d14/f16 [4078286,21853] 0 2024-02-16T22:56:20.365 INFO:tasks.workunit.client.0.smithi064.stdout:8/335: fsync df/d31/d2f/d38/d48/f49 0 2024-02-16T22:56:20.367 INFO:tasks.workunit.client.0.smithi064.stdout:8/336: readlink df/d31/d2f/l25 0 2024-02-16T22:56:20.367 INFO:tasks.workunit.client.0.smithi064.stdout:6/417: link d0/d5/df/d17/d18/l52 d0/d6/l80 0 2024-02-16T22:56:20.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:20 smithi064 ceph-mon[29226]: Adjusting osd_memory_target on smithi064 to 1557M 2024-02-16T22:56:20.369 INFO:tasks.workunit.client.0.smithi064.stdout:6/418: write d0/d6/d10/f40 [3115757,127648] 0 2024-02-16T22:56:20.371 INFO:tasks.workunit.client.1.smithi192.stdout:8/445: mkdir d5/d26/d83 0 2024-02-16T22:56:20.373 INFO:tasks.workunit.client.0.smithi064.stdout:0/318: rmdir da/dc/de/d12/d1d 39 2024-02-16T22:56:20.374 INFO:tasks.workunit.client.1.smithi192.stdout:1/438: truncate d3/f46 703812 0 2024-02-16T22:56:20.375 INFO:tasks.workunit.client.1.smithi192.stdout:1/439: fsync d3/de/d27/d34/d52/d63/d81/f8d 0 2024-02-16T22:56:20.376 INFO:tasks.workunit.client.1.smithi192.stdout:1/440: readlink d3/de/d27/d34/d36/l5f 0 2024-02-16T22:56:20.377 INFO:tasks.workunit.client.1.smithi192.stdout:0/458: creat d1/d3/de/d22/d37/d47/f93 x:0 0 0 2024-02-16T22:56:20.377 INFO:tasks.workunit.client.0.smithi064.stdout:1/341: creat d2/df/f76 x:0 0 0 2024-02-16T22:56:20.378 INFO:tasks.workunit.client.1.smithi192.stdout:1/441: truncate d3/d5b/f71 320641 0 2024-02-16T22:56:20.379 INFO:tasks.workunit.client.1.smithi192.stdout:5/431: symlink d6/d20/l91 0 2024-02-16T22:56:20.381 INFO:tasks.workunit.client.1.smithi192.stdout:2/504: link d22/d24/c5d d22/ca6 0 2024-02-16T22:56:20.382 INFO:tasks.workunit.client.1.smithi192.stdout:4/528: link d1/f19 d1/d15/d48/d8d/fba 0 2024-02-16T22:56:20.383 INFO:tasks.workunit.client.1.smithi192.stdout:4/529: write d1/d15/d23/f5c [2273725,34776] 0 2024-02-16T22:56:20.384 INFO:tasks.workunit.client.1.smithi192.stdout:4/530: chown d1/d26/d66/d89/d11/fb7 3955 1 2024-02-16T22:56:20.384 INFO:tasks.workunit.client.0.smithi064.stdout:8/337: dread df/f1b [0,4194304] 0 2024-02-16T22:56:20.384 INFO:tasks.workunit.client.1.smithi192.stdout:7/535: creat d1/fc1 x:0 0 0 2024-02-16T22:56:20.385 INFO:tasks.workunit.client.1.smithi192.stdout:7/536: chown d1/fac 2 1 2024-02-16T22:56:20.385 INFO:tasks.workunit.client.0.smithi064.stdout:8/338: fsync df/d31/d2f/f44 0 2024-02-16T22:56:20.386 INFO:tasks.workunit.client.0.smithi064.stdout:9/346: truncate d3/d11/f28 867401 0 2024-02-16T22:56:20.386 INFO:tasks.workunit.client.1.smithi192.stdout:7/537: chown d1/d2/de/d2c/f96 36869 1 2024-02-16T22:56:20.388 INFO:tasks.workunit.client.0.smithi064.stdout:9/347: chown d3/d11/c4c 322993414 1 2024-02-16T22:56:20.388 INFO:tasks.workunit.client.1.smithi192.stdout:9/570: dread d7/d19/d37/f9e [0,4194304] 0 2024-02-16T22:56:20.389 INFO:tasks.workunit.client.1.smithi192.stdout:9/571: read d7/f83 [202431,5667] 0 2024-02-16T22:56:20.390 INFO:tasks.workunit.client.0.smithi064.stdout:1/342: truncate d2/df/d14/f52 1219009 0 2024-02-16T22:56:20.390 INFO:tasks.workunit.client.0.smithi064.stdout:0/319: dwrite da/dc/de/d12/f2a [0,4194304] 0 2024-02-16T22:56:20.391 INFO:tasks.workunit.client.1.smithi192.stdout:1/442: creat d3/d40/f94 x:0 0 0 2024-02-16T22:56:20.392 INFO:tasks.workunit.client.0.smithi064.stdout:2/404: write d7/de/f2f [647175,13175] 0 2024-02-16T22:56:20.397 INFO:tasks.workunit.client.1.smithi192.stdout:1/443: dwrite d3/de/d27/d34/f38 [0,4194304] 0 2024-02-16T22:56:20.400 INFO:tasks.workunit.client.0.smithi064.stdout:3/439: write d4/d1f/f32 [507481,52111] 0 2024-02-16T22:56:20.400 INFO:tasks.workunit.client.0.smithi064.stdout:2/405: dwrite d7/d14/d3e/f75 [0,4194304] 0 2024-02-16T22:56:20.402 INFO:tasks.workunit.client.0.smithi064.stdout:3/440: dread - d4/d13/d1b/d27/d61/f66 zero size 2024-02-16T22:56:20.405 INFO:tasks.workunit.client.0.smithi064.stdout:3/441: fsync d4/d1f/d5b/d51/f6a 0 2024-02-16T22:56:20.408 INFO:tasks.workunit.client.1.smithi192.stdout:7/538: symlink d1/d2/de/d2c/lc2 0 2024-02-16T22:56:20.408 INFO:tasks.workunit.client.0.smithi064.stdout:3/442: chown d4/d13/d1b/d4f/f6f 103 1 2024-02-16T22:56:20.408 INFO:tasks.workunit.client.0.smithi064.stdout:2/406: dwrite d7/d14/f36 [0,4194304] 0 2024-02-16T22:56:20.410 INFO:tasks.workunit.client.0.smithi064.stdout:2/407: write d7/d14/f2c [6048635,10933] 0 2024-02-16T22:56:20.420 INFO:tasks.workunit.client.1.smithi192.stdout:2/505: rename d22/d24/c5d to d22/ca7 0 2024-02-16T22:56:20.420 INFO:tasks.workunit.client.1.smithi192.stdout:0/459: rmdir d1/d3/de/d22/d37/d47 39 2024-02-16T22:56:20.421 INFO:tasks.workunit.client.1.smithi192.stdout:0/460: chown d1/d3/d29/f70 113 1 2024-02-16T22:56:20.421 INFO:tasks.workunit.client.1.smithi192.stdout:0/461: chown d1/d3/d29/d43 0 1 2024-02-16T22:56:20.424 INFO:tasks.workunit.client.1.smithi192.stdout:8/446: creat d5/f84 x:0 0 0 2024-02-16T22:56:20.426 INFO:tasks.workunit.client.1.smithi192.stdout:4/531: symlink d1/d15/d29/lbb 0 2024-02-16T22:56:20.429 INFO:tasks.workunit.client.0.smithi064.stdout:6/419: truncate d0/f41 4535080 0 2024-02-16T22:56:20.431 INFO:tasks.workunit.client.1.smithi192.stdout:7/539: creat d1/d2/d37/fc3 x:0 0 0 2024-02-16T22:56:20.432 INFO:tasks.workunit.client.0.smithi064.stdout:1/343: creat d2/df/d14/d18/d48/d6b/d75/f77 x:0 0 0 2024-02-16T22:56:20.435 INFO:tasks.workunit.client.0.smithi064.stdout:2/408: creat d7/d14/d25/d37/d44/f80 x:0 0 0 2024-02-16T22:56:20.437 INFO:tasks.workunit.client.1.smithi192.stdout:9/572: link d7/d19/d5b/f89 d7/d15/d36/d38/d3f/fc3 0 2024-02-16T22:56:20.437 INFO:tasks.workunit.client.1.smithi192.stdout:9/573: chown d7/d19/d37/d5c/f87 0 1 2024-02-16T22:56:20.438 INFO:tasks.workunit.client.0.smithi064.stdout:8/339: creat df/f74 x:0 0 0 2024-02-16T22:56:20.449 INFO:tasks.workunit.client.0.smithi064.stdout:6/420: truncate d0/d6/d10/d33/f2a 3804012 0 2024-02-16T22:56:20.455 INFO:tasks.workunit.client.0.smithi064.stdout:1/344: symlink d2/df/d14/d2c/d62/l78 0 2024-02-16T22:56:20.456 INFO:tasks.workunit.client.0.smithi064.stdout:1/345: write d2/df/d14/d18/d40/f68 [134847,4375] 0 2024-02-16T22:56:20.456 INFO:tasks.workunit.client.0.smithi064.stdout:2/409: creat d7/d14/d1e/f81 x:0 0 0 2024-02-16T22:56:20.456 INFO:tasks.workunit.client.0.smithi064.stdout:8/340: creat df/d31/d2f/d4f/f75 x:0 0 0 2024-02-16T22:56:20.456 INFO:tasks.workunit.client.0.smithi064.stdout:9/348: getdents d3/d9/d1e/d42 0 2024-02-16T22:56:20.459 INFO:tasks.workunit.client.0.smithi064.stdout:6/421: unlink d0/d5/df/d17/c59 0 2024-02-16T22:56:20.465 INFO:tasks.workunit.client.0.smithi064.stdout:9/349: chown d3/fc 70358405 1 2024-02-16T22:56:20.465 INFO:tasks.workunit.client.0.smithi064.stdout:3/443: truncate d4/d13/d1b/d4f/f6b 57660 0 2024-02-16T22:56:20.466 INFO:tasks.workunit.client.0.smithi064.stdout:0/320: getdents da/dc/de/d12/d1d 0 2024-02-16T22:56:20.466 INFO:tasks.workunit.client.0.smithi064.stdout:9/350: read d3/d11/d15/d5a/f52 [233963,116869] 0 2024-02-16T22:56:20.469 INFO:tasks.workunit.client.0.smithi064.stdout:2/410: symlink d7/d14/d1e/d34/l82 0 2024-02-16T22:56:20.471 INFO:tasks.workunit.client.0.smithi064.stdout:6/422: getdents d0/d12/d7b 0 2024-02-16T22:56:20.475 INFO:tasks.workunit.client.1.smithi192.stdout:2/506: symlink d22/d24/d39/d42/d65/la8 0 2024-02-16T22:56:20.475 INFO:tasks.workunit.client.1.smithi192.stdout:1/444: symlink d3/de/d27/d34/d69/d85/d89/l95 0 2024-02-16T22:56:20.476 INFO:tasks.workunit.client.1.smithi192.stdout:3/474: symlink da/d11/la4 0 2024-02-16T22:56:20.476 INFO:tasks.workunit.client.0.smithi064.stdout:8/341: write df/f12 [668220,115263] 0 2024-02-16T22:56:20.478 INFO:tasks.workunit.client.1.smithi192.stdout:9/574: dwrite d7/d15/d36/d77/faf [0,4194304] 0 2024-02-16T22:56:20.482 INFO:tasks.workunit.client.0.smithi064.stdout:0/321: dread da/dc/de/d12/f61 [0,4194304] 0 2024-02-16T22:56:20.487 INFO:tasks.workunit.client.0.smithi064.stdout:9/351: rename d3/d11/d15/d5a/d2a/c71 to d3/d9/d1e/d42/c77 0 2024-02-16T22:56:20.496 INFO:tasks.workunit.client.0.smithi064.stdout:0/322: rmdir da/dc/de/d12/d2c/d2e/d38 39 2024-02-16T22:56:20.497 INFO:tasks.workunit.client.0.smithi064.stdout:2/411: dwrite d7/d14/d25/f4a [4194304,4194304] 0 2024-02-16T22:56:20.499 INFO:tasks.workunit.client.0.smithi064.stdout:2/412: write d7/d14/d25/d37/d44/f6a [226377,120577] 0 2024-02-16T22:56:20.500 INFO:tasks.workunit.client.0.smithi064.stdout:6/423: link d0/d12/f78 d0/d6/da/f81 0 2024-02-16T22:56:20.501 INFO:tasks.workunit.client.0.smithi064.stdout:2/413: fdatasync d7/d14/d25/d37/d44/f5c 0 2024-02-16T22:56:20.502 INFO:tasks.workunit.client.0.smithi064.stdout:2/414: write d7/de/f2f [86528,88018] 0 2024-02-16T22:56:20.506 INFO:tasks.workunit.client.1.smithi192.stdout:4/532: dread d1/d26/f36 [0,4194304] 0 2024-02-16T22:56:20.508 INFO:tasks.workunit.client.0.smithi064.stdout:0/323: rmdir da/dc/de 39 2024-02-16T22:56:20.508 INFO:tasks.workunit.client.0.smithi064.stdout:8/342: dwrite df/d31/d2f/d38/d48/f49 [0,4194304] 0 2024-02-16T22:56:20.511 INFO:tasks.workunit.client.0.smithi064.stdout:8/343: dread df/f15 [0,4194304] 0 2024-02-16T22:56:20.512 INFO:tasks.workunit.client.0.smithi064.stdout:8/344: dread df/f15 [0,4194304] 0 2024-02-16T22:56:20.516 INFO:tasks.workunit.client.1.smithi192.stdout:5/432: link d6/d12/d4e/d5b/l82 d6/d8/d80/l92 0 2024-02-16T22:56:20.520 INFO:tasks.workunit.client.0.smithi064.stdout:6/424: creat d0/d5/d54/d60/f82 x:0 0 0 2024-02-16T22:56:20.522 INFO:tasks.workunit.client.0.smithi064.stdout:9/352: mknod d3/d9/d1e/d3e/d76/c78 0 2024-02-16T22:56:20.523 INFO:tasks.workunit.client.0.smithi064.stdout:6/425: dread d0/d5/d54/d60/f6d [0,4194304] 0 2024-02-16T22:56:20.524 INFO:tasks.workunit.client.1.smithi192.stdout:8/447: link d5/d16/f4f d5/db/d47/d43/d67/f85 0 2024-02-16T22:56:20.524 INFO:tasks.workunit.client.0.smithi064.stdout:2/415: symlink d7/d14/d4b/d63/l83 0 2024-02-16T22:56:20.525 INFO:tasks.workunit.client.1.smithi192.stdout:2/507: creat d22/fa9 x:0 0 0 2024-02-16T22:56:20.527 INFO:tasks.workunit.client.1.smithi192.stdout:7/540: symlink d1/d2/d2d/d61/d8a/dbb/lc4 0 2024-02-16T22:56:20.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/541: truncate d1/d2/f1e 5123423 0 2024-02-16T22:56:20.529 INFO:tasks.workunit.client.1.smithi192.stdout:3/475: dread - da/d11/d63/d6b/d66/d75/f9f zero size 2024-02-16T22:56:20.530 INFO:tasks.workunit.client.0.smithi064.stdout:8/345: creat df/d31/d2f/d38/d48/f76 x:0 0 0 2024-02-16T22:56:20.531 INFO:tasks.workunit.client.0.smithi064.stdout:8/346: fdatasync df/d31/d2f/d38/f46 0 2024-02-16T22:56:20.531 INFO:tasks.workunit.client.1.smithi192.stdout:9/575: creat d7/d15/d36/d77/db1/fc4 x:0 0 0 2024-02-16T22:56:20.536 INFO:tasks.workunit.client.0.smithi064.stdout:6/426: mknod d0/d5/d54/d60/c83 0 2024-02-16T22:56:20.538 INFO:tasks.workunit.client.1.smithi192.stdout:4/533: dwrite d1/d26/d66/d89/f3a [0,4194304] 0 2024-02-16T22:56:20.538 INFO:tasks.workunit.client.1.smithi192.stdout:1/445: truncate d3/f2d 2040622 0 2024-02-16T22:56:20.539 INFO:tasks.workunit.client.1.smithi192.stdout:4/534: dread - d1/d26/d66/fac zero size 2024-02-16T22:56:20.540 INFO:tasks.workunit.client.0.smithi064.stdout:0/324: unlink da/dc/de/d12/d2c/l53 0 2024-02-16T22:56:20.543 INFO:tasks.workunit.client.1.smithi192.stdout:7/542: dread d1/d2/f58 [0,4194304] 0 2024-02-16T22:56:20.543 INFO:tasks.workunit.client.0.smithi064.stdout:8/347: creat df/d31/d2f/d3e/f77 x:0 0 0 2024-02-16T22:56:20.550 INFO:tasks.workunit.client.1.smithi192.stdout:5/433: chown d6/d12/d4e/d46/c87 1217101 1 2024-02-16T22:56:20.551 INFO:tasks.workunit.client.1.smithi192.stdout:5/434: readlink d6/d8/d75/l7c 0 2024-02-16T22:56:20.551 INFO:tasks.workunit.client.1.smithi192.stdout:5/435: stat d6/d12/l76 0 2024-02-16T22:56:20.552 INFO:tasks.workunit.client.1.smithi192.stdout:6/457: sync 2024-02-16T22:56:20.552 INFO:tasks.workunit.client.0.smithi064.stdout:6/427: rmdir d0/d5/d54 39 2024-02-16T22:56:20.553 INFO:tasks.workunit.client.1.smithi192.stdout:8/448: rename d5/d3e to d5/d26/d83/d86 0 2024-02-16T22:56:20.554 INFO:tasks.workunit.client.0.smithi064.stdout:2/416: mknod d7/d14/d1e/d34/c84 0 2024-02-16T22:56:20.556 INFO:tasks.workunit.client.1.smithi192.stdout:4/535: rmdir d1/d15/d48/d8d 39 2024-02-16T22:56:20.556 INFO:tasks.workunit.client.1.smithi192.stdout:2/508: dwrite d22/d24/d39/d42/d65/d69/d4f/f97 [0,4194304] 0 2024-02-16T22:56:20.557 INFO:tasks.workunit.client.0.smithi064.stdout:8/348: symlink df/d31/d2f/d38/d48/l78 0 2024-02-16T22:56:20.557 INFO:tasks.workunit.client.0.smithi064.stdout:8/349: readlink df/d31/d2f/d38/d48/l78 0 2024-02-16T22:56:20.558 INFO:tasks.workunit.client.0.smithi064.stdout:8/350: write df/f12 [1372894,69165] 0 2024-02-16T22:56:20.563 INFO:tasks.workunit.client.1.smithi192.stdout:8/449: dwrite d5/d16/f7d [0,4194304] 0 2024-02-16T22:56:20.566 INFO:tasks.workunit.client.1.smithi192.stdout:4/536: dwrite d1/d26/d66/d89/d11/fb7 [0,4194304] 0 2024-02-16T22:56:20.606 INFO:tasks.workunit.client.1.smithi192.stdout:7/543: unlink d1/d69/c72 0 2024-02-16T22:56:20.607 INFO:tasks.workunit.client.0.smithi064.stdout:0/325: creat da/dc/de/d12/d2c/d2e/d38/f67 x:0 0 0 2024-02-16T22:56:20.607 INFO:tasks.workunit.client.1.smithi192.stdout:9/576: mknod d7/d15/d36/d77/dbd/cc5 0 2024-02-16T22:56:20.608 INFO:tasks.workunit.client.0.smithi064.stdout:9/353: truncate d3/d11/d15/d5a/f56 3456779 0 2024-02-16T22:56:20.608 INFO:tasks.workunit.client.1.smithi192.stdout:9/577: chown d7/d15/d36/d38/d3f/f59 6375912 1 2024-02-16T22:56:20.628 INFO:tasks.workunit.client.1.smithi192.stdout:0/462: sync 2024-02-16T22:56:20.659 INFO:tasks.workunit.client.0.smithi064.stdout:7/299: sync 2024-02-16T22:56:20.661 INFO:tasks.workunit.client.0.smithi064.stdout:8/351: rmdir df/d2a 39 2024-02-16T22:56:20.662 INFO:tasks.workunit.client.1.smithi192.stdout:5/436: mkdir d6/d20/d93 0 2024-02-16T22:56:20.663 INFO:tasks.workunit.client.0.smithi064.stdout:2/417: unlink d7/d14/f33 0 2024-02-16T22:56:20.664 INFO:tasks.workunit.client.1.smithi192.stdout:6/458: chown da/df/d6b/d74 130535 1 2024-02-16T22:56:20.668 INFO:tasks.workunit.client.1.smithi192.stdout:3/476: rename da/f14 to da/d11/d63/d6b/d4e/fa5 0 2024-02-16T22:56:20.669 INFO:tasks.workunit.client.1.smithi192.stdout:6/459: dwrite da/df/d12/d57/f50 [0,4194304] 0 2024-02-16T22:56:20.669 INFO:tasks.workunit.client.0.smithi064.stdout:8/352: mkdir df/d31/d61/d79 0 2024-02-16T22:56:20.670 INFO:tasks.workunit.client.0.smithi064.stdout:2/418: dwrite d7/d14/d3e/f5d [0,4194304] 0 2024-02-16T22:56:20.670 INFO:tasks.workunit.client.1.smithi192.stdout:6/460: fsync da/df/d58/f91 0 2024-02-16T22:56:20.672 INFO:tasks.workunit.client.0.smithi064.stdout:2/419: truncate d7/d14/d1e/f81 562489 0 2024-02-16T22:56:20.673 INFO:tasks.workunit.client.0.smithi064.stdout:2/420: write d7/de/f11 [2049021,104760] 0 2024-02-16T22:56:20.674 INFO:tasks.workunit.client.0.smithi064.stdout:0/326: symlink da/d26/l68 0 2024-02-16T22:56:20.674 INFO:tasks.workunit.client.0.smithi064.stdout:0/327: read - da/dc/de/d12/f42 zero size 2024-02-16T22:56:20.686 INFO:tasks.workunit.client.1.smithi192.stdout:4/537: chown d1/d15/d48/d8d/f9a 9077 1 2024-02-16T22:56:20.687 INFO:tasks.workunit.client.0.smithi064.stdout:2/421: unlink d7/d14/l5b 0 2024-02-16T22:56:20.688 INFO:tasks.workunit.client.1.smithi192.stdout:8/450: read d5/db/de/df/f2b [2908184,100543] 0 2024-02-16T22:56:20.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/578: symlink d7/d15/d36/d77/lc6 0 2024-02-16T22:56:20.690 INFO:tasks.workunit.client.1.smithi192.stdout:8/451: chown d5/d26/d83/d86/d5f/f4e 571403648 1 2024-02-16T22:56:20.690 INFO:tasks.workunit.client.1.smithi192.stdout:8/452: chown d5 50448 1 2024-02-16T22:56:20.691 INFO:tasks.workunit.client.1.smithi192.stdout:0/463: symlink d1/d3/de/d22/d37/d67/d34/l94 0 2024-02-16T22:56:20.691 INFO:tasks.workunit.client.1.smithi192.stdout:4/538: dwrite d1/d15/d48/d8d/d9b/fb0 [0,4194304] 0 2024-02-16T22:56:20.693 INFO:tasks.workunit.client.0.smithi064.stdout:2/422: mkdir d7/d14/d1e/d85 0 2024-02-16T22:56:20.695 INFO:tasks.workunit.client.0.smithi064.stdout:0/328: mknod da/d26/c69 0 2024-02-16T22:56:20.695 INFO:tasks.workunit.client.0.smithi064.stdout:2/423: write d7/d14/d1e/d34/d3f/f79 [139837,7572] 0 2024-02-16T22:56:20.696 INFO:tasks.workunit.client.1.smithi192.stdout:6/461: symlink da/d40/l93 0 2024-02-16T22:56:20.697 INFO:tasks.workunit.client.0.smithi064.stdout:9/354: getdents d3/d11/d15/d5a/d2a 0 2024-02-16T22:56:20.698 INFO:tasks.workunit.client.1.smithi192.stdout:4/539: dread d1/d26/d66/d89/f64 [0,4194304] 0 2024-02-16T22:56:20.698 INFO:tasks.workunit.client.0.smithi064.stdout:7/300: write d5/d6/f1f [947171,60543] 0 2024-02-16T22:56:20.701 INFO:tasks.workunit.client.1.smithi192.stdout:2/509: symlink d22/d24/d39/d42/d52/laa 0 2024-02-16T22:56:20.701 INFO:tasks.workunit.client.0.smithi064.stdout:2/424: dread d7/d14/d3e/f5d [0,4194304] 0 2024-02-16T22:56:20.702 INFO:tasks.workunit.client.0.smithi064.stdout:2/425: chown d7/d14/d4b/d63 1 1 2024-02-16T22:56:20.704 INFO:tasks.workunit.client.0.smithi064.stdout:2/426: write d7/d14/d25/d46/d55/f7d [277506,22585] 0 2024-02-16T22:56:20.705 INFO:tasks.workunit.client.1.smithi192.stdout:7/544: write d1/d2/d2d/d61/d88/d90/f1d [9146563,114405] 0 2024-02-16T22:56:20.707 INFO:tasks.workunit.client.1.smithi192.stdout:7/545: fdatasync d1/d2/d2d/d61/d88/d90/d2a/f87 0 2024-02-16T22:56:20.707 INFO:tasks.workunit.client.1.smithi192.stdout:7/546: chown d1/d69 2432 1 2024-02-16T22:56:20.708 INFO:tasks.workunit.client.0.smithi064.stdout:7/301: write d5/d6/f22 [4011005,57351] 0 2024-02-16T22:56:20.712 INFO:tasks.workunit.client.1.smithi192.stdout:8/453: truncate d5/db/d47/f78 317918 0 2024-02-16T22:56:20.712 INFO:tasks.workunit.client.0.smithi064.stdout:7/302: dread d5/d6/f22 [0,4194304] 0 2024-02-16T22:56:20.720 INFO:tasks.workunit.client.0.smithi064.stdout:9/355: dwrite d3/d11/d15/d5a/f26 [0,4194304] 0 2024-02-16T22:56:20.720 INFO:tasks.workunit.client.1.smithi192.stdout:6/462: creat da/d40/d85/f94 x:0 0 0 2024-02-16T22:56:20.722 INFO:tasks.workunit.client.0.smithi064.stdout:7/303: creat d5/d6/d54/f63 x:0 0 0 2024-02-16T22:56:20.724 INFO:tasks.workunit.client.0.smithi064.stdout:7/304: truncate d5/d6/d54/f3f 4966981 0 2024-02-16T22:56:20.725 INFO:tasks.workunit.client.1.smithi192.stdout:4/540: creat d1/d26/d6f/fbc x:0 0 0 2024-02-16T22:56:20.726 INFO:tasks.workunit.client.0.smithi064.stdout:0/329: getdents da/d26 0 2024-02-16T22:56:20.727 INFO:tasks.workunit.client.1.smithi192.stdout:4/541: readlink d1/d26/d66/d89/d11/l33 0 2024-02-16T22:56:20.727 INFO:tasks.workunit.client.1.smithi192.stdout:6/463: dwrite da/df/d6b/f88 [0,4194304] 0 2024-02-16T22:56:20.727 INFO:tasks.workunit.client.0.smithi064.stdout:2/427: dwrite d7/d14/d1e/d34/d3f/f43 [0,4194304] 0 2024-02-16T22:56:20.728 INFO:tasks.workunit.client.1.smithi192.stdout:6/464: fsync da/f34 0 2024-02-16T22:56:20.728 INFO:tasks.workunit.client.0.smithi064.stdout:2/428: readlink d7/d14/d1e/l2b 0 2024-02-16T22:56:20.728 INFO:tasks.workunit.client.0.smithi064.stdout:0/330: dread - da/dc/d3b/f5f zero size 2024-02-16T22:56:20.729 INFO:tasks.workunit.client.0.smithi064.stdout:2/429: dread - d7/d14/d1e/d34/f62 zero size 2024-02-16T22:56:20.729 INFO:tasks.workunit.client.0.smithi064.stdout:0/331: chown da/dc/de/d12/d1d/f3d 9 1 2024-02-16T22:56:20.731 INFO:tasks.workunit.client.0.smithi064.stdout:2/430: fdatasync d7/d14/d3e/f75 0 2024-02-16T22:56:20.734 INFO:tasks.workunit.client.0.smithi064.stdout:7/305: rename d5/d6/d49 to d5/d33/d45/d64 0 2024-02-16T22:56:20.737 INFO:tasks.workunit.client.0.smithi064.stdout:2/431: dread d7/d14/d25/f4a [4194304,4194304] 0 2024-02-16T22:56:20.737 INFO:tasks.workunit.client.0.smithi064.stdout:7/306: truncate d5/d6/d54/d41/f4b 73482 0 2024-02-16T22:56:20.738 INFO:tasks.workunit.client.0.smithi064.stdout:0/332: write da/dc/de/d12/f61 [5206575,56615] 0 2024-02-16T22:56:20.738 INFO:tasks.workunit.client.1.smithi192.stdout:8/454: symlink d5/d26/d83/l87 0 2024-02-16T22:56:20.739 INFO:tasks.workunit.client.0.smithi064.stdout:2/432: fdatasync d7/d14/d25/d37/d44/f5c 0 2024-02-16T22:56:20.742 INFO:tasks.workunit.client.1.smithi192.stdout:1/446: getdents d3/de/d27/d34/d52 0 2024-02-16T22:56:20.743 INFO:tasks.workunit.client.1.smithi192.stdout:1/447: chown d3/de/d27/f4a 116400482 1 2024-02-16T22:56:20.746 INFO:tasks.workunit.client.0.smithi064.stdout:7/307: dwrite d5/d6/d54/f63 [0,4194304] 0 2024-02-16T22:56:20.748 INFO:tasks.workunit.client.1.smithi192.stdout:2/510: mknod d22/d24/d39/d42/d52/d7c/cab 0 2024-02-16T22:56:20.753 INFO:tasks.workunit.client.0.smithi064.stdout:2/433: rmdir d7/d14/d1e/d85 0 2024-02-16T22:56:20.754 INFO:tasks.workunit.client.1.smithi192.stdout:9/579: creat d7/d19/d28/fc7 x:0 0 0 2024-02-16T22:56:20.755 INFO:tasks.workunit.client.0.smithi064.stdout:7/308: mkdir d5/d6/d65 0 2024-02-16T22:56:20.756 INFO:tasks.workunit.client.1.smithi192.stdout:8/455: write d5/db/d47/f3b [9328012,39153] 0 2024-02-16T22:56:20.758 INFO:tasks.workunit.client.1.smithi192.stdout:5/437: rename d6/d12/c26 to d6/c94 0 2024-02-16T22:56:20.759 INFO:tasks.workunit.client.1.smithi192.stdout:3/477: getdents da/d11/d63 0 2024-02-16T22:56:20.760 INFO:tasks.workunit.client.1.smithi192.stdout:1/448: symlink d3/de/d27/l96 0 2024-02-16T22:56:20.762 INFO:tasks.workunit.client.1.smithi192.stdout:9/580: fsync d7/d19/d28/f8b 0 2024-02-16T22:56:20.763 INFO:tasks.workunit.client.1.smithi192.stdout:8/456: creat d5/d26/f88 x:0 0 0 2024-02-16T22:56:20.764 INFO:tasks.workunit.client.0.smithi064.stdout:2/434: getdents d7/d14 0 2024-02-16T22:56:20.765 INFO:tasks.workunit.client.1.smithi192.stdout:5/438: dwrite d6/d12/d1d/f3a [0,4194304] 0 2024-02-16T22:56:20.765 INFO:tasks.workunit.client.0.smithi064.stdout:9/356: dread d3/d11/d15/d5a/f56 [0,4194304] 0 2024-02-16T22:56:20.766 INFO:tasks.workunit.client.0.smithi064.stdout:2/435: write d7/d14/d1e/d34/d3f/f79 [79379,22273] 0 2024-02-16T22:56:20.766 INFO:tasks.workunit.client.1.smithi192.stdout:3/478: creat da/d11/d63/fa6 x:0 0 0 2024-02-16T22:56:20.767 INFO:tasks.workunit.client.1.smithi192.stdout:4/542: creat d1/d15/d48/fbd x:0 0 0 2024-02-16T22:56:20.769 INFO:tasks.workunit.client.0.smithi064.stdout:9/357: unlink d3/d9/d1e/d3e/f49 0 2024-02-16T22:56:20.771 INFO:tasks.workunit.client.1.smithi192.stdout:8/457: dread d5/d16/f7d [0,4194304] 0 2024-02-16T22:56:20.771 INFO:tasks.workunit.client.0.smithi064.stdout:2/436: fdatasync d7/d14/d1e/f21 0 2024-02-16T22:56:20.773 INFO:tasks.workunit.client.1.smithi192.stdout:9/581: unlink d7/d19/d28/fc7 0 2024-02-16T22:56:20.777 INFO:tasks.workunit.client.1.smithi192.stdout:8/458: dwrite d5/d9/f69 [0,4194304] 0 2024-02-16T22:56:20.778 INFO:tasks.workunit.client.0.smithi064.stdout:9/358: creat d3/d9/d1e/d3e/d76/f79 x:0 0 0 2024-02-16T22:56:20.779 INFO:tasks.workunit.client.0.smithi064.stdout:5/284: sync 2024-02-16T22:56:20.779 INFO:tasks.workunit.client.0.smithi064.stdout:9/359: read - d3/d11/d15/d23/f6e zero size 2024-02-16T22:56:20.780 INFO:tasks.workunit.client.1.smithi192.stdout:0/464: rename d1/c11 to d1/d3/de/d22/d37/d67/c95 0 2024-02-16T22:56:20.783 INFO:tasks.workunit.client.1.smithi192.stdout:5/439: unlink d6/d12/d1d/f36 0 2024-02-16T22:56:20.787 INFO:tasks.workunit.client.0.smithi064.stdout:7/309: dread d5/d6/f3b [0,4194304] 0 2024-02-16T22:56:20.787 INFO:tasks.workunit.client.0.smithi064.stdout:5/285: rename d6/d18/d19/d1a/c46 to d6/d25/c63 0 2024-02-16T22:56:20.789 INFO:tasks.workunit.client.0.smithi064.stdout:9/360: chown d3/d11/d15/d5a/d38/f66 9 1 2024-02-16T22:56:20.792 INFO:tasks.workunit.client.1.smithi192.stdout:9/582: creat d7/d15/d36/d38/d3f/fc8 x:0 0 0 2024-02-16T22:56:20.793 INFO:tasks.workunit.client.0.smithi064.stdout:7/310: mkdir d5/d33/d45/d64/d56/d66 0 2024-02-16T22:56:20.794 INFO:tasks.workunit.client.0.smithi064.stdout:7/311: write d5/d33/f57 [11358,119559] 0 2024-02-16T22:56:20.807 INFO:tasks.workunit.client.1.smithi192.stdout:9/583: dread d7/d19/d28/d43/f57 [0,4194304] 0 2024-02-16T22:56:20.834 INFO:tasks.workunit.client.1.smithi192.stdout:2/511: sync 2024-02-16T22:56:20.834 INFO:tasks.workunit.client.1.smithi192.stdout:2/512: stat d22/d24/d39/d42/d65/d69/f68 0 2024-02-16T22:56:20.865 INFO:tasks.workunit.client.0.smithi064.stdout:2/437: creat d7/f86 x:0 0 0 2024-02-16T22:56:20.866 INFO:tasks.workunit.client.1.smithi192.stdout:7/547: truncate d1/d2/de/d2c/d49/da4/f85 3117333 0 2024-02-16T22:56:20.866 INFO:tasks.workunit.client.1.smithi192.stdout:7/548: readlink d1/lae 0 2024-02-16T22:56:20.869 INFO:tasks.workunit.client.1.smithi192.stdout:8/459: symlink d5/d26/d83/d86/d5f/l89 0 2024-02-16T22:56:20.870 INFO:tasks.workunit.client.0.smithi064.stdout:2/438: dwrite d7/de/f11 [0,4194304] 0 2024-02-16T22:56:20.872 INFO:tasks.workunit.client.1.smithi192.stdout:6/465: rename da/df/d12/d15/d1b/c2b to da/d40/c95 0 2024-02-16T22:56:20.873 INFO:tasks.workunit.client.1.smithi192.stdout:3/479: rename da to da/d11/d63/d6b/d4e/da7 22 2024-02-16T22:56:20.874 INFO:tasks.workunit.client.1.smithi192.stdout:0/465: mknod d1/d3/de/d22/c96 0 2024-02-16T22:56:20.877 INFO:tasks.workunit.client.1.smithi192.stdout:5/440: dread - d6/d12/d4e/d46/d4f/f62 zero size 2024-02-16T22:56:20.879 INFO:tasks.workunit.client.1.smithi192.stdout:5/441: write d6/d12/f1e [7015998,77962] 0 2024-02-16T22:56:20.880 INFO:tasks.workunit.client.1.smithi192.stdout:1/449: creat d3/f97 x:0 0 0 2024-02-16T22:56:20.881 INFO:tasks.workunit.client.1.smithi192.stdout:1/450: chown d3/de/d27/d34/d52/d63/l92 455640 1 2024-02-16T22:56:20.882 INFO:tasks.workunit.client.1.smithi192.stdout:4/543: mkdir d1/d26/dbe 0 2024-02-16T22:56:20.882 INFO:tasks.workunit.client.1.smithi192.stdout:4/544: fdatasync d1/d15/d23/f52 0 2024-02-16T22:56:20.884 INFO:tasks.workunit.client.1.smithi192.stdout:4/545: write d1/d26/d66/d89/f3a [992820,22759] 0 2024-02-16T22:56:20.884 INFO:tasks.workunit.client.1.smithi192.stdout:9/584: creat d7/d19/d28/d43/fc9 x:0 0 0 2024-02-16T22:56:20.885 INFO:tasks.workunit.client.1.smithi192.stdout:4/546: read d1/d26/d66/d89/f68 [376092,93809] 0 2024-02-16T22:56:20.886 INFO:tasks.workunit.client.0.smithi064.stdout:2/439: link d7/de/f2f d7/d14/d1e/d34/f87 0 2024-02-16T22:56:20.887 INFO:tasks.workunit.client.0.smithi064.stdout:2/440: fdatasync d7/d14/d1e/d34/d65/f67 0 2024-02-16T22:56:20.888 INFO:tasks.workunit.client.1.smithi192.stdout:7/549: mknod d1/d2/d37/cc5 0 2024-02-16T22:56:20.890 INFO:tasks.workunit.client.0.smithi064.stdout:2/441: dread d7/d14/f36 [0,4194304] 0 2024-02-16T22:56:20.894 INFO:tasks.workunit.client.1.smithi192.stdout:3/480: dread - da/d11/d35/f70 zero size 2024-02-16T22:56:20.902 INFO:tasks.workunit.client.0.smithi064.stdout:2/442: link d7/d14/d1e/d34/f50 d7/d14/d4b/f88 0 2024-02-16T22:56:20.903 INFO:tasks.workunit.client.0.smithi064.stdout:2/443: write d7/de/f11 [3764206,52547] 0 2024-02-16T22:56:20.904 INFO:tasks.workunit.client.0.smithi064.stdout:2/444: write d7/de/f11 [4880883,83288] 0 2024-02-16T22:56:20.906 INFO:tasks.workunit.client.1.smithi192.stdout:6/466: rmdir da/d40 39 2024-02-16T22:56:20.907 INFO:tasks.workunit.client.0.smithi064.stdout:2/445: truncate d7/d14/d25/d46/d55/f5a 5013735 0 2024-02-16T22:56:20.908 INFO:tasks.workunit.client.0.smithi064.stdout:2/446: write d7/d14/f2e [3287167,86097] 0 2024-02-16T22:56:20.908 INFO:tasks.workunit.client.0.smithi064.stdout:7/312: truncate d5/d6/d54/d41/f50 1825982 0 2024-02-16T22:56:20.909 INFO:tasks.workunit.client.1.smithi192.stdout:0/466: rmdir d1/d3/de/d22 39 2024-02-16T22:56:20.912 INFO:tasks.workunit.client.1.smithi192.stdout:9/585: mknod d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/cca 0 2024-02-16T22:56:20.913 INFO:tasks.workunit.client.0.smithi064.stdout:4/373: sync 2024-02-16T22:56:20.913 INFO:tasks.workunit.client.0.smithi064.stdout:1/346: sync 2024-02-16T22:56:20.914 INFO:tasks.workunit.client.1.smithi192.stdout:4/547: truncate d1/d26/f36 1299581 0 2024-02-16T22:56:20.915 INFO:tasks.workunit.client.0.smithi064.stdout:1/347: fdatasync d2/df/d14/f2d 0 2024-02-16T22:56:20.915 INFO:tasks.workunit.client.0.smithi064.stdout:4/374: chown dc/de/l2a 293087 1 2024-02-16T22:56:20.915 INFO:tasks.workunit.client.0.smithi064.stdout:7/313: creat d5/d33/f67 x:0 0 0 2024-02-16T22:56:20.915 INFO:tasks.workunit.client.0.smithi064.stdout:4/375: stat dc/de/d28/l50 0 2024-02-16T22:56:20.916 INFO:tasks.workunit.client.0.smithi064.stdout:4/376: dread - dc/dd/d32/d46/d5a/f66 zero size 2024-02-16T22:56:20.916 INFO:tasks.workunit.client.1.smithi192.stdout:2/513: mkdir d22/d24/d2b/d2c/d3f/dac 0 2024-02-16T22:56:20.918 INFO:tasks.workunit.client.0.smithi064.stdout:2/447: symlink d7/d14/d1e/d34/d3f/l89 0 2024-02-16T22:56:20.919 INFO:tasks.workunit.client.1.smithi192.stdout:7/550: symlink d1/d2/d37/d86/lc6 0 2024-02-16T22:56:20.920 INFO:tasks.workunit.client.1.smithi192.stdout:8/460: symlink d5/db/de/l8a 0 2024-02-16T22:56:20.920 INFO:tasks.workunit.client.1.smithi192.stdout:8/461: fdatasync d5/d9/f5a 0 2024-02-16T22:56:20.921 INFO:tasks.workunit.client.0.smithi064.stdout:7/314: dwrite d5/d33/f3d [4194304,4194304] 0 2024-02-16T22:56:20.971 INFO:tasks.workunit.client.0.smithi064.stdout:4/377: mknod dc/dd/d32/d22/c71 0 2024-02-16T22:56:20.973 INFO:tasks.workunit.client.0.smithi064.stdout:2/448: creat d7/d14/d25/f8a x:0 0 0 2024-02-16T22:56:20.977 INFO:tasks.workunit.client.1.smithi192.stdout:3/481: chown da/d11/d63/d6b/d66/f86 8406964 1 2024-02-16T22:56:20.978 INFO:tasks.workunit.client.0.smithi064.stdout:7/315: readlink d5/d6/d54/l51 0 2024-02-16T22:56:20.978 INFO:tasks.workunit.client.0.smithi064.stdout:7/316: stat d5/d33/d53 0 2024-02-16T22:56:20.982 INFO:tasks.workunit.client.1.smithi192.stdout:1/451: symlink d3/de/d27/d34/d52/d72/l98 0 2024-02-16T22:56:20.983 INFO:tasks.workunit.client.1.smithi192.stdout:6/467: mknod da/df/d12/d15/d1b/c96 0 2024-02-16T22:56:20.990 INFO:tasks.workunit.client.1.smithi192.stdout:0/467: read - d1/d3/de/d22/f85 zero size 2024-02-16T22:56:20.991 INFO:tasks.workunit.client.1.smithi192.stdout:0/468: chown d1 2 1 2024-02-16T22:56:20.992 INFO:tasks.workunit.client.1.smithi192.stdout:9/586: mknod d7/d19/ccb 0 2024-02-16T22:56:20.994 INFO:tasks.workunit.client.1.smithi192.stdout:4/548: unlink d1/f35 0 2024-02-16T22:56:20.997 INFO:tasks.workunit.client.1.smithi192.stdout:7/551: creat d1/d2/de/fc7 x:0 0 0 2024-02-16T22:56:21.000 INFO:tasks.workunit.client.1.smithi192.stdout:8/462: creat d5/db/d47/d43/d67/f8b x:0 0 0 2024-02-16T22:56:21.002 INFO:tasks.workunit.client.1.smithi192.stdout:3/482: rmdir da/d11/d63/d6b/d80 39 2024-02-16T22:56:21.006 INFO:tasks.workunit.client.1.smithi192.stdout:5/442: rename d6/d12/d2d/c37 to d6/d12/d4e/c95 0 2024-02-16T22:56:21.007 INFO:tasks.workunit.client.1.smithi192.stdout:1/452: mknod d3/d5b/c99 0 2024-02-16T22:56:21.008 INFO:tasks.workunit.client.0.smithi064.stdout:7/317: creat d5/d6/d54/d41/f68 x:0 0 0 2024-02-16T22:56:21.009 INFO:tasks.workunit.client.1.smithi192.stdout:6/468: creat da/df/d12/d15/d1b/f97 x:0 0 0 2024-02-16T22:56:21.010 INFO:tasks.workunit.client.1.smithi192.stdout:6/469: chown da/df/d58/c5b 419 1 2024-02-16T22:56:21.012 INFO:tasks.workunit.client.1.smithi192.stdout:0/469: unlink d1/d3/de/f4e 0 2024-02-16T22:56:21.013 INFO:tasks.workunit.client.1.smithi192.stdout:0/470: fdatasync d1/d3/d29/f8d 0 2024-02-16T22:56:21.014 INFO:tasks.workunit.client.1.smithi192.stdout:0/471: write d1/d3/de/d22/d37/d67/d75/f8e [213708,74740] 0 2024-02-16T22:56:21.015 INFO:tasks.workunit.client.1.smithi192.stdout:0/472: fdatasync d1/d3/d18/f6f 0 2024-02-16T22:56:21.019 INFO:tasks.workunit.client.0.smithi064.stdout:7/318: symlink d5/d6/l69 0 2024-02-16T22:56:21.020 INFO:tasks.workunit.client.0.smithi064.stdout:2/449: getdents d7/d14/d25 0 2024-02-16T22:56:21.022 INFO:tasks.workunit.client.1.smithi192.stdout:7/552: creat d1/d2/de/d2c/d49/da1/fc8 x:0 0 0 2024-02-16T22:56:21.022 INFO:tasks.workunit.client.1.smithi192.stdout:1/453: dread d3/de/d27/f3b [4194304,4194304] 0 2024-02-16T22:56:21.023 INFO:tasks.workunit.client.1.smithi192.stdout:7/553: chown d1/d2/de/d2c/d49/f71 28721 1 2024-02-16T22:56:21.024 INFO:tasks.workunit.client.0.smithi064.stdout:3/444: sync 2024-02-16T22:56:21.024 INFO:tasks.workunit.client.1.smithi192.stdout:7/554: chown d1/d2/de/d2c/f96 135 1 2024-02-16T22:56:21.027 INFO:tasks.workunit.client.0.smithi064.stdout:3/445: write d4/d13/d1b/d27/d44/f4b [1044713,122372] 0 2024-02-16T22:56:21.027 INFO:tasks.workunit.client.1.smithi192.stdout:5/443: write d6/fd [3810685,116693] 0 2024-02-16T22:56:21.028 INFO:tasks.workunit.client.0.smithi064.stdout:2/450: dwrite d7/d14/d25/d46/d55/f71 [0,4194304] 0 2024-02-16T22:56:21.031 INFO:tasks.workunit.client.1.smithi192.stdout:6/470: symlink da/df/d12/d15/d1b/d70/l98 0 2024-02-16T22:56:21.031 INFO:tasks.workunit.client.0.smithi064.stdout:7/319: dread d5/d6/fa [0,4194304] 0 2024-02-16T22:56:21.032 INFO:tasks.workunit.client.1.smithi192.stdout:1/454: dread d3/fb [4194304,4194304] 0 2024-02-16T22:56:21.033 INFO:tasks.workunit.client.1.smithi192.stdout:6/471: read da/df/d18/d52/f55 [405472,98570] 0 2024-02-16T22:56:21.034 INFO:tasks.workunit.client.1.smithi192.stdout:6/472: stat da/df/d12/d15/f31 0 2024-02-16T22:56:21.036 INFO:tasks.workunit.client.1.smithi192.stdout:9/587: link d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/fa1 d7/d19/d5e/d81/fcc 0 2024-02-16T22:56:21.037 INFO:tasks.workunit.client.1.smithi192.stdout:9/588: chown d7/d15/d36/d38/d3f/d8a/d98/l76 24915 1 2024-02-16T22:56:21.039 INFO:tasks.workunit.client.1.smithi192.stdout:4/549: mknod d1/d26/dbe/cbf 0 2024-02-16T22:56:21.040 INFO:tasks.workunit.client.0.smithi064.stdout:7/320: creat d5/d6/d54/f6a x:0 0 0 2024-02-16T22:56:21.043 INFO:tasks.workunit.client.1.smithi192.stdout:9/589: dwrite d7/d15/d36/d77/faf [0,4194304] 0 2024-02-16T22:56:21.044 INFO:tasks.workunit.client.1.smithi192.stdout:8/463: fdatasync d5/d9/f17 0 2024-02-16T22:56:21.045 INFO:tasks.workunit.client.1.smithi192.stdout:9/590: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/fa1 2 1 2024-02-16T22:56:21.045 INFO:tasks.workunit.client.1.smithi192.stdout:8/464: stat d5/d16 0 2024-02-16T22:56:21.047 INFO:tasks.workunit.client.1.smithi192.stdout:3/483: write da/d11/f96 [3674390,7234] 0 2024-02-16T22:56:21.051 INFO:tasks.workunit.client.1.smithi192.stdout:5/444: creat d6/d12/d2d/f96 x:0 0 0 2024-02-16T22:56:21.052 INFO:tasks.workunit.client.1.smithi192.stdout:5/445: write d6/d12/d4e/d46/f6b [919549,18646] 0 2024-02-16T22:56:21.053 INFO:tasks.workunit.client.0.smithi064.stdout:7/321: symlink d5/d33/d45/d64/d56/d66/l6b 0 2024-02-16T22:56:21.053 INFO:tasks.workunit.client.0.smithi064.stdout:7/322: read d5/d33/f57 [92256,46458] 0 2024-02-16T22:56:21.056 INFO:tasks.workunit.client.1.smithi192.stdout:1/455: rmdir d3/de 39 2024-02-16T22:56:21.059 INFO:tasks.workunit.client.0.smithi064.stdout:7/323: dwrite d5/d6/f1d [0,4194304] 0 2024-02-16T22:56:21.069 INFO:tasks.workunit.client.0.smithi064.stdout:7/324: mknod d5/d33/d42/c6c 0 2024-02-16T22:56:21.069 INFO:tasks.workunit.client.0.smithi064.stdout:2/451: dwrite d7/de/f15 [0,4194304] 0 2024-02-16T22:56:21.070 INFO:tasks.workunit.client.0.smithi064.stdout:3/446: truncate d4/d14/f46 3574884 0 2024-02-16T22:56:21.070 INFO:tasks.workunit.client.1.smithi192.stdout:2/514: getdents d22/d24/d2b/d67 0 2024-02-16T22:56:21.070 INFO:tasks.workunit.client.1.smithi192.stdout:9/591: rmdir d7/d19/d28/d43/d70/d8e 39 2024-02-16T22:56:21.071 INFO:tasks.workunit.client.1.smithi192.stdout:8/465: mkdir d5/db/d47/d8c 0 2024-02-16T22:56:21.076 INFO:tasks.workunit.client.0.smithi064.stdout:3/447: rename d4/d14/d2b/d72 to d4/d13/d1b/d27/d99 0 2024-02-16T22:56:21.078 INFO:tasks.workunit.client.0.smithi064.stdout:2/452: symlink d7/d14/d1e/d34/l8b 0 2024-02-16T22:56:21.078 INFO:tasks.workunit.client.1.smithi192.stdout:0/473: dwrite d1/d3/de/d22/d37/d67/d34/f6e [4194304,4194304] 0 2024-02-16T22:56:21.085 INFO:tasks.workunit.client.1.smithi192.stdout:1/456: readlink d3/de/d27/d34/d36/l5f 0 2024-02-16T22:56:21.085 INFO:tasks.workunit.client.0.smithi064.stdout:2/453: dwrite d7/de/f24 [0,4194304] 0 2024-02-16T22:56:21.086 INFO:tasks.workunit.client.0.smithi064.stdout:3/448: mknod d4/d13/c9a 0 2024-02-16T22:56:21.087 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:21 smithi192 ceph-mon[36254]: pgmap v15: 65 pgs: 65 active+clean; 1.5 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 220 MiB/s wr, 332 op/s 2024-02-16T22:56:21.087 INFO:tasks.workunit.client.1.smithi192.stdout:6/473: symlink da/df/d6b/l99 0 2024-02-16T22:56:21.089 INFO:tasks.workunit.client.1.smithi192.stdout:4/550: symlink d1/d26/d66/d89/da8/da9/lc0 0 2024-02-16T22:56:21.090 INFO:tasks.workunit.client.0.smithi064.stdout:2/454: dread d7/d14/f2c [0,4194304] 0 2024-02-16T22:56:21.090 INFO:tasks.workunit.client.1.smithi192.stdout:4/551: write d1/d15/f45 [3294842,106021] 0 2024-02-16T22:56:21.091 INFO:tasks.workunit.client.0.smithi064.stdout:7/325: dread d5/d6/d54/d41/f50 [0,4194304] 0 2024-02-16T22:56:21.099 INFO:tasks.workunit.client.0.smithi064.stdout:7/326: dwrite d5/d33/f3d [4194304,4194304] 0 2024-02-16T22:56:21.103 INFO:tasks.workunit.client.1.smithi192.stdout:0/474: symlink d1/d3/d29/d43/l97 0 2024-02-16T22:56:21.105 INFO:tasks.workunit.client.1.smithi192.stdout:1/457: symlink d3/de/d27/d34/d69/d85/l9a 0 2024-02-16T22:56:21.107 INFO:tasks.workunit.client.0.smithi064.stdout:7/327: unlink d5/d33/d45/d64/c58 0 2024-02-16T22:56:21.110 INFO:tasks.workunit.client.0.smithi064.stdout:3/449: mknod d4/d13/d1b/d27/d44/c9b 0 2024-02-16T22:56:21.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/446: dread d6/d12/d4e/d5b/f5e [0,4194304] 0 2024-02-16T22:56:21.112 INFO:tasks.workunit.client.0.smithi064.stdout:8/353: sync 2024-02-16T22:56:21.113 INFO:tasks.workunit.client.0.smithi064.stdout:9/361: sync 2024-02-16T22:56:21.113 INFO:tasks.workunit.client.0.smithi064.stdout:6/428: sync 2024-02-16T22:56:21.113 INFO:tasks.workunit.client.0.smithi064.stdout:0/333: sync 2024-02-16T22:56:21.113 INFO:tasks.workunit.client.0.smithi064.stdout:5/286: sync 2024-02-16T22:56:21.114 INFO:tasks.workunit.client.1.smithi192.stdout:4/552: symlink d1/d15/d29/d3c/lc1 0 2024-02-16T22:56:21.115 INFO:tasks.workunit.client.0.smithi064.stdout:0/334: dread - da/dc/d3b/f5f zero size 2024-02-16T22:56:21.118 INFO:tasks.workunit.client.0.smithi064.stdout:5/287: chown d6/d18/d19/d1a/f3b 513997 1 2024-02-16T22:56:21.118 INFO:tasks.workunit.client.0.smithi064.stdout:5/288: chown d6/d18/d5d 97697 1 2024-02-16T22:56:21.120 INFO:tasks.workunit.client.1.smithi192.stdout:8/466: link d5/f84 d5/d26/d83/d86/d5f/f8d 0 2024-02-16T22:56:21.120 INFO:tasks.workunit.client.1.smithi192.stdout:6/474: dwrite da/df/d12/d15/d1b/f37 [0,4194304] 0 2024-02-16T22:56:21.120 INFO:tasks.workunit.client.0.smithi064.stdout:6/429: readlink d0/d6/d10/d33/l20 0 2024-02-16T22:56:21.121 INFO:tasks.workunit.client.0.smithi064.stdout:7/328: write d5/d33/f57 [199906,53934] 0 2024-02-16T22:56:21.121 INFO:tasks.workunit.client.0.smithi064.stdout:6/430: chown d0/d6/d10/d33/d72 2 1 2024-02-16T22:56:21.122 INFO:tasks.workunit.client.0.smithi064.stdout:9/362: symlink d3/d11/d15/d5a/d2a/d2e/l7a 0 2024-02-16T22:56:21.123 INFO:tasks.workunit.client.1.smithi192.stdout:7/555: getdents d1/d2/de/d2c/d49/da4 0 2024-02-16T22:56:21.124 INFO:tasks.workunit.client.0.smithi064.stdout:2/455: dread d7/d14/d25/f52 [0,4194304] 0 2024-02-16T22:56:21.126 INFO:tasks.workunit.client.0.smithi064.stdout:3/450: creat d4/d13/d1b/d27/d99/d7d/f9c x:0 0 0 2024-02-16T22:56:21.127 INFO:tasks.workunit.client.1.smithi192.stdout:1/458: mknod d3/de/d27/d34/d52/d63/c9b 0 2024-02-16T22:56:21.129 INFO:tasks.workunit.client.0.smithi064.stdout:0/335: rename da/d26/d3f/c45 to da/dc/de/d12/d1d/c6a 0 2024-02-16T22:56:21.133 INFO:tasks.workunit.client.1.smithi192.stdout:4/553: creat d1/d26/d5e/fc2 x:0 0 0 2024-02-16T22:56:21.135 INFO:tasks.workunit.client.1.smithi192.stdout:2/515: rename d22/d24/d39/d42/d52/c73 to d22/d24/d39/d42/d52/cad 0 2024-02-16T22:56:21.136 INFO:tasks.workunit.client.0.smithi064.stdout:5/289: dread d6/d1c/f62 [0,4194304] 0 2024-02-16T22:56:21.137 INFO:tasks.workunit.client.0.smithi064.stdout:2/456: mkdir d7/d14/d1e/d34/d73/d8c 0 2024-02-16T22:56:21.138 INFO:tasks.workunit.client.0.smithi064.stdout:4/378: sync 2024-02-16T22:56:21.138 INFO:tasks.workunit.client.0.smithi064.stdout:1/348: sync 2024-02-16T22:56:21.140 INFO:tasks.workunit.client.0.smithi064.stdout:3/451: creat d4/d1f/d5b/f9d x:0 0 0 2024-02-16T22:56:21.140 INFO:tasks.workunit.client.1.smithi192.stdout:4/554: read d1/d26/d66/d89/d11/d58/f7e [3193272,50690] 0 2024-02-16T22:56:21.141 INFO:tasks.workunit.client.1.smithi192.stdout:4/555: chown d1/d26/d66/d89/d11/d58/f90 224216 1 2024-02-16T22:56:21.142 INFO:tasks.workunit.client.1.smithi192.stdout:6/475: creat da/df/d58/f9a x:0 0 0 2024-02-16T22:56:21.142 INFO:tasks.workunit.client.1.smithi192.stdout:8/467: read d5/db/de/df/f2b [977873,72626] 0 2024-02-16T22:56:21.143 INFO:tasks.workunit.client.0.smithi064.stdout:7/329: creat d5/d6/d65/f6d x:0 0 0 2024-02-16T22:56:21.143 INFO:tasks.workunit.client.0.smithi064.stdout:4/379: truncate dc/dd/d32/d58/f70 7349583 0 2024-02-16T22:56:21.146 INFO:tasks.workunit.client.1.smithi192.stdout:3/484: link da/d11/d20/d39/d53/f62 da/fa8 0 2024-02-16T22:56:21.147 INFO:tasks.workunit.client.0.smithi064.stdout:4/380: chown dc/d1d/d43/l48 3760 1 2024-02-16T22:56:21.148 INFO:tasks.workunit.client.1.smithi192.stdout:7/556: chown d1/d2/d2d/d55/c7b 269778 1 2024-02-16T22:56:21.149 INFO:tasks.workunit.client.0.smithi064.stdout:1/349: dwrite d2/df/f13 [0,4194304] 0 2024-02-16T22:56:21.149 INFO:tasks.workunit.client.1.smithi192.stdout:6/476: dread da/df/d12/d15/f33 [0,4194304] 0 2024-02-16T22:56:21.150 INFO:tasks.workunit.client.0.smithi064.stdout:4/381: write dc/dd/d32/d22/f24 [4154204,96516] 0 2024-02-16T22:56:21.153 INFO:tasks.workunit.client.1.smithi192.stdout:0/475: mknod d1/d3/de/d22/d37/d47/c98 0 2024-02-16T22:56:21.153 INFO:tasks.workunit.client.1.smithi192.stdout:7/557: dwrite d1/d2/d2d/d61/d88/d90/d4c/f6b [0,4194304] 0 2024-02-16T22:56:21.154 INFO:tasks.workunit.client.0.smithi064.stdout:7/330: dread d5/f10 [0,4194304] 0 2024-02-16T22:56:21.154 INFO:tasks.workunit.client.0.smithi064.stdout:6/431: dread d0/d6/d10/d33/f2a [0,4194304] 0 2024-02-16T22:56:21.154 INFO:tasks.workunit.client.0.smithi064.stdout:1/350: fsync d2/df/f76 0 2024-02-16T22:56:21.155 INFO:tasks.workunit.client.1.smithi192.stdout:0/476: dread - d1/d3/de/d22/d37/d67/d34/f7f zero size 2024-02-16T22:56:21.155 INFO:tasks.workunit.client.1.smithi192.stdout:7/558: chown d1/d2/de/d2c/d49/f71 5 1 2024-02-16T22:56:21.155 INFO:tasks.workunit.client.0.smithi064.stdout:5/290: stat d6/f13 0 2024-02-16T22:56:21.157 INFO:tasks.workunit.client.1.smithi192.stdout:1/459: mkdir d3/d5b/d9c 0 2024-02-16T22:56:21.158 INFO:tasks.workunit.client.0.smithi064.stdout:4/382: dread dc/dd/d16/f51 [0,4194304] 0 2024-02-16T22:56:21.160 INFO:tasks.workunit.client.0.smithi064.stdout:2/457: fsync d7/f8 0 2024-02-16T22:56:21.161 INFO:tasks.workunit.client.0.smithi064.stdout:0/336: dread f3 [0,4194304] 0 2024-02-16T22:56:21.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/447: mknod d6/d12/d1d/c97 0 2024-02-16T22:56:21.166 INFO:tasks.workunit.client.0.smithi064.stdout:6/432: dwrite d0/d1d/f61 [0,4194304] 0 2024-02-16T22:56:21.166 INFO:tasks.workunit.client.1.smithi192.stdout:2/516: rename d22/f31 to d22/d24/d39/d42/d65/d69/d4f/fae 0 2024-02-16T22:56:21.167 INFO:tasks.workunit.client.0.smithi064.stdout:9/363: dwrite d3/d11/f1f [0,4194304] 0 2024-02-16T22:56:21.168 INFO:tasks.workunit.client.1.smithi192.stdout:5/448: dwrite d6/fc [4194304,4194304] 0 2024-02-16T22:56:21.168 INFO:tasks.workunit.client.1.smithi192.stdout:9/592: link d7/d15/f20 d7/d19/d5e/fcd 0 2024-02-16T22:56:21.171 INFO:tasks.workunit.client.1.smithi192.stdout:9/593: truncate d7/d15/d36/d38/d3f/d8a/d98/d6e/f91 862200 0 2024-02-16T22:56:21.172 INFO:tasks.workunit.client.0.smithi064.stdout:9/364: write d3/d9/f4a [2042259,128194] 0 2024-02-16T22:56:21.175 INFO:tasks.workunit.client.0.smithi064.stdout:9/365: dread d3/d11/f1f [0,4194304] 0 2024-02-16T22:56:21.177 INFO:tasks.workunit.client.1.smithi192.stdout:5/449: dwrite d6/d12/d4e/d46/f5d [0,4194304] 0 2024-02-16T22:56:21.179 INFO:tasks.workunit.client.1.smithi192.stdout:8/468: mknod d5/d9/c8e 0 2024-02-16T22:56:21.181 INFO:tasks.workunit.client.1.smithi192.stdout:3/485: dread - da/d11/d2b/d43/f88 zero size 2024-02-16T22:56:21.181 INFO:tasks.workunit.client.0.smithi064.stdout:8/354: getdents df/d31/d2f/d4f/d6a 0 2024-02-16T22:56:21.182 INFO:tasks.workunit.client.0.smithi064.stdout:8/355: chown df/d31/d47/l4e 795 1 2024-02-16T22:56:21.184 INFO:tasks.workunit.client.0.smithi064.stdout:8/356: write df/f12 [1331716,24554] 0 2024-02-16T22:56:21.186 INFO:tasks.workunit.client.0.smithi064.stdout:8/357: chown df/d31/d2f/d38/f5d 13171962 1 2024-02-16T22:56:21.191 INFO:tasks.workunit.client.0.smithi064.stdout:5/291: mknod d6/d1c/c64 0 2024-02-16T22:56:21.192 INFO:tasks.workunit.client.1.smithi192.stdout:1/460: creat d3/de/d27/d34/d69/f9d x:0 0 0 2024-02-16T22:56:21.192 INFO:tasks.workunit.client.0.smithi064.stdout:8/358: dwrite df/f74 [0,4194304] 0 2024-02-16T22:56:21.198 INFO:tasks.workunit.client.0.smithi064.stdout:0/337: unlink da/dc/d3b/d5a/l65 0 2024-02-16T22:56:21.199 INFO:tasks.workunit.client.1.smithi192.stdout:4/556: mkdir d1/d15/d48/dc3 0 2024-02-16T22:56:21.200 INFO:tasks.workunit.client.0.smithi064.stdout:6/433: chown d0/d5/df/d17/d18/l52 14 1 2024-02-16T22:56:21.202 INFO:tasks.workunit.client.0.smithi064.stdout:6/434: chown d0/d12/f78 35036646 1 2024-02-16T22:56:21.203 INFO:tasks.workunit.client.0.smithi064.stdout:3/452: symlink d4/d13/d1b/d27/l9e 0 2024-02-16T22:56:21.204 INFO:tasks.workunit.client.0.smithi064.stdout:0/338: dwrite da/dc/de/d37/f46 [0,4194304] 0 2024-02-16T22:56:21.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/450: rmdir d6/d12 39 2024-02-16T22:56:21.214 INFO:tasks.workunit.client.1.smithi192.stdout:2/517: mknod d22/d24/d2b/d2c/d3f/dac/caf 0 2024-02-16T22:56:21.216 INFO:tasks.workunit.client.1.smithi192.stdout:8/469: dread d5/d16/f18 [0,4194304] 0 2024-02-16T22:56:21.216 INFO:tasks.workunit.client.1.smithi192.stdout:8/470: fsync d5/db/d47/f3b 0 2024-02-16T22:56:21.217 INFO:tasks.workunit.client.1.smithi192.stdout:4/557: creat d1/d26/dbe/fc4 x:0 0 0 2024-02-16T22:56:21.218 INFO:tasks.workunit.client.0.smithi064.stdout:7/331: write d5/d6/fa [404597,97521] 0 2024-02-16T22:56:21.219 INFO:tasks.workunit.client.1.smithi192.stdout:0/477: write d1/d3/de/d22/f28 [2497344,90452] 0 2024-02-16T22:56:21.220 INFO:tasks.workunit.client.1.smithi192.stdout:6/477: creat da/df/d12/f9b x:0 0 0 2024-02-16T22:56:21.223 INFO:tasks.workunit.client.1.smithi192.stdout:7/559: dwrite d1/f4b [0,4194304] 0 2024-02-16T22:56:21.225 INFO:tasks.workunit.client.1.smithi192.stdout:1/461: symlink d3/de/d27/d34/d52/d63/d7a/l9e 0 2024-02-16T22:56:21.226 INFO:tasks.workunit.client.0.smithi064.stdout:8/359: mknod df/d31/d2f/d68/c7a 0 2024-02-16T22:56:21.230 INFO:tasks.workunit.client.0.smithi064.stdout:4/383: dwrite dc/dd/d32/f1f [4194304,4194304] 0 2024-02-16T22:56:21.230 INFO:tasks.workunit.client.0.smithi064.stdout:4/384: readlink dc/l6f 0 2024-02-16T22:56:21.231 INFO:tasks.workunit.client.1.smithi192.stdout:2/518: mkdir d22/d24/d39/d42/d85/d98/db0 0 2024-02-16T22:56:21.235 INFO:tasks.workunit.client.0.smithi064.stdout:6/435: creat d0/d1d/f84 x:0 0 0 2024-02-16T22:56:21.238 INFO:tasks.workunit.client.1.smithi192.stdout:8/471: rename d5/d3a to d5/d26/d8f 0 2024-02-16T22:56:21.240 INFO:tasks.workunit.client.1.smithi192.stdout:2/519: dwrite d22/d24/f49 [4194304,4194304] 0 2024-02-16T22:56:21.241 INFO:tasks.workunit.client.1.smithi192.stdout:4/558: mkdir d1/d26/d66/d89/da8/dc5 0 2024-02-16T22:56:21.242 INFO:tasks.workunit.client.1.smithi192.stdout:4/559: dread - d1/d15/d48/fbd zero size 2024-02-16T22:56:21.243 INFO:tasks.workunit.client.1.smithi192.stdout:0/478: write d1/f3e [4442987,57468] 0 2024-02-16T22:56:21.245 INFO:tasks.workunit.client.0.smithi064.stdout:0/339: truncate da/dc/de/d12/d1d/f3d 2501356 0 2024-02-16T22:56:21.250 INFO:tasks.workunit.client.0.smithi064.stdout:5/292: dwrite d6/f1b [0,4194304] 0 2024-02-16T22:56:21.251 INFO:tasks.workunit.client.1.smithi192.stdout:5/451: symlink d6/d12/d4e/d5b/d67/l98 0 2024-02-16T22:56:21.251 INFO:tasks.workunit.client.1.smithi192.stdout:7/560: rmdir d1/d2/d2d/d61/d88 39 2024-02-16T22:56:21.252 INFO:tasks.workunit.client.1.smithi192.stdout:7/561: write d1/d2/de/f99 [434675,82693] 0 2024-02-16T22:56:21.254 INFO:tasks.workunit.client.0.smithi064.stdout:8/360: symlink df/d31/d47/l7b 0 2024-02-16T22:56:21.255 INFO:tasks.workunit.client.0.smithi064.stdout:2/458: dwrite d7/f1a [0,4194304] 0 2024-02-16T22:56:21.255 INFO:tasks.workunit.client.1.smithi192.stdout:1/462: mkdir d3/d40/d9f 0 2024-02-16T22:56:21.256 INFO:tasks.workunit.client.1.smithi192.stdout:1/463: chown d3/l60 1065 1 2024-02-16T22:56:21.258 INFO:tasks.workunit.client.1.smithi192.stdout:9/594: getdents d7/d15 0 2024-02-16T22:56:21.262 INFO:tasks.workunit.client.0.smithi064.stdout:4/385: fdatasync dc/d1d/f4c 0 2024-02-16T22:56:21.263 INFO:tasks.workunit.client.0.smithi064.stdout:2/459: dwrite d7/d14/d1e/d34/d73/f77 [0,4194304] 0 2024-02-16T22:56:21.264 INFO:tasks.workunit.client.0.smithi064.stdout:4/386: chown dc/d1d/d43/f5f 1 1 2024-02-16T22:56:21.264 INFO:tasks.workunit.client.1.smithi192.stdout:3/486: getdents da/df 0 2024-02-16T22:56:21.267 INFO:tasks.workunit.client.0.smithi064.stdout:6/436: mkdir d0/d85 0 2024-02-16T22:56:21.269 INFO:tasks.workunit.client.1.smithi192.stdout:8/472: fsync d5/d26/d83/d86/d5f/f8d 0 2024-02-16T22:56:21.271 INFO:tasks.workunit.client.0.smithi064.stdout:4/387: dwrite dc/d1d/f2b [0,4194304] 0 2024-02-16T22:56:21.272 INFO:tasks.workunit.client.0.smithi064.stdout:3/453: mknod d4/d14/c9f 0 2024-02-16T22:56:21.273 INFO:tasks.workunit.client.0.smithi064.stdout:4/388: chown dc/dd/d32/d58/c37 3949 1 2024-02-16T22:56:21.274 INFO:tasks.workunit.client.0.smithi064.stdout:4/389: readlink dc/dd/d32/l26 0 2024-02-16T22:56:21.277 INFO:tasks.workunit.client.0.smithi064.stdout:1/351: getdents d2/df/d11 0 2024-02-16T22:56:21.280 INFO:tasks.workunit.client.1.smithi192.stdout:4/560: rename d1/l71 to d1/d26/d5e/lc6 0 2024-02-16T22:56:21.280 INFO:tasks.workunit.client.0.smithi064.stdout:7/332: creat d5/d33/d53/f6e x:0 0 0 2024-02-16T22:56:21.282 INFO:tasks.workunit.client.1.smithi192.stdout:4/561: read d1/d26/f31 [562545,111574] 0 2024-02-16T22:56:21.282 INFO:tasks.workunit.client.1.smithi192.stdout:0/479: mknod d1/d3/de/d22/d37/d67/d34/c99 0 2024-02-16T22:56:21.284 INFO:tasks.workunit.client.0.smithi064.stdout:5/293: symlink d6/d18/d19/l65 0 2024-02-16T22:56:21.285 INFO:tasks.workunit.client.1.smithi192.stdout:5/452: dread - d6/d12/d4e/d46/f57 zero size 2024-02-16T22:56:21.289 INFO:tasks.workunit.client.1.smithi192.stdout:7/562: creat d1/d2/d2d/d61/d8a/dbb/fc9 x:0 0 0 2024-02-16T22:56:21.291 INFO:tasks.workunit.client.1.smithi192.stdout:1/464: mknod d3/de/d27/d34/d69/d85/d89/ca0 0 2024-02-16T22:56:21.291 INFO:tasks.workunit.client.1.smithi192.stdout:2/520: dwrite d22/d24/d2b/d2c/f43 [0,4194304] 0 2024-02-16T22:56:21.292 INFO:tasks.workunit.client.1.smithi192.stdout:1/465: stat d3/de/d27/d34/d36/d47/d5d/c76 0 2024-02-16T22:56:21.299 INFO:tasks.workunit.client.0.smithi064.stdout:2/460: fsync d7/de/f2f 0 2024-02-16T22:56:21.302 INFO:tasks.workunit.client.0.smithi064.stdout:3/454: dread - d4/d13/f62 zero size 2024-02-16T22:56:21.303 INFO:tasks.workunit.client.0.smithi064.stdout:9/366: getdents d3/d9/d1e/d42 0 2024-02-16T22:56:21.304 INFO:tasks.workunit.client.1.smithi192.stdout:4/562: mkdir d1/d26/d34/dc7 0 2024-02-16T22:56:21.305 INFO:tasks.workunit.client.1.smithi192.stdout:6/478: creat da/df/d12/f9c x:0 0 0 2024-02-16T22:56:21.306 INFO:tasks.workunit.client.1.smithi192.stdout:5/453: creat d6/d12/d2d/d5c/f99 x:0 0 0 2024-02-16T22:56:21.307 INFO:tasks.workunit.client.1.smithi192.stdout:5/454: write d6/d8/d75/f7b [175837,66319] 0 2024-02-16T22:56:21.308 INFO:tasks.workunit.client.1.smithi192.stdout:8/473: write d5/d26/f3f [1832669,97582] 0 2024-02-16T22:56:21.311 INFO:tasks.workunit.client.0.smithi064.stdout:0/340: link da/f5c da/d26/d3f/f6b 0 2024-02-16T22:56:21.314 INFO:tasks.workunit.client.0.smithi064.stdout:8/361: dread f3 [0,4194304] 0 2024-02-16T22:56:21.316 INFO:tasks.workunit.client.0.smithi064.stdout:5/294: truncate d6/fa 293606 0 2024-02-16T22:56:21.317 INFO:tasks.workunit.client.1.smithi192.stdout:2/521: mkdir d22/d24/d39/d42/d65/d69/db1 0 2024-02-16T22:56:21.320 INFO:tasks.workunit.client.1.smithi192.stdout:9/595: rename d7/d19/d28/f3e to d7/d15/fce 0 2024-02-16T22:56:21.322 INFO:tasks.workunit.client.0.smithi064.stdout:7/333: dwrite d5/f23 [0,4194304] 0 2024-02-16T22:56:21.323 INFO:tasks.workunit.client.1.smithi192.stdout:9/596: write d7/d19/d37/d5a/f63 [1885934,61390] 0 2024-02-16T22:56:21.325 INFO:tasks.workunit.client.0.smithi064.stdout:6/437: link d0/d1d/d21/f7f d0/d5/df/f86 0 2024-02-16T22:56:21.326 INFO:tasks.workunit.client.1.smithi192.stdout:2/522: dwrite d22/d24/d2b/f38 [0,4194304] 0 2024-02-16T22:56:21.327 INFO:tasks.workunit.client.0.smithi064.stdout:3/455: mknod d4/d13/d1b/d27/d99/ca0 0 2024-02-16T22:56:21.329 INFO:tasks.workunit.client.1.smithi192.stdout:2/523: write d22/d24/d2b/d67/f90 [741287,104592] 0 2024-02-16T22:56:21.330 INFO:tasks.workunit.client.0.smithi064.stdout:7/334: dwrite d5/d6/f5b [0,4194304] 0 2024-02-16T22:56:21.331 INFO:tasks.workunit.client.0.smithi064.stdout:9/367: truncate d3/f18 4956176 0 2024-02-16T22:56:21.332 INFO:tasks.workunit.client.0.smithi064.stdout:9/368: chown d3 0 1 2024-02-16T22:56:21.332 INFO:tasks.workunit.client.1.smithi192.stdout:7/563: dwrite d1/d2/f1b [0,4194304] 0 2024-02-16T22:56:21.334 INFO:tasks.workunit.client.1.smithi192.stdout:0/480: dread d1/f36 [4194304,4194304] 0 2024-02-16T22:56:21.336 INFO:tasks.workunit.client.1.smithi192.stdout:8/474: rmdir d5 39 2024-02-16T22:56:21.337 INFO:tasks.workunit.client.0.smithi064.stdout:4/390: symlink dc/dd/d16/d6e/l72 0 2024-02-16T22:56:21.338 INFO:tasks.workunit.client.0.smithi064.stdout:2/461: write d7/d14/d25/f27 [4953112,34413] 0 2024-02-16T22:56:21.340 INFO:tasks.workunit.client.0.smithi064.stdout:4/391: stat dc/de/d28 0 2024-02-16T22:56:21.340 INFO:tasks.workunit.client.0.smithi064.stdout:3/456: dwrite d4/d13/d1b/d27/d99/d7d/f9c [0,4194304] 0 2024-02-16T22:56:21.344 INFO:tasks.workunit.client.0.smithi064.stdout:1/352: mknod d2/c79 0 2024-02-16T22:56:21.344 INFO:tasks.workunit.client.1.smithi192.stdout:3/487: rename da/d11/d35/l36 to da/d11/d63/d6b/d71/la9 0 2024-02-16T22:56:21.346 INFO:tasks.workunit.client.0.smithi064.stdout:7/335: dwrite d5/d33/f57 [0,4194304] 0 2024-02-16T22:56:21.346 INFO:tasks.workunit.client.1.smithi192.stdout:3/488: write da/d11/d63/d6b/d71/f7b [1001109,41940] 0 2024-02-16T22:56:21.350 INFO:tasks.workunit.client.1.smithi192.stdout:9/597: creat d7/d19/d5e/fcf x:0 0 0 2024-02-16T22:56:21.351 INFO:tasks.workunit.client.0.smithi064.stdout:3/457: write d4/d1f/d34/f45 [2411130,36201] 0 2024-02-16T22:56:21.353 INFO:tasks.workunit.client.0.smithi064.stdout:2/462: dwrite d7/de/f1f [0,4194304] 0 2024-02-16T22:56:21.354 INFO:tasks.workunit.client.0.smithi064.stdout:5/295: chown d6/c36 1107700238 1 2024-02-16T22:56:21.354 INFO:tasks.workunit.client.1.smithi192.stdout:7/564: rmdir d1/d2/de/d2c 39 2024-02-16T22:56:21.355 INFO:tasks.workunit.client.0.smithi064.stdout:5/296: write d6/d18/d19/f26 [2869180,126468] 0 2024-02-16T22:56:21.360 INFO:tasks.workunit.client.1.smithi192.stdout:2/524: truncate d22/d24/d2b/f64 2397048 0 2024-02-16T22:56:21.361 INFO:tasks.workunit.client.1.smithi192.stdout:6/479: truncate da/d40/f4c 3754091 0 2024-02-16T22:56:21.363 INFO:tasks.workunit.client.0.smithi064.stdout:6/438: truncate d0/d1b/f7a 582743 0 2024-02-16T22:56:21.367 INFO:tasks.workunit.client.1.smithi192.stdout:3/489: fdatasync da/d11/d63/d6b/d71/f98 0 2024-02-16T22:56:21.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:21 smithi064 ceph-mon[29226]: pgmap v15: 65 pgs: 65 active+clean; 1.5 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 220 MiB/s wr, 332 op/s 2024-02-16T22:56:21.368 INFO:tasks.workunit.client.1.smithi192.stdout:8/475: dwrite d5/d26/d8f/d4d/f4b [0,4194304] 0 2024-02-16T22:56:21.368 INFO:tasks.workunit.client.0.smithi064.stdout:9/369: mkdir d3/d9/d1e/d42/d7b 0 2024-02-16T22:56:21.369 INFO:tasks.workunit.client.1.smithi192.stdout:9/598: read d7/d19/d5b/fa5 [710691,29669] 0 2024-02-16T22:56:21.373 INFO:tasks.workunit.client.0.smithi064.stdout:4/392: mknod dc/d1d/c73 0 2024-02-16T22:56:21.375 INFO:tasks.workunit.client.1.smithi192.stdout:2/525: unlink d22/f27 0 2024-02-16T22:56:21.375 INFO:tasks.workunit.client.1.smithi192.stdout:7/565: write d1/d2/de/d2c/d49/d54/d6a/f95 [41008,5467] 0 2024-02-16T22:56:21.380 INFO:tasks.workunit.client.1.smithi192.stdout:2/526: truncate d22/d47/f95 720346 0 2024-02-16T22:56:21.380 INFO:tasks.workunit.client.0.smithi064.stdout:1/353: symlink d2/d4e/d56/l7a 0 2024-02-16T22:56:21.381 INFO:tasks.workunit.client.0.smithi064.stdout:1/354: chown d2/df/d14/f16 30 1 2024-02-16T22:56:21.381 INFO:tasks.workunit.client.1.smithi192.stdout:3/490: dwrite da/d11/d20/f85 [0,4194304] 0 2024-02-16T22:56:21.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/599: dwrite d7/d19/d37/f55 [0,4194304] 0 2024-02-16T22:56:21.382 INFO:tasks.workunit.client.0.smithi064.stdout:1/355: write d2/df/d14/d2c/f5e [571034,38840] 0 2024-02-16T22:56:21.384 INFO:tasks.workunit.client.1.smithi192.stdout:9/600: stat d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/c9c 0 2024-02-16T22:56:21.386 INFO:tasks.workunit.client.1.smithi192.stdout:4/563: rename d1/d26/d66/d89/d11/d58/l87 to d1/d26/d34/lc8 0 2024-02-16T22:56:21.389 INFO:tasks.workunit.client.1.smithi192.stdout:8/476: mkdir d5/db/d47/d43/d90 0 2024-02-16T22:56:21.391 INFO:tasks.workunit.client.0.smithi064.stdout:6/439: symlink d0/d3c/l87 0 2024-02-16T22:56:21.394 INFO:tasks.workunit.client.0.smithi064.stdout:6/440: chown d0/d5/df/d17/d18/l39 1843334 1 2024-02-16T22:56:21.394 INFO:tasks.workunit.client.1.smithi192.stdout:3/491: chown da/d11/d20/c3e 22937 1 2024-02-16T22:56:21.395 INFO:tasks.workunit.client.1.smithi192.stdout:3/492: fdatasync da/d11/f96 0 2024-02-16T22:56:21.397 INFO:tasks.workunit.client.1.smithi192.stdout:9/601: creat d7/d19/d5e/d81/fd0 x:0 0 0 2024-02-16T22:56:21.400 INFO:tasks.workunit.client.0.smithi064.stdout:0/341: creat da/dc/de/d12/f6c x:0 0 0 2024-02-16T22:56:21.401 INFO:tasks.workunit.client.1.smithi192.stdout:3/493: dread da/fa8 [0,4194304] 0 2024-02-16T22:56:21.402 INFO:tasks.workunit.client.1.smithi192.stdout:4/564: mknod d1/d15/d23/cc9 0 2024-02-16T22:56:21.403 INFO:tasks.workunit.client.0.smithi064.stdout:1/356: creat d2/d4e/d56/f7b x:0 0 0 2024-02-16T22:56:21.404 INFO:tasks.workunit.client.0.smithi064.stdout:1/357: chown d2/df/d14/d2a/l3a 52 1 2024-02-16T22:56:21.404 INFO:tasks.workunit.client.1.smithi192.stdout:9/602: dwrite d7/d19/d37/f9e [0,4194304] 0 2024-02-16T22:56:21.405 INFO:tasks.workunit.client.0.smithi064.stdout:9/370: dread d3/d9/ff [0,4194304] 0 2024-02-16T22:56:21.406 INFO:tasks.workunit.client.1.smithi192.stdout:5/455: rename d6/d12/c1a to d6/d8/d75/c9a 0 2024-02-16T22:56:21.407 INFO:tasks.workunit.client.1.smithi192.stdout:5/456: write d6/d12/f83 [337228,35671] 0 2024-02-16T22:56:21.408 INFO:tasks.workunit.client.0.smithi064.stdout:8/362: creat df/d31/f7c x:0 0 0 2024-02-16T22:56:21.408 INFO:tasks.workunit.client.0.smithi064.stdout:3/458: dread d4/d1f/d34/f45 [0,4194304] 0 2024-02-16T22:56:21.409 INFO:tasks.workunit.client.0.smithi064.stdout:9/371: dread d3/d11/d15/d5a/d38/f66 [0,4194304] 0 2024-02-16T22:56:21.410 INFO:tasks.workunit.client.1.smithi192.stdout:8/477: mknod d5/db/d47/d43/d67/c91 0 2024-02-16T22:56:21.410 INFO:tasks.workunit.client.0.smithi064.stdout:9/372: chown d3/d11/d15/d5a/d38 3547 1 2024-02-16T22:56:21.411 INFO:tasks.workunit.client.1.smithi192.stdout:8/478: dread - d5/db/de/df/f62 zero size 2024-02-16T22:56:21.412 INFO:tasks.workunit.client.1.smithi192.stdout:2/527: symlink d22/d24/d2b/d2c/d78/lb2 0 2024-02-16T22:56:21.413 INFO:tasks.workunit.client.1.smithi192.stdout:5/457: dwrite d6/f6f [0,4194304] 0 2024-02-16T22:56:21.425 INFO:tasks.workunit.client.1.smithi192.stdout:9/603: mknod d7/d19/d5e/d6f/cd1 0 2024-02-16T22:56:21.429 INFO:tasks.workunit.client.1.smithi192.stdout:7/566: rename d1/d2/d2d/d61/d88/d90/l36 to d1/d2/d2d/d61/d88/d90/d4c/d60/lca 0 2024-02-16T22:56:21.429 INFO:tasks.workunit.client.1.smithi192.stdout:7/567: fdatasync d1/f11 0 2024-02-16T22:56:21.430 INFO:tasks.workunit.client.1.smithi192.stdout:0/481: truncate d1/d3/de/d22/d37/d67/d34/f92 739334 0 2024-02-16T22:56:21.436 INFO:tasks.workunit.client.1.smithi192.stdout:8/479: creat d5/db/d47/f92 x:0 0 0 2024-02-16T22:56:21.439 INFO:tasks.workunit.client.1.smithi192.stdout:5/458: symlink d6/d20/l9b 0 2024-02-16T22:56:21.440 INFO:tasks.workunit.client.1.smithi192.stdout:4/565: getdents d1/d26/d66/d89/da8/dc5 0 2024-02-16T22:56:21.440 INFO:tasks.workunit.client.1.smithi192.stdout:5/459: truncate d6/d8/d80/f8a 4515537 0 2024-02-16T22:56:21.444 INFO:tasks.workunit.client.1.smithi192.stdout:0/482: creat d1/d3/d29/f9a x:0 0 0 2024-02-16T22:56:21.445 INFO:tasks.workunit.client.1.smithi192.stdout:2/528: symlink d22/d24/d39/d42/d84/lb3 0 2024-02-16T22:56:21.448 INFO:tasks.workunit.client.0.smithi064.stdout:1/358: mknod d2/df/d14/d2c/d62/c7c 0 2024-02-16T22:56:21.448 INFO:tasks.workunit.client.0.smithi064.stdout:9/373: dread d3/d9/f36 [0,4194304] 0 2024-02-16T22:56:21.449 INFO:tasks.workunit.client.0.smithi064.stdout:2/463: rename d7/d14/d25/d37/d44/f6a to d7/de/f8d 0 2024-02-16T22:56:21.450 INFO:tasks.workunit.client.1.smithi192.stdout:4/566: mkdir d1/d26/d34/dca 0 2024-02-16T22:56:21.450 INFO:tasks.workunit.client.0.smithi064.stdout:7/336: dread d5/d6/f18 [0,4194304] 0 2024-02-16T22:56:21.452 INFO:tasks.workunit.client.0.smithi064.stdout:4/393: creat dc/de/f74 x:0 0 0 2024-02-16T22:56:21.452 INFO:tasks.workunit.client.0.smithi064.stdout:4/394: fdatasync dc/d1d/f69 0 2024-02-16T22:56:21.455 INFO:tasks.workunit.client.0.smithi064.stdout:6/441: link d0/d1d/f84 d0/d1d/f88 0 2024-02-16T22:56:21.457 INFO:tasks.workunit.client.1.smithi192.stdout:5/460: dread d6/f33 [0,4194304] 0 2024-02-16T22:56:21.458 INFO:tasks.workunit.client.1.smithi192.stdout:5/461: write d6/f10 [2791039,15223] 0 2024-02-16T22:56:21.459 INFO:tasks.workunit.client.0.smithi064.stdout:5/297: dwrite d6/f29 [0,4194304] 0 2024-02-16T22:56:21.461 INFO:tasks.workunit.client.1.smithi192.stdout:9/604: link d7/d15/d36/d77/c7f d7/d15/d36/d77/cd2 0 2024-02-16T22:56:21.463 INFO:tasks.workunit.client.1.smithi192.stdout:5/462: dwrite d6/fc [0,4194304] 0 2024-02-16T22:56:21.466 INFO:tasks.workunit.client.0.smithi064.stdout:3/459: dread d4/f8 [0,4194304] 0 2024-02-16T22:56:21.469 INFO:tasks.workunit.client.1.smithi192.stdout:3/494: rename da/d11/d20/f30 to da/d11/d63/d6b/faa 0 2024-02-16T22:56:21.471 INFO:tasks.workunit.client.1.smithi192.stdout:3/495: truncate da/d11/d63/fa6 514893 0 2024-02-16T22:56:21.475 INFO:tasks.workunit.client.0.smithi064.stdout:6/442: mkdir d0/d89 0 2024-02-16T22:56:21.476 INFO:tasks.workunit.client.0.smithi064.stdout:8/363: fdatasync df/d2a/f2d 0 2024-02-16T22:56:21.476 INFO:tasks.workunit.client.0.smithi064.stdout:4/395: dwrite dc/dd/d16/f3a [0,4194304] 0 2024-02-16T22:56:21.478 INFO:tasks.workunit.client.0.smithi064.stdout:4/396: readlink dc/dd/l53 0 2024-02-16T22:56:21.481 INFO:tasks.workunit.client.1.smithi192.stdout:0/483: link d1/d3/de/d22/d37/d69/f72 d1/d3/de/d22/d37/d67/f9b 0 2024-02-16T22:56:21.483 INFO:tasks.workunit.client.0.smithi064.stdout:8/364: dwrite df/f74 [0,4194304] 0 2024-02-16T22:56:21.485 INFO:tasks.workunit.client.0.smithi064.stdout:8/365: readlink df/d31/d2f/d38/d48/d69/l6e 0 2024-02-16T22:56:21.485 INFO:tasks.workunit.client.0.smithi064.stdout:5/298: unlink d6/d1c/c64 0 2024-02-16T22:56:21.487 INFO:tasks.workunit.client.0.smithi064.stdout:5/299: dread - d6/d1c/d43/f58 zero size 2024-02-16T22:56:21.489 INFO:tasks.workunit.client.1.smithi192.stdout:0/484: dread d1/d3/d39/f4f [0,4194304] 0 2024-02-16T22:56:21.489 INFO:tasks.workunit.client.0.smithi064.stdout:1/359: symlink d2/df/d14/d18/l7d 0 2024-02-16T22:56:21.489 INFO:tasks.workunit.client.1.smithi192.stdout:7/568: write d1/d2/f46 [3328879,16725] 0 2024-02-16T22:56:21.494 INFO:tasks.workunit.client.1.smithi192.stdout:9/605: creat d7/d15/d36/dbe/fd3 x:0 0 0 2024-02-16T22:56:21.495 INFO:tasks.workunit.client.0.smithi064.stdout:2/464: symlink d7/d14/d25/d46/d55/d7f/l8e 0 2024-02-16T22:56:21.499 INFO:tasks.workunit.client.1.smithi192.stdout:5/463: write d6/d8/d77/f85 [684666,16289] 0 2024-02-16T22:56:21.500 INFO:tasks.workunit.client.1.smithi192.stdout:3/496: dread da/df/f1f [0,4194304] 0 2024-02-16T22:56:21.501 INFO:tasks.workunit.client.0.smithi064.stdout:6/443: rename d0/d6/f3b to d0/d85/f8a 0 2024-02-16T22:56:21.502 INFO:tasks.workunit.client.0.smithi064.stdout:2/465: dwrite d7/f86 [0,4194304] 0 2024-02-16T22:56:21.505 INFO:tasks.workunit.client.1.smithi192.stdout:8/480: getdents d5/d26/d83/d86/d5f 0 2024-02-16T22:56:21.508 INFO:tasks.workunit.client.0.smithi064.stdout:8/366: creat df/d31/d2f/f7d x:0 0 0 2024-02-16T22:56:21.509 INFO:tasks.workunit.client.0.smithi064.stdout:8/367: fdatasync df/d31/f7c 0 2024-02-16T22:56:21.511 INFO:tasks.workunit.client.1.smithi192.stdout:1/466: sync 2024-02-16T22:56:21.511 INFO:tasks.workunit.client.0.smithi064.stdout:5/300: symlink d6/d18/d39/l66 0 2024-02-16T22:56:21.512 INFO:tasks.workunit.client.1.smithi192.stdout:1/467: fsync d3/de/d27/d34/d52/f57 0 2024-02-16T22:56:21.513 INFO:tasks.workunit.client.1.smithi192.stdout:1/468: readlink d3/de/d27/d34/d69/d85/d89/l95 0 2024-02-16T22:56:21.514 INFO:tasks.workunit.client.0.smithi064.stdout:1/360: mknod d2/d4e/d56/c7e 0 2024-02-16T22:56:21.514 INFO:tasks.workunit.client.1.smithi192.stdout:1/469: chown d3/de/l13 281290 1 2024-02-16T22:56:21.517 INFO:tasks.workunit.client.0.smithi064.stdout:7/337: write d5/d6/f19 [868884,8793] 0 2024-02-16T22:56:21.518 INFO:tasks.workunit.client.0.smithi064.stdout:3/460: mkdir d4/d1f/d75/da1 0 2024-02-16T22:56:21.520 INFO:tasks.workunit.client.1.smithi192.stdout:4/567: write d1/f19 [2889304,104143] 0 2024-02-16T22:56:21.524 INFO:tasks.workunit.client.0.smithi064.stdout:7/338: dwrite d5/f23 [0,4194304] 0 2024-02-16T22:56:21.533 INFO:tasks.workunit.client.0.smithi064.stdout:5/301: mkdir d6/d25/d48/d67 0 2024-02-16T22:56:21.537 INFO:tasks.workunit.client.0.smithi064.stdout:8/368: dread df/d31/d61/f65 [0,4194304] 0 2024-02-16T22:56:21.541 INFO:tasks.workunit.client.0.smithi064.stdout:3/461: rename d4/d1f/d34/f3c to d4/d13/d1b/d4f/fa2 0 2024-02-16T22:56:21.542 INFO:tasks.workunit.client.0.smithi064.stdout:3/462: write d4/fd [8436621,123953] 0 2024-02-16T22:56:21.542 INFO:tasks.workunit.client.0.smithi064.stdout:7/339: fdatasync d5/d6/d54/f40 0 2024-02-16T22:56:21.544 INFO:tasks.workunit.client.0.smithi064.stdout:7/340: read d5/d6/f18 [2768451,33982] 0 2024-02-16T22:56:21.545 INFO:tasks.workunit.client.0.smithi064.stdout:6/444: fdatasync d0/d6/d10/f5e 0 2024-02-16T22:56:21.546 INFO:tasks.workunit.client.0.smithi064.stdout:7/341: chown d5/f27 26016 1 2024-02-16T22:56:21.548 INFO:tasks.workunit.client.0.smithi064.stdout:7/342: truncate d5/d33/d53/f6e 1011651 0 2024-02-16T22:56:21.549 INFO:tasks.workunit.client.0.smithi064.stdout:2/466: dwrite d7/d14/d3e/f5d [4194304,4194304] 0 2024-02-16T22:56:21.550 INFO:tasks.workunit.client.0.smithi064.stdout:4/397: link dc/dd/d4f/f5c dc/de/f75 0 2024-02-16T22:56:21.550 INFO:tasks.workunit.client.0.smithi064.stdout:7/343: dread d5/d6/d54/d41/f4b [0,4194304] 0 2024-02-16T22:56:21.551 INFO:tasks.workunit.client.0.smithi064.stdout:7/344: dread - d5/d33/d42/f4f zero size 2024-02-16T22:56:21.552 INFO:tasks.workunit.client.0.smithi064.stdout:7/345: readlink d5/l3e 0 2024-02-16T22:56:21.553 INFO:tasks.workunit.client.0.smithi064.stdout:5/302: rmdir d6/d18/d19/d1a 39 2024-02-16T22:56:21.564 INFO:tasks.workunit.client.0.smithi064.stdout:1/361: truncate d2/df/d14/d2c/f5f 1226441 0 2024-02-16T22:56:21.572 INFO:tasks.workunit.client.1.smithi192.stdout:0/485: chown d1/f5 556909 1 2024-02-16T22:56:21.573 INFO:tasks.workunit.client.0.smithi064.stdout:8/369: dwrite df/f37 [0,4194304] 0 2024-02-16T22:56:21.574 INFO:tasks.workunit.client.1.smithi192.stdout:0/486: write d1/d3/d39/f76 [1799584,193] 0 2024-02-16T22:56:21.576 INFO:tasks.workunit.client.0.smithi064.stdout:6/445: dwrite d0/d5/d54/d60/f7c [0,4194304] 0 2024-02-16T22:56:21.576 INFO:tasks.workunit.client.1.smithi192.stdout:2/529: dwrite d22/d24/f96 [0,4194304] 0 2024-02-16T22:56:21.578 INFO:tasks.workunit.client.1.smithi192.stdout:2/530: chown d22/d24/d39/d42/d52/d7c/cab 24417825 1 2024-02-16T22:56:21.578 INFO:tasks.workunit.client.0.smithi064.stdout:2/467: dread d7/d14/f36 [0,4194304] 0 2024-02-16T22:56:21.579 INFO:tasks.workunit.client.1.smithi192.stdout:2/531: dread - d22/d24/d39/d42/d65/f8e zero size 2024-02-16T22:56:21.580 INFO:tasks.workunit.client.0.smithi064.stdout:2/468: truncate d7/d14/d1e/d34/f62 55938 0 2024-02-16T22:56:21.581 INFO:tasks.workunit.client.1.smithi192.stdout:9/606: creat d7/d15/fd4 x:0 0 0 2024-02-16T22:56:21.587 INFO:tasks.workunit.client.0.smithi064.stdout:3/463: unlink d4/l23 0 2024-02-16T22:56:21.591 INFO:tasks.workunit.client.1.smithi192.stdout:3/497: creat da/d11/d20/fab x:0 0 0 2024-02-16T22:56:21.592 INFO:tasks.workunit.client.1.smithi192.stdout:8/481: rmdir d5/d9 39 2024-02-16T22:56:21.594 INFO:tasks.workunit.client.0.smithi064.stdout:7/346: dwrite d5/d6/f3b [4194304,4194304] 0 2024-02-16T22:56:21.597 INFO:tasks.workunit.client.1.smithi192.stdout:4/568: creat d1/d15/d29/d3c/fcb x:0 0 0 2024-02-16T22:56:21.597 INFO:tasks.workunit.client.1.smithi192.stdout:4/569: fdatasync d1/d26/dbe/fc4 0 2024-02-16T22:56:21.598 INFO:tasks.workunit.client.0.smithi064.stdout:4/398: mknod dc/dd/d5b/c76 0 2024-02-16T22:56:21.598 INFO:tasks.workunit.client.1.smithi192.stdout:4/570: dread - d1/d26/d6f/fbc zero size 2024-02-16T22:56:21.603 INFO:tasks.workunit.client.1.smithi192.stdout:7/569: rename d1/d2/de/d2c/d49/l75 to d1/d2/d2d/d92/d9b/lcb 0 2024-02-16T22:56:21.606 INFO:tasks.workunit.client.1.smithi192.stdout:9/607: mknod d7/d19/d5b/cd5 0 2024-02-16T22:56:21.609 INFO:tasks.workunit.client.1.smithi192.stdout:7/570: dwrite d1/d2/d2d/d61/d8a/dbb/fc9 [0,4194304] 0 2024-02-16T22:56:21.610 INFO:tasks.workunit.client.0.smithi064.stdout:5/303: rename d6/c38 to d6/d18/d19/d1a/c68 0 2024-02-16T22:56:21.612 INFO:tasks.workunit.client.0.smithi064.stdout:3/464: write d4/d13/d1b/d27/f7f [723700,2335] 0 2024-02-16T22:56:21.614 INFO:tasks.workunit.client.0.smithi064.stdout:3/465: write d4/d13/d1b/d4f/f5d [4993947,10899] 0 2024-02-16T22:56:21.615 INFO:tasks.workunit.client.0.smithi064.stdout:3/466: write d4/f12 [1511087,118972] 0 2024-02-16T22:56:21.616 INFO:tasks.workunit.client.0.smithi064.stdout:3/467: write d4/d13/d1b/d4f/f6f [316910,81447] 0 2024-02-16T22:56:21.617 INFO:tasks.workunit.client.1.smithi192.stdout:1/470: dwrite d3/f19 [4194304,4194304] 0 2024-02-16T22:56:21.619 INFO:tasks.workunit.client.1.smithi192.stdout:8/482: dread - d5/d26/d83/d86/d5f/f8d zero size 2024-02-16T22:56:21.621 INFO:tasks.workunit.client.1.smithi192.stdout:5/464: truncate d6/d8/d86/f89 2007860 0 2024-02-16T22:56:21.622 INFO:tasks.workunit.client.1.smithi192.stdout:4/571: symlink d1/d15/d29/d3c/lcc 0 2024-02-16T22:56:21.623 INFO:tasks.workunit.client.0.smithi064.stdout:6/446: dwrite d0/d12/f78 [0,4194304] 0 2024-02-16T22:56:21.625 INFO:tasks.workunit.client.1.smithi192.stdout:6/480: sync 2024-02-16T22:56:21.625 INFO:tasks.workunit.client.0.smithi064.stdout:4/399: mkdir dc/de/d28/d57/d77 0 2024-02-16T22:56:21.627 INFO:tasks.workunit.client.0.smithi064.stdout:4/400: chown dc/dd/d32/f1f 246685274 1 2024-02-16T22:56:21.628 INFO:tasks.workunit.client.1.smithi192.stdout:0/487: rename d1/d3/de/d22/d37/d69 to d1/d3/d29/d43/d9c 0 2024-02-16T22:56:21.630 INFO:tasks.workunit.client.1.smithi192.stdout:0/488: readlink d1/d3/d29/d43/l7d 0 2024-02-16T22:56:21.631 INFO:tasks.workunit.client.1.smithi192.stdout:6/481: dwrite da/d40/f4e [0,4194304] 0 2024-02-16T22:56:21.631 INFO:tasks.workunit.client.0.smithi064.stdout:5/304: creat d6/d18/d19/d1a/f69 x:0 0 0 2024-02-16T22:56:21.631 INFO:tasks.workunit.client.0.smithi064.stdout:5/305: chown d6/f37 193251161 1 2024-02-16T22:56:21.633 INFO:tasks.workunit.client.1.smithi192.stdout:0/489: chown d1/d3/de/d22/d37/d67/d34/l94 112166 1 2024-02-16T22:56:21.633 INFO:tasks.workunit.client.1.smithi192.stdout:0/490: readlink d1/d3/de/d22/d37/d67/l2d 0 2024-02-16T22:56:21.636 INFO:tasks.workunit.client.1.smithi192.stdout:0/491: chown d1/f3e 0 1 2024-02-16T22:56:21.636 INFO:tasks.workunit.client.0.smithi064.stdout:3/468: mknod d4/d14/d2b/ca3 0 2024-02-16T22:56:21.637 INFO:tasks.workunit.client.1.smithi192.stdout:3/498: creat da/d11/d63/d6b/d71/da1/fac x:0 0 0 2024-02-16T22:56:21.638 INFO:tasks.workunit.client.1.smithi192.stdout:0/492: chown d1/d3/de/d22/d37/d67/d34/c99 59036 1 2024-02-16T22:56:21.639 INFO:tasks.workunit.client.1.smithi192.stdout:0/493: write d1/d3/de/f3c [1015643,40050] 0 2024-02-16T22:56:21.640 INFO:tasks.workunit.client.1.smithi192.stdout:0/494: dread - d1/d3/d29/d43/f68 zero size 2024-02-16T22:56:21.641 INFO:tasks.workunit.client.0.smithi064.stdout:5/306: dwrite d6/d25/d48/f5e [0,4194304] 0 2024-02-16T22:56:21.642 INFO:tasks.workunit.client.0.smithi064.stdout:7/347: mknod d5/d6/c6f 0 2024-02-16T22:56:21.643 INFO:tasks.workunit.client.1.smithi192.stdout:8/483: dread d5/d9/f69 [0,4194304] 0 2024-02-16T22:56:21.644 INFO:tasks.workunit.client.1.smithi192.stdout:5/465: rmdir d6/d20 39 2024-02-16T22:56:21.644 INFO:tasks.workunit.client.1.smithi192.stdout:4/572: mknod d1/d26/d34/ccd 0 2024-02-16T22:56:21.645 INFO:tasks.workunit.client.0.smithi064.stdout:8/370: creat df/d31/f7e x:0 0 0 2024-02-16T22:56:21.646 INFO:tasks.workunit.client.1.smithi192.stdout:4/573: write d1/d26/d66/d89/fb [598251,30387] 0 2024-02-16T22:56:21.647 INFO:tasks.workunit.client.1.smithi192.stdout:4/574: fsync d1/d26/d66/d89/d11/d58/f5d 0 2024-02-16T22:56:21.648 INFO:tasks.workunit.client.1.smithi192.stdout:4/575: stat d1/d15/d48/d8d/d9b/fad 0 2024-02-16T22:56:21.652 INFO:tasks.workunit.client.1.smithi192.stdout:2/532: rename d22/d24/d39/d42/d85/d98/c9e to d22/d24/d39/d3d/cb4 0 2024-02-16T22:56:21.663 INFO:tasks.workunit.client.0.smithi064.stdout:7/348: readlink d5/d6/l2e 0 2024-02-16T22:56:21.664 INFO:tasks.workunit.client.0.smithi064.stdout:7/349: readlink d5/d6/d54/l51 0 2024-02-16T22:56:21.664 INFO:tasks.workunit.client.0.smithi064.stdout:8/371: mkdir df/d31/d2f/d7f 0 2024-02-16T22:56:21.664 INFO:tasks.workunit.client.1.smithi192.stdout:3/499: rmdir da/d11/d2b/d43 39 2024-02-16T22:56:21.665 INFO:tasks.workunit.client.0.smithi064.stdout:7/350: write d5/d6/f19 [451674,25379] 0 2024-02-16T22:56:21.669 INFO:tasks.workunit.client.1.smithi192.stdout:6/482: dread da/df/f1c [0,4194304] 0 2024-02-16T22:56:21.671 INFO:tasks.workunit.client.1.smithi192.stdout:0/495: write d1/fd [10599707,70107] 0 2024-02-16T22:56:21.671 INFO:tasks.workunit.client.1.smithi192.stdout:9/608: dwrite d7/d19/d28/d43/d70/f8d [0,4194304] 0 2024-02-16T22:56:21.672 INFO:tasks.workunit.client.0.smithi064.stdout:2/469: getdents d7/d14/d4b 0 2024-02-16T22:56:21.673 INFO:tasks.workunit.client.0.smithi064.stdout:2/470: readlink d7/l6e 0 2024-02-16T22:56:21.673 INFO:tasks.workunit.client.1.smithi192.stdout:7/571: dwrite d1/d2/de/d2c/f96 [0,4194304] 0 2024-02-16T22:56:21.673 INFO:tasks.workunit.client.0.smithi064.stdout:2/471: chown d7/d14/d4b/d63 13454 1 2024-02-16T22:56:21.675 INFO:tasks.workunit.client.0.smithi064.stdout:3/469: rmdir d4/d13/d1b/d27/d61/d63 39 2024-02-16T22:56:21.675 INFO:tasks.workunit.client.1.smithi192.stdout:6/483: truncate da/df/d12/d15/d1b/f37 5204428 0 2024-02-16T22:56:21.676 INFO:tasks.workunit.client.0.smithi064.stdout:3/470: fdatasync d4/f17 0 2024-02-16T22:56:21.676 INFO:tasks.workunit.client.1.smithi192.stdout:7/572: stat d1/d2/de/d2c/d49/f71 0 2024-02-16T22:56:21.679 INFO:tasks.workunit.client.1.smithi192.stdout:3/500: dwrite da/d11/d20/f4a [0,4194304] 0 2024-02-16T22:56:21.679 INFO:tasks.workunit.client.0.smithi064.stdout:4/401: write dc/d1d/f4c [1926442,11065] 0 2024-02-16T22:56:21.679 INFO:tasks.workunit.client.0.smithi064.stdout:5/307: mknod d6/d25/d48/d67/c6a 0 2024-02-16T22:56:21.680 INFO:tasks.workunit.client.0.smithi064.stdout:4/402: write dc/dd/d32/d22/f52 [1533165,95672] 0 2024-02-16T22:56:21.686 INFO:tasks.workunit.client.0.smithi064.stdout:7/351: creat d5/d33/d53/f70 x:0 0 0 2024-02-16T22:56:21.687 INFO:tasks.workunit.client.1.smithi192.stdout:6/484: dwrite da/d40/d85/f94 [0,4194304] 0 2024-02-16T22:56:21.687 INFO:tasks.workunit.client.0.smithi064.stdout:7/352: readlink d5/l2b 0 2024-02-16T22:56:21.688 INFO:tasks.workunit.client.0.smithi064.stdout:6/447: truncate d0/d5/df/f6f 1039034 0 2024-02-16T22:56:21.690 INFO:tasks.workunit.client.0.smithi064.stdout:7/353: read d5/f2f [130059,99609] 0 2024-02-16T22:56:21.691 INFO:tasks.workunit.client.0.smithi064.stdout:8/372: mkdir df/d31/d2f/d38/d48/d69/d80 0 2024-02-16T22:56:21.692 INFO:tasks.workunit.client.0.smithi064.stdout:8/373: readlink df/l6c 0 2024-02-16T22:56:21.692 INFO:tasks.workunit.client.0.smithi064.stdout:6/448: truncate d0/d5/df/d17/f7d 1019250 0 2024-02-16T22:56:21.692 INFO:tasks.workunit.client.0.smithi064.stdout:7/354: chown d5/d33/d42/c55 124 1 2024-02-16T22:56:21.694 INFO:tasks.workunit.client.0.smithi064.stdout:6/449: write d0/d12/f49 [1811083,70776] 0 2024-02-16T22:56:21.695 INFO:tasks.workunit.client.0.smithi064.stdout:1/362: truncate d2/df/d11/d1a/d1d/f24 645486 0 2024-02-16T22:56:21.700 INFO:tasks.workunit.client.0.smithi064.stdout:3/471: rmdir d4/d13/d1b/d48 39 2024-02-16T22:56:21.700 INFO:tasks.workunit.client.1.smithi192.stdout:4/576: dwrite d1/d26/d34/f76 [0,4194304] 0 2024-02-16T22:56:21.701 INFO:tasks.workunit.client.1.smithi192.stdout:5/466: symlink d6/d8/d75/d8c/l9c 0 2024-02-16T22:56:21.703 INFO:tasks.workunit.client.1.smithi192.stdout:5/467: dread - d6/d12/d2d/d5c/f99 zero size 2024-02-16T22:56:21.703 INFO:tasks.workunit.client.0.smithi064.stdout:8/374: dwrite df/d31/d5e/f64 [0,4194304] 0 2024-02-16T22:56:21.703 INFO:tasks.workunit.client.0.smithi064.stdout:5/308: symlink d6/d1c/l6b 0 2024-02-16T22:56:21.703 INFO:tasks.workunit.client.0.smithi064.stdout:5/309: chown d6/f31 668669 1 2024-02-16T22:56:21.704 INFO:tasks.workunit.client.0.smithi064.stdout:2/472: dread d7/d14/d25/d37/d44/f5c [0,4194304] 0 2024-02-16T22:56:21.705 INFO:tasks.workunit.client.1.smithi192.stdout:0/496: creat d1/d3/d29/d43/d9c/f9d x:0 0 0 2024-02-16T22:56:21.711 INFO:tasks.workunit.client.1.smithi192.stdout:4/577: dwrite d1/d15/d48/d8d/fba [0,4194304] 0 2024-02-16T22:56:21.711 INFO:tasks.workunit.client.1.smithi192.stdout:7/573: mknod d1/d2/de/d83/d9f/ccc 0 2024-02-16T22:56:21.713 INFO:tasks.workunit.client.1.smithi192.stdout:4/578: write d1/d15/d48/d8d/f9a [800043,75632] 0 2024-02-16T22:56:21.713 INFO:tasks.workunit.client.1.smithi192.stdout:3/501: chown da/d11/d20/d39/l90 5608 1 2024-02-16T22:56:21.714 INFO:tasks.workunit.client.0.smithi064.stdout:7/355: chown d5/d33/c36 2579 1 2024-02-16T22:56:21.715 INFO:tasks.workunit.client.0.smithi064.stdout:7/356: fsync d5/d33/f60 0 2024-02-16T22:56:21.716 INFO:tasks.workunit.client.0.smithi064.stdout:7/357: read - d5/d33/d42/f4f zero size 2024-02-16T22:56:21.718 INFO:tasks.workunit.client.1.smithi192.stdout:1/471: link d3/f46 d3/d40/fa1 0 2024-02-16T22:56:21.719 INFO:tasks.workunit.client.0.smithi064.stdout:3/472: creat d4/d1f/d5b/d51/fa4 x:0 0 0 2024-02-16T22:56:21.720 INFO:tasks.workunit.client.1.smithi192.stdout:6/485: creat da/df/d6b/d74/f9d x:0 0 0 2024-02-16T22:56:21.721 INFO:tasks.workunit.client.0.smithi064.stdout:8/375: dread - df/d2a/f60 zero size 2024-02-16T22:56:21.722 INFO:tasks.workunit.client.1.smithi192.stdout:1/472: dread d3/d5b/d77/f88 [0,4194304] 0 2024-02-16T22:56:21.723 INFO:tasks.workunit.client.1.smithi192.stdout:1/473: fsync d3/de/d27/f4a 0 2024-02-16T22:56:21.726 INFO:tasks.workunit.client.0.smithi064.stdout:7/358: dwrite d5/d33/f3d [8388608,4194304] 0 2024-02-16T22:56:21.727 INFO:tasks.workunit.client.0.smithi064.stdout:5/310: rename d6/d18/f5a to d6/d25/d48/d67/f6c 0 2024-02-16T22:56:21.729 INFO:tasks.workunit.client.1.smithi192.stdout:5/468: dread d6/d12/d1d/f34 [0,4194304] 0 2024-02-16T22:56:21.730 INFO:tasks.workunit.client.0.smithi064.stdout:7/359: dread d5/d6/fa [0,4194304] 0 2024-02-16T22:56:21.735 INFO:tasks.workunit.client.1.smithi192.stdout:7/574: mkdir d1/d2/d2d/d92/dcd 0 2024-02-16T22:56:21.736 INFO:tasks.workunit.client.0.smithi064.stdout:1/363: symlink d2/df/d11/d1a/l7f 0 2024-02-16T22:56:21.739 INFO:tasks.workunit.client.1.smithi192.stdout:9/609: write d7/d15/f58 [208721,17360] 0 2024-02-16T22:56:21.740 INFO:tasks.workunit.client.1.smithi192.stdout:2/533: creat d22/d24/d39/d42/d52/fb5 x:0 0 0 2024-02-16T22:56:21.742 INFO:tasks.workunit.client.0.smithi064.stdout:4/403: write dc/de/f75 [624381,28990] 0 2024-02-16T22:56:21.744 INFO:tasks.workunit.client.1.smithi192.stdout:7/575: dwrite d1/d2/de/d2c/d49/da4/fa3 [0,4194304] 0 2024-02-16T22:56:21.751 INFO:tasks.workunit.client.0.smithi064.stdout:7/360: mknod d5/d33/d53/c71 0 2024-02-16T22:56:21.751 INFO:tasks.workunit.client.1.smithi192.stdout:1/474: creat d3/de/d27/d34/d52/d5a/fa2 x:0 0 0 2024-02-16T22:56:21.752 INFO:tasks.workunit.client.1.smithi192.stdout:2/534: dwrite d22/d24/d39/d42/d65/d66/f83 [0,4194304] 0 2024-02-16T22:56:21.753 INFO:tasks.workunit.client.1.smithi192.stdout:3/502: dread da/d11/d2b/f5f [0,4194304] 0 2024-02-16T22:56:21.753 INFO:tasks.workunit.client.0.smithi064.stdout:6/450: dwrite d0/d6/d10/f5e [0,4194304] 0 2024-02-16T22:56:21.756 INFO:tasks.workunit.client.0.smithi064.stdout:3/473: readlink d4/d13/d1b/d48/l70 0 2024-02-16T22:56:21.758 INFO:tasks.workunit.client.0.smithi064.stdout:3/474: write d4/d13/d1b/d27/d99/f87 [4305168,78296] 0 2024-02-16T22:56:21.759 INFO:tasks.workunit.client.0.smithi064.stdout:2/473: link d7/d14/d1e/d34/l8b d7/d14/d25/l8f 0 2024-02-16T22:56:21.761 INFO:tasks.workunit.client.0.smithi064.stdout:4/404: rmdir dc/dd/d32/d46 39 2024-02-16T22:56:21.764 INFO:tasks.workunit.client.0.smithi064.stdout:8/376: mkdir df/d31/d61/d79/d81 0 2024-02-16T22:56:21.764 INFO:tasks.workunit.client.1.smithi192.stdout:2/535: dwrite d22/d24/d39/d42/f5f [0,4194304] 0 2024-02-16T22:56:21.766 INFO:tasks.workunit.client.1.smithi192.stdout:9/610: unlink d7/d15/d36/d77/cb7 0 2024-02-16T22:56:21.770 INFO:tasks.workunit.client.0.smithi064.stdout:4/405: dwrite dc/dd/d16/f5d [0,4194304] 0 2024-02-16T22:56:21.771 INFO:tasks.workunit.client.1.smithi192.stdout:9/611: dwrite d7/d19/d28/f8b [0,4194304] 0 2024-02-16T22:56:21.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/612: write f1 [2760634,97391] 0 2024-02-16T22:56:21.774 INFO:tasks.workunit.client.1.smithi192.stdout:4/579: truncate d1/d26/d66/d89/d11/d58/f7e 1037028 0 2024-02-16T22:56:21.774 INFO:tasks.workunit.client.1.smithi192.stdout:9/613: write d7/d15/d36/d54/f65 [857332,807] 0 2024-02-16T22:56:21.774 INFO:tasks.workunit.client.0.smithi064.stdout:8/377: dwrite df/d31/d5e/f64 [0,4194304] 0 2024-02-16T22:56:21.781 INFO:tasks.workunit.client.1.smithi192.stdout:9/614: dread d7/d15/f33 [4194304,4194304] 0 2024-02-16T22:56:21.782 INFO:tasks.workunit.client.0.smithi064.stdout:7/361: mknod d5/d6/d54/c72 0 2024-02-16T22:56:21.783 INFO:tasks.workunit.client.1.smithi192.stdout:7/576: creat d1/d69/fce x:0 0 0 2024-02-16T22:56:21.784 INFO:tasks.workunit.client.0.smithi064.stdout:6/451: creat d0/d12/f8b x:0 0 0 2024-02-16T22:56:21.786 INFO:tasks.workunit.client.0.smithi064.stdout:1/364: getdents d2/df/d14/d2a 0 2024-02-16T22:56:21.787 INFO:tasks.workunit.client.1.smithi192.stdout:9/615: read d7/d15/f5d [1945422,99799] 0 2024-02-16T22:56:21.788 INFO:tasks.workunit.client.1.smithi192.stdout:5/469: write d6/f33 [1345659,7253] 0 2024-02-16T22:56:21.789 INFO:tasks.workunit.client.1.smithi192.stdout:0/497: truncate d1/f3e 1456762 0 2024-02-16T22:56:21.790 INFO:tasks.workunit.client.1.smithi192.stdout:1/475: mknod d3/de/d27/d34/d52/d72/ca3 0 2024-02-16T22:56:21.791 INFO:tasks.workunit.client.0.smithi064.stdout:3/475: fsync d4/d1f/f20 0 2024-02-16T22:56:21.796 INFO:tasks.workunit.client.0.smithi064.stdout:5/311: link d6/d25/c28 d6/d18/d39/c6d 0 2024-02-16T22:56:21.799 INFO:tasks.workunit.client.0.smithi064.stdout:8/378: unlink df/f74 0 2024-02-16T22:56:21.800 INFO:tasks.workunit.client.0.smithi064.stdout:8/379: write df/d31/d5e/f64 [1780216,15063] 0 2024-02-16T22:56:21.802 INFO:tasks.workunit.client.0.smithi064.stdout:7/362: rmdir d5/d33/d45/d64 39 2024-02-16T22:56:21.803 INFO:tasks.workunit.client.0.smithi064.stdout:7/363: chown d5/d6/f43 52684592 1 2024-02-16T22:56:21.807 INFO:tasks.workunit.client.0.smithi064.stdout:8/380: dwrite df/f1a [0,4194304] 0 2024-02-16T22:56:21.808 INFO:tasks.workunit.client.0.smithi064.stdout:8/381: write df/d31/d2f/f59 [304758,97053] 0 2024-02-16T22:56:21.809 INFO:tasks.workunit.client.1.smithi192.stdout:7/577: creat d1/d2/de/d2c/d49/da4/fcf x:0 0 0 2024-02-16T22:56:21.810 INFO:tasks.workunit.client.0.smithi064.stdout:8/382: truncate df/d31/f3f 5203882 0 2024-02-16T22:56:21.810 INFO:tasks.workunit.client.1.smithi192.stdout:7/578: write d1/d2/de/d83/f8d [1866588,87860] 0 2024-02-16T22:56:21.812 INFO:tasks.workunit.client.0.smithi064.stdout:1/365: dread d2/f42 [0,4194304] 0 2024-02-16T22:56:21.815 INFO:tasks.workunit.client.0.smithi064.stdout:3/476: mknod d4/d13/d1b/d27/d44/ca5 0 2024-02-16T22:56:21.816 INFO:tasks.workunit.client.1.smithi192.stdout:4/580: rmdir d1/d26/d66/d89/d11 39 2024-02-16T22:56:21.821 INFO:tasks.workunit.client.1.smithi192.stdout:9/616: rename d7/d15/d36/d77/fb2 to d7/d15/d36/fd6 0 2024-02-16T22:56:21.822 INFO:tasks.workunit.client.1.smithi192.stdout:9/617: dread - d7/d19/f86 zero size 2024-02-16T22:56:21.822 INFO:tasks.workunit.client.0.smithi064.stdout:3/477: dwrite d4/d13/d41/f8d [0,4194304] 0 2024-02-16T22:56:21.823 INFO:tasks.workunit.client.1.smithi192.stdout:4/581: dwrite d1/d26/faf [0,4194304] 0 2024-02-16T22:56:21.829 INFO:tasks.workunit.client.0.smithi064.stdout:6/452: mknod d0/d1b/c8c 0 2024-02-16T22:56:21.829 INFO:tasks.workunit.client.0.smithi064.stdout:9/374: sync 2024-02-16T22:56:21.829 INFO:tasks.workunit.client.0.smithi064.stdout:0/342: sync 2024-02-16T22:56:21.832 INFO:tasks.workunit.client.0.smithi064.stdout:9/375: dread - d3/d11/d15/d5a/f6d zero size 2024-02-16T22:56:21.835 INFO:tasks.workunit.client.1.smithi192.stdout:7/579: mkdir d1/d2/d2d/d61/d88/d90/d2a/dd0 0 2024-02-16T22:56:21.835 INFO:tasks.workunit.client.1.smithi192.stdout:7/580: read d1/d2/de/f8f [6827,75927] 0 2024-02-16T22:56:21.836 INFO:tasks.workunit.client.1.smithi192.stdout:6/486: link da/df/c32 da/df/d12/d57/d81/c9e 0 2024-02-16T22:56:21.837 INFO:tasks.workunit.client.1.smithi192.stdout:6/487: write da/f5f [973304,116846] 0 2024-02-16T22:56:21.841 INFO:tasks.workunit.client.0.smithi064.stdout:5/312: unlink d6/d1c/c44 0 2024-02-16T22:56:21.843 INFO:tasks.workunit.client.0.smithi064.stdout:3/478: mkdir d4/d1f/d34/da6 0 2024-02-16T22:56:21.844 INFO:tasks.workunit.client.1.smithi192.stdout:8/484: sync 2024-02-16T22:56:21.844 INFO:tasks.workunit.client.0.smithi064.stdout:0/343: mknod da/dc/de/d37/c6d 0 2024-02-16T22:56:21.844 INFO:tasks.workunit.client.0.smithi064.stdout:0/344: chown da/dc/de/d12/f27 1978 1 2024-02-16T22:56:21.845 INFO:tasks.workunit.client.1.smithi192.stdout:4/582: creat d1/d26/d66/d89/da8/da9/fce x:0 0 0 2024-02-16T22:56:21.848 INFO:tasks.workunit.client.1.smithi192.stdout:2/536: rmdir d22/d24/d39/d42/d84/d89 0 2024-02-16T22:56:21.849 INFO:tasks.workunit.client.0.smithi064.stdout:7/364: mkdir d5/d33/d45/d73 0 2024-02-16T22:56:21.849 INFO:tasks.workunit.client.0.smithi064.stdout:7/365: chown d5/d33 1679878 1 2024-02-16T22:56:21.849 INFO:tasks.workunit.client.1.smithi192.stdout:8/485: dwrite d5/db/d47/f92 [0,4194304] 0 2024-02-16T22:56:21.851 INFO:tasks.workunit.client.0.smithi064.stdout:8/383: creat df/d31/d61/d79/d81/f82 x:0 0 0 2024-02-16T22:56:21.853 INFO:tasks.workunit.client.0.smithi064.stdout:4/406: link dc/dd/d32/l3f dc/dd/d32/d58/l78 0 2024-02-16T22:56:21.857 INFO:tasks.workunit.client.0.smithi064.stdout:5/313: creat d6/d1c/d4b/f6e x:0 0 0 2024-02-16T22:56:21.860 INFO:tasks.workunit.client.1.smithi192.stdout:5/470: creat d6/f9d x:0 0 0 2024-02-16T22:56:21.861 INFO:tasks.workunit.client.1.smithi192.stdout:5/471: dread - d6/d12/d4e/d46/f57 zero size 2024-02-16T22:56:21.863 INFO:tasks.workunit.client.0.smithi064.stdout:7/366: rename d5/f2f to d5/d33/d42/f74 0 2024-02-16T22:56:21.864 INFO:tasks.workunit.client.1.smithi192.stdout:4/583: unlink d1/d15/f45 0 2024-02-16T22:56:21.865 INFO:tasks.workunit.client.1.smithi192.stdout:4/584: write d1/d26/d66/d89/da8/fb8 [111263,55966] 0 2024-02-16T22:56:21.866 INFO:tasks.workunit.client.0.smithi064.stdout:8/384: creat df/d31/d2f/d3e/f83 x:0 0 0 2024-02-16T22:56:21.866 INFO:tasks.workunit.client.0.smithi064.stdout:8/385: write df/d31/d2f/d4f/f66 [2379486,26167] 0 2024-02-16T22:56:21.872 INFO:tasks.workunit.client.1.smithi192.stdout:8/486: creat d5/d26/d83/d86/f93 x:0 0 0 2024-02-16T22:56:21.876 INFO:tasks.workunit.client.1.smithi192.stdout:7/581: symlink d1/ld1 0 2024-02-16T22:56:21.876 INFO:tasks.workunit.client.1.smithi192.stdout:7/582: chown d1/d2/de/d2c/d49/f80 697 1 2024-02-16T22:56:21.877 INFO:tasks.workunit.client.1.smithi192.stdout:7/583: fdatasync d1/d2/f46 0 2024-02-16T22:56:21.877 INFO:tasks.workunit.client.0.smithi064.stdout:6/453: dwrite d0/d6/d10/d33/f2a [0,4194304] 0 2024-02-16T22:56:21.877 INFO:tasks.workunit.client.1.smithi192.stdout:7/584: read - d1/d2/d37/fc3 zero size 2024-02-16T22:56:21.879 INFO:tasks.workunit.client.0.smithi064.stdout:8/386: readlink df/d31/d2f/d38/d48/l56 0 2024-02-16T22:56:21.881 INFO:tasks.workunit.client.1.smithi192.stdout:0/498: rename d1/d3/de/d22/d37/d67/d34/c99 to d1/d3/de/c9e 0 2024-02-16T22:56:21.882 INFO:tasks.workunit.client.0.smithi064.stdout:4/407: rmdir dc/d4a 39 2024-02-16T22:56:21.888 INFO:tasks.workunit.client.0.smithi064.stdout:3/479: creat d4/d13/fa7 x:0 0 0 2024-02-16T22:56:21.888 INFO:tasks.workunit.client.1.smithi192.stdout:9/618: link d7/d19/l2b d7/d15/d36/d38/d3f/d8a/d98/d6e/ld7 0 2024-02-16T22:56:21.889 INFO:tasks.workunit.client.1.smithi192.stdout:9/619: chown d7/d15/d36/d38/d3f/fc8 120049 1 2024-02-16T22:56:21.893 INFO:tasks.workunit.client.0.smithi064.stdout:5/314: dwrite d6/d18/d19/d1a/f3b [0,4194304] 0 2024-02-16T22:56:21.893 INFO:tasks.workunit.client.0.smithi064.stdout:6/454: creat d0/d5/df/f8d x:0 0 0 2024-02-16T22:56:21.898 INFO:tasks.workunit.client.0.smithi064.stdout:0/345: getdents da 0 2024-02-16T22:56:21.899 INFO:tasks.workunit.client.0.smithi064.stdout:0/346: chown da/dc 868727 1 2024-02-16T22:56:21.900 INFO:tasks.workunit.client.0.smithi064.stdout:0/347: fsync da/dc/de/d12/f49 0 2024-02-16T22:56:21.902 INFO:tasks.workunit.client.1.smithi192.stdout:0/499: dwrite d1/f7a [0,4194304] 0 2024-02-16T22:56:21.905 INFO:tasks.workunit.client.0.smithi064.stdout:6/455: unlink d0/d1b/f50 0 2024-02-16T22:56:21.906 INFO:tasks.workunit.client.0.smithi064.stdout:6/456: fdatasync d0/d5/df/f79 0 2024-02-16T22:56:21.908 INFO:tasks.workunit.client.0.smithi064.stdout:8/387: creat df/d31/d2f/d38/d48/d69/d80/f84 x:0 0 0 2024-02-16T22:56:21.908 INFO:tasks.workunit.client.0.smithi064.stdout:7/367: write f0 [1004133,53354] 0 2024-02-16T22:56:21.908 INFO:tasks.workunit.client.1.smithi192.stdout:9/620: read d7/f83 [157180,8100] 0 2024-02-16T22:56:21.909 INFO:tasks.workunit.client.1.smithi192.stdout:7/585: creat d1/d2/d2d/d61/d88/d90/d2a/dd0/fd2 x:0 0 0 2024-02-16T22:56:21.910 INFO:tasks.workunit.client.1.smithi192.stdout:7/586: dread - d1/d2/de/d83/d9f/fad zero size 2024-02-16T22:56:21.911 INFO:tasks.workunit.client.0.smithi064.stdout:5/315: rename d6/d18/d39/l3d to d6/d25/d48/d67/l6f 0 2024-02-16T22:56:21.912 INFO:tasks.workunit.client.1.smithi192.stdout:7/587: fsync d1/d2/d2d/d61/d88/d90/d2a/f3c 0 2024-02-16T22:56:21.913 INFO:tasks.workunit.client.0.smithi064.stdout:4/408: mkdir dc/dd/d32/d46/d79 0 2024-02-16T22:56:21.914 INFO:tasks.workunit.client.1.smithi192.stdout:5/472: truncate d6/d12/f83 343785 0 2024-02-16T22:56:21.914 INFO:tasks.workunit.client.0.smithi064.stdout:8/388: dwrite df/d31/f7c [0,4194304] 0 2024-02-16T22:56:21.915 INFO:tasks.workunit.client.1.smithi192.stdout:0/500: dread - d1/d3/d39/f5f zero size 2024-02-16T22:56:21.921 INFO:tasks.workunit.client.1.smithi192.stdout:6/488: rename da/df/d18/d52/f55 to da/df/d18/f9f 0 2024-02-16T22:56:21.921 INFO:tasks.workunit.client.1.smithi192.stdout:6/489: fsync da/df/d12/f47 0 2024-02-16T22:56:21.922 INFO:tasks.workunit.client.1.smithi192.stdout:6/490: readlink da/l22 0 2024-02-16T22:56:21.924 INFO:tasks.workunit.client.1.smithi192.stdout:7/588: creat d1/d2/d2d/d55/fd3 x:0 0 0 2024-02-16T22:56:21.924 INFO:tasks.workunit.client.1.smithi192.stdout:5/473: creat d6/d8/f9e x:0 0 0 2024-02-16T22:56:21.925 INFO:tasks.workunit.client.1.smithi192.stdout:0/501: dread - d1/d3/de/d22/d37/d67/d34/f56 zero size 2024-02-16T22:56:21.926 INFO:tasks.workunit.client.1.smithi192.stdout:8/487: getdents d5 0 2024-02-16T22:56:21.928 INFO:tasks.workunit.client.1.smithi192.stdout:7/589: unlink d1/d2/de/d2c/d49/da4/f4f 0 2024-02-16T22:56:21.928 INFO:tasks.workunit.client.1.smithi192.stdout:6/491: creat da/df/d12/fa0 x:0 0 0 2024-02-16T22:56:21.929 INFO:tasks.workunit.client.1.smithi192.stdout:0/502: mknod d1/c9f 0 2024-02-16T22:56:21.929 INFO:tasks.workunit.client.1.smithi192.stdout:6/492: fsync da/df/d12/d15/d1b/f60 0 2024-02-16T22:56:21.930 INFO:tasks.workunit.client.1.smithi192.stdout:0/503: readlink d1/d3/d18/l5a 0 2024-02-16T22:56:21.930 INFO:tasks.workunit.client.1.smithi192.stdout:7/590: chown d1/d2/c18 3773880 1 2024-02-16T22:56:21.932 INFO:tasks.workunit.client.1.smithi192.stdout:6/493: symlink da/la1 0 2024-02-16T22:56:21.932 INFO:tasks.workunit.client.1.smithi192.stdout:0/504: truncate d1/d3/d29/d43/f8b 84069 0 2024-02-16T22:56:21.932 INFO:tasks.workunit.client.1.smithi192.stdout:7/591: mknod d1/d2/d2d/d61/d8a/cd4 0 2024-02-16T22:56:21.933 INFO:tasks.workunit.client.1.smithi192.stdout:7/592: write d1/fac [696123,13566] 0 2024-02-16T22:56:21.933 INFO:tasks.workunit.client.1.smithi192.stdout:6/494: symlink da/df/d6b/d83/la2 0 2024-02-16T22:56:21.936 INFO:tasks.workunit.client.1.smithi192.stdout:6/495: creat da/df/d12/d15/d1b/fa3 x:0 0 0 2024-02-16T22:56:21.937 INFO:tasks.workunit.client.1.smithi192.stdout:7/593: symlink d1/d2/d2d/d61/d88/d90/db2/ld5 0 2024-02-16T22:56:21.938 INFO:tasks.workunit.client.1.smithi192.stdout:6/496: creat da/df/d18/d46/fa4 x:0 0 0 2024-02-16T22:56:21.938 INFO:tasks.workunit.client.1.smithi192.stdout:7/594: symlink d1/d2/d2d/d55/ld6 0 2024-02-16T22:56:21.939 INFO:tasks.workunit.client.1.smithi192.stdout:6/497: dread - da/df/d6b/d74/f8b zero size 2024-02-16T22:56:21.941 INFO:tasks.workunit.client.1.smithi192.stdout:6/498: unlink da/df/d12/d57/f3a 0 2024-02-16T22:56:21.942 INFO:tasks.workunit.client.1.smithi192.stdout:6/499: chown da/df/d12/d15 3893126 1 2024-02-16T22:56:21.942 INFO:tasks.workunit.client.1.smithi192.stdout:7/595: truncate d1/d2/d2d/d61/d88/d90/d2a/f7c 338548 0 2024-02-16T22:56:21.948 INFO:tasks.workunit.client.1.smithi192.stdout:7/596: dwrite d1/d2/de/f20 [0,4194304] 0 2024-02-16T22:56:21.950 INFO:tasks.workunit.client.1.smithi192.stdout:7/597: creat d1/d2/de/d2c/d49/fd7 x:0 0 0 2024-02-16T22:56:21.971 INFO:tasks.workunit.client.0.smithi064.stdout:3/480: symlink d4/d13/d1b/d27/d61/la8 0 2024-02-16T22:56:21.979 INFO:tasks.workunit.client.0.smithi064.stdout:0/348: dread da/dc/f4b [0,4194304] 0 2024-02-16T22:56:21.996 INFO:tasks.workunit.client.1.smithi192.stdout:9/621: write d7/f83 [887465,100057] 0 2024-02-16T22:56:21.997 INFO:tasks.workunit.client.1.smithi192.stdout:9/622: write d7/d19/d5e/d81/fd0 [726165,25661] 0 2024-02-16T22:56:21.998 INFO:tasks.workunit.client.0.smithi064.stdout:5/316: fsync d6/fc 0 2024-02-16T22:56:22.001 INFO:tasks.workunit.client.0.smithi064.stdout:7/368: dwrite d5/d6/d54/d41/f4b [0,4194304] 0 2024-02-16T22:56:22.002 INFO:tasks.workunit.client.0.smithi064.stdout:7/369: read d5/d6/d54/d41/f4b [1667656,2556] 0 2024-02-16T22:56:22.004 INFO:tasks.workunit.client.0.smithi064.stdout:7/370: stat d5/d6/d54/l51 0 2024-02-16T22:56:22.006 INFO:tasks.workunit.client.0.smithi064.stdout:4/409: write f3 [2470296,2289] 0 2024-02-16T22:56:22.006 INFO:tasks.workunit.client.1.smithi192.stdout:9/623: rename d7/d19/d37/d39/c51 to d7/d19/d37/d5c/cd8 0 2024-02-16T22:56:22.007 INFO:tasks.workunit.client.0.smithi064.stdout:0/349: dread da/f5c [4194304,4194304] 0 2024-02-16T22:56:22.010 INFO:tasks.workunit.client.1.smithi192.stdout:8/488: dread d5/d16/f45 [0,4194304] 0 2024-02-16T22:56:22.010 INFO:tasks.workunit.client.1.smithi192.stdout:3/503: sync 2024-02-16T22:56:22.012 INFO:tasks.workunit.client.1.smithi192.stdout:8/489: readlink d5/db/d47/l40 0 2024-02-16T22:56:22.018 INFO:tasks.workunit.client.1.smithi192.stdout:5/474: dwrite d6/d12/f44 [0,4194304] 0 2024-02-16T22:56:22.018 INFO:tasks.workunit.client.1.smithi192.stdout:3/504: creat da/d11/d20/d39/fad x:0 0 0 2024-02-16T22:56:22.019 INFO:tasks.workunit.client.0.smithi064.stdout:7/371: creat d5/d33/d53/f75 x:0 0 0 2024-02-16T22:56:22.019 INFO:tasks.workunit.client.1.smithi192.stdout:5/475: chown d6/d12/d1d/l25 11 1 2024-02-16T22:56:22.020 INFO:tasks.workunit.client.0.smithi064.stdout:7/372: fsync d5/d33/d53/f70 0 2024-02-16T22:56:22.023 INFO:tasks.workunit.client.0.smithi064.stdout:8/389: symlink df/d2a/d5c/l85 0 2024-02-16T22:56:22.026 INFO:tasks.workunit.client.1.smithi192.stdout:0/505: dwrite d1/d3/d29/f70 [0,4194304] 0 2024-02-16T22:56:22.026 INFO:tasks.workunit.client.0.smithi064.stdout:4/410: mkdir dc/d1d/d43/d7a 0 2024-02-16T22:56:22.029 INFO:tasks.workunit.client.0.smithi064.stdout:4/411: write dc/f6c [611654,1665] 0 2024-02-16T22:56:22.029 INFO:tasks.workunit.client.0.smithi064.stdout:7/373: dwrite d5/d6/f1f [0,4194304] 0 2024-02-16T22:56:22.030 INFO:tasks.workunit.client.1.smithi192.stdout:3/505: rename da/d11/d20/f8b to da/d11/d63/fae 0 2024-02-16T22:56:22.032 INFO:tasks.workunit.client.0.smithi064.stdout:7/374: truncate d5/d6/d54/d41/f4a 834020 0 2024-02-16T22:56:22.036 INFO:tasks.workunit.client.1.smithi192.stdout:5/476: read d6/d12/d1d/f3a [797468,55374] 0 2024-02-16T22:56:22.039 INFO:tasks.workunit.client.0.smithi064.stdout:0/350: write da/dc/f1e [3593542,105182] 0 2024-02-16T22:56:22.041 INFO:tasks.workunit.client.1.smithi192.stdout:7/598: write d1/d2/d2d/d61/d88/d90/f78 [2422364,56569] 0 2024-02-16T22:56:22.044 INFO:tasks.workunit.client.1.smithi192.stdout:3/506: mknod da/d11/d20/d39/caf 0 2024-02-16T22:56:22.048 INFO:tasks.workunit.client.1.smithi192.stdout:7/599: write d1/d2/de/d2c/d49/d54/d9d/fa9 [401160,23856] 0 2024-02-16T22:56:22.050 INFO:tasks.workunit.client.1.smithi192.stdout:0/506: fsync d1/d3/de/d22/d37/d67/f66 0 2024-02-16T22:56:22.053 INFO:tasks.workunit.client.1.smithi192.stdout:3/507: rename da/f99 to da/d11/d63/d6b/d71/da1/fb0 0 2024-02-16T22:56:22.054 INFO:tasks.workunit.client.0.smithi064.stdout:3/481: getdents d4/d13/d1b/d4f 0 2024-02-16T22:56:22.055 INFO:tasks.workunit.client.0.smithi064.stdout:3/482: write d4/d13/d1b/d27/d99/f87 [366413,77176] 0 2024-02-16T22:56:22.056 INFO:tasks.workunit.client.0.smithi064.stdout:0/351: creat da/dc/de/d12/d2c/f6e x:0 0 0 2024-02-16T22:56:22.058 INFO:tasks.workunit.client.1.smithi192.stdout:7/600: mkdir d1/d2/de/d2c/d49/d54/dd8 0 2024-02-16T22:56:22.064 INFO:tasks.workunit.client.0.smithi064.stdout:6/457: dwrite d0/f41 [0,4194304] 0 2024-02-16T22:56:22.065 INFO:tasks.workunit.client.0.smithi064.stdout:7/375: dread d5/d6/f30 [0,4194304] 0 2024-02-16T22:56:22.066 INFO:tasks.workunit.client.0.smithi064.stdout:7/376: dread - d5/d6/d54/d41/f68 zero size 2024-02-16T22:56:22.067 INFO:tasks.workunit.client.0.smithi064.stdout:0/352: mknod da/c6f 0 2024-02-16T22:56:22.073 INFO:tasks.workunit.client.0.smithi064.stdout:3/483: symlink d4/d13/d1b/d27/d61/la9 0 2024-02-16T22:56:22.077 INFO:tasks.workunit.client.1.smithi192.stdout:3/508: symlink da/d6a/lb1 0 2024-02-16T22:56:22.078 INFO:tasks.workunit.client.1.smithi192.stdout:9/624: dwrite d7/d19/d37/d39/f44 [0,4194304] 0 2024-02-16T22:56:22.080 INFO:tasks.workunit.client.1.smithi192.stdout:5/477: link d6/d8/d75/c7d d6/d8/d75/c9f 0 2024-02-16T22:56:22.084 INFO:tasks.workunit.client.1.smithi192.stdout:1/476: sync 2024-02-16T22:56:22.085 INFO:tasks.workunit.client.1.smithi192.stdout:0/507: creat d1/d3/de/d22/d37/fa0 x:0 0 0 2024-02-16T22:56:22.086 INFO:tasks.workunit.client.1.smithi192.stdout:8/490: dwrite d5/d26/d83/d86/d5f/f8d [0,4194304] 0 2024-02-16T22:56:22.086 INFO:tasks.workunit.client.1.smithi192.stdout:5/478: dread d6/d12/f44 [0,4194304] 0 2024-02-16T22:56:22.087 INFO:tasks.workunit.client.0.smithi064.stdout:7/377: rename d5/d6/c28 to d5/d6/d54/c76 0 2024-02-16T22:56:22.088 INFO:tasks.workunit.client.0.smithi064.stdout:7/378: read d5/d6/fb [3917823,89382] 0 2024-02-16T22:56:22.090 INFO:tasks.workunit.client.0.smithi064.stdout:5/317: truncate d6/d1c/f62 4059875 0 2024-02-16T22:56:22.090 INFO:tasks.workunit.client.0.smithi064.stdout:5/318: chown d6/d25/l50 0 1 2024-02-16T22:56:22.095 INFO:tasks.workunit.client.1.smithi192.stdout:3/509: dread - da/d11/d63/d6b/d4e/f6d zero size 2024-02-16T22:56:22.096 INFO:tasks.workunit.client.0.smithi064.stdout:8/390: write df/f4c [71021,75701] 0 2024-02-16T22:56:22.096 INFO:tasks.workunit.client.0.smithi064.stdout:5/319: read d6/fa [155621,79206] 0 2024-02-16T22:56:22.097 INFO:tasks.workunit.client.0.smithi064.stdout:2/474: sync 2024-02-16T22:56:22.099 INFO:tasks.workunit.client.0.smithi064.stdout:7/379: rmdir d5/d6/d65 39 2024-02-16T22:56:22.100 INFO:tasks.workunit.client.0.smithi064.stdout:4/412: write dc/dd/d32/d22/f56 [1697658,129073] 0 2024-02-16T22:56:22.101 INFO:tasks.workunit.client.1.smithi192.stdout:0/508: write d1/f51 [2477642,17833] 0 2024-02-16T22:56:22.102 INFO:tasks.workunit.client.0.smithi064.stdout:6/458: getdents d0/d6/d10/d33 0 2024-02-16T22:56:22.105 INFO:tasks.workunit.client.0.smithi064.stdout:4/413: write dc/dd/d32/d58/f38 [494779,19361] 0 2024-02-16T22:56:22.105 INFO:tasks.workunit.client.1.smithi192.stdout:8/491: chown d5/d26/d83/d86/d5f/f5e 0 1 2024-02-16T22:56:22.107 INFO:tasks.workunit.client.0.smithi064.stdout:7/380: dwrite d5/d6/f3b [4194304,4194304] 0 2024-02-16T22:56:22.107 INFO:tasks.workunit.client.0.smithi064.stdout:4/414: write dc/dd/d16/f2e [3612605,118531] 0 2024-02-16T22:56:22.108 INFO:tasks.workunit.client.1.smithi192.stdout:5/479: symlink d6/d12/d2d/d5c/la0 0 2024-02-16T22:56:22.108 INFO:tasks.workunit.client.0.smithi064.stdout:7/381: chown d5/d33/d53 3509973 1 2024-02-16T22:56:22.109 INFO:tasks.workunit.client.0.smithi064.stdout:4/415: chown dc/dd/d32/d58 1357862447 1 2024-02-16T22:56:22.109 INFO:tasks.workunit.client.0.smithi064.stdout:8/391: unlink df/d31/d2f/c35 0 2024-02-16T22:56:22.111 INFO:tasks.workunit.client.1.smithi192.stdout:0/509: dwrite d1/d3/de/d22/d37/d67/d75/f8e [0,4194304] 0 2024-02-16T22:56:22.112 INFO:tasks.workunit.client.0.smithi064.stdout:8/392: write df/d31/d2f/d3e/f4a [932591,2781] 0 2024-02-16T22:56:22.112 INFO:tasks.workunit.client.0.smithi064.stdout:5/320: readlink d6/d25/d48/d67/l6f 0 2024-02-16T22:56:22.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/480: read d6/d12/d2d/d5c/f65 [1537287,67109] 0 2024-02-16T22:56:22.114 INFO:tasks.workunit.client.1.smithi192.stdout:3/510: mknod da/d11/d20/cb2 0 2024-02-16T22:56:22.114 INFO:tasks.workunit.client.1.smithi192.stdout:0/510: write d1/f7a [2442872,70867] 0 2024-02-16T22:56:22.115 INFO:tasks.workunit.client.0.smithi064.stdout:3/484: getdents d4/d1f/d75 0 2024-02-16T22:56:22.119 INFO:tasks.workunit.client.1.smithi192.stdout:0/511: dread d1/d3/d29/d43/f8b [0,4194304] 0 2024-02-16T22:56:22.119 INFO:tasks.workunit.client.1.smithi192.stdout:1/477: dread d3/f22 [0,4194304] 0 2024-02-16T22:56:22.120 INFO:tasks.workunit.client.1.smithi192.stdout:7/601: getdents d1/d2/de/d2c/d49/da1 0 2024-02-16T22:56:22.121 INFO:tasks.workunit.client.1.smithi192.stdout:3/511: dread - da/d11/d20/d39/d53/f93 zero size 2024-02-16T22:56:22.121 INFO:tasks.workunit.client.1.smithi192.stdout:9/625: dread d7/d19/d5e/d6f/f7d [0,4194304] 0 2024-02-16T22:56:22.122 INFO:tasks.workunit.client.1.smithi192.stdout:0/512: write d1/fd [6707505,18951] 0 2024-02-16T22:56:22.124 INFO:tasks.workunit.client.1.smithi192.stdout:8/492: symlink d5/d26/d83/d86/l94 0 2024-02-16T22:56:22.124 INFO:tasks.workunit.client.1.smithi192.stdout:0/513: fdatasync d1/d3/d39/f76 0 2024-02-16T22:56:22.126 INFO:tasks.workunit.client.1.smithi192.stdout:1/478: truncate d3/de/d27/d34/d52/d5a/f90 264565 0 2024-02-16T22:56:22.128 INFO:tasks.workunit.client.1.smithi192.stdout:8/493: truncate d5/db/de/df/f62 6182 0 2024-02-16T22:56:22.131 INFO:tasks.workunit.client.0.smithi064.stdout:1/366: sync 2024-02-16T22:56:22.131 INFO:tasks.workunit.client.0.smithi064.stdout:9/376: sync 2024-02-16T22:56:22.132 INFO:tasks.workunit.client.0.smithi064.stdout:7/382: mknod d5/d33/c77 0 2024-02-16T22:56:22.133 INFO:tasks.workunit.client.0.smithi064.stdout:7/383: stat d5/d6/f30 0 2024-02-16T22:56:22.134 INFO:tasks.workunit.client.0.smithi064.stdout:2/475: dread d7/d14/d3e/f75 [0,4194304] 0 2024-02-16T22:56:22.134 INFO:tasks.workunit.client.0.smithi064.stdout:4/416: creat dc/dd/d16/d6e/f7b x:0 0 0 2024-02-16T22:56:22.136 INFO:tasks.workunit.client.0.smithi064.stdout:8/393: symlink df/d31/d47/l86 0 2024-02-16T22:56:22.136 INFO:tasks.workunit.client.0.smithi064.stdout:2/476: chown d7/d14/d25/f52 0 1 2024-02-16T22:56:22.139 INFO:tasks.workunit.client.0.smithi064.stdout:3/485: symlink d4/d14/d2b/laa 0 2024-02-16T22:56:22.140 INFO:tasks.workunit.client.0.smithi064.stdout:0/353: symlink da/l70 0 2024-02-16T22:56:22.141 INFO:tasks.workunit.client.0.smithi064.stdout:7/384: dwrite d5/d6/d54/f63 [0,4194304] 0 2024-02-16T22:56:22.142 INFO:tasks.workunit.client.1.smithi192.stdout:3/512: dread - da/d11/d63/d6b/f47 zero size 2024-02-16T22:56:22.142 INFO:tasks.workunit.client.1.smithi192.stdout:6/500: sync 2024-02-16T22:56:22.142 INFO:tasks.workunit.client.1.smithi192.stdout:2/537: sync 2024-02-16T22:56:22.143 INFO:tasks.workunit.client.1.smithi192.stdout:4/585: sync 2024-02-16T22:56:22.143 INFO:tasks.workunit.client.1.smithi192.stdout:2/538: fdatasync d22/d24/d39/d42/d65/d69/f7a 0 2024-02-16T22:56:22.143 INFO:tasks.workunit.client.0.smithi064.stdout:3/486: write d4/f18 [3521140,64935] 0 2024-02-16T22:56:22.144 INFO:tasks.workunit.client.0.smithi064.stdout:6/459: rename d0/d85/f8a to d0/d89/f8e 0 2024-02-16T22:56:22.144 INFO:tasks.workunit.client.0.smithi064.stdout:7/385: chown d5/d33/c77 457 1 2024-02-16T22:56:22.144 INFO:tasks.workunit.client.1.smithi192.stdout:4/586: dread d1/d15/d48/d8d/d9b/fad [0,4194304] 0 2024-02-16T22:56:22.145 INFO:tasks.workunit.client.0.smithi064.stdout:3/487: dread - d4/d13/d1b/d27/d61/f66 zero size 2024-02-16T22:56:22.145 INFO:tasks.workunit.client.1.smithi192.stdout:4/587: fdatasync d1/d26/d66/fac 0 2024-02-16T22:56:22.146 INFO:tasks.workunit.client.0.smithi064.stdout:9/377: creat d3/d9/d1e/d42/f7c x:0 0 0 2024-02-16T22:56:22.146 INFO:tasks.workunit.client.0.smithi064.stdout:3/488: stat d4/f1e 0 2024-02-16T22:56:22.147 INFO:tasks.workunit.client.1.smithi192.stdout:4/588: write d1/d15/d23/f52 [3545569,91086] 0 2024-02-16T22:56:22.148 INFO:tasks.workunit.client.1.smithi192.stdout:4/589: truncate d1/d26/fa5 144351 0 2024-02-16T22:56:22.150 INFO:tasks.workunit.client.0.smithi064.stdout:9/378: readlink d3/d11/d15/d5a/d2a/d2e/l70 0 2024-02-16T22:56:22.150 INFO:tasks.workunit.client.1.smithi192.stdout:4/590: chown d1/d15/d48/d8d/c9d 70815 1 2024-02-16T22:56:22.151 INFO:tasks.workunit.client.0.smithi064.stdout:4/417: chown dc/d4a/l64 778283 1 2024-02-16T22:56:22.152 INFO:tasks.workunit.client.1.smithi192.stdout:8/494: mknod d5/d26/d83/d86/d5f/c95 0 2024-02-16T22:56:22.153 INFO:tasks.workunit.client.0.smithi064.stdout:8/394: rmdir df/d31/d2f/d38/d48 39 2024-02-16T22:56:22.153 INFO:tasks.workunit.client.0.smithi064.stdout:6/460: dwrite d0/d12/f74 [0,4194304] 0 2024-02-16T22:56:22.154 INFO:tasks.workunit.client.0.smithi064.stdout:9/379: stat d3/d11/d15/d5a/d2a/d2e/f59 0 2024-02-16T22:56:22.155 INFO:tasks.workunit.client.0.smithi064.stdout:9/380: fsync d3/d11/d15/d5a/f27 0 2024-02-16T22:56:22.156 INFO:tasks.workunit.client.1.smithi192.stdout:1/479: dwrite d3/de/d27/f3b [4194304,4194304] 0 2024-02-16T22:56:22.158 INFO:tasks.workunit.client.0.smithi064.stdout:5/321: dread d6/d25/f32 [0,4194304] 0 2024-02-16T22:56:22.158 INFO:tasks.workunit.client.1.smithi192.stdout:1/480: stat d3/de/d27/d34/d52/d5a/fa2 0 2024-02-16T22:56:22.162 INFO:tasks.workunit.client.0.smithi064.stdout:0/354: dread da/d26/f32 [0,4194304] 0 2024-02-16T22:56:22.167 INFO:tasks.workunit.client.1.smithi192.stdout:1/481: dwrite d3/de/d27/f39 [8388608,4194304] 0 2024-02-16T22:56:22.168 INFO:tasks.workunit.client.0.smithi064.stdout:5/322: dwrite d6/d18/f56 [0,4194304] 0 2024-02-16T22:56:22.170 INFO:tasks.workunit.client.0.smithi064.stdout:5/323: readlink d6/ld 0 2024-02-16T22:56:22.175 INFO:tasks.workunit.client.0.smithi064.stdout:4/418: mkdir dc/dd/d16/d7c 0 2024-02-16T22:56:22.177 INFO:tasks.workunit.client.1.smithi192.stdout:9/626: creat d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/da2/fd9 x:0 0 0 2024-02-16T22:56:22.179 INFO:tasks.workunit.client.0.smithi064.stdout:4/419: chown dc/dd/d32/d58/l30 0 1 2024-02-16T22:56:22.180 INFO:tasks.workunit.client.1.smithi192.stdout:4/591: dread d1/d15/d48/d8d/f7b [4194304,4194304] 0 2024-02-16T22:56:22.182 INFO:tasks.workunit.client.1.smithi192.stdout:7/602: write d1/d2/f29 [4290360,90305] 0 2024-02-16T22:56:22.182 INFO:tasks.workunit.client.0.smithi064.stdout:5/324: dwrite d6/d25/d48/f5e [0,4194304] 0 2024-02-16T22:56:22.188 INFO:tasks.workunit.client.0.smithi064.stdout:9/381: symlink d3/d11/d15/d5a/d38/l7d 0 2024-02-16T22:56:22.189 INFO:tasks.workunit.client.1.smithi192.stdout:6/501: mknod da/ca5 0 2024-02-16T22:56:22.190 INFO:tasks.workunit.client.1.smithi192.stdout:4/592: dwrite d1/d26/d66/d89/f97 [0,4194304] 0 2024-02-16T22:56:22.192 INFO:tasks.workunit.client.0.smithi064.stdout:5/325: dwrite d6/d18/d19/d1a/f3b [4194304,4194304] 0 2024-02-16T22:56:22.192 INFO:tasks.workunit.client.1.smithi192.stdout:6/502: write da/df/d12/d15/d1b/f60 [3652600,80481] 0 2024-02-16T22:56:22.193 INFO:tasks.workunit.client.0.smithi064.stdout:5/326: write d6/d18/d19/d1a/f69 [426598,83572] 0 2024-02-16T22:56:22.196 INFO:tasks.workunit.client.0.smithi064.stdout:2/477: write d7/d14/d25/d46/d55/f71 [4435105,122559] 0 2024-02-16T22:56:22.197 INFO:tasks.workunit.client.1.smithi192.stdout:2/539: mkdir d22/d24/d39/d42/d85/d98/db0/db6 0 2024-02-16T22:56:22.197 INFO:tasks.workunit.client.1.smithi192.stdout:6/503: chown da/df/d12/d15/l23 5228326 1 2024-02-16T22:56:22.199 INFO:tasks.workunit.client.0.smithi064.stdout:3/489: dread - d4/d13/d35/f3e zero size 2024-02-16T22:56:22.199 INFO:tasks.workunit.client.1.smithi192.stdout:6/504: chown da/df/d12/d15/d1b/c6f 523563 1 2024-02-16T22:56:22.204 INFO:tasks.workunit.client.1.smithi192.stdout:7/603: mkdir d1/d2/d2d/d61/d88/dd9 0 2024-02-16T22:56:22.205 INFO:tasks.workunit.client.0.smithi064.stdout:4/420: dread dc/dd/d16/f51 [0,4194304] 0 2024-02-16T22:56:22.206 INFO:tasks.workunit.client.0.smithi064.stdout:6/461: dread d0/d5/df/f6f [0,4194304] 0 2024-02-16T22:56:22.206 INFO:tasks.workunit.client.1.smithi192.stdout:5/481: dwrite d6/d20/f4d [0,4194304] 0 2024-02-16T22:56:22.206 INFO:tasks.workunit.client.0.smithi064.stdout:6/462: chown d0/d12 1 1 2024-02-16T22:56:22.207 INFO:tasks.workunit.client.1.smithi192.stdout:4/593: creat d1/d26/d66/fcf x:0 0 0 2024-02-16T22:56:22.207 INFO:tasks.workunit.client.0.smithi064.stdout:6/463: chown d0/d5/df/f86 3767 1 2024-02-16T22:56:22.207 INFO:tasks.workunit.client.1.smithi192.stdout:3/513: rename da/d11/d20/d39/caf to da/d11/d2b/cb3 0 2024-02-16T22:56:22.209 INFO:tasks.workunit.client.0.smithi064.stdout:9/382: rmdir d3/d11/d15/d5a/d2a/d2e/d31 39 2024-02-16T22:56:22.210 INFO:tasks.workunit.client.1.smithi192.stdout:0/514: dread d1/f19 [0,4194304] 0 2024-02-16T22:56:22.210 INFO:tasks.workunit.client.1.smithi192.stdout:2/540: readlink d22/d24/d39/d42/l77 0 2024-02-16T22:56:22.211 INFO:tasks.workunit.client.1.smithi192.stdout:7/604: truncate d1/d2/de/d2c/d49/f80 136432 0 2024-02-16T22:56:22.212 INFO:tasks.workunit.client.1.smithi192.stdout:0/515: chown d1/d3/de/d22/d37/d47/f81 11993 1 2024-02-16T22:56:22.213 INFO:tasks.workunit.client.1.smithi192.stdout:3/514: read da/d6a/f81 [1035466,97378] 0 2024-02-16T22:56:22.213 INFO:tasks.workunit.client.1.smithi192.stdout:4/594: read d1/d15/d48/f55 [3056663,59413] 0 2024-02-16T22:56:22.215 INFO:tasks.workunit.client.0.smithi064.stdout:1/367: rename d2/f3 to d2/df/d11/d25/f80 0 2024-02-16T22:56:22.217 INFO:tasks.workunit.client.1.smithi192.stdout:8/495: rename d5/d26/d83/d86/l94 to d5/db/d47/d43/l96 0 2024-02-16T22:56:22.218 INFO:tasks.workunit.client.0.smithi064.stdout:5/327: mknod d6/d25/d48/c70 0 2024-02-16T22:56:22.218 INFO:tasks.workunit.client.1.smithi192.stdout:6/505: dread da/df/d12/d57/f2c [0,4194304] 0 2024-02-16T22:56:22.219 INFO:tasks.workunit.client.0.smithi064.stdout:0/355: mkdir da/d71 0 2024-02-16T22:56:22.219 INFO:tasks.workunit.client.1.smithi192.stdout:5/482: dwrite d6/d8/d75/f7b [0,4194304] 0 2024-02-16T22:56:22.221 INFO:tasks.workunit.client.0.smithi064.stdout:2/478: creat d7/d14/d25/f90 x:0 0 0 2024-02-16T22:56:22.222 INFO:tasks.workunit.client.1.smithi192.stdout:5/483: write d6/fc [1673932,44579] 0 2024-02-16T22:56:22.223 INFO:tasks.workunit.client.0.smithi064.stdout:3/490: read d4/d13/d4d/d5c/f6c [52681,51787] 0 2024-02-16T22:56:22.224 INFO:tasks.workunit.client.0.smithi064.stdout:8/395: creat df/d2a/f87 x:0 0 0 2024-02-16T22:56:22.226 INFO:tasks.workunit.client.1.smithi192.stdout:0/516: mkdir d1/d3/de/d22/d37/d67/d34/da1 0 2024-02-16T22:56:22.227 INFO:tasks.workunit.client.0.smithi064.stdout:4/421: dread dc/d1d/f36 [0,4194304] 0 2024-02-16T22:56:22.228 INFO:tasks.workunit.client.1.smithi192.stdout:6/506: dwrite da/df/d12/d15/d1b/f8a [0,4194304] 0 2024-02-16T22:56:22.229 INFO:tasks.workunit.client.0.smithi064.stdout:6/464: creat d0/d6/da/d56/f8f x:0 0 0 2024-02-16T22:56:22.229 INFO:tasks.workunit.client.1.smithi192.stdout:6/507: stat da/df/d12/c92 0 2024-02-16T22:56:22.230 INFO:tasks.workunit.client.0.smithi064.stdout:9/383: chown d3/ce 1332167 1 2024-02-16T22:56:22.231 INFO:tasks.workunit.client.0.smithi064.stdout:9/384: fsync d3/f69 0 2024-02-16T22:56:22.234 INFO:tasks.workunit.client.0.smithi064.stdout:9/385: read - d3/d11/d15/d23/f6e zero size 2024-02-16T22:56:22.235 INFO:tasks.workunit.client.0.smithi064.stdout:6/465: dread d0/d5/f42 [0,4194304] 0 2024-02-16T22:56:22.235 INFO:tasks.workunit.client.0.smithi064.stdout:7/386: rename d5/d6/fa to d5/d6/d54/f78 0 2024-02-16T22:56:22.236 INFO:tasks.workunit.client.1.smithi192.stdout:8/496: mkdir d5/d26/d83/d86/d97 0 2024-02-16T22:56:22.238 INFO:tasks.workunit.client.0.smithi064.stdout:7/387: dread - d5/d6/d54/d41/f68 zero size 2024-02-16T22:56:22.239 INFO:tasks.workunit.client.0.smithi064.stdout:6/466: write d0/d1b/f7a [1426351,118537] 0 2024-02-16T22:56:22.246 INFO:tasks.workunit.client.1.smithi192.stdout:5/484: creat d6/d8/fa1 x:0 0 0 2024-02-16T22:56:22.247 INFO:tasks.workunit.client.0.smithi064.stdout:8/396: creat df/d31/d61/d79/d81/f88 x:0 0 0 2024-02-16T22:56:22.248 INFO:tasks.workunit.client.0.smithi064.stdout:2/479: dread d7/d14/d1e/d34/d3f/f43 [0,4194304] 0 2024-02-16T22:56:22.249 INFO:tasks.workunit.client.1.smithi192.stdout:7/605: dread d1/d2/de/d83/f8d [0,4194304] 0 2024-02-16T22:56:22.253 INFO:tasks.workunit.client.0.smithi064.stdout:9/386: mknod d3/d11/d15/c7e 0 2024-02-16T22:56:22.258 INFO:tasks.workunit.client.0.smithi064.stdout:4/422: rename dc/dd/d16/f2e to dc/d4a/f7d 0 2024-02-16T22:56:22.259 INFO:tasks.workunit.client.0.smithi064.stdout:4/423: stat dc/dd/d5b/c76 0 2024-02-16T22:56:22.259 INFO:tasks.workunit.client.0.smithi064.stdout:4/424: chown dc/ff 5712934 1 2024-02-16T22:56:22.263 INFO:tasks.workunit.client.0.smithi064.stdout:7/388: symlink d5/d33/d53/l79 0 2024-02-16T22:56:22.264 INFO:tasks.workunit.client.0.smithi064.stdout:7/389: chown d5/d33/d45 3159 1 2024-02-16T22:56:22.265 INFO:tasks.workunit.client.0.smithi064.stdout:7/390: write d5/d33/d42/f4f [471969,79635] 0 2024-02-16T22:56:22.266 INFO:tasks.workunit.client.1.smithi192.stdout:2/541: write d22/f3c [1175985,32967] 0 2024-02-16T22:56:22.267 INFO:tasks.workunit.client.0.smithi064.stdout:6/467: chown d0/d6/d10/l2f 204 1 2024-02-16T22:56:22.270 INFO:tasks.workunit.client.0.smithi064.stdout:0/356: dwrite da/fb [4194304,4194304] 0 2024-02-16T22:56:22.271 INFO:tasks.workunit.client.0.smithi064.stdout:3/491: symlink d4/d13/d1b/lab 0 2024-02-16T22:56:22.271 INFO:tasks.workunit.client.0.smithi064.stdout:3/492: fdatasync d4/d1f/d5b/d51/fa4 0 2024-02-16T22:56:22.272 INFO:tasks.workunit.client.1.smithi192.stdout:0/517: symlink d1/d3/la2 0 2024-02-16T22:56:22.278 INFO:tasks.workunit.client.0.smithi064.stdout:8/397: creat df/d31/d2f/d4f/f89 x:0 0 0 2024-02-16T22:56:22.279 INFO:tasks.workunit.client.1.smithi192.stdout:3/515: creat da/d11/d63/d6b/d66/fb4 x:0 0 0 2024-02-16T22:56:22.283 INFO:tasks.workunit.client.1.smithi192.stdout:6/508: truncate da/df/d12/d15/f33 45620 0 2024-02-16T22:56:22.284 INFO:tasks.workunit.client.1.smithi192.stdout:6/509: write da/d40/d85/f87 [538082,79642] 0 2024-02-16T22:56:22.285 INFO:tasks.workunit.client.0.smithi064.stdout:9/387: creat d3/d11/d15/f7f x:0 0 0 2024-02-16T22:56:22.287 INFO:tasks.workunit.client.1.smithi192.stdout:1/482: rename d3/d5b/d9c to d3/de/da4 0 2024-02-16T22:56:22.289 INFO:tasks.workunit.client.1.smithi192.stdout:6/510: dwrite da/df/d12/f9c [0,4194304] 0 2024-02-16T22:56:22.289 INFO:tasks.workunit.client.1.smithi192.stdout:6/511: dread - da/d40/f82 zero size 2024-02-16T22:56:22.290 INFO:tasks.workunit.client.1.smithi192.stdout:8/497: mknod d5/d68/c98 0 2024-02-16T22:56:22.292 INFO:tasks.workunit.client.0.smithi064.stdout:2/480: dread d7/f10 [0,4194304] 0 2024-02-16T22:56:22.294 INFO:tasks.workunit.client.1.smithi192.stdout:5/485: creat d6/d12/d4e/d58/d8f/fa2 x:0 0 0 2024-02-16T22:56:22.299 INFO:tasks.workunit.client.0.smithi064.stdout:7/391: chown d5/d6/d54/f78 15068 1 2024-02-16T22:56:22.300 INFO:tasks.workunit.client.1.smithi192.stdout:7/606: fsync d1/f12 0 2024-02-16T22:56:22.300 INFO:tasks.workunit.client.1.smithi192.stdout:7/607: write d1/d2/de/f99 [1133421,93040] 0 2024-02-16T22:56:22.300 INFO:tasks.workunit.client.1.smithi192.stdout:7/608: chown d1/d2/fb4 212795 1 2024-02-16T22:56:22.300 INFO:tasks.workunit.client.1.smithi192.stdout:4/595: getdents d1/d26/d5e 0 2024-02-16T22:56:22.302 INFO:tasks.workunit.client.0.smithi064.stdout:0/357: read da/dc/f1e [3692047,52238] 0 2024-02-16T22:56:22.304 INFO:tasks.workunit.client.0.smithi064.stdout:1/368: getdents d2/df 0 2024-02-16T22:56:22.306 INFO:tasks.workunit.client.0.smithi064.stdout:9/388: chown d3/f18 48843103 1 2024-02-16T22:56:22.308 INFO:tasks.workunit.client.1.smithi192.stdout:9/627: rename d7/d19/d37/d5c to d7/d19/d5e/d81/dda 0 2024-02-16T22:56:22.312 INFO:tasks.workunit.client.1.smithi192.stdout:8/498: unlink d5/d26/d83/d86/d5f/f4e 0 2024-02-16T22:56:22.313 INFO:tasks.workunit.client.1.smithi192.stdout:8/499: read - d5/d26/f88 zero size 2024-02-16T22:56:22.313 INFO:tasks.workunit.client.0.smithi064.stdout:5/328: getdents d6/d25/d48 0 2024-02-16T22:56:22.314 INFO:tasks.workunit.client.1.smithi192.stdout:5/486: creat d6/d8/fa3 x:0 0 0 2024-02-16T22:56:22.315 INFO:tasks.workunit.client.1.smithi192.stdout:5/487: chown d6/d8/f9e 0 1 2024-02-16T22:56:22.316 INFO:tasks.workunit.client.0.smithi064.stdout:2/481: dwrite d7/f35 [0,4194304] 0 2024-02-16T22:56:22.316 INFO:tasks.workunit.client.0.smithi064.stdout:7/392: symlink d5/d6/d54/d41/l7a 0 2024-02-16T22:56:22.317 INFO:tasks.workunit.client.1.smithi192.stdout:8/500: dread d5/d26/d83/d86/d5f/f8d [0,4194304] 0 2024-02-16T22:56:22.317 INFO:tasks.workunit.client.0.smithi064.stdout:4/425: truncate dc/dd/d32/d58/f70 1599785 0 2024-02-16T22:56:22.318 INFO:tasks.workunit.client.1.smithi192.stdout:8/501: dread d5/d16/f4f [0,4194304] 0 2024-02-16T22:56:22.323 INFO:tasks.workunit.client.0.smithi064.stdout:7/393: dwrite d5/d6/f1f [0,4194304] 0 2024-02-16T22:56:22.323 INFO:tasks.workunit.client.0.smithi064.stdout:0/358: rmdir da/dc/de/d12 39 2024-02-16T22:56:22.327 INFO:tasks.workunit.client.1.smithi192.stdout:7/609: dwrite d1/d2/de/d83/f8d [0,4194304] 0 2024-02-16T22:56:22.339 INFO:tasks.workunit.client.0.smithi064.stdout:5/329: dread d6/d18/d19/f1f [0,4194304] 0 2024-02-16T22:56:22.340 INFO:tasks.workunit.client.0.smithi064.stdout:8/398: truncate f1 2577620 0 2024-02-16T22:56:22.341 INFO:tasks.workunit.client.1.smithi192.stdout:6/512: symlink da/df/d12/d57/d81/la6 0 2024-02-16T22:56:22.348 INFO:tasks.workunit.client.1.smithi192.stdout:2/542: creat d22/d24/d39/fb7 x:0 0 0 2024-02-16T22:56:22.350 INFO:tasks.workunit.client.0.smithi064.stdout:7/394: mkdir d5/d6/d54/d7b 0 2024-02-16T22:56:22.350 INFO:tasks.workunit.client.1.smithi192.stdout:3/516: dwrite f9 [0,4194304] 0 2024-02-16T22:56:22.351 INFO:tasks.workunit.client.0.smithi064.stdout:0/359: mkdir da/dc/d3b/d72 0 2024-02-16T22:56:22.352 INFO:tasks.workunit.client.0.smithi064.stdout:4/426: dread dc/dd/d32/f3d [0,4194304] 0 2024-02-16T22:56:22.356 INFO:tasks.workunit.client.0.smithi064.stdout:9/389: mkdir d3/d11/d15/d5a/d2a/d2e/d31/d3f/d80 0 2024-02-16T22:56:22.357 INFO:tasks.workunit.client.1.smithi192.stdout:2/543: dwrite d22/f41 [4194304,4194304] 0 2024-02-16T22:56:22.360 INFO:tasks.workunit.client.1.smithi192.stdout:0/518: rename d1/f5d to d1/d3/d39/fa3 0 2024-02-16T22:56:22.361 INFO:tasks.workunit.client.1.smithi192.stdout:5/488: write d6/f29 [990846,128124] 0 2024-02-16T22:56:22.362 INFO:tasks.workunit.client.1.smithi192.stdout:5/489: chown d6/d8/c63 217 1 2024-02-16T22:56:22.362 INFO:tasks.workunit.client.0.smithi064.stdout:5/330: unlink d6/c15 0 2024-02-16T22:56:22.364 INFO:tasks.workunit.client.1.smithi192.stdout:6/513: rmdir da/df/d71 39 2024-02-16T22:56:22.365 INFO:tasks.workunit.client.1.smithi192.stdout:6/514: write da/df/f84 [827352,32133] 0 2024-02-16T22:56:22.367 INFO:tasks.workunit.client.0.smithi064.stdout:2/482: mknod d7/de/c91 0 2024-02-16T22:56:22.367 INFO:tasks.workunit.client.0.smithi064.stdout:2/483: readlink d7/de/l42 0 2024-02-16T22:56:22.368 INFO:tasks.workunit.client.0.smithi064.stdout:2/484: fsync d7/d14/d3e/f75 0 2024-02-16T22:56:22.368 INFO:tasks.workunit.client.0.smithi064.stdout:6/468: getdents d0/d1d 0 2024-02-16T22:56:22.371 INFO:tasks.workunit.client.0.smithi064.stdout:3/493: link d4/d13/d1b/d27/d61/d63/f71 d4/d13/d35/fac 0 2024-02-16T22:56:22.373 INFO:tasks.workunit.client.1.smithi192.stdout:9/628: dread d7/d15/f33 [0,4194304] 0 2024-02-16T22:56:22.373 INFO:tasks.workunit.client.0.smithi064.stdout:2/485: dread d7/d14/d25/d37/d44/f5c [0,4194304] 0 2024-02-16T22:56:22.374 INFO:tasks.workunit.client.1.smithi192.stdout:9/629: write d7/d19/d5e/d81/fcc [4333424,104476] 0 2024-02-16T22:56:22.378 INFO:tasks.workunit.client.0.smithi064.stdout:1/369: rename d2/df/d14/d2c/d62/c7c to d2/df/d11/d1a/c81 0 2024-02-16T22:56:22.380 INFO:tasks.workunit.client.0.smithi064.stdout:3/494: dwrite d4/d13/d1b/d27/f7f [0,4194304] 0 2024-02-16T22:56:22.382 INFO:tasks.workunit.client.0.smithi064.stdout:9/390: symlink d3/d11/l81 0 2024-02-16T22:56:22.383 INFO:tasks.workunit.client.1.smithi192.stdout:7/610: dwrite d1/d2/de/d2c/d49/d54/d6a/f95 [4194304,4194304] 0 2024-02-16T22:56:22.392 INFO:tasks.workunit.client.1.smithi192.stdout:3/517: dread da/d11/d63/d6b/d66/f86 [0,4194304] 0 2024-02-16T22:56:22.392 INFO:tasks.workunit.client.1.smithi192.stdout:8/502: creat d5/db/de/f99 x:0 0 0 2024-02-16T22:56:22.396 INFO:tasks.workunit.client.1.smithi192.stdout:1/483: rename d3/de/l6e to d3/de/d27/d34/d52/d63/d81/la5 0 2024-02-16T22:56:22.397 INFO:tasks.workunit.client.1.smithi192.stdout:1/484: dread - d3/de/d27/d34/d52/d63/d81/f8d zero size 2024-02-16T22:56:22.398 INFO:tasks.workunit.client.1.smithi192.stdout:5/490: write d6/d8/d86/f89 [2595694,89353] 0 2024-02-16T22:56:22.399 INFO:tasks.workunit.client.1.smithi192.stdout:7/611: truncate d1/d2/de/f8f 336576 0 2024-02-16T22:56:22.402 INFO:tasks.workunit.client.1.smithi192.stdout:9/630: write d7/d19/d5e/d81/f95 [78512,28906] 0 2024-02-16T22:56:22.404 INFO:tasks.workunit.client.1.smithi192.stdout:9/631: truncate d7/d19/d5e/d81/fd0 1618158 0 2024-02-16T22:56:22.405 INFO:tasks.workunit.client.1.smithi192.stdout:4/596: rename d1/d26/d66/d89/d11/d58/cb1 to d1/d15/d29/d3c/cd0 0 2024-02-16T22:56:22.405 INFO:tasks.workunit.client.1.smithi192.stdout:9/632: truncate d7/d19/f86 756877 0 2024-02-16T22:56:22.407 INFO:tasks.workunit.client.1.smithi192.stdout:4/597: chown d1/d26/d66/d89/d11/l33 638128 1 2024-02-16T22:56:22.408 INFO:tasks.workunit.client.1.smithi192.stdout:3/518: creat da/d11/d63/d6b/d80/fb5 x:0 0 0 2024-02-16T22:56:22.408 INFO:tasks.workunit.client.1.smithi192.stdout:9/633: dread - d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/da2/fd9 zero size 2024-02-16T22:56:22.411 INFO:tasks.workunit.client.1.smithi192.stdout:0/519: rename d1/c9f to d1/d3/d39/ca4 0 2024-02-16T22:56:22.412 INFO:tasks.workunit.client.0.smithi064.stdout:5/331: fsync d6/f13 0 2024-02-16T22:56:22.417 INFO:tasks.workunit.client.1.smithi192.stdout:1/485: rename d3/f16 to d3/fa6 0 2024-02-16T22:56:22.417 INFO:tasks.workunit.client.1.smithi192.stdout:8/503: dread d5/d9/f44 [0,4194304] 0 2024-02-16T22:56:22.417 INFO:tasks.workunit.client.0.smithi064.stdout:3/495: dwrite d4/d1f/f67 [0,4194304] 0 2024-02-16T22:56:22.419 INFO:tasks.workunit.client.1.smithi192.stdout:4/598: dwrite d1/d26/d5e/f96 [0,4194304] 0 2024-02-16T22:56:22.419 INFO:tasks.workunit.client.0.smithi064.stdout:4/427: dread dc/dd/d32/d22/f52 [0,4194304] 0 2024-02-16T22:56:22.419 INFO:tasks.workunit.client.0.smithi064.stdout:3/496: write d4/f17 [6640180,93086] 0 2024-02-16T22:56:22.420 INFO:tasks.workunit.client.1.smithi192.stdout:9/634: dwrite d7/d15/fd4 [0,4194304] 0 2024-02-16T22:56:22.420 INFO:tasks.workunit.client.1.smithi192.stdout:1/486: write d3/de/d27/f6a [1019435,41551] 0 2024-02-16T22:56:22.423 INFO:tasks.workunit.client.1.smithi192.stdout:4/599: chown d1/d26/d66/fac 479913 1 2024-02-16T22:56:22.423 INFO:tasks.workunit.client.1.smithi192.stdout:6/515: rename da/df/d12/d57/d81/la6 to da/df/d58/la7 0 2024-02-16T22:56:22.427 INFO:tasks.workunit.client.0.smithi064.stdout:9/391: dread d3/d9/f48 [0,4194304] 0 2024-02-16T22:56:22.427 INFO:tasks.workunit.client.1.smithi192.stdout:5/491: truncate d6/d8/d80/f8a 1874209 0 2024-02-16T22:56:22.435 INFO:tasks.workunit.client.1.smithi192.stdout:4/600: dwrite d1/d15/d48/f95 [0,4194304] 0 2024-02-16T22:56:22.437 INFO:tasks.workunit.client.1.smithi192.stdout:3/519: dread f4 [0,4194304] 0 2024-02-16T22:56:22.437 INFO:tasks.workunit.client.1.smithi192.stdout:4/601: write d1/d26/d6f/f86 [1974563,11289] 0 2024-02-16T22:56:22.446 INFO:tasks.workunit.client.1.smithi192.stdout:4/602: dwrite d1/d26/d66/d89/d11/f98 [0,4194304] 0 2024-02-16T22:56:22.466 INFO:tasks.workunit.client.1.smithi192.stdout:0/520: dread d1/d3/d29/f27 [0,4194304] 0 2024-02-16T22:56:22.509 INFO:tasks.workunit.client.1.smithi192.stdout:7/612: getdents d1/d2/d2d/d61/d88/d90/db2 0 2024-02-16T22:56:22.510 INFO:tasks.workunit.client.1.smithi192.stdout:7/613: chown d1/d2/de/d2c/d49/da1/lb1 603 1 2024-02-16T22:56:22.512 INFO:tasks.workunit.client.0.smithi064.stdout:1/370: truncate d2/df/d14/d2c/f3c 3103706 0 2024-02-16T22:56:22.514 INFO:tasks.workunit.client.0.smithi064.stdout:5/332: mknod d6/d25/d48/d67/c71 0 2024-02-16T22:56:22.515 INFO:tasks.workunit.client.0.smithi064.stdout:5/333: chown d6/d1c/d43 1801775 1 2024-02-16T22:56:22.515 INFO:tasks.workunit.client.0.smithi064.stdout:5/334: stat d6/d1c/d4b/c61 0 2024-02-16T22:56:22.519 INFO:tasks.workunit.client.1.smithi192.stdout:9/635: fsync d7/d19/d37/f34 0 2024-02-16T22:56:22.522 INFO:tasks.workunit.client.0.smithi064.stdout:2/486: link d7/f13 d7/d14/d1e/f92 0 2024-02-16T22:56:22.524 INFO:tasks.workunit.client.0.smithi064.stdout:2/487: read d7/d14/d1e/f92 [6347,112107] 0 2024-02-16T22:56:22.531 INFO:tasks.workunit.client.0.smithi064.stdout:2/488: dread d7/d14/f2c [4194304,4194304] 0 2024-02-16T22:56:22.534 INFO:tasks.workunit.client.0.smithi064.stdout:2/489: read d7/d14/d4b/f64 [6644628,9505] 0 2024-02-16T22:56:22.535 INFO:tasks.workunit.client.0.smithi064.stdout:6/469: rename d0/d6/da/d56/f8f to d0/d5/f90 0 2024-02-16T22:56:22.542 INFO:tasks.workunit.client.0.smithi064.stdout:1/371: symlink d2/df/d11/d1a/d30/d3d/l82 0 2024-02-16T22:56:22.545 INFO:tasks.workunit.client.0.smithi064.stdout:5/335: creat d6/d18/f72 x:0 0 0 2024-02-16T22:56:22.545 INFO:tasks.workunit.client.1.smithi192.stdout:4/603: fsync d1/d15/d48/d8d/d9b/fb0 0 2024-02-16T22:56:22.549 INFO:tasks.workunit.client.0.smithi064.stdout:1/372: dwrite d2/d4e/f64 [0,4194304] 0 2024-02-16T22:56:22.554 INFO:tasks.workunit.client.0.smithi064.stdout:5/336: dread d6/f31 [0,4194304] 0 2024-02-16T22:56:22.556 INFO:tasks.workunit.client.0.smithi064.stdout:0/360: getdents da/dc/de/d12/d2c 0 2024-02-16T22:56:22.564 INFO:tasks.workunit.client.0.smithi064.stdout:7/395: rename d5/d33/d45/d64/c61 to d5/d33/d45/d64/d56/d5d/c7c 0 2024-02-16T22:56:22.571 INFO:tasks.workunit.client.1.smithi192.stdout:0/521: creat d1/d3/de/d22/d37/d47/fa5 x:0 0 0 2024-02-16T22:56:22.572 INFO:tasks.workunit.client.1.smithi192.stdout:2/544: sync 2024-02-16T22:56:22.572 INFO:tasks.workunit.client.1.smithi192.stdout:0/522: fsync d1/d3/d29/d43/f68 0 2024-02-16T22:56:22.573 INFO:tasks.workunit.client.1.smithi192.stdout:0/523: fdatasync d1/d3/d39/f79 0 2024-02-16T22:56:22.577 INFO:tasks.workunit.client.0.smithi064.stdout:5/337: mknod d6/d1c/d43/c73 0 2024-02-16T22:56:22.591 INFO:tasks.workunit.client.0.smithi064.stdout:7/396: unlink d5/d6/d54/f6a 0 2024-02-16T22:56:22.591 INFO:tasks.workunit.client.0.smithi064.stdout:7/397: fsync d5/d33/f60 0 2024-02-16T22:56:22.598 INFO:tasks.workunit.client.1.smithi192.stdout:3/520: symlink da/d11/d20/d39/lb6 0 2024-02-16T22:56:22.600 INFO:tasks.workunit.client.0.smithi064.stdout:9/392: getdents d3/d9 0 2024-02-16T22:56:22.600 INFO:tasks.workunit.client.1.smithi192.stdout:9/636: symlink d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/ldb 0 2024-02-16T22:56:22.602 INFO:tasks.workunit.client.0.smithi064.stdout:5/338: chown d6/d25/l3a 0 1 2024-02-16T22:56:22.615 INFO:tasks.workunit.client.0.smithi064.stdout:1/373: link d2/df/d14/d18/l7d d2/d4e/d56/d5b/l83 0 2024-02-16T22:56:22.617 INFO:tasks.workunit.client.1.smithi192.stdout:2/545: mknod d22/d24/d39/d42/d52/d7c/cb8 0 2024-02-16T22:56:22.618 INFO:tasks.workunit.client.1.smithi192.stdout:2/546: chown d22/d24/d39/d42/d65/d69/d4f/l6d 493619 1 2024-02-16T22:56:22.618 INFO:tasks.workunit.client.1.smithi192.stdout:2/547: write d22/f3c [1380344,21549] 0 2024-02-16T22:56:22.619 INFO:tasks.workunit.client.0.smithi064.stdout:5/339: mknod d6/d25/d48/d67/c74 0 2024-02-16T22:56:22.620 INFO:tasks.workunit.client.0.smithi064.stdout:5/340: write d6/d18/d19/f2e [603622,71856] 0 2024-02-16T22:56:22.621 INFO:tasks.workunit.client.0.smithi064.stdout:5/341: chown f4 361081 1 2024-02-16T22:56:22.621 INFO:tasks.workunit.client.0.smithi064.stdout:2/490: getdents d7/d14/d1e/d34/d65 0 2024-02-16T22:56:22.622 INFO:tasks.workunit.client.0.smithi064.stdout:2/491: chown d7/de/l29 384 1 2024-02-16T22:56:22.627 INFO:tasks.workunit.client.0.smithi064.stdout:7/398: mkdir d5/d7d 0 2024-02-16T22:56:22.629 INFO:tasks.workunit.client.0.smithi064.stdout:1/374: rmdir d2/d4e/d56/d5b 39 2024-02-16T22:56:22.630 INFO:tasks.workunit.client.0.smithi064.stdout:1/375: chown d2/df/f54 276692373 1 2024-02-16T22:56:22.630 INFO:tasks.workunit.client.0.smithi064.stdout:7/399: dread d5/d33/d42/f4f [0,4194304] 0 2024-02-16T22:56:22.631 INFO:tasks.workunit.client.0.smithi064.stdout:7/400: dread - d5/d33/d45/f52 zero size 2024-02-16T22:56:22.632 INFO:tasks.workunit.client.1.smithi192.stdout:9/637: rename d7/d19/d37/f9e to d7/d19/d37/fdc 0 2024-02-16T22:56:22.633 INFO:tasks.workunit.client.1.smithi192.stdout:9/638: chown d7/d19/d37/d5a/f63 262071 1 2024-02-16T22:56:22.633 INFO:tasks.workunit.client.0.smithi064.stdout:9/393: mknod d3/d9/d1e/d42/d7b/c82 0 2024-02-16T22:56:22.635 INFO:tasks.workunit.client.1.smithi192.stdout:6/516: creat da/df/d12/d15/fa8 x:0 0 0 2024-02-16T22:56:22.639 INFO:tasks.workunit.client.0.smithi064.stdout:2/492: mknod d7/d14/d1e/d34/d3f/c93 0 2024-02-16T22:56:22.641 INFO:tasks.workunit.client.0.smithi064.stdout:3/497: rename d4/d13/d1b/d4f/f6b to d4/d13/fad 0 2024-02-16T22:56:22.646 INFO:tasks.workunit.client.1.smithi192.stdout:2/548: unlink d22/f92 0 2024-02-16T22:56:22.647 INFO:tasks.workunit.client.1.smithi192.stdout:8/504: link d5/d26/d8f/l7e d5/d26/d8f/l9a 0 2024-02-16T22:56:22.648 INFO:tasks.workunit.client.0.smithi064.stdout:9/394: fsync d3/d9/d1e/d42/f54 0 2024-02-16T22:56:22.649 INFO:tasks.workunit.client.0.smithi064.stdout:9/395: chown d3/d11/d15/d5a/d2a 82960 1 2024-02-16T22:56:22.652 INFO:tasks.workunit.client.1.smithi192.stdout:9/639: creat d7/d19/d37/d5a/fdd x:0 0 0 2024-02-16T22:56:22.655 INFO:tasks.workunit.client.1.smithi192.stdout:6/517: mkdir da/df/d18/d52/da9 0 2024-02-16T22:56:22.656 INFO:tasks.workunit.client.0.smithi064.stdout:9/396: dwrite d3/d11/d15/d5a/f29 [0,4194304] 0 2024-02-16T22:56:22.657 INFO:tasks.workunit.client.0.smithi064.stdout:3/498: dread d4/d13/d1b/f1c [0,4194304] 0 2024-02-16T22:56:22.665 INFO:tasks.workunit.client.1.smithi192.stdout:9/640: symlink d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/da2/lde 0 2024-02-16T22:56:22.666 INFO:tasks.workunit.client.1.smithi192.stdout:1/487: dwrite d3/f21 [4194304,4194304] 0 2024-02-16T22:56:22.670 INFO:tasks.workunit.client.1.smithi192.stdout:0/524: creat d1/fa6 x:0 0 0 2024-02-16T22:56:22.672 INFO:tasks.workunit.client.0.smithi064.stdout:3/499: dread - d4/d13/d1b/d27/d44/f5f zero size 2024-02-16T22:56:22.676 INFO:tasks.workunit.client.0.smithi064.stdout:9/397: fsync d3/f63 0 2024-02-16T22:56:22.678 INFO:tasks.workunit.client.1.smithi192.stdout:9/641: mkdir d7/d19/d37/ddf 0 2024-02-16T22:56:22.679 INFO:tasks.workunit.client.1.smithi192.stdout:9/642: read - d7/d19/d5e/fcf zero size 2024-02-16T22:56:22.680 INFO:tasks.workunit.client.1.smithi192.stdout:9/643: write d7/d19/d37/d5a/f63 [3319246,40890] 0 2024-02-16T22:56:22.680 INFO:tasks.workunit.client.0.smithi064.stdout:3/500: unlink d4/d13/c9a 0 2024-02-16T22:56:22.683 INFO:tasks.workunit.client.1.smithi192.stdout:1/488: creat d3/de/da4/fa7 x:0 0 0 2024-02-16T22:56:22.684 INFO:tasks.workunit.client.0.smithi064.stdout:4/428: write dc/dd/d32/d58/f70 [947416,120795] 0 2024-02-16T22:56:22.686 INFO:tasks.workunit.client.0.smithi064.stdout:4/429: write dc/dd/f59 [467690,129181] 0 2024-02-16T22:56:22.687 INFO:tasks.workunit.client.0.smithi064.stdout:4/430: write dc/d1d/f4c [116849,75824] 0 2024-02-16T22:56:22.687 INFO:tasks.workunit.client.1.smithi192.stdout:2/549: creat d22/d24/fb9 x:0 0 0 2024-02-16T22:56:22.688 INFO:tasks.workunit.client.1.smithi192.stdout:9/644: dread d7/d15/f58 [0,4194304] 0 2024-02-16T22:56:22.689 INFO:tasks.workunit.client.1.smithi192.stdout:2/550: dread d22/d24/d39/f4c [0,4194304] 0 2024-02-16T22:56:22.691 INFO:tasks.workunit.client.1.smithi192.stdout:8/505: creat d5/db/f9b x:0 0 0 2024-02-16T22:56:22.693 INFO:tasks.workunit.client.0.smithi064.stdout:3/501: symlink d4/d13/d4d/d5c/d78/d96/lae 0 2024-02-16T22:56:22.694 INFO:tasks.workunit.client.0.smithi064.stdout:1/376: dread d2/df/d14/d2c/f3c [0,4194304] 0 2024-02-16T22:56:22.696 INFO:tasks.workunit.client.0.smithi064.stdout:4/431: mkdir dc/dd/d32/d46/d5a/d7e 0 2024-02-16T22:56:22.697 INFO:tasks.workunit.client.0.smithi064.stdout:4/432: dread - dc/dd/d32/d46/d5a/f65 zero size 2024-02-16T22:56:22.697 INFO:tasks.workunit.client.0.smithi064.stdout:9/398: link d3/ce d3/d9/d1e/c83 0 2024-02-16T22:56:22.699 INFO:tasks.workunit.client.1.smithi192.stdout:0/525: mkdir d1/d3/de/d22/da7 0 2024-02-16T22:56:22.700 INFO:tasks.workunit.client.0.smithi064.stdout:3/502: mkdir d4/d1f/daf 0 2024-02-16T22:56:22.701 INFO:tasks.workunit.client.0.smithi064.stdout:3/503: dread - d4/d13/d1b/d27/d61/f66 zero size 2024-02-16T22:56:22.701 INFO:tasks.workunit.client.1.smithi192.stdout:9/645: mkdir d7/d15/d36/d77/dbd/de0 0 2024-02-16T22:56:22.702 INFO:tasks.workunit.client.0.smithi064.stdout:3/504: write d4/d13/d1b/d27/d99/f87 [3244725,107800] 0 2024-02-16T22:56:22.703 INFO:tasks.workunit.client.1.smithi192.stdout:9/646: chown d7/d15/d36/d38/d3f/d8a/d98/la4 213 1 2024-02-16T22:56:22.706 INFO:tasks.workunit.client.0.smithi064.stdout:4/433: symlink dc/dd/d32/d46/d5a/l7f 0 2024-02-16T22:56:22.707 INFO:tasks.workunit.client.1.smithi192.stdout:1/489: symlink d3/d40/d9f/la8 0 2024-02-16T22:56:22.708 INFO:tasks.workunit.client.1.smithi192.stdout:1/490: truncate d3/de/d27/f30 13201367 0 2024-02-16T22:56:22.711 INFO:tasks.workunit.client.0.smithi064.stdout:4/434: mknod dc/dd/d32/d4b/c80 0 2024-02-16T22:56:22.715 INFO:tasks.workunit.client.1.smithi192.stdout:0/526: mknod d1/ca8 0 2024-02-16T22:56:22.718 INFO:tasks.workunit.client.0.smithi064.stdout:3/505: mkdir d4/d13/d1b/d27/d82/db0 0 2024-02-16T22:56:22.719 INFO:tasks.workunit.client.1.smithi192.stdout:1/491: creat d3/de/d87/fa9 x:0 0 0 2024-02-16T22:56:22.722 INFO:tasks.workunit.client.0.smithi064.stdout:0/361: dwrite da/dc/de/d12/d1d/f34 [0,4194304] 0 2024-02-16T22:56:22.724 INFO:tasks.workunit.client.0.smithi064.stdout:3/506: mknod d4/d1f/d75/cb1 0 2024-02-16T22:56:22.724 INFO:tasks.workunit.client.0.smithi064.stdout:3/507: readlink d4/d13/l6d 0 2024-02-16T22:56:22.727 INFO:tasks.workunit.client.1.smithi192.stdout:1/492: mkdir d3/de/d27/d34/d69/d85/d89/daa 0 2024-02-16T22:56:22.727 INFO:tasks.workunit.client.0.smithi064.stdout:4/435: creat dc/f81 x:0 0 0 2024-02-16T22:56:22.729 INFO:tasks.workunit.client.1.smithi192.stdout:0/527: creat d1/d3/de/d22/d37/d67/d34/da1/fa9 x:0 0 0 2024-02-16T22:56:22.730 INFO:tasks.workunit.client.0.smithi064.stdout:0/362: creat da/dc/d3b/d5a/f73 x:0 0 0 2024-02-16T22:56:22.730 INFO:tasks.workunit.client.1.smithi192.stdout:0/528: chown d1/d3/de/d22 4782665 1 2024-02-16T22:56:22.732 INFO:tasks.workunit.client.1.smithi192.stdout:0/529: read d1/d3/f7c [105413,64866] 0 2024-02-16T22:56:22.732 INFO:tasks.workunit.client.0.smithi064.stdout:4/436: unlink dc/dd/d32/d4b/c80 0 2024-02-16T22:56:22.733 INFO:tasks.workunit.client.1.smithi192.stdout:0/530: truncate d1/d3/d29/d43/d9c/f72 267054 0 2024-02-16T22:56:22.735 INFO:tasks.workunit.client.0.smithi064.stdout:0/363: mknod da/dc/de/d12/d2c/d2e/d38/c74 0 2024-02-16T22:56:22.736 INFO:tasks.workunit.client.0.smithi064.stdout:4/437: mkdir dc/dd/d32/d46/d82 0 2024-02-16T22:56:22.739 INFO:tasks.workunit.client.0.smithi064.stdout:3/508: rmdir d4/d1f/daf 0 2024-02-16T22:56:22.739 INFO:tasks.workunit.client.0.smithi064.stdout:0/364: fdatasync da/d26/f28 0 2024-02-16T22:56:22.742 INFO:tasks.workunit.client.0.smithi064.stdout:4/438: creat dc/dd/d32/d46/d5a/f83 x:0 0 0 2024-02-16T22:56:22.744 INFO:tasks.workunit.client.0.smithi064.stdout:3/509: write d4/d13/d1b/d27/f36 [4414275,115575] 0 2024-02-16T22:56:22.745 INFO:tasks.workunit.client.0.smithi064.stdout:3/510: write d4/d1f/d34/f74 [2569400,68815] 0 2024-02-16T22:56:22.751 INFO:tasks.workunit.client.0.smithi064.stdout:4/439: dread dc/d4a/f7d [0,4194304] 0 2024-02-16T22:56:22.754 INFO:tasks.workunit.client.0.smithi064.stdout:3/511: fsync d4/d13/d1b/d27/f58 0 2024-02-16T22:56:22.758 INFO:tasks.workunit.client.0.smithi064.stdout:3/512: unlink d4/d13/d4d/f8f 0 2024-02-16T22:56:22.758 INFO:tasks.workunit.client.0.smithi064.stdout:4/440: dwrite dc/dd/d32/d58/f12 [4194304,4194304] 0 2024-02-16T22:56:22.758 INFO:tasks.workunit.client.0.smithi064.stdout:2/493: symlink d7/d14/d25/l94 0 2024-02-16T22:56:22.760 INFO:tasks.workunit.client.0.smithi064.stdout:4/441: fdatasync dc/dd/d16/d6e/f7b 0 2024-02-16T22:56:22.760 INFO:tasks.workunit.client.0.smithi064.stdout:3/513: rmdir d4/d13/d1b/d27 39 2024-02-16T22:56:22.767 INFO:tasks.workunit.client.0.smithi064.stdout:3/514: truncate d4/d13/fa7 614622 0 2024-02-16T22:56:22.767 INFO:tasks.workunit.client.0.smithi064.stdout:4/442: creat dc/dd/d32/d46/d5a/f84 x:0 0 0 2024-02-16T22:56:22.767 INFO:tasks.workunit.client.0.smithi064.stdout:4/443: fsync dc/de/f75 0 2024-02-16T22:56:22.767 INFO:tasks.workunit.client.1.smithi192.stdout:7/614: truncate d1/d2/de/d2c/d49/f6f 2832551 0 2024-02-16T22:56:22.767 INFO:tasks.workunit.client.1.smithi192.stdout:4/604: dwrite d1/fd [0,4194304] 0 2024-02-16T22:56:22.768 INFO:tasks.workunit.client.1.smithi192.stdout:7/615: mknod d1/d2/de/d83/cda 0 2024-02-16T22:56:22.770 INFO:tasks.workunit.client.0.smithi064.stdout:2/494: rename d7/d14/d25/d37 to d7/de/d95 0 2024-02-16T22:56:22.773 INFO:tasks.workunit.client.0.smithi064.stdout:4/444: dwrite dc/f6c [0,4194304] 0 2024-02-16T22:56:22.773 INFO:tasks.workunit.client.1.smithi192.stdout:7/616: symlink d1/d2/d37/d86/ldb 0 2024-02-16T22:56:22.781 INFO:tasks.workunit.client.0.smithi064.stdout:2/495: dwrite d7/d14/d1e/f81 [0,4194304] 0 2024-02-16T22:56:22.784 INFO:tasks.workunit.client.0.smithi064.stdout:2/496: write d7/de/f15 [2650760,24765] 0 2024-02-16T22:56:22.784 INFO:tasks.workunit.client.0.smithi064.stdout:4/445: dwrite dc/dd/d16/f5d [0,4194304] 0 2024-02-16T22:56:22.788 INFO:tasks.workunit.client.1.smithi192.stdout:5/492: rename d6/d8/d80/l48 to d6/d8/la4 0 2024-02-16T22:56:22.790 INFO:tasks.workunit.client.0.smithi064.stdout:3/515: rename d4/d1f/d5b/f9d to d4/d13/d1b/fb2 0 2024-02-16T22:56:22.793 INFO:tasks.workunit.client.0.smithi064.stdout:2/497: unlink d7/d14/d1e/c58 0 2024-02-16T22:56:22.795 INFO:tasks.workunit.client.1.smithi192.stdout:3/521: rename da/d11/d2b/d43/l57 to da/d11/d63/d6b/d66/d75/lb7 0 2024-02-16T22:56:22.797 INFO:tasks.workunit.client.1.smithi192.stdout:2/551: getdents d22 0 2024-02-16T22:56:22.799 INFO:tasks.workunit.client.1.smithi192.stdout:8/506: rename d5/db/de/df/f2b to d5/db/d47/d8c/f9c 0 2024-02-16T22:56:22.800 INFO:tasks.workunit.client.1.smithi192.stdout:8/507: dread - d5/d26/d83/d86/f93 zero size 2024-02-16T22:56:22.800 INFO:tasks.workunit.client.0.smithi064.stdout:3/516: unlink d4/d13/c1a 0 2024-02-16T22:56:22.801 INFO:tasks.workunit.client.1.smithi192.stdout:3/522: mknod da/d11/d35/cb8 0 2024-02-16T22:56:22.805 INFO:tasks.workunit.client.1.smithi192.stdout:5/493: mkdir d6/d12/d4e/da5 0 2024-02-16T22:56:22.805 INFO:tasks.workunit.client.0.smithi064.stdout:4/446: getdents dc/dd/d4f 0 2024-02-16T22:56:22.806 INFO:tasks.workunit.client.0.smithi064.stdout:4/447: fdatasync dc/dd/d32/d46/f55 0 2024-02-16T22:56:22.807 INFO:tasks.workunit.client.0.smithi064.stdout:4/448: rename dc/dd/d16 to dc/dd/d16/d6e/d85 22 2024-02-16T22:56:22.807 INFO:tasks.workunit.client.0.smithi064.stdout:4/449: truncate dc/f19 6242819 0 2024-02-16T22:56:22.808 INFO:tasks.workunit.client.0.smithi064.stdout:3/517: creat d4/d13/d1b/d4f/fb3 x:0 0 0 2024-02-16T22:56:22.811 INFO:tasks.workunit.client.1.smithi192.stdout:2/552: dread f4 [0,4194304] 0 2024-02-16T22:56:22.811 INFO:tasks.workunit.client.1.smithi192.stdout:9/647: rename d7/d15/d36/d54 to d7/d19/d5e/d6f/de1 0 2024-02-16T22:56:22.812 INFO:tasks.workunit.client.0.smithi064.stdout:4/450: fdatasync dc/dd/d16/f51 0 2024-02-16T22:56:22.813 INFO:tasks.workunit.client.1.smithi192.stdout:6/518: write da/df/d12/f51 [4969008,5665] 0 2024-02-16T22:56:22.813 INFO:tasks.workunit.client.1.smithi192.stdout:2/553: chown d22/d24/d39/d42/d65/d69/l79 1 1 2024-02-16T22:56:22.814 INFO:tasks.workunit.client.1.smithi192.stdout:8/508: mkdir d5/db/d47/d8c/d9d 0 2024-02-16T22:56:22.814 INFO:tasks.workunit.client.0.smithi064.stdout:6/470: creat d0/f91 x:0 0 0 2024-02-16T22:56:22.815 INFO:tasks.workunit.client.1.smithi192.stdout:2/554: dread - d22/d24/d39/d42/d65/f8e zero size 2024-02-16T22:56:22.816 INFO:tasks.workunit.client.0.smithi064.stdout:6/471: chown d0/d6/d10/f5e 21793 1 2024-02-16T22:56:22.821 INFO:tasks.workunit.client.1.smithi192.stdout:1/493: rename d3/de/d27/d34/d36/d47/d5d/l7b to d3/d40/lab 0 2024-02-16T22:56:22.823 INFO:tasks.workunit.client.0.smithi064.stdout:6/472: chown d0/d5/df/d17/d18/f3f 5066 1 2024-02-16T22:56:22.824 INFO:tasks.workunit.client.0.smithi064.stdout:4/451: creat dc/dd/d16/d7c/f86 x:0 0 0 2024-02-16T22:56:22.829 INFO:tasks.workunit.client.1.smithi192.stdout:6/519: rename da/df/d12/fa0 to da/df/d12/d15/d1b/faa 0 2024-02-16T22:56:22.835 INFO:tasks.workunit.client.0.smithi064.stdout:6/473: getdents d0/d5/df/d17 0 2024-02-16T22:56:22.836 INFO:tasks.workunit.client.0.smithi064.stdout:9/399: dwrite d3/d11/d15/d5a/f52 [0,4194304] 0 2024-02-16T22:56:22.837 INFO:tasks.workunit.client.0.smithi064.stdout:6/474: write d0/f91 [763772,98804] 0 2024-02-16T22:56:22.838 INFO:tasks.workunit.client.0.smithi064.stdout:9/400: fsync d3/d9/d1e/d42/f7c 0 2024-02-16T22:56:22.838 INFO:tasks.workunit.client.1.smithi192.stdout:1/494: mknod d3/cac 0 2024-02-16T22:56:22.843 INFO:tasks.workunit.client.1.smithi192.stdout:9/648: creat d7/d19/d28/fe2 x:0 0 0 2024-02-16T22:56:22.843 INFO:tasks.workunit.client.0.smithi064.stdout:6/475: dread d0/f91 [0,4194304] 0 2024-02-16T22:56:22.844 INFO:tasks.workunit.client.0.smithi064.stdout:9/401: creat d3/d11/d15/d5a/d2a/d2e/d31/f84 x:0 0 0 2024-02-16T22:56:22.844 INFO:tasks.workunit.client.0.smithi064.stdout:6/476: chown d0/d12/l53 22987170 1 2024-02-16T22:56:22.846 INFO:tasks.workunit.client.1.smithi192.stdout:3/523: getdents da 0 2024-02-16T22:56:22.846 INFO:tasks.workunit.client.1.smithi192.stdout:3/524: stat da/df/f3a 0 2024-02-16T22:56:22.847 INFO:tasks.workunit.client.1.smithi192.stdout:5/494: getdents d6/d20 0 2024-02-16T22:56:22.849 INFO:tasks.workunit.client.0.smithi064.stdout:6/477: rmdir d0/d6/da/d56 39 2024-02-16T22:56:22.851 INFO:tasks.workunit.client.1.smithi192.stdout:1/495: symlink d3/de/d27/d34/d36/d47/d5d/lad 0 2024-02-16T22:56:22.852 INFO:tasks.workunit.client.1.smithi192.stdout:0/531: write d1/d3/d39/f5f [394103,118246] 0 2024-02-16T22:56:22.853 INFO:tasks.workunit.client.1.smithi192.stdout:6/520: mkdir da/df/d71/dab 0 2024-02-16T22:56:22.854 INFO:tasks.workunit.client.1.smithi192.stdout:0/532: chown d1/d3/de/d22/d37/d67/d34/f8a 187 1 2024-02-16T22:56:22.855 INFO:tasks.workunit.client.0.smithi064.stdout:6/478: truncate d0/d6/d10/d33/f55 215819 0 2024-02-16T22:56:22.855 INFO:tasks.workunit.client.1.smithi192.stdout:9/649: truncate d7/d19/f94 599002 0 2024-02-16T22:56:22.857 INFO:tasks.workunit.client.1.smithi192.stdout:3/525: dread - da/d11/d20/f41 zero size 2024-02-16T22:56:22.858 INFO:tasks.workunit.client.0.smithi064.stdout:6/479: creat d0/d85/f92 x:0 0 0 2024-02-16T22:56:22.859 INFO:tasks.workunit.client.1.smithi192.stdout:9/650: dread d7/d15/d36/f7b [0,4194304] 0 2024-02-16T22:56:22.859 INFO:tasks.workunit.client.1.smithi192.stdout:5/495: chown d6/d8/d75/c9f 1287211117 1 2024-02-16T22:56:22.862 INFO:tasks.workunit.client.1.smithi192.stdout:1/496: rename d3/l1c to d3/d40/lae 0 2024-02-16T22:56:22.862 INFO:tasks.workunit.client.1.smithi192.stdout:1/497: dread - d3/de/d27/d34/d69/f9d zero size 2024-02-16T22:56:22.863 INFO:tasks.workunit.client.0.smithi064.stdout:0/365: dwrite da/f10 [0,4194304] 0 2024-02-16T22:56:22.865 INFO:tasks.workunit.client.0.smithi064.stdout:9/402: link d3/d9/f12 d3/d11/d15/d5a/f85 0 2024-02-16T22:56:22.871 INFO:tasks.workunit.client.0.smithi064.stdout:0/366: chown da/f4d 7111 1 2024-02-16T22:56:22.872 INFO:tasks.workunit.client.0.smithi064.stdout:1/377: truncate d2/df/d14/d2c/f3c 192194 0 2024-02-16T22:56:22.873 INFO:tasks.workunit.client.0.smithi064.stdout:1/378: fdatasync d2/df/d14/d2c/d62/f72 0 2024-02-16T22:56:22.874 INFO:tasks.workunit.client.0.smithi064.stdout:9/403: mkdir d3/d9/d1e/d42/d7b/d86 0 2024-02-16T22:56:22.875 INFO:tasks.workunit.client.1.smithi192.stdout:5/496: symlink d6/d8/d77/la6 0 2024-02-16T22:56:22.877 INFO:tasks.workunit.client.1.smithi192.stdout:1/498: creat d3/de/d27/d34/d36/d47/faf x:0 0 0 2024-02-16T22:56:22.878 INFO:tasks.workunit.client.1.smithi192.stdout:6/521: symlink da/d76/lac 0 2024-02-16T22:56:22.879 INFO:tasks.workunit.client.1.smithi192.stdout:6/522: dread - da/df/d12/d15/d1b/fa3 zero size 2024-02-16T22:56:22.879 INFO:tasks.workunit.client.1.smithi192.stdout:6/523: chown da/df/d6b 3415784 1 2024-02-16T22:56:22.879 INFO:tasks.workunit.client.0.smithi064.stdout:9/404: creat d3/d9/d1e/d3e/f87 x:0 0 0 2024-02-16T22:56:22.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/524: dread - da/df/d18/d46/f80 zero size 2024-02-16T22:56:22.881 INFO:tasks.workunit.client.0.smithi064.stdout:9/405: read d3/d11/d15/d5a/f56 [1417156,75389] 0 2024-02-16T22:56:22.883 INFO:tasks.workunit.client.0.smithi064.stdout:9/406: read d3/d11/d15/d5a/f27 [3420553,11602] 0 2024-02-16T22:56:22.884 INFO:tasks.workunit.client.0.smithi064.stdout:9/407: write d3/d9/f60 [4539944,95493] 0 2024-02-16T22:56:22.886 INFO:tasks.workunit.client.0.smithi064.stdout:1/379: dread d2/df/d11/d1a/d1d/f24 [0,4194304] 0 2024-02-16T22:56:22.886 INFO:tasks.workunit.client.1.smithi192.stdout:9/651: mknod d7/d19/d28/ce3 0 2024-02-16T22:56:22.887 INFO:tasks.workunit.client.0.smithi064.stdout:2/498: rmdir d7 39 2024-02-16T22:56:22.888 INFO:tasks.workunit.client.1.smithi192.stdout:9/652: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/da2/lde 7786 1 2024-02-16T22:56:22.888 INFO:tasks.workunit.client.0.smithi064.stdout:0/367: getdents da/d26 0 2024-02-16T22:56:22.892 INFO:tasks.workunit.client.1.smithi192.stdout:1/499: mkdir d3/de/d27/d34/d52/d63/d7a/db0 0 2024-02-16T22:56:22.896 INFO:tasks.workunit.client.1.smithi192.stdout:6/525: rename da/df/d18/d52/f66 to da/df/d71/dab/fad 0 2024-02-16T22:56:22.897 INFO:tasks.workunit.client.0.smithi064.stdout:1/380: mkdir d2/df/d14/d18/d48/d84 0 2024-02-16T22:56:22.901 INFO:tasks.workunit.client.0.smithi064.stdout:0/368: dread da/d26/d3f/f47 [0,4194304] 0 2024-02-16T22:56:22.902 INFO:tasks.workunit.client.1.smithi192.stdout:6/526: dwrite da/df/d18/d52/f7c [0,4194304] 0 2024-02-16T22:56:22.902 INFO:tasks.workunit.client.0.smithi064.stdout:9/408: mkdir d3/d88 0 2024-02-16T22:56:22.903 INFO:tasks.workunit.client.0.smithi064.stdout:3/518: truncate d4/d1f/f67 3238493 0 2024-02-16T22:56:22.908 INFO:tasks.workunit.client.0.smithi064.stdout:4/452: truncate dc/dd/f59 262640 0 2024-02-16T22:56:22.912 INFO:tasks.workunit.client.1.smithi192.stdout:1/500: mkdir d3/d5b/db1 0 2024-02-16T22:56:22.912 INFO:tasks.workunit.client.1.smithi192.stdout:6/527: dwrite da/df/d12/f75 [0,4194304] 0 2024-02-16T22:56:22.912 INFO:tasks.workunit.client.1.smithi192.stdout:2/555: truncate d22/f41 519620 0 2024-02-16T22:56:22.913 INFO:tasks.workunit.client.0.smithi064.stdout:0/369: creat da/d26/d3f/f75 x:0 0 0 2024-02-16T22:56:22.913 INFO:tasks.workunit.client.0.smithi064.stdout:0/370: fdatasync da/d26/d3f/f75 0 2024-02-16T22:56:22.915 INFO:tasks.workunit.client.0.smithi064.stdout:8/399: sync 2024-02-16T22:56:22.920 INFO:tasks.workunit.client.1.smithi192.stdout:4/605: sync 2024-02-16T22:56:22.920 INFO:tasks.workunit.client.0.smithi064.stdout:0/371: dwrite da/dc/de/d37/f46 [4194304,4194304] 0 2024-02-16T22:56:22.922 INFO:tasks.workunit.client.1.smithi192.stdout:1/501: symlink d3/d5b/lb2 0 2024-02-16T22:56:22.922 INFO:tasks.workunit.client.0.smithi064.stdout:0/372: fsync da/dc/de/d12/d1d/f34 0 2024-02-16T22:56:22.925 INFO:tasks.workunit.client.1.smithi192.stdout:2/556: chown l17 102037016 1 2024-02-16T22:56:22.925 INFO:tasks.workunit.client.0.smithi064.stdout:6/480: write d0/d1d/d21/f7f [584096,64611] 0 2024-02-16T22:56:22.927 INFO:tasks.workunit.client.1.smithi192.stdout:6/528: symlink da/df/d18/d52/lae 0 2024-02-16T22:56:22.928 INFO:tasks.workunit.client.1.smithi192.stdout:4/606: dwrite d1/d15/d48/d8d/fba [0,4194304] 0 2024-02-16T22:56:22.931 INFO:tasks.workunit.client.1.smithi192.stdout:6/529: chown da/df/d12/d15/d1b/d70/l98 155312 1 2024-02-16T22:56:22.932 INFO:tasks.workunit.client.0.smithi064.stdout:3/519: symlink d4/d13/d1b/d27/d44/lb4 0 2024-02-16T22:56:22.935 INFO:tasks.workunit.client.1.smithi192.stdout:0/533: write d1/d3/d39/fa3 [579474,45015] 0 2024-02-16T22:56:22.935 INFO:tasks.workunit.client.1.smithi192.stdout:3/526: write da/d11/d63/fae [443908,63886] 0 2024-02-16T22:56:22.938 INFO:tasks.workunit.client.1.smithi192.stdout:2/557: dread fa [0,4194304] 0 2024-02-16T22:56:22.939 INFO:tasks.workunit.client.0.smithi064.stdout:1/381: creat d2/df/d11/d25/f85 x:0 0 0 2024-02-16T22:56:22.940 INFO:tasks.workunit.client.0.smithi064.stdout:1/382: chown d2/df/c2e 1 1 2024-02-16T22:56:22.941 INFO:tasks.workunit.client.0.smithi064.stdout:1/383: write d2/df/d14/f2d [3330957,68604] 0 2024-02-16T22:56:22.944 INFO:tasks.workunit.client.1.smithi192.stdout:6/530: mkdir da/df/d12/d57/daf 0 2024-02-16T22:56:22.945 INFO:tasks.workunit.client.1.smithi192.stdout:6/531: fdatasync da/df/d12/d15/d1b/f8a 0 2024-02-16T22:56:22.945 INFO:tasks.workunit.client.0.smithi064.stdout:4/453: link dc/dd/d32/d46/d5a/f66 dc/de/d6d/f87 0 2024-02-16T22:56:22.945 INFO:tasks.workunit.client.1.smithi192.stdout:9/653: write d7/d15/d36/d38/d3f/d8a/d98/f22 [45015,81849] 0 2024-02-16T22:56:22.946 INFO:tasks.workunit.client.1.smithi192.stdout:1/502: creat d3/de/d27/d34/d69/d85/d89/daa/fb3 x:0 0 0 2024-02-16T22:56:22.947 INFO:tasks.workunit.client.0.smithi064.stdout:0/373: read - da/dc/de/d37/f4e zero size 2024-02-16T22:56:22.949 INFO:tasks.workunit.client.1.smithi192.stdout:5/497: truncate d6/f33 538586 0 2024-02-16T22:56:22.951 INFO:tasks.workunit.client.1.smithi192.stdout:6/532: dwrite da/df/d12/f75 [0,4194304] 0 2024-02-16T22:56:22.951 INFO:tasks.workunit.client.0.smithi064.stdout:6/481: truncate d0/d1b/f23 4465195 0 2024-02-16T22:56:22.952 INFO:tasks.workunit.client.0.smithi064.stdout:6/482: write d0/d5/df/d17/f62 [414561,76274] 0 2024-02-16T22:56:22.953 INFO:tasks.workunit.client.0.smithi064.stdout:6/483: read d0/d5/df/f6f [862647,20487] 0 2024-02-16T22:56:22.955 INFO:tasks.workunit.client.1.smithi192.stdout:3/527: mkdir da/d11/d63/d6b/d71/da1/db9 0 2024-02-16T22:56:22.958 INFO:tasks.workunit.client.0.smithi064.stdout:8/400: mknod df/d31/d2f/d38/d48/d69/c8a 0 2024-02-16T22:56:22.959 INFO:tasks.workunit.client.1.smithi192.stdout:2/558: dread - d22/d24/d2b/d2c/f75 zero size 2024-02-16T22:56:22.960 INFO:tasks.workunit.client.0.smithi064.stdout:3/520: mknod d4/d13/d1b/d27/d61/d63/cb5 0 2024-02-16T22:56:22.962 INFO:tasks.workunit.client.1.smithi192.stdout:4/607: creat d1/d26/d34/dca/fd1 x:0 0 0 2024-02-16T22:56:22.963 INFO:tasks.workunit.client.0.smithi064.stdout:2/499: getdents d7/de/d95/d44 0 2024-02-16T22:56:22.965 INFO:tasks.workunit.client.0.smithi064.stdout:7/401: sync 2024-02-16T22:56:22.965 INFO:tasks.workunit.client.0.smithi064.stdout:5/342: sync 2024-02-16T22:56:22.965 INFO:tasks.workunit.client.0.smithi064.stdout:8/401: dwrite df/d31/d2f/d38/f5d [0,4194304] 0 2024-02-16T22:56:22.967 INFO:tasks.workunit.client.0.smithi064.stdout:5/343: dread - d6/d1c/d43/f4c zero size 2024-02-16T22:56:22.969 INFO:tasks.workunit.client.0.smithi064.stdout:4/454: mknod dc/de/d28/c88 0 2024-02-16T22:56:22.970 INFO:tasks.workunit.client.0.smithi064.stdout:5/344: write d6/d1c/d43/f4d [176948,80657] 0 2024-02-16T22:56:22.977 INFO:tasks.workunit.client.1.smithi192.stdout:5/498: stat d6/d12/d4e/d5b/l90 0 2024-02-16T22:56:22.979 INFO:tasks.workunit.client.1.smithi192.stdout:0/534: mknod d1/d3/de/d22/da7/caa 0 2024-02-16T22:56:22.979 INFO:tasks.workunit.client.1.smithi192.stdout:8/509: sync 2024-02-16T22:56:22.979 INFO:tasks.workunit.client.1.smithi192.stdout:7/617: sync 2024-02-16T22:56:22.981 INFO:tasks.workunit.client.0.smithi064.stdout:6/484: creat d0/d89/f93 x:0 0 0 2024-02-16T22:56:22.981 INFO:tasks.workunit.client.1.smithi192.stdout:0/535: truncate d1/d3/de/d22/d37/f87 503829 0 2024-02-16T22:56:22.982 INFO:tasks.workunit.client.0.smithi064.stdout:3/521: symlink d4/d13/d1b/d27/d44/lb6 0 2024-02-16T22:56:22.982 INFO:tasks.workunit.client.1.smithi192.stdout:6/533: write da/df/d71/dab/fad [468092,71114] 0 2024-02-16T22:56:22.984 INFO:tasks.workunit.client.0.smithi064.stdout:5/345: dwrite d6/d18/d19/f47 [0,4194304] 0 2024-02-16T22:56:22.985 INFO:tasks.workunit.client.0.smithi064.stdout:0/374: dwrite da/f4d [0,4194304] 0 2024-02-16T22:56:22.986 INFO:tasks.workunit.client.1.smithi192.stdout:3/528: symlink da/df/lba 0 2024-02-16T22:56:22.987 INFO:tasks.workunit.client.0.smithi064.stdout:2/500: rmdir d7/de/d95 39 2024-02-16T22:56:22.989 INFO:tasks.workunit.client.1.smithi192.stdout:2/559: mknod d22/d24/d39/d42/d84/cba 0 2024-02-16T22:56:22.990 INFO:tasks.workunit.client.1.smithi192.stdout:2/560: dread - d22/d24/fb9 zero size 2024-02-16T22:56:22.992 INFO:tasks.workunit.client.1.smithi192.stdout:4/608: mknod d1/d26/dbe/cd2 0 2024-02-16T22:56:22.994 INFO:tasks.workunit.client.0.smithi064.stdout:2/501: dwrite d7/d14/d1e/d34/d3f/f4e [0,4194304] 0 2024-02-16T22:56:22.995 INFO:tasks.workunit.client.1.smithi192.stdout:9/654: symlink d7/d19/d37/dbf/le4 0 2024-02-16T22:56:22.999 INFO:tasks.workunit.client.1.smithi192.stdout:1/503: fsync d3/de/f1b 0 2024-02-16T22:56:23.004 INFO:tasks.workunit.client.0.smithi064.stdout:7/402: rename d5/d6/d54/d41/f4a to d5/d6/d54/d41/f7e 0 2024-02-16T22:56:23.005 INFO:tasks.workunit.client.0.smithi064.stdout:2/502: dwrite d7/d14/d25/d46/d55/f71 [0,4194304] 0 2024-02-16T22:56:23.007 INFO:tasks.workunit.client.0.smithi064.stdout:8/402: rename df/d31/d2f/d4f/d6a/c6f to df/d31/d2f/d38/d48/d69/c8b 0 2024-02-16T22:56:23.009 INFO:tasks.workunit.client.1.smithi192.stdout:8/510: creat d5/db/d47/d43/d67/f9e x:0 0 0 2024-02-16T22:56:23.011 INFO:tasks.workunit.client.0.smithi064.stdout:4/455: unlink dc/de/c11 0 2024-02-16T22:56:23.012 INFO:tasks.workunit.client.0.smithi064.stdout:1/384: write d2/f42 [1491771,33359] 0 2024-02-16T22:56:23.016 INFO:tasks.workunit.client.1.smithi192.stdout:7/618: rename d1/d2/d2d/d61/d88/d90/d4c/c8e to d1/d2/d2d/d61/d88/d90/d2a/cdc 0 2024-02-16T22:56:23.020 INFO:tasks.workunit.client.1.smithi192.stdout:3/529: mknod da/d11/d35/cbb 0 2024-02-16T22:56:23.022 INFO:tasks.workunit.client.1.smithi192.stdout:2/561: creat d22/d24/d39/d3d/fbb x:0 0 0 2024-02-16T22:56:23.026 INFO:tasks.workunit.client.1.smithi192.stdout:1/504: creat d3/de/d27/d34/d52/fb4 x:0 0 0 2024-02-16T22:56:23.026 INFO:tasks.workunit.client.1.smithi192.stdout:1/505: stat d3/de/f1a 0 2024-02-16T22:56:23.028 INFO:tasks.workunit.client.1.smithi192.stdout:9/655: dread d7/d19/d28/d43/f57 [0,4194304] 0 2024-02-16T22:56:23.028 INFO:tasks.workunit.client.1.smithi192.stdout:5/499: creat d6/d20/d93/fa7 x:0 0 0 2024-02-16T22:56:23.030 INFO:tasks.workunit.client.1.smithi192.stdout:5/500: write d6/d8/fa1 [578394,21886] 0 2024-02-16T22:56:23.030 INFO:tasks.workunit.client.1.smithi192.stdout:8/511: chown d5/d26/d8f/l7e 387124694 1 2024-02-16T22:56:23.032 INFO:tasks.workunit.client.1.smithi192.stdout:0/536: truncate d1/f24 1436426 0 2024-02-16T22:56:23.032 INFO:tasks.workunit.client.1.smithi192.stdout:5/501: read d6/d12/d4e/d46/f6b [763063,4274] 0 2024-02-16T22:56:23.032 INFO:tasks.workunit.client.1.smithi192.stdout:5/502: chown d6/d12/d2d/l5a 54 1 2024-02-16T22:56:23.034 INFO:tasks.workunit.client.1.smithi192.stdout:5/503: truncate d6/d12/d2d/d5c/f99 757321 0 2024-02-16T22:56:23.035 INFO:tasks.workunit.client.1.smithi192.stdout:4/609: dread d1/d26/d66/d89/d11/d58/f7e [0,4194304] 0 2024-02-16T22:56:23.036 INFO:tasks.workunit.client.1.smithi192.stdout:4/610: chown d1/d26/dbe 34278597 1 2024-02-16T22:56:23.036 INFO:tasks.workunit.client.0.smithi064.stdout:9/409: getdents d3/d11 0 2024-02-16T22:56:23.037 INFO:tasks.workunit.client.1.smithi192.stdout:2/562: symlink d22/d24/d2b/d67/lbc 0 2024-02-16T22:56:23.040 INFO:tasks.workunit.client.1.smithi192.stdout:9/656: creat d7/d19/fe5 x:0 0 0 2024-02-16T22:56:23.041 INFO:tasks.workunit.client.0.smithi064.stdout:5/346: symlink d6/d1c/l75 0 2024-02-16T22:56:23.049 INFO:tasks.workunit.client.0.smithi064.stdout:7/403: fsync d5/d6/d54/f78 0 2024-02-16T22:56:23.053 INFO:tasks.workunit.client.0.smithi064.stdout:8/403: creat df/d31/d61/d79/f8c x:0 0 0 2024-02-16T22:56:23.055 INFO:tasks.workunit.client.0.smithi064.stdout:4/456: symlink dc/dd/d32/d46/d5a/l89 0 2024-02-16T22:56:23.056 INFO:tasks.workunit.client.0.smithi064.stdout:4/457: rename dc/d1d/d43 to dc/d1d/d43/d8a 22 2024-02-16T22:56:23.057 INFO:tasks.workunit.client.0.smithi064.stdout:1/385: chown d2/df/d14/d18/l5d 1344 1 2024-02-16T22:56:23.059 INFO:tasks.workunit.client.0.smithi064.stdout:8/404: dread df/d31/f29 [0,4194304] 0 2024-02-16T22:56:23.059 INFO:tasks.workunit.client.0.smithi064.stdout:1/386: truncate d2/df/d14/d2c/d62/f72 536642 0 2024-02-16T22:56:23.060 INFO:tasks.workunit.client.0.smithi064.stdout:5/347: unlink d6/f29 0 2024-02-16T22:56:23.062 INFO:tasks.workunit.client.0.smithi064.stdout:2/503: fsync d7/de/d95/f47 0 2024-02-16T22:56:23.064 INFO:tasks.workunit.client.0.smithi064.stdout:4/458: creat dc/d1d/f8b x:0 0 0 2024-02-16T22:56:23.066 INFO:tasks.workunit.client.0.smithi064.stdout:9/410: mkdir d3/d89 0 2024-02-16T22:56:23.067 INFO:tasks.workunit.client.0.smithi064.stdout:3/522: creat d4/d14/fb7 x:0 0 0 2024-02-16T22:56:23.069 INFO:tasks.workunit.client.0.smithi064.stdout:0/375: creat da/dc/de/f76 x:0 0 0 2024-02-16T22:56:23.070 INFO:tasks.workunit.client.0.smithi064.stdout:8/405: truncate df/f15 1081540 0 2024-02-16T22:56:23.073 INFO:tasks.workunit.client.0.smithi064.stdout:1/387: rename d2/df/d11/d1a to d2/df/d11/d1a/d86 22 2024-02-16T22:56:23.075 INFO:tasks.workunit.client.0.smithi064.stdout:7/404: mkdir d5/d6/d65/d7f 0 2024-02-16T22:56:23.075 INFO:tasks.workunit.client.0.smithi064.stdout:1/388: readlink d2/df/d14/l20 0 2024-02-16T22:56:23.076 INFO:tasks.workunit.client.0.smithi064.stdout:7/405: chown d5/d33/d45/f46 10563933 1 2024-02-16T22:56:23.076 INFO:tasks.workunit.client.0.smithi064.stdout:1/389: readlink d2/df/d14/d2a/l3a 0 2024-02-16T22:56:23.077 INFO:tasks.workunit.client.0.smithi064.stdout:2/504: mknod d7/d14/d25/d46/c96 0 2024-02-16T22:56:23.077 INFO:tasks.workunit.client.0.smithi064.stdout:0/376: dwrite da/dc/de/d12/d2c/f31 [0,4194304] 0 2024-02-16T22:56:23.083 INFO:tasks.workunit.client.0.smithi064.stdout:3/523: dread d4/f18 [0,4194304] 0 2024-02-16T22:56:23.084 INFO:tasks.workunit.client.0.smithi064.stdout:8/406: dread - df/d31/d2f/d38/d48/f54 zero size 2024-02-16T22:56:23.086 INFO:tasks.workunit.client.0.smithi064.stdout:4/459: symlink dc/de/d28/d57/d77/l8c 0 2024-02-16T22:56:23.087 INFO:tasks.workunit.client.0.smithi064.stdout:4/460: truncate dc/dd/d32/d46/d5a/f66 392689 0 2024-02-16T22:56:23.089 INFO:tasks.workunit.client.0.smithi064.stdout:4/461: write dc/dd/d16/f5e [941147,91912] 0 2024-02-16T22:56:23.090 INFO:tasks.workunit.client.0.smithi064.stdout:7/406: creat d5/d33/d45/d64/d56/d5d/f80 x:0 0 0 2024-02-16T22:56:23.090 INFO:tasks.workunit.client.0.smithi064.stdout:4/462: write dc/dd/d32/d46/f55 [3611075,36832] 0 2024-02-16T22:56:23.091 INFO:tasks.workunit.client.0.smithi064.stdout:7/407: fdatasync d5/d33/d53/f75 0 2024-02-16T22:56:23.092 INFO:tasks.workunit.client.0.smithi064.stdout:2/505: creat d7/d14/d25/d46/f97 x:0 0 0 2024-02-16T22:56:23.093 INFO:tasks.workunit.client.0.smithi064.stdout:7/408: dread - d5/d33/d45/d64/d56/d5d/f80 zero size 2024-02-16T22:56:23.093 INFO:tasks.workunit.client.0.smithi064.stdout:7/409: readlink d5/d6/l2e 0 2024-02-16T22:56:23.094 INFO:tasks.workunit.client.0.smithi064.stdout:7/410: chown d5/d33/d45/d64/d56/d5d 11 1 2024-02-16T22:56:23.095 INFO:tasks.workunit.client.0.smithi064.stdout:7/411: stat d5/l14 0 2024-02-16T22:56:23.096 INFO:tasks.workunit.client.0.smithi064.stdout:3/524: creat d4/d13/d4d/fb8 x:0 0 0 2024-02-16T22:56:23.097 INFO:tasks.workunit.client.1.smithi192.stdout:1/506: truncate d3/de/d27/d34/f4e 687914 0 2024-02-16T22:56:23.097 INFO:tasks.workunit.client.1.smithi192.stdout:8/512: creat d5/d26/d83/d86/d5f/f9f x:0 0 0 2024-02-16T22:56:23.099 INFO:tasks.workunit.client.0.smithi064.stdout:8/407: unlink df/d31/d2f/d4f/l5b 0 2024-02-16T22:56:23.099 INFO:tasks.workunit.client.1.smithi192.stdout:9/657: truncate d7/d19/d28/d43/fc9 128699 0 2024-02-16T22:56:23.099 INFO:tasks.workunit.client.1.smithi192.stdout:3/530: truncate da/d11/d63/d6b/d71/f7b 93938 0 2024-02-16T22:56:23.100 INFO:tasks.workunit.client.0.smithi064.stdout:0/377: mknod da/c77 0 2024-02-16T22:56:23.101 INFO:tasks.workunit.client.1.smithi192.stdout:9/658: chown d7/d15/d36/d38/d3f/d8a/d98/l1f 5 1 2024-02-16T22:56:23.101 INFO:tasks.workunit.client.0.smithi064.stdout:7/412: dwrite d5/d6/f3b [0,4194304] 0 2024-02-16T22:56:23.102 INFO:tasks.workunit.client.1.smithi192.stdout:3/531: chown da/d11/d20/fab 50494399 1 2024-02-16T22:56:23.102 INFO:tasks.workunit.client.1.smithi192.stdout:7/619: dread d1/d2/d2d/d61/d88/d90/d2a/fb8 [0,4194304] 0 2024-02-16T22:56:23.102 INFO:tasks.workunit.client.0.smithi064.stdout:7/413: write d5/d33/f67 [526220,20735] 0 2024-02-16T22:56:23.103 INFO:tasks.workunit.client.1.smithi192.stdout:6/534: link da/df/d12/d15/c78 da/df/d12/d57/d81/cb0 0 2024-02-16T22:56:23.105 INFO:tasks.workunit.client.1.smithi192.stdout:1/507: dwrite d3/f50 [0,4194304] 0 2024-02-16T22:56:23.105 INFO:tasks.workunit.client.0.smithi064.stdout:7/414: chown d5/d33/d45/d64/d56/d66/l6b 1065 1 2024-02-16T22:56:23.109 INFO:tasks.workunit.client.0.smithi064.stdout:5/348: dwrite d6/d1c/f20 [0,4194304] 0 2024-02-16T22:56:23.113 INFO:tasks.workunit.client.1.smithi192.stdout:2/563: symlink d22/d24/d39/d42/d65/d69/db1/lbd 0 2024-02-16T22:56:23.116 INFO:tasks.workunit.client.0.smithi064.stdout:2/506: truncate d7/de/f2f 443601 0 2024-02-16T22:56:23.120 INFO:tasks.workunit.client.0.smithi064.stdout:7/415: dread d5/d6/d54/f78 [0,4194304] 0 2024-02-16T22:56:23.122 INFO:tasks.workunit.client.1.smithi192.stdout:4/611: rename d1/d15/d29/d3c/cd0 to d1/d15/d48/dc3/cd3 0 2024-02-16T22:56:23.128 INFO:tasks.workunit.client.1.smithi192.stdout:9/659: mkdir d7/d19/d5b/de6 0 2024-02-16T22:56:23.131 INFO:tasks.workunit.client.0.smithi064.stdout:9/411: link d3/d11/d15/f51 d3/f8a 0 2024-02-16T22:56:23.134 INFO:tasks.workunit.client.0.smithi064.stdout:1/390: link d2/df/d11/d1a/d1d/f4d d2/df/d14/d2a/f87 0 2024-02-16T22:56:23.136 INFO:tasks.workunit.client.1.smithi192.stdout:3/532: dwrite da/d11/d2b/f5f [0,4194304] 0 2024-02-16T22:56:23.140 INFO:tasks.workunit.client.0.smithi064.stdout:7/416: write d5/d6/d54/d41/f7e [1650771,10690] 0 2024-02-16T22:56:23.143 INFO:tasks.workunit.client.1.smithi192.stdout:6/535: rename da/df/d18/d52/f4a to da/df/d12/d15/d1b/d70/fb1 0 2024-02-16T22:56:23.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/536: chown da/d40/d85/f87 1 1 2024-02-16T22:56:23.145 INFO:tasks.workunit.client.0.smithi064.stdout:8/408: dwrite df/f1b [0,4194304] 0 2024-02-16T22:56:23.146 INFO:tasks.workunit.client.1.smithi192.stdout:9/660: creat d7/d15/d36/fe7 x:0 0 0 2024-02-16T22:56:23.151 INFO:tasks.workunit.client.0.smithi064.stdout:4/463: link dc/dd/d32/d58/f70 dc/dd/d32/f8d 0 2024-02-16T22:56:23.153 INFO:tasks.workunit.client.0.smithi064.stdout:4/464: dread dc/dd/d32/d46/d5a/f66 [0,4194304] 0 2024-02-16T22:56:23.155 INFO:tasks.workunit.client.0.smithi064.stdout:5/349: chown d6/d18/d19/l22 82 1 2024-02-16T22:56:23.157 INFO:tasks.workunit.client.0.smithi064.stdout:4/465: dread dc/dd/d16/f5d [0,4194304] 0 2024-02-16T22:56:23.158 INFO:tasks.workunit.client.1.smithi192.stdout:2/564: dread f19 [0,4194304] 0 2024-02-16T22:56:23.158 INFO:tasks.workunit.client.1.smithi192.stdout:7/620: truncate d1/d2/d2d/d61/d88/d90/d2a/f63 4105093 0 2024-02-16T22:56:23.160 INFO:tasks.workunit.client.1.smithi192.stdout:5/504: getdents d6/d12/d4e/d46 0 2024-02-16T22:56:23.160 INFO:tasks.workunit.client.0.smithi064.stdout:3/525: creat d4/fb9 x:0 0 0 2024-02-16T22:56:23.161 INFO:tasks.workunit.client.1.smithi192.stdout:5/505: chown d6/d8/fa3 80966 1 2024-02-16T22:56:23.161 INFO:tasks.workunit.client.1.smithi192.stdout:3/533: mknod da/d11/d2b/d43/cbc 0 2024-02-16T22:56:23.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/506: fdatasync d6/fc 0 2024-02-16T22:56:23.163 INFO:tasks.workunit.client.0.smithi064.stdout:7/417: mkdir d5/d33/d42/d81 0 2024-02-16T22:56:23.163 INFO:tasks.workunit.client.1.smithi192.stdout:3/534: read - da/d11/d20/d39/fad zero size 2024-02-16T22:56:23.165 INFO:tasks.workunit.client.1.smithi192.stdout:2/565: dwrite d22/f3c [0,4194304] 0 2024-02-16T22:56:23.167 INFO:tasks.workunit.client.0.smithi064.stdout:0/378: link da/dc/de/d12/d2c/l58 da/d71/l78 0 2024-02-16T22:56:23.167 INFO:tasks.workunit.client.1.smithi192.stdout:6/537: creat da/df/d12/d57/d81/fb2 x:0 0 0 2024-02-16T22:56:23.174 INFO:tasks.workunit.client.0.smithi064.stdout:1/391: fdatasync d2/df/d14/f59 0 2024-02-16T22:56:23.180 INFO:tasks.workunit.client.1.smithi192.stdout:4/612: write d1/f4 [160067,23657] 0 2024-02-16T22:56:23.182 INFO:tasks.workunit.client.1.smithi192.stdout:8/513: getdents d5/d26/d83/d86 0 2024-02-16T22:56:23.183 INFO:tasks.workunit.client.0.smithi064.stdout:5/350: creat d6/d18/d19/d1a/f76 x:0 0 0 2024-02-16T22:56:23.183 INFO:tasks.workunit.client.0.smithi064.stdout:9/412: write d3/d9/ff [1667375,113800] 0 2024-02-16T22:56:23.185 INFO:tasks.workunit.client.0.smithi064.stdout:4/466: creat dc/dd/d32/d58/f8e x:0 0 0 2024-02-16T22:56:23.185 INFO:tasks.workunit.client.0.smithi064.stdout:4/467: fsync dc/dd/d32/d58/f38 0 2024-02-16T22:56:23.187 INFO:tasks.workunit.client.0.smithi064.stdout:2/507: link d7/de/c2a d7/d14/d25/d46/d55/c98 0 2024-02-16T22:56:23.188 INFO:tasks.workunit.client.1.smithi192.stdout:6/538: mknod da/df/d12/d15/d1b/cb3 0 2024-02-16T22:56:23.190 INFO:tasks.workunit.client.1.smithi192.stdout:1/508: getdents d3/de/d27/d34/d52/d5a 0 2024-02-16T22:56:23.191 INFO:tasks.workunit.client.1.smithi192.stdout:7/621: mkdir d1/d2/de/d2c/d49/d54/d9d/ddd 0 2024-02-16T22:56:23.192 INFO:tasks.workunit.client.0.smithi064.stdout:9/413: dwrite d3/d11/d15/d23/f73 [0,4194304] 0 2024-02-16T22:56:23.192 INFO:tasks.workunit.client.0.smithi064.stdout:0/379: rmdir da/dc/de/d12 39 2024-02-16T22:56:23.196 INFO:tasks.workunit.client.0.smithi064.stdout:8/409: link df/d31/d2f/f44 df/d31/d2f/d4f/f8d 0 2024-02-16T22:56:23.197 INFO:tasks.workunit.client.0.smithi064.stdout:3/526: dwrite d4/d13/d1b/d4f/fa2 [0,4194304] 0 2024-02-16T22:56:23.197 INFO:tasks.workunit.client.1.smithi192.stdout:8/514: dread d5/d26/d83/d86/f74 [0,4194304] 0 2024-02-16T22:56:23.197 INFO:tasks.workunit.client.1.smithi192.stdout:9/661: link d7/d15/d36/c4f d7/d19/d5e/ce8 0 2024-02-16T22:56:23.198 INFO:tasks.workunit.client.0.smithi064.stdout:9/414: dread - d3/d9/d1e/d3e/d76/f79 zero size 2024-02-16T22:56:23.199 INFO:tasks.workunit.client.0.smithi064.stdout:9/415: rename d3/d11 to d3/d11/d8b 22 2024-02-16T22:56:23.201 INFO:tasks.workunit.client.1.smithi192.stdout:6/539: read - da/df/f6e zero size 2024-02-16T22:56:23.203 INFO:tasks.workunit.client.1.smithi192.stdout:6/540: dread - da/df/d12/d57/d81/fb2 zero size 2024-02-16T22:56:23.203 INFO:tasks.workunit.client.1.smithi192.stdout:1/509: creat d3/de/d27/d34/d69/d85/d89/daa/fb5 x:0 0 0 2024-02-16T22:56:23.204 INFO:tasks.workunit.client.0.smithi064.stdout:5/351: creat d6/d18/d39/f77 x:0 0 0 2024-02-16T22:56:23.204 INFO:tasks.workunit.client.0.smithi064.stdout:5/352: symlink d6/d1c/d4b/l78 2 2024-02-16T22:56:23.205 INFO:tasks.workunit.client.1.smithi192.stdout:1/510: fsync d3/d5b/f84 0 2024-02-16T22:56:23.207 INFO:tasks.workunit.client.1.smithi192.stdout:2/566: dread d22/d24/d39/d3d/f40 [0,4194304] 0 2024-02-16T22:56:23.207 INFO:tasks.workunit.client.1.smithi192.stdout:8/515: dwrite d5/db/de/df/f1b [0,4194304] 0 2024-02-16T22:56:23.211 INFO:tasks.workunit.client.1.smithi192.stdout:2/567: write d22/d24/d39/d42/d65/f8e [609754,72786] 0 2024-02-16T22:56:23.212 INFO:tasks.workunit.client.1.smithi192.stdout:8/516: chown d5/db/d47/d43/c6a 1141 1 2024-02-16T22:56:23.212 INFO:tasks.workunit.client.1.smithi192.stdout:4/613: write d1/d15/d48/f55 [3923171,35619] 0 2024-02-16T22:56:23.213 INFO:tasks.workunit.client.0.smithi064.stdout:2/508: creat d7/d14/d25/d46/d55/d7f/f99 x:0 0 0 2024-02-16T22:56:23.214 INFO:tasks.workunit.client.1.smithi192.stdout:9/662: unlink d7/d15/d36/fd6 0 2024-02-16T22:56:23.215 INFO:tasks.workunit.client.0.smithi064.stdout:7/418: mkdir d5/d33/d42/d81/d82 0 2024-02-16T22:56:23.216 INFO:tasks.workunit.client.0.smithi064.stdout:7/419: chown d5/d6/d65/d7f 898 1 2024-02-16T22:56:23.217 INFO:tasks.workunit.client.0.smithi064.stdout:0/380: symlink da/dc/de/d37/l79 0 2024-02-16T22:56:23.218 INFO:tasks.workunit.client.1.smithi192.stdout:8/517: creat d5/d26/fa0 x:0 0 0 2024-02-16T22:56:23.218 INFO:tasks.workunit.client.1.smithi192.stdout:1/511: dwrite d3/de/d27/d34/d69/d85/d89/daa/fb5 [0,4194304] 0 2024-02-16T22:56:23.223 INFO:tasks.workunit.client.1.smithi192.stdout:4/614: mkdir d1/d15/d29/dd4 0 2024-02-16T22:56:23.224 INFO:tasks.workunit.client.0.smithi064.stdout:9/416: truncate d3/d11/d15/d5a/d2a/f4b 2476127 0 2024-02-16T22:56:23.225 INFO:tasks.workunit.client.1.smithi192.stdout:7/622: dread d1/d2/de/d2c/f8b [0,4194304] 0 2024-02-16T22:56:23.226 INFO:tasks.workunit.client.1.smithi192.stdout:2/568: write d22/d24/d39/d42/d65/d69/d4f/fae [460324,116098] 0 2024-02-16T22:56:23.227 INFO:tasks.workunit.client.1.smithi192.stdout:8/518: dwrite d5/db/d47/d43/d67/f8b [0,4194304] 0 2024-02-16T22:56:23.229 INFO:tasks.workunit.client.1.smithi192.stdout:2/569: write d22/d24/d39/d42/d65/d69/d4f/fae [760835,99011] 0 2024-02-16T22:56:23.231 INFO:tasks.workunit.client.1.smithi192.stdout:9/663: rename d7/d19/d28/ce3 to d7/d15/d36/d77/db1/ce9 0 2024-02-16T22:56:23.231 INFO:tasks.workunit.client.0.smithi064.stdout:4/468: symlink dc/de/d6d/l8f 0 2024-02-16T22:56:23.234 INFO:tasks.workunit.client.0.smithi064.stdout:2/509: dread - d7/de/d95/f66 zero size 2024-02-16T22:56:23.235 INFO:tasks.workunit.client.1.smithi192.stdout:9/664: dread - d7/d15/d36/d38/d3f/fc3 zero size 2024-02-16T22:56:23.237 INFO:tasks.workunit.client.1.smithi192.stdout:8/519: dwrite d5/f38 [0,4194304] 0 2024-02-16T22:56:23.238 INFO:tasks.workunit.client.1.smithi192.stdout:8/520: dread d5/d16/f4f [0,4194304] 0 2024-02-16T22:56:23.240 INFO:tasks.workunit.client.0.smithi064.stdout:7/420: dread d5/d6/d54/d41/f50 [0,4194304] 0 2024-02-16T22:56:23.241 INFO:tasks.workunit.client.1.smithi192.stdout:0/537: sync 2024-02-16T22:56:23.241 INFO:tasks.workunit.client.0.smithi064.stdout:1/392: creat d2/df/d14/f88 x:0 0 0 2024-02-16T22:56:23.243 INFO:tasks.workunit.client.0.smithi064.stdout:8/410: symlink df/d31/d2f/d38/l8e 0 2024-02-16T22:56:23.243 INFO:tasks.workunit.client.0.smithi064.stdout:1/393: truncate d2/d4e/f64 4812749 0 2024-02-16T22:56:23.244 INFO:tasks.workunit.client.0.smithi064.stdout:1/394: write d2/f42 [1701771,12810] 0 2024-02-16T22:56:23.246 INFO:tasks.workunit.client.1.smithi192.stdout:7/623: rename d1/d2/f1b to d1/d2/d2d/d61/d8a/fde 0 2024-02-16T22:56:23.248 INFO:tasks.workunit.client.0.smithi064.stdout:4/469: fsync dc/dd/d32/d22/f52 0 2024-02-16T22:56:23.250 INFO:tasks.workunit.client.0.smithi064.stdout:2/510: mkdir d7/d14/d1e/d34/d3f/d9a 0 2024-02-16T22:56:23.251 INFO:tasks.workunit.client.1.smithi192.stdout:1/512: truncate d3/f46 491674 0 2024-02-16T22:56:23.252 INFO:tasks.workunit.client.1.smithi192.stdout:1/513: read - d3/d40/f94 zero size 2024-02-16T22:56:23.252 INFO:tasks.workunit.client.0.smithi064.stdout:0/381: creat da/dc/de/d12/d2c/d2e/d38/f7a x:0 0 0 2024-02-16T22:56:23.253 INFO:tasks.workunit.client.0.smithi064.stdout:0/382: chown da/fb 22563 1 2024-02-16T22:56:23.254 INFO:tasks.workunit.client.1.smithi192.stdout:9/665: chown d7/ca 321205 1 2024-02-16T22:56:23.255 INFO:tasks.workunit.client.0.smithi064.stdout:7/421: symlink d5/d33/d45/d64/l83 0 2024-02-16T22:56:23.256 INFO:tasks.workunit.client.1.smithi192.stdout:4/615: mkdir d1/d15/d48/db5/dd5 0 2024-02-16T22:56:23.256 INFO:tasks.workunit.client.0.smithi064.stdout:7/422: write d5/d6/f5b [4816573,23389] 0 2024-02-16T22:56:23.257 INFO:tasks.workunit.client.1.smithi192.stdout:6/541: dwrite da/df/d12/d57/f35 [0,4194304] 0 2024-02-16T22:56:23.258 INFO:tasks.workunit.client.1.smithi192.stdout:8/521: mkdir d5/d26/d8f/d4d/da1 0 2024-02-16T22:56:23.259 INFO:tasks.workunit.client.1.smithi192.stdout:6/542: chown da/df/d12/d15/f5c 14 1 2024-02-16T22:56:23.260 INFO:tasks.workunit.client.1.smithi192.stdout:8/522: chown d5/db/d47/d43/d67/c53 405 1 2024-02-16T22:56:23.262 INFO:tasks.workunit.client.1.smithi192.stdout:0/538: truncate d1/d3/de/d22/d37/d67/f66 278272 0 2024-02-16T22:56:23.267 INFO:tasks.workunit.client.0.smithi064.stdout:1/395: write d2/df/d11/d1a/f34 [259841,116183] 0 2024-02-16T22:56:23.267 INFO:tasks.workunit.client.0.smithi064.stdout:8/411: dread df/f37 [0,4194304] 0 2024-02-16T22:56:23.268 INFO:tasks.workunit.client.0.smithi064.stdout:8/412: write df/f1b [266667,73309] 0 2024-02-16T22:56:23.269 INFO:tasks.workunit.client.1.smithi192.stdout:2/570: rename d22/d24/d2b/f64 to d22/d24/d39/d42/d65/d69/db1/fbe 0 2024-02-16T22:56:23.271 INFO:tasks.workunit.client.1.smithi192.stdout:2/571: write d22/d24/fa0 [620093,81983] 0 2024-02-16T22:56:23.271 INFO:tasks.workunit.client.0.smithi064.stdout:5/353: link d6/d1c/f20 d6/d1c/d43/f79 0 2024-02-16T22:56:23.272 INFO:tasks.workunit.client.1.smithi192.stdout:7/624: fdatasync d1/d69/f93 0 2024-02-16T22:56:23.274 INFO:tasks.workunit.client.0.smithi064.stdout:4/470: unlink dc/de/d28/l31 0 2024-02-16T22:56:23.275 INFO:tasks.workunit.client.0.smithi064.stdout:4/471: chown dc/dd/d32/f1f 9 1 2024-02-16T22:56:23.277 INFO:tasks.workunit.client.0.smithi064.stdout:2/511: symlink d7/d14/d1e/d34/d73/l9b 0 2024-02-16T22:56:23.278 INFO:tasks.workunit.client.0.smithi064.stdout:2/512: write d7/d14/d25/d46/d55/d57/f7b [1007142,105775] 0 2024-02-16T22:56:23.281 INFO:tasks.workunit.client.1.smithi192.stdout:9/666: mkdir d7/d19/d28/d43/dea 0 2024-02-16T22:56:23.285 INFO:tasks.workunit.client.0.smithi064.stdout:7/423: truncate d5/f10 2780052 0 2024-02-16T22:56:23.286 INFO:tasks.workunit.client.1.smithi192.stdout:8/523: truncate d5/d9/f17 1044542 0 2024-02-16T22:56:23.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/524: readlink d5/d26/l31 0 2024-02-16T22:56:23.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/525: readlink d5/d26/d83/d86/l70 0 2024-02-16T22:56:23.288 INFO:tasks.workunit.client.1.smithi192.stdout:0/539: fsync d1/d3/de/d22/d37/d47/d53/f65 0 2024-02-16T22:56:23.288 INFO:tasks.workunit.client.1.smithi192.stdout:8/526: dread - d5/db/f64 zero size 2024-02-16T22:56:23.297 INFO:tasks.workunit.client.1.smithi192.stdout:5/507: sync 2024-02-16T22:56:23.297 INFO:tasks.workunit.client.1.smithi192.stdout:9/667: dread d7/d19/d5e/d6f/f7a [0,4194304] 0 2024-02-16T22:56:23.297 INFO:tasks.workunit.client.0.smithi064.stdout:8/413: truncate df/d31/d2f/f55 661558 0 2024-02-16T22:56:23.298 INFO:tasks.workunit.client.1.smithi192.stdout:9/668: fsync d7/d19/d37/d5a/f63 0 2024-02-16T22:56:23.299 INFO:tasks.workunit.client.0.smithi064.stdout:5/354: mknod d6/d1c/c7a 0 2024-02-16T22:56:23.301 INFO:tasks.workunit.client.0.smithi064.stdout:2/513: creat d7/d14/d4b/d63/f9c x:0 0 0 2024-02-16T22:56:23.301 INFO:tasks.workunit.client.0.smithi064.stdout:0/383: write da/dc/f1e [3035691,33368] 0 2024-02-16T22:56:23.302 INFO:tasks.workunit.client.0.smithi064.stdout:2/514: readlink d7/d14/d1e/d34/d73/l9b 0 2024-02-16T22:56:23.306 INFO:tasks.workunit.client.1.smithi192.stdout:6/543: symlink da/lb4 0 2024-02-16T22:56:23.307 INFO:tasks.workunit.client.0.smithi064.stdout:9/417: getdents d3/d9/d1e/d3e/d76 0 2024-02-16T22:56:23.309 INFO:tasks.workunit.client.0.smithi064.stdout:1/396: mknod d2/d4e/d56/c89 0 2024-02-16T22:56:23.309 INFO:tasks.workunit.client.1.smithi192.stdout:6/544: dread da/df/d12/f9c [0,4194304] 0 2024-02-16T22:56:23.310 INFO:tasks.workunit.client.1.smithi192.stdout:0/540: fsync d1/f19 0 2024-02-16T22:56:23.311 INFO:tasks.workunit.client.1.smithi192.stdout:0/541: read - d1/d3/de/d22/d37/d67/d34/f56 zero size 2024-02-16T22:56:23.312 INFO:tasks.workunit.client.1.smithi192.stdout:8/527: symlink d5/d26/d8f/la2 0 2024-02-16T22:56:23.314 INFO:tasks.workunit.client.0.smithi064.stdout:0/384: creat da/dc/d3b/f7b x:0 0 0 2024-02-16T22:56:23.314 INFO:tasks.workunit.client.0.smithi064.stdout:0/385: chown da 261104684 1 2024-02-16T22:56:23.316 INFO:tasks.workunit.client.1.smithi192.stdout:2/572: dwrite d22/d24/d2b/d2c/f2e [4194304,4194304] 0 2024-02-16T22:56:23.316 INFO:tasks.workunit.client.0.smithi064.stdout:2/515: symlink d7/d14/d25/d46/l9d 0 2024-02-16T22:56:23.317 INFO:tasks.workunit.client.0.smithi064.stdout:2/516: dread - d7/d14/d25/f69 zero size 2024-02-16T22:56:23.318 INFO:tasks.workunit.client.0.smithi064.stdout:7/424: mknod d5/d6/c84 0 2024-02-16T22:56:23.320 INFO:tasks.workunit.client.1.smithi192.stdout:7/625: mkdir d1/d2/d2d/ddf 0 2024-02-16T22:56:23.321 INFO:tasks.workunit.client.0.smithi064.stdout:6/485: sync 2024-02-16T22:56:23.323 INFO:tasks.workunit.client.0.smithi064.stdout:6/486: truncate d0/d5/f90 559862 0 2024-02-16T22:56:23.323 INFO:tasks.workunit.client.1.smithi192.stdout:9/669: creat d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/feb x:0 0 0 2024-02-16T22:56:23.329 INFO:tasks.workunit.client.1.smithi192.stdout:4/616: creat d1/d26/d66/d89/fd6 x:0 0 0 2024-02-16T22:56:23.329 INFO:tasks.workunit.client.1.smithi192.stdout:4/617: dread - d1/d26/d66/d89/fd6 zero size 2024-02-16T22:56:23.329 INFO:tasks.workunit.client.0.smithi064.stdout:2/517: mknod d7/d14/d1e/d34/d65/c9e 0 2024-02-16T22:56:23.330 INFO:tasks.workunit.client.0.smithi064.stdout:2/518: chown d7/de/f8d 15018 1 2024-02-16T22:56:23.330 INFO:tasks.workunit.client.1.smithi192.stdout:4/618: fsync d1/d26/d66/d89/d11/d58/f5d 0 2024-02-16T22:56:23.336 INFO:tasks.workunit.client.0.smithi064.stdout:8/414: rename df/d31/d2f/d38/d48/d69 to df/d31/d8f 0 2024-02-16T22:56:23.340 INFO:tasks.workunit.client.0.smithi064.stdout:0/386: write da/dc/de/d37/f4e [605191,22677] 0 2024-02-16T22:56:23.341 INFO:tasks.workunit.client.1.smithi192.stdout:8/528: read d5/d16/f7d [550215,104245] 0 2024-02-16T22:56:23.341 INFO:tasks.workunit.client.0.smithi064.stdout:6/487: dread d0/d12/f78 [0,4194304] 0 2024-02-16T22:56:23.343 INFO:tasks.workunit.client.0.smithi064.stdout:9/418: dwrite f1 [0,4194304] 0 2024-02-16T22:56:23.345 INFO:tasks.workunit.client.1.smithi192.stdout:2/573: write fa [1365074,114864] 0 2024-02-16T22:56:23.347 INFO:tasks.workunit.client.1.smithi192.stdout:8/529: dwrite d5/db/d47/d43/d67/f8b [0,4194304] 0 2024-02-16T22:56:23.348 INFO:tasks.workunit.client.0.smithi064.stdout:0/387: write da/dc/de/d37/f46 [8575702,80339] 0 2024-02-16T22:56:23.348 INFO:tasks.workunit.client.0.smithi064.stdout:8/415: fsync df/f37 0 2024-02-16T22:56:23.349 INFO:tasks.workunit.client.0.smithi064.stdout:1/397: rename d2/df/d11/d1a/d30/d3d/l46 to d2/df/d14/d2a/l8a 0 2024-02-16T22:56:23.351 INFO:tasks.workunit.client.0.smithi064.stdout:8/416: write df/d2a/f87 [688054,33428] 0 2024-02-16T22:56:23.352 INFO:tasks.workunit.client.0.smithi064.stdout:5/355: getdents d6/d18 0 2024-02-16T22:56:23.353 INFO:tasks.workunit.client.0.smithi064.stdout:8/417: chown df/d31/d5e/f64 7845 1 2024-02-16T22:56:23.353 INFO:tasks.workunit.client.1.smithi192.stdout:3/535: sync 2024-02-16T22:56:23.353 INFO:tasks.workunit.client.0.smithi064.stdout:5/356: fsync d6/d18/d19/d1a/f60 0 2024-02-16T22:56:23.354 INFO:tasks.workunit.client.0.smithi064.stdout:2/519: dread d7/de/f2f [0,4194304] 0 2024-02-16T22:56:23.355 INFO:tasks.workunit.client.0.smithi064.stdout:6/488: symlink d0/d3c/l94 0 2024-02-16T22:56:23.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:23 smithi192 ceph-mon[36254]: pgmap v16: 65 pgs: 65 active+clean; 1.5 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 220 MiB/s wr, 332 op/s 2024-02-16T22:56:23.357 INFO:tasks.workunit.client.1.smithi192.stdout:1/514: getdents d3/de/d27/d34 0 2024-02-16T22:56:23.358 INFO:tasks.workunit.client.0.smithi064.stdout:9/419: fsync d3/f8a 0 2024-02-16T22:56:23.362 INFO:tasks.workunit.client.0.smithi064.stdout:7/425: rename d5/d6/d54/d41/f50 to d5/d33/d45/d64/f85 0 2024-02-16T22:56:23.363 INFO:tasks.workunit.client.1.smithi192.stdout:6/545: mknod da/df/d12/d15/cb5 0 2024-02-16T22:56:23.363 INFO:tasks.workunit.client.0.smithi064.stdout:2/520: dwrite d7/d14/d25/d46/d55/d57/f7b [0,4194304] 0 2024-02-16T22:56:23.363 INFO:tasks.workunit.client.1.smithi192.stdout:6/546: dread - da/df/d12/d15/d1b/faa zero size 2024-02-16T22:56:23.364 INFO:tasks.workunit.client.0.smithi064.stdout:7/426: chown d5/d6/d54/d7b 270 1 2024-02-16T22:56:23.364 INFO:tasks.workunit.client.0.smithi064.stdout:6/489: dwrite d0/d12/f49 [0,4194304] 0 2024-02-16T22:56:23.365 INFO:tasks.workunit.client.0.smithi064.stdout:2/521: dread - d7/d14/d1e/d34/d65/f67 zero size 2024-02-16T22:56:23.367 INFO:tasks.workunit.client.0.smithi064.stdout:2/522: chown d7/d14/d25/d46/d55/d7f 10597 1 2024-02-16T22:56:23.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:23 smithi064 ceph-mon[29226]: pgmap v16: 65 pgs: 65 active+clean; 1.5 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 220 MiB/s wr, 332 op/s 2024-02-16T22:56:23.368 INFO:tasks.workunit.client.0.smithi064.stdout:2/523: write d7/d14/d25/d46/d55/f71 [2171270,107451] 0 2024-02-16T22:56:23.372 INFO:tasks.workunit.client.0.smithi064.stdout:8/418: symlink df/d31/d8f/d80/l90 0 2024-02-16T22:56:23.374 INFO:tasks.workunit.client.0.smithi064.stdout:8/419: chown df/d31/d2f/d68/f71 5300080 1 2024-02-16T22:56:23.375 INFO:tasks.workunit.client.1.smithi192.stdout:7/626: creat d1/d2/d2d/d92/dcd/fe0 x:0 0 0 2024-02-16T22:56:23.377 INFO:tasks.workunit.client.0.smithi064.stdout:2/524: dwrite d7/d14/d1e/d34/f61 [0,4194304] 0 2024-02-16T22:56:23.379 INFO:tasks.workunit.client.0.smithi064.stdout:2/525: fdatasync d7/d14/d1e/d34/d3f/f4e 0 2024-02-16T22:56:23.381 INFO:tasks.workunit.client.0.smithi064.stdout:2/526: truncate d7/de/f24 4379044 0 2024-02-16T22:56:23.384 INFO:tasks.workunit.client.1.smithi192.stdout:0/542: write d1/d3/de/d22/d37/f71 [3729738,20504] 0 2024-02-16T22:56:23.386 INFO:tasks.workunit.client.1.smithi192.stdout:0/543: read - d1/d3/d29/d43/f68 zero size 2024-02-16T22:56:23.386 INFO:tasks.workunit.client.0.smithi064.stdout:1/398: symlink d2/df/d14/d18/l8b 0 2024-02-16T22:56:23.387 INFO:tasks.workunit.client.1.smithi192.stdout:3/536: creat da/d11/d63/d6b/d71/da1/fbd x:0 0 0 2024-02-16T22:56:23.387 INFO:tasks.workunit.client.0.smithi064.stdout:0/388: mkdir da/dc/d3b/d72/d7c 0 2024-02-16T22:56:23.390 INFO:tasks.workunit.client.1.smithi192.stdout:6/547: mkdir da/d40/db6 0 2024-02-16T22:56:23.392 INFO:tasks.workunit.client.1.smithi192.stdout:5/508: link d6/d8/d80/l55 d6/d8/la8 0 2024-02-16T22:56:23.398 INFO:tasks.workunit.client.1.smithi192.stdout:2/574: write d22/f4a [213070,17030] 0 2024-02-16T22:56:23.398 INFO:tasks.workunit.client.1.smithi192.stdout:2/575: fsync f5 0 2024-02-16T22:56:23.399 INFO:tasks.workunit.client.0.smithi064.stdout:2/527: rename d7/d14/d25/d46/l9d to d7/d14/d25/d46/d55/d7f/l9f 0 2024-02-16T22:56:23.401 INFO:tasks.workunit.client.1.smithi192.stdout:9/670: truncate d7/d19/d37/f55 2951915 0 2024-02-16T22:56:23.402 INFO:tasks.workunit.client.1.smithi192.stdout:1/515: mkdir d3/d5b/db1/db6 0 2024-02-16T22:56:23.404 INFO:tasks.workunit.client.0.smithi064.stdout:1/399: creat d2/df/d14/d18/d48/d6b/d75/f8c x:0 0 0 2024-02-16T22:56:23.405 INFO:tasks.workunit.client.1.smithi192.stdout:4/619: creat d1/d26/fd7 x:0 0 0 2024-02-16T22:56:23.408 INFO:tasks.workunit.client.0.smithi064.stdout:0/389: rmdir da/dc/d3b/d5a 39 2024-02-16T22:56:23.409 INFO:tasks.workunit.client.0.smithi064.stdout:0/390: readlink da/l70 0 2024-02-16T22:56:23.410 INFO:tasks.workunit.client.1.smithi192.stdout:5/509: creat d6/d12/d4e/d58/d8f/fa9 x:0 0 0 2024-02-16T22:56:23.410 INFO:tasks.workunit.client.1.smithi192.stdout:5/510: write d6/f29 [895436,103518] 0 2024-02-16T22:56:23.411 INFO:tasks.workunit.client.0.smithi064.stdout:5/357: dwrite d6/fa [0,4194304] 0 2024-02-16T22:56:23.412 INFO:tasks.workunit.client.1.smithi192.stdout:5/511: write d6/d12/d4e/d58/d8f/fa9 [835337,97769] 0 2024-02-16T22:56:23.413 INFO:tasks.workunit.client.0.smithi064.stdout:5/358: chown d6/d18/d19/l65 3 1 2024-02-16T22:56:23.415 INFO:tasks.workunit.client.1.smithi192.stdout:8/530: rmdir d5/d26/d83/d86/d97 0 2024-02-16T22:56:23.419 INFO:tasks.workunit.client.1.smithi192.stdout:8/531: dread d5/db/d47/d43/d67/f8b [0,4194304] 0 2024-02-16T22:56:23.420 INFO:tasks.workunit.client.1.smithi192.stdout:8/532: chown d5/d26/d8f/d4d/l75 11044172 1 2024-02-16T22:56:23.421 INFO:tasks.workunit.client.0.smithi064.stdout:2/528: symlink d7/d14/d1e/d34/d3f/la0 0 2024-02-16T22:56:23.422 INFO:tasks.workunit.client.0.smithi064.stdout:9/420: link d3/d11/f28 d3/d9/f8c 0 2024-02-16T22:56:23.425 INFO:tasks.workunit.client.1.smithi192.stdout:4/620: rmdir d1/d15/d29/d3c 39 2024-02-16T22:56:23.425 INFO:tasks.workunit.client.0.smithi064.stdout:3/527: sync 2024-02-16T22:56:23.426 INFO:tasks.workunit.client.0.smithi064.stdout:6/490: creat d0/d6/da/f95 x:0 0 0 2024-02-16T22:56:23.428 INFO:tasks.workunit.client.1.smithi192.stdout:6/548: link da/df/d12/d15/d1b/f37 da/df/d12/d57/d81/fb7 0 2024-02-16T22:56:23.428 INFO:tasks.workunit.client.1.smithi192.stdout:6/549: fdatasync da/f5f 0 2024-02-16T22:56:23.429 INFO:tasks.workunit.client.1.smithi192.stdout:6/550: dread - da/df/d12/d15/f68 zero size 2024-02-16T22:56:23.431 INFO:tasks.workunit.client.0.smithi064.stdout:8/420: truncate df/f1b 1495749 0 2024-02-16T22:56:23.433 INFO:tasks.workunit.client.0.smithi064.stdout:6/491: dwrite d0/d5/df/f8d [0,4194304] 0 2024-02-16T22:56:23.438 INFO:tasks.workunit.client.1.smithi192.stdout:7/627: rename d1/d2/d2d/d61/d88/d90/d2a to d1/de1 0 2024-02-16T22:56:23.439 INFO:tasks.workunit.client.1.smithi192.stdout:5/512: stat d6/d8/d80/l55 0 2024-02-16T22:56:23.445 INFO:tasks.workunit.client.1.smithi192.stdout:5/513: write d6/d12/d2d/d5c/f99 [906073,81002] 0 2024-02-16T22:56:23.446 INFO:tasks.workunit.client.1.smithi192.stdout:8/533: mknod d5/d26/d8f/ca3 0 2024-02-16T22:56:23.447 INFO:tasks.workunit.client.1.smithi192.stdout:0/544: dwrite d1/f3e [0,4194304] 0 2024-02-16T22:56:23.448 INFO:tasks.workunit.client.0.smithi064.stdout:8/421: symlink df/d31/d2f/d38/d48/l91 0 2024-02-16T22:56:23.448 INFO:tasks.workunit.client.0.smithi064.stdout:3/528: read d4/d14/f22 [22267,10326] 0 2024-02-16T22:56:23.448 INFO:tasks.workunit.client.1.smithi192.stdout:7/628: dwrite d1/d2/d2d/d61/d8a/f94 [0,4194304] 0 2024-02-16T22:56:23.450 INFO:tasks.workunit.client.0.smithi064.stdout:0/391: rename da/dc/d3b/d5a/f5d to da/d71/f7d 0 2024-02-16T22:56:23.451 INFO:tasks.workunit.client.0.smithi064.stdout:6/492: truncate d0/d5/d54/d60/f6d 5168445 0 2024-02-16T22:56:23.453 INFO:tasks.workunit.client.0.smithi064.stdout:7/427: getdents d5/d33/d45/d64 0 2024-02-16T22:56:23.453 INFO:tasks.workunit.client.0.smithi064.stdout:6/493: read d0/d5/d54/d60/f7c [2376238,100970] 0 2024-02-16T22:56:23.457 INFO:tasks.workunit.client.1.smithi192.stdout:3/537: rename da/l9d to da/d11/d20/d39/d53/lbe 0 2024-02-16T22:56:23.461 INFO:tasks.workunit.client.0.smithi064.stdout:1/400: creat d2/df/d14/f8d x:0 0 0 2024-02-16T22:56:23.466 INFO:tasks.workunit.client.1.smithi192.stdout:9/671: truncate d7/d15/f6a 2610735 0 2024-02-16T22:56:23.466 INFO:tasks.workunit.client.1.smithi192.stdout:9/672: fdatasync d7/d19/d5e/fcf 0 2024-02-16T22:56:23.467 INFO:tasks.workunit.client.1.smithi192.stdout:9/673: read - d7/d15/d36/fb4 zero size 2024-02-16T22:56:23.467 INFO:tasks.workunit.client.0.smithi064.stdout:8/422: mkdir df/d31/d2f/d4f/d6a/d92 0 2024-02-16T22:56:23.467 INFO:tasks.workunit.client.0.smithi064.stdout:1/401: dwrite d2/df/d14/f16 [4194304,4194304] 0 2024-02-16T22:56:23.468 INFO:tasks.workunit.client.0.smithi064.stdout:3/529: fsync d4/d13/d1b/fb2 0 2024-02-16T22:56:23.469 INFO:tasks.workunit.client.0.smithi064.stdout:1/402: dread - d2/df/d11/d1a/d1d/f61 zero size 2024-02-16T22:56:23.469 INFO:tasks.workunit.client.0.smithi064.stdout:3/530: chown d4/d13/d1b/d27/d82 631285 1 2024-02-16T22:56:23.471 INFO:tasks.workunit.client.1.smithi192.stdout:7/629: rmdir d1/de1/dd0 39 2024-02-16T22:56:23.472 INFO:tasks.workunit.client.1.smithi192.stdout:6/551: mkdir da/d40/db6/db8 0 2024-02-16T22:56:23.474 INFO:tasks.workunit.client.1.smithi192.stdout:6/552: write da/f5f [2074596,75944] 0 2024-02-16T22:56:23.474 INFO:tasks.workunit.client.0.smithi064.stdout:6/494: creat d0/d5/df/f96 x:0 0 0 2024-02-16T22:56:23.478 INFO:tasks.workunit.client.1.smithi192.stdout:2/576: getdents d22/d24/d2b/d2c/d78 0 2024-02-16T22:56:23.479 INFO:tasks.workunit.client.0.smithi064.stdout:9/421: creat d3/d9/d1e/f8d x:0 0 0 2024-02-16T22:56:23.479 INFO:tasks.workunit.client.0.smithi064.stdout:5/359: write d6/d1c/f62 [3626732,23261] 0 2024-02-16T22:56:23.480 INFO:tasks.workunit.client.1.smithi192.stdout:0/545: dread d1/f8 [0,4194304] 0 2024-02-16T22:56:23.481 INFO:tasks.workunit.client.1.smithi192.stdout:0/546: chown d1/d3/d18 505 1 2024-02-16T22:56:23.481 INFO:tasks.workunit.client.0.smithi064.stdout:9/422: write d3/d11/d15/d23/f3d [4609516,93851] 0 2024-02-16T22:56:23.487 INFO:tasks.workunit.client.1.smithi192.stdout:2/577: fsync f4 0 2024-02-16T22:56:23.488 INFO:tasks.workunit.client.1.smithi192.stdout:2/578: fsync d22/d24/d39/d42/d65/d66/fa4 0 2024-02-16T22:56:23.491 INFO:tasks.workunit.client.1.smithi192.stdout:5/514: creat d6/d12/d4e/faa x:0 0 0 2024-02-16T22:56:23.491 INFO:tasks.workunit.client.0.smithi064.stdout:3/531: creat d4/d1f/d34/fba x:0 0 0 2024-02-16T22:56:23.492 INFO:tasks.workunit.client.1.smithi192.stdout:3/538: write da/df/f2e [3718614,291] 0 2024-02-16T22:56:23.496 INFO:tasks.workunit.client.0.smithi064.stdout:7/428: creat d5/d33/d45/d73/f86 x:0 0 0 2024-02-16T22:56:23.498 INFO:tasks.workunit.client.0.smithi064.stdout:6/495: unlink d0/d6/da/f81 0 2024-02-16T22:56:23.500 INFO:tasks.workunit.client.0.smithi064.stdout:2/529: link d7/d14/d25/d46/d55/d7f/l9f d7/d14/d25/d46/d55/la1 0 2024-02-16T22:56:23.501 INFO:tasks.workunit.client.0.smithi064.stdout:5/360: dread d6/d18/f56 [0,4194304] 0 2024-02-16T22:56:23.502 INFO:tasks.workunit.client.0.smithi064.stdout:1/403: write d2/df/d14/f59 [1397520,45123] 0 2024-02-16T22:56:23.504 INFO:tasks.workunit.client.0.smithi064.stdout:5/361: truncate d6/d18/f72 628443 0 2024-02-16T22:56:23.508 INFO:tasks.workunit.client.0.smithi064.stdout:8/423: truncate f1 797403 0 2024-02-16T22:56:23.509 INFO:tasks.workunit.client.0.smithi064.stdout:2/530: dwrite d7/d14/d1e/d34/d3f/f4e [0,4194304] 0 2024-02-16T22:56:23.509 INFO:tasks.workunit.client.0.smithi064.stdout:2/531: readlink d7/de/l42 0 2024-02-16T22:56:23.513 INFO:tasks.workunit.client.0.smithi064.stdout:3/532: rename d4/d13/d35/c95 to d4/d13/d1b/d27/d61/d63/cbb 0 2024-02-16T22:56:23.514 INFO:tasks.workunit.client.0.smithi064.stdout:3/533: dread - d4/fb9 zero size 2024-02-16T22:56:23.516 INFO:tasks.workunit.client.0.smithi064.stdout:3/534: read - d4/d13/d35/f3e zero size 2024-02-16T22:56:23.517 INFO:tasks.workunit.client.0.smithi064.stdout:8/424: dwrite df/d31/f41 [0,4194304] 0 2024-02-16T22:56:23.522 INFO:tasks.workunit.client.0.smithi064.stdout:6/496: symlink d0/d5/df/l97 0 2024-02-16T22:56:23.526 INFO:tasks.workunit.client.0.smithi064.stdout:7/429: dwrite d5/d6/f30 [4194304,4194304] 0 2024-02-16T22:56:23.527 INFO:tasks.workunit.client.0.smithi064.stdout:5/362: rmdir d6/d18/d19/d1a 39 2024-02-16T22:56:23.530 INFO:tasks.workunit.client.0.smithi064.stdout:9/423: dread d3/d11/d15/d5a/f85 [0,4194304] 0 2024-02-16T22:56:23.535 INFO:tasks.workunit.client.0.smithi064.stdout:9/424: chown d3/d11/d15/d5a/d2a 13 1 2024-02-16T22:56:23.535 INFO:tasks.workunit.client.0.smithi064.stdout:2/532: creat d7/d14/d1e/fa2 x:0 0 0 2024-02-16T22:56:23.535 INFO:tasks.workunit.client.0.smithi064.stdout:3/535: creat d4/d14/d2b/fbc x:0 0 0 2024-02-16T22:56:23.538 INFO:tasks.workunit.client.0.smithi064.stdout:8/425: unlink df/d31/d2f/d3e/f4a 0 2024-02-16T22:56:23.540 INFO:tasks.workunit.client.0.smithi064.stdout:6/497: mkdir d0/d1d/d21/d98 0 2024-02-16T22:56:23.540 INFO:tasks.workunit.client.0.smithi064.stdout:8/426: write df/d31/d61/d79/d81/f82 [708049,35770] 0 2024-02-16T22:56:23.541 INFO:tasks.workunit.client.0.smithi064.stdout:2/533: dwrite d7/de/f1f [0,4194304] 0 2024-02-16T22:56:23.544 INFO:tasks.workunit.client.0.smithi064.stdout:1/404: mknod d2/df/d14/d18/c8e 0 2024-02-16T22:56:23.552 INFO:tasks.workunit.client.0.smithi064.stdout:2/534: dwrite d7/d14/d25/f90 [0,4194304] 0 2024-02-16T22:56:23.557 INFO:tasks.workunit.client.1.smithi192.stdout:8/534: rename d5/db/d47/d43/d67/f9e to d5/db/de/fa4 0 2024-02-16T22:56:23.559 INFO:tasks.workunit.client.1.smithi192.stdout:2/579: rmdir d22/d24/d39/d42/d84 39 2024-02-16T22:56:23.565 INFO:tasks.workunit.client.0.smithi064.stdout:1/405: creat d2/df/d11/f8f x:0 0 0 2024-02-16T22:56:23.566 INFO:tasks.workunit.client.0.smithi064.stdout:1/406: readlink d2/d4e/d56/l6f 0 2024-02-16T22:56:23.571 INFO:tasks.workunit.client.0.smithi064.stdout:5/363: mknod d6/d25/c7b 0 2024-02-16T22:56:23.572 INFO:tasks.workunit.client.1.smithi192.stdout:4/621: rename d1/d15/d29/d3c/fcb to d1/d26/d6f/fd8 0 2024-02-16T22:56:23.575 INFO:tasks.workunit.client.1.smithi192.stdout:8/535: unlink d5/d26/c66 0 2024-02-16T22:56:23.576 INFO:tasks.workunit.client.1.smithi192.stdout:8/536: chown d5/d16/c6d 334208138 1 2024-02-16T22:56:23.577 INFO:tasks.workunit.client.1.smithi192.stdout:2/580: creat d22/d24/d39/d42/d65/d69/d4f/fbf x:0 0 0 2024-02-16T22:56:23.578 INFO:tasks.workunit.client.0.smithi064.stdout:6/498: creat d0/d12/d7b/f99 x:0 0 0 2024-02-16T22:56:23.579 INFO:tasks.workunit.client.1.smithi192.stdout:5/515: symlink d6/d12/d1d/lab 0 2024-02-16T22:56:23.580 INFO:tasks.workunit.client.1.smithi192.stdout:3/539: mkdir da/d11/d63/dbf 0 2024-02-16T22:56:23.582 INFO:tasks.workunit.client.1.smithi192.stdout:9/674: getdents d7/d15/d36/d77 0 2024-02-16T22:56:23.582 INFO:tasks.workunit.client.0.smithi064.stdout:7/430: write d5/f10 [2230539,43941] 0 2024-02-16T22:56:23.583 INFO:tasks.workunit.client.1.smithi192.stdout:2/581: dwrite d22/d24/d39/d42/f5f [0,4194304] 0 2024-02-16T22:56:23.585 INFO:tasks.workunit.client.0.smithi064.stdout:9/425: write d3/d11/d15/d5a/d2a/f4b [488736,53362] 0 2024-02-16T22:56:23.586 INFO:tasks.workunit.client.0.smithi064.stdout:9/426: fdatasync d3/d11/d15/f7f 0 2024-02-16T22:56:23.587 INFO:tasks.workunit.client.0.smithi064.stdout:9/427: chown d3/d11 5 1 2024-02-16T22:56:23.587 INFO:tasks.workunit.client.1.smithi192.stdout:0/547: rename d1/d3/d39/f82 to d1/d3/de/d22/d37/d67/d34/da1/fab 0 2024-02-16T22:56:23.589 INFO:tasks.workunit.client.1.smithi192.stdout:2/582: dwrite d22/d24/d2b/f38 [0,4194304] 0 2024-02-16T22:56:23.590 INFO:tasks.workunit.client.0.smithi064.stdout:9/428: chown d3/d9/d1e/d3e/d76 5006356 1 2024-02-16T22:56:23.590 INFO:tasks.workunit.client.0.smithi064.stdout:7/431: dwrite d5/d33/f3d [4194304,4194304] 0 2024-02-16T22:56:23.591 INFO:tasks.workunit.client.1.smithi192.stdout:2/583: write d22/d24/d2b/d2c/f43 [1836876,8501] 0 2024-02-16T22:56:23.591 INFO:tasks.workunit.client.1.smithi192.stdout:9/675: dread d7/d19/d5b/fa5 [0,4194304] 0 2024-02-16T22:56:23.592 INFO:tasks.workunit.client.0.smithi064.stdout:3/536: link d4/d13/d1b/d48/c7a d4/d13/d4d/cbd 0 2024-02-16T22:56:23.592 INFO:tasks.workunit.client.1.smithi192.stdout:4/622: symlink d1/d26/d66/ld9 0 2024-02-16T22:56:23.593 INFO:tasks.workunit.client.0.smithi064.stdout:3/537: chown d4/d13/d1b/d27/d99/f87 46 1 2024-02-16T22:56:23.596 INFO:tasks.workunit.client.1.smithi192.stdout:5/516: creat d6/d8/d75/fac x:0 0 0 2024-02-16T22:56:23.598 INFO:tasks.workunit.client.1.smithi192.stdout:5/517: read d6/d8/d86/f89 [76707,10140] 0 2024-02-16T22:56:23.600 INFO:tasks.workunit.client.0.smithi064.stdout:5/364: symlink d6/l7c 0 2024-02-16T22:56:23.602 INFO:tasks.workunit.client.0.smithi064.stdout:6/499: creat d0/d5/df/d17/d18/d5f/d76/f9a x:0 0 0 2024-02-16T22:56:23.604 INFO:tasks.workunit.client.0.smithi064.stdout:8/427: getdents df/d2a/d5c 0 2024-02-16T22:56:23.605 INFO:tasks.workunit.client.0.smithi064.stdout:8/428: fdatasync df/d31/d2f/f59 0 2024-02-16T22:56:23.606 INFO:tasks.workunit.client.0.smithi064.stdout:1/407: link d2/df/l33 d2/df/d14/d18/d48/d84/l90 0 2024-02-16T22:56:23.606 INFO:tasks.workunit.client.1.smithi192.stdout:8/537: truncate d5/db/d47/f3b 944837 0 2024-02-16T22:56:23.607 INFO:tasks.workunit.client.1.smithi192.stdout:8/538: read - d5/d26/fa0 zero size 2024-02-16T22:56:23.608 INFO:tasks.workunit.client.0.smithi064.stdout:1/408: truncate d2/df/d11/d1a/d1d/f61 764540 0 2024-02-16T22:56:23.609 INFO:tasks.workunit.client.0.smithi064.stdout:4/472: sync 2024-02-16T22:56:23.614 INFO:tasks.workunit.client.0.smithi064.stdout:3/538: unlink d4/d13/d35/c40 0 2024-02-16T22:56:23.616 INFO:tasks.workunit.client.0.smithi064.stdout:3/539: stat d4/d1f/d34/l98 0 2024-02-16T22:56:23.618 INFO:tasks.workunit.client.0.smithi064.stdout:5/365: dwrite d6/d18/d19/d1a/f3b [8388608,4194304] 0 2024-02-16T22:56:23.618 INFO:tasks.workunit.client.1.smithi192.stdout:5/518: readlink d6/d12/d4e/d46/d4f/l7a 0 2024-02-16T22:56:23.619 INFO:tasks.workunit.client.0.smithi064.stdout:3/540: write d4/f12 [1110116,10389] 0 2024-02-16T22:56:23.621 INFO:tasks.workunit.client.1.smithi192.stdout:0/548: mkdir d1/d3/dac 0 2024-02-16T22:56:23.622 INFO:tasks.workunit.client.1.smithi192.stdout:3/540: dread da/f51 [0,4194304] 0 2024-02-16T22:56:23.622 INFO:tasks.workunit.client.1.smithi192.stdout:0/549: chown d1/d3/d29/f70 941 1 2024-02-16T22:56:23.622 INFO:tasks.workunit.client.0.smithi064.stdout:4/473: dwrite dc/dd/d16/f3a [0,4194304] 0 2024-02-16T22:56:23.623 INFO:tasks.workunit.client.1.smithi192.stdout:3/541: fsync da/d11/d63/d6b/d66/f60 0 2024-02-16T22:56:23.631 INFO:tasks.workunit.client.1.smithi192.stdout:8/539: mknod d5/d26/ca5 0 2024-02-16T22:56:23.633 INFO:tasks.workunit.client.0.smithi064.stdout:8/429: dread df/f19 [0,4194304] 0 2024-02-16T22:56:23.635 INFO:tasks.workunit.client.0.smithi064.stdout:1/409: rename d2/d4e to d2/df/d14/d2c/d62/d91 0 2024-02-16T22:56:23.639 INFO:tasks.workunit.client.1.smithi192.stdout:6/553: sync 2024-02-16T22:56:23.639 INFO:tasks.workunit.client.1.smithi192.stdout:7/630: sync 2024-02-16T22:56:23.639 INFO:tasks.workunit.client.1.smithi192.stdout:1/516: sync 2024-02-16T22:56:23.640 INFO:tasks.workunit.client.0.smithi064.stdout:5/366: rmdir d6/d1c/d4b 39 2024-02-16T22:56:23.642 INFO:tasks.workunit.client.1.smithi192.stdout:5/519: mknod d6/d12/cad 0 2024-02-16T22:56:23.646 INFO:tasks.workunit.client.1.smithi192.stdout:9/676: write d7/d19/d37/f8c [451848,14952] 0 2024-02-16T22:56:23.647 INFO:tasks.workunit.client.1.smithi192.stdout:7/631: dwrite d1/f11 [0,4194304] 0 2024-02-16T22:56:23.647 INFO:tasks.workunit.client.1.smithi192.stdout:6/554: dwrite da/df/d12/d57/f2f [0,4194304] 0 2024-02-16T22:56:23.650 INFO:tasks.workunit.client.0.smithi064.stdout:5/367: dwrite d6/d1c/f54 [0,4194304] 0 2024-02-16T22:56:23.650 INFO:tasks.workunit.client.1.smithi192.stdout:0/550: creat d1/d3/d29/d43/d9c/fad x:0 0 0 2024-02-16T22:56:23.651 INFO:tasks.workunit.client.1.smithi192.stdout:9/677: chown d7/d15/d36/d38/d3f/d8a/d98/cb3 132211249 1 2024-02-16T22:56:23.653 INFO:tasks.workunit.client.1.smithi192.stdout:3/542: rename da/d11/d2b/l74 to da/df/lc0 0 2024-02-16T22:56:23.654 INFO:tasks.workunit.client.0.smithi064.stdout:9/429: dwrite d3/f63 [0,4194304] 0 2024-02-16T22:56:23.656 INFO:tasks.workunit.client.1.smithi192.stdout:4/623: dwrite d1/d26/d66/d89/d11/d58/f90 [0,4194304] 0 2024-02-16T22:56:23.659 INFO:tasks.workunit.client.1.smithi192.stdout:7/632: dwrite d1/d2/fb4 [0,4194304] 0 2024-02-16T22:56:23.659 INFO:tasks.workunit.client.0.smithi064.stdout:1/410: unlink d2/df/d14/d18/l51 0 2024-02-16T22:56:23.661 INFO:tasks.workunit.client.0.smithi064.stdout:9/430: chown d3/d11/d15/d23/f6e 32 1 2024-02-16T22:56:23.663 INFO:tasks.workunit.client.1.smithi192.stdout:8/540: truncate d5/db/d47/f78 297003 0 2024-02-16T22:56:23.665 INFO:tasks.workunit.client.0.smithi064.stdout:8/430: dread f3 [0,4194304] 0 2024-02-16T22:56:23.666 INFO:tasks.workunit.client.1.smithi192.stdout:8/541: write d5/db/de/f80 [960054,106139] 0 2024-02-16T22:56:23.666 INFO:tasks.workunit.client.1.smithi192.stdout:1/517: rmdir d3/de/d27 39 2024-02-16T22:56:23.673 INFO:tasks.workunit.client.0.smithi064.stdout:6/500: dwrite d0/d6/d10/f14 [4194304,4194304] 0 2024-02-16T22:56:23.676 INFO:tasks.workunit.client.1.smithi192.stdout:5/520: truncate d6/d12/d4e/d46/f57 447020 0 2024-02-16T22:56:23.681 INFO:tasks.workunit.client.0.smithi064.stdout:8/431: dwrite df/d31/d61/d79/d81/f88 [0,4194304] 0 2024-02-16T22:56:23.682 INFO:tasks.workunit.client.0.smithi064.stdout:9/431: truncate d3/d11/f28 743121 0 2024-02-16T22:56:23.684 INFO:tasks.workunit.client.0.smithi064.stdout:8/432: write df/d31/f7c [1506977,125292] 0 2024-02-16T22:56:23.685 INFO:tasks.workunit.client.0.smithi064.stdout:9/432: rename d3/d11/d15/d5a/d2a/d2e to d3/d11/d15/d5a/d2a/d2e/d31/d8e 22 2024-02-16T22:56:23.688 INFO:tasks.workunit.client.0.smithi064.stdout:3/541: getdents d4/d13/d1b/d27/d61 0 2024-02-16T22:56:23.688 INFO:tasks.workunit.client.1.smithi192.stdout:0/551: read d1/d3/fc [2812041,5728] 0 2024-02-16T22:56:23.689 INFO:tasks.workunit.client.0.smithi064.stdout:3/542: readlink d4/d13/d1b/lab 0 2024-02-16T22:56:23.690 INFO:tasks.workunit.client.0.smithi064.stdout:3/543: readlink d4/d13/d4d/d5c/d78/d96/lae 0 2024-02-16T22:56:23.693 INFO:tasks.workunit.client.1.smithi192.stdout:2/584: getdents d22/d24/d39/d3d 0 2024-02-16T22:56:23.693 INFO:tasks.workunit.client.0.smithi064.stdout:9/433: read d3/d11/d15/d5a/f27 [3103783,91435] 0 2024-02-16T22:56:23.695 INFO:tasks.workunit.client.1.smithi192.stdout:7/633: rename d1/d2/de/d2c/d49/da4/c52 to d1/d2/d2d/d61/d88/ce2 0 2024-02-16T22:56:23.696 INFO:tasks.workunit.client.0.smithi064.stdout:4/474: dwrite dc/dd/d32/d58/f70 [0,4194304] 0 2024-02-16T22:56:23.698 INFO:tasks.workunit.client.1.smithi192.stdout:7/634: chown d1/d2/d2d/d61/d8a/la6 13018 1 2024-02-16T22:56:23.698 INFO:tasks.workunit.client.1.smithi192.stdout:0/552: dwrite d1/d3/f8c [0,4194304] 0 2024-02-16T22:56:23.699 INFO:tasks.workunit.client.1.smithi192.stdout:0/553: chown d1/d3/d29/f9a 7914 1 2024-02-16T22:56:23.700 INFO:tasks.workunit.client.0.smithi064.stdout:5/368: fsync d6/d1c/f54 0 2024-02-16T22:56:23.700 INFO:tasks.workunit.client.1.smithi192.stdout:8/542: mknod d5/db/d47/d43/ca6 0 2024-02-16T22:56:23.702 INFO:tasks.workunit.client.0.smithi064.stdout:5/369: write d6/fa [3260366,113085] 0 2024-02-16T22:56:23.703 INFO:tasks.workunit.client.1.smithi192.stdout:8/543: write d5/f25 [236795,41009] 0 2024-02-16T22:56:23.705 INFO:tasks.workunit.client.0.smithi064.stdout:3/544: mknod d4/d13/d1b/d27/d61/d63/cbe 0 2024-02-16T22:56:23.707 INFO:tasks.workunit.client.0.smithi064.stdout:3/545: chown d4/d1f/d5b/d51/l80 14 1 2024-02-16T22:56:23.708 INFO:tasks.workunit.client.0.smithi064.stdout:3/546: truncate d4/d13/d4d/fb8 474822 0 2024-02-16T22:56:23.709 INFO:tasks.workunit.client.1.smithi192.stdout:3/543: symlink da/d11/lc1 0 2024-02-16T22:56:23.711 INFO:tasks.workunit.client.1.smithi192.stdout:0/554: dwrite d1/d3/de/d22/d37/d67/d75/f80 [0,4194304] 0 2024-02-16T22:56:23.711 INFO:tasks.workunit.client.1.smithi192.stdout:2/585: rmdir d22/d24/d39 39 2024-02-16T22:56:23.712 INFO:tasks.workunit.client.0.smithi064.stdout:6/501: dread d0/d1d/f61 [0,4194304] 0 2024-02-16T22:56:23.713 INFO:tasks.workunit.client.1.smithi192.stdout:0/555: write d1/d3/de/f3c [339325,22078] 0 2024-02-16T22:56:23.714 INFO:tasks.workunit.client.1.smithi192.stdout:0/556: read - d1/d3/de/d22/d37/d47/f93 zero size 2024-02-16T22:56:23.718 INFO:tasks.workunit.client.1.smithi192.stdout:6/555: dread da/df/d12/d15/f3e [0,4194304] 0 2024-02-16T22:56:23.719 INFO:tasks.workunit.client.1.smithi192.stdout:6/556: chown da/df/d12/f5a 364 1 2024-02-16T22:56:23.720 INFO:tasks.workunit.client.1.smithi192.stdout:6/557: fdatasync da/df/d12/f9b 0 2024-02-16T22:56:23.720 INFO:tasks.workunit.client.0.smithi064.stdout:6/502: dread d0/d5/df/d17/d18/d5f/f67 [0,4194304] 0 2024-02-16T22:56:23.720 INFO:tasks.workunit.client.0.smithi064.stdout:1/411: link d2/df/d14/d18/l7d d2/df/d14/d2c/l92 0 2024-02-16T22:56:23.723 INFO:tasks.workunit.client.0.smithi064.stdout:5/370: mkdir d6/d1c/d43/d7d 0 2024-02-16T22:56:23.726 INFO:tasks.workunit.client.1.smithi192.stdout:9/678: write d7/d15/d36/f7b [426394,39553] 0 2024-02-16T22:56:23.727 INFO:tasks.workunit.client.0.smithi064.stdout:3/547: creat d4/d13/d1b/d27/d99/d7d/fbf x:0 0 0 2024-02-16T22:56:23.727 INFO:tasks.workunit.client.0.smithi064.stdout:3/548: chown d4/d1f/d5b/d51/fa4 22104 1 2024-02-16T22:56:23.728 INFO:tasks.workunit.client.1.smithi192.stdout:6/558: dwrite da/df/d12/f9b [0,4194304] 0 2024-02-16T22:56:23.728 INFO:tasks.workunit.client.1.smithi192.stdout:5/521: mkdir d6/d12/d4e/d5b/d67/d78/dae 0 2024-02-16T22:56:23.729 INFO:tasks.workunit.client.0.smithi064.stdout:6/503: fdatasync d0/d1b/f36 0 2024-02-16T22:56:23.729 INFO:tasks.workunit.client.1.smithi192.stdout:6/559: chown da/df/d6b/d74/f8b 2 1 2024-02-16T22:56:23.732 INFO:tasks.workunit.client.1.smithi192.stdout:6/560: chown da/df/d12/d57/d81/cb0 11580 1 2024-02-16T22:56:23.735 INFO:tasks.workunit.client.0.smithi064.stdout:0/392: sync 2024-02-16T22:56:23.735 INFO:tasks.workunit.client.0.smithi064.stdout:0/393: chown da/d26 1878322 1 2024-02-16T22:56:23.736 INFO:tasks.workunit.client.0.smithi064.stdout:0/394: read da/f10 [2256977,56330] 0 2024-02-16T22:56:23.736 INFO:tasks.workunit.client.0.smithi064.stdout:5/371: mknod d6/d18/c7e 0 2024-02-16T22:56:23.737 INFO:tasks.workunit.client.1.smithi192.stdout:1/518: dwrite d3/d40/f59 [0,4194304] 0 2024-02-16T22:56:23.740 INFO:tasks.workunit.client.0.smithi064.stdout:3/549: chown d4/d13/d1b/d27/d61/d63/cbb 0 1 2024-02-16T22:56:23.742 INFO:tasks.workunit.client.1.smithi192.stdout:0/557: fdatasync d1/d3/d29/d43/f8b 0 2024-02-16T22:56:23.742 INFO:tasks.workunit.client.0.smithi064.stdout:3/550: truncate d4/d13/d1b/d4f/fb3 784288 0 2024-02-16T22:56:23.743 INFO:tasks.workunit.client.1.smithi192.stdout:0/558: write d1/f51 [2243210,60765] 0 2024-02-16T22:56:23.743 INFO:tasks.workunit.client.0.smithi064.stdout:8/433: truncate df/d2a/f70 1225704 0 2024-02-16T22:56:23.743 INFO:tasks.workunit.client.0.smithi064.stdout:3/551: chown d4/d13/d35 1158673 1 2024-02-16T22:56:23.744 INFO:tasks.workunit.client.1.smithi192.stdout:8/544: dread d5/d9/f17 [0,4194304] 0 2024-02-16T22:56:23.744 INFO:tasks.workunit.client.0.smithi064.stdout:3/552: truncate d4/d13/d1b/fb2 253776 0 2024-02-16T22:56:23.746 INFO:tasks.workunit.client.1.smithi192.stdout:7/635: write d1/d69/f93 [812810,37993] 0 2024-02-16T22:56:23.750 INFO:tasks.workunit.client.1.smithi192.stdout:5/522: mkdir d6/d12/d4e/d5b/daf 0 2024-02-16T22:56:23.751 INFO:tasks.workunit.client.0.smithi064.stdout:3/553: dread d4/d13/d1b/d4f/fa2 [0,4194304] 0 2024-02-16T22:56:23.752 INFO:tasks.workunit.client.0.smithi064.stdout:0/395: mkdir da/dc/de/d12/d2c/d2e/d38/d7e 0 2024-02-16T22:56:23.753 INFO:tasks.workunit.client.0.smithi064.stdout:0/396: dread - da/d26/d3f/f75 zero size 2024-02-16T22:56:23.754 INFO:tasks.workunit.client.1.smithi192.stdout:9/679: symlink d7/d19/d5e/d81/dda/lec 0 2024-02-16T22:56:23.755 INFO:tasks.workunit.client.0.smithi064.stdout:5/372: creat d6/d25/d48/d67/f7f x:0 0 0 2024-02-16T22:56:23.755 INFO:tasks.workunit.client.0.smithi064.stdout:5/373: stat d6/d25/d48/c5b 0 2024-02-16T22:56:23.758 INFO:tasks.workunit.client.0.smithi064.stdout:8/434: unlink df/c50 0 2024-02-16T22:56:23.758 INFO:tasks.workunit.client.1.smithi192.stdout:3/544: dwrite da/df/f23 [0,4194304] 0 2024-02-16T22:56:23.759 INFO:tasks.workunit.client.0.smithi064.stdout:2/535: sync 2024-02-16T22:56:23.759 INFO:tasks.workunit.client.0.smithi064.stdout:7/432: sync 2024-02-16T22:56:23.760 INFO:tasks.workunit.client.0.smithi064.stdout:8/435: truncate df/d31/d2f/d3e/f83 480748 0 2024-02-16T22:56:23.761 INFO:tasks.workunit.client.0.smithi064.stdout:8/436: chown df/d31/d47/l4e 1 1 2024-02-16T22:56:23.761 INFO:tasks.workunit.client.0.smithi064.stdout:1/412: truncate d2/df/d14/d2c/d62/f72 114799 0 2024-02-16T22:56:23.763 INFO:tasks.workunit.client.0.smithi064.stdout:8/437: write df/d31/f7c [60406,25561] 0 2024-02-16T22:56:23.766 INFO:tasks.workunit.client.1.smithi192.stdout:2/586: unlink d22/d24/d39/d42/c5c 0 2024-02-16T22:56:23.769 INFO:tasks.workunit.client.1.smithi192.stdout:1/519: chown d3/d40/lae 15172997 1 2024-02-16T22:56:23.770 INFO:tasks.workunit.client.0.smithi064.stdout:1/413: dwrite d2/df/d14/d18/d40/f68 [0,4194304] 0 2024-02-16T22:56:23.776 INFO:tasks.workunit.client.0.smithi064.stdout:3/554: mkdir d4/d13/d1b/d48/dc0 0 2024-02-16T22:56:23.778 INFO:tasks.workunit.client.0.smithi064.stdout:3/555: write d4/d13/d1b/d4f/f6f [657923,31246] 0 2024-02-16T22:56:23.778 INFO:tasks.workunit.client.1.smithi192.stdout:0/559: symlink d1/d3/de/d22/lae 0 2024-02-16T22:56:23.779 INFO:tasks.workunit.client.0.smithi064.stdout:6/504: truncate d0/d6/d10/f14 626179 0 2024-02-16T22:56:23.780 INFO:tasks.workunit.client.0.smithi064.stdout:6/505: fdatasync d0/d89/f93 0 2024-02-16T22:56:23.782 INFO:tasks.workunit.client.1.smithi192.stdout:8/545: rename d5/db/de/fa4 to d5/fa7 0 2024-02-16T22:56:23.782 INFO:tasks.workunit.client.0.smithi064.stdout:5/374: unlink d6/d18/c2c 0 2024-02-16T22:56:23.784 INFO:tasks.workunit.client.0.smithi064.stdout:9/434: rename d3/d9/f8c to d3/f8f 0 2024-02-16T22:56:23.788 INFO:tasks.workunit.client.1.smithi192.stdout:6/561: mkdir da/db9 0 2024-02-16T22:56:23.790 INFO:tasks.workunit.client.1.smithi192.stdout:6/562: write da/df/d12/d15/d1b/f8a [3213865,123376] 0 2024-02-16T22:56:23.792 INFO:tasks.workunit.client.0.smithi064.stdout:9/435: dwrite d3/d11/d15/d23/f6e [0,4194304] 0 2024-02-16T22:56:23.793 INFO:tasks.workunit.client.1.smithi192.stdout:2/587: mknod d22/d24/d2b/d2c/d3f/dac/cc0 0 2024-02-16T22:56:23.797 INFO:tasks.workunit.client.0.smithi064.stdout:8/438: truncate df/d31/d2f/d38/d48/f54 97728 0 2024-02-16T22:56:23.798 INFO:tasks.workunit.client.1.smithi192.stdout:3/545: dread da/f1a [0,4194304] 0 2024-02-16T22:56:23.799 INFO:tasks.workunit.client.1.smithi192.stdout:3/546: stat da/d11/d63/dbf 0 2024-02-16T22:56:23.801 INFO:tasks.workunit.client.1.smithi192.stdout:3/547: dread da/f1a [0,4194304] 0 2024-02-16T22:56:23.802 INFO:tasks.workunit.client.0.smithi064.stdout:1/414: symlink d2/df/d11/l93 0 2024-02-16T22:56:23.803 INFO:tasks.workunit.client.1.smithi192.stdout:3/548: dread - da/d11/d20/d39/d53/f93 zero size 2024-02-16T22:56:23.807 INFO:tasks.workunit.client.0.smithi064.stdout:7/433: write d5/f27 [2515521,43670] 0 2024-02-16T22:56:23.808 INFO:tasks.workunit.client.1.smithi192.stdout:9/680: mkdir d7/d19/d28/ded 0 2024-02-16T22:56:23.812 INFO:tasks.workunit.client.0.smithi064.stdout:6/506: creat d0/d12/f9b x:0 0 0 2024-02-16T22:56:23.815 INFO:tasks.workunit.client.0.smithi064.stdout:3/556: dwrite d4/d1f/d34/f45 [0,4194304] 0 2024-02-16T22:56:23.816 INFO:tasks.workunit.client.1.smithi192.stdout:2/588: dread d22/f4a [0,4194304] 0 2024-02-16T22:56:23.816 INFO:tasks.workunit.client.0.smithi064.stdout:5/375: mkdir d6/d25/d48/d67/d80 0 2024-02-16T22:56:23.819 INFO:tasks.workunit.client.0.smithi064.stdout:5/376: dread d6/d18/f72 [0,4194304] 0 2024-02-16T22:56:23.819 INFO:tasks.workunit.client.0.smithi064.stdout:4/475: rename dc/d1d/d43/l48 to dc/de/d6d/l90 0 2024-02-16T22:56:23.820 INFO:tasks.workunit.client.0.smithi064.stdout:5/377: dread - d6/d25/d48/d67/f6c zero size 2024-02-16T22:56:23.821 INFO:tasks.workunit.client.0.smithi064.stdout:5/378: dread - d6/d25/d48/d67/f7f zero size 2024-02-16T22:56:23.823 INFO:tasks.workunit.client.1.smithi192.stdout:3/549: symlink da/d11/d20/d39/lc2 0 2024-02-16T22:56:23.824 INFO:tasks.workunit.client.1.smithi192.stdout:5/523: creat d6/d12/d1d/fb0 x:0 0 0 2024-02-16T22:56:23.824 INFO:tasks.workunit.client.1.smithi192.stdout:3/550: readlink da/d11/d63/d6b/d66/d75/l8e 0 2024-02-16T22:56:23.825 INFO:tasks.workunit.client.1.smithi192.stdout:5/524: dread - d6/d12/d2d/f96 zero size 2024-02-16T22:56:23.825 INFO:tasks.workunit.client.1.smithi192.stdout:9/681: rmdir d7/d19/d37/d5a 39 2024-02-16T22:56:23.826 INFO:tasks.workunit.client.1.smithi192.stdout:5/525: fsync d6/d12/d4e/d58/d8f/fa2 0 2024-02-16T22:56:23.826 INFO:tasks.workunit.client.1.smithi192.stdout:6/563: write da/df/d12/d15/d1b/f60 [3457867,115531] 0 2024-02-16T22:56:23.827 INFO:tasks.workunit.client.1.smithi192.stdout:5/526: chown d6/fc 485048913 1 2024-02-16T22:56:23.830 INFO:tasks.workunit.client.0.smithi064.stdout:8/439: mkdir df/d31/d2f/d3e/d5f/d93 0 2024-02-16T22:56:23.833 INFO:tasks.workunit.client.0.smithi064.stdout:7/434: creat d5/d6/d54/d41/f87 x:0 0 0 2024-02-16T22:56:23.835 INFO:tasks.workunit.client.1.smithi192.stdout:7/636: rename d1/d2/d2d/d55 to d1/d2/de/d2c/d49/d54/de3 0 2024-02-16T22:56:23.836 INFO:tasks.workunit.client.0.smithi064.stdout:9/436: dread d3/d9/ff [0,4194304] 0 2024-02-16T22:56:23.837 INFO:tasks.workunit.client.1.smithi192.stdout:3/551: symlink da/d11/d63/d6b/d66/d75/lc3 0 2024-02-16T22:56:23.847 INFO:tasks.workunit.client.1.smithi192.stdout:1/520: dwrite d3/de/d27/d34/d52/f54 [0,4194304] 0 2024-02-16T22:56:23.847 INFO:tasks.workunit.client.0.smithi064.stdout:1/415: rename d2/df/d14/d18/l5d to d2/df/l94 0 2024-02-16T22:56:23.849 INFO:tasks.workunit.client.0.smithi064.stdout:4/476: mknod dc/d1d/d43/c91 0 2024-02-16T22:56:23.854 INFO:tasks.workunit.client.1.smithi192.stdout:2/589: truncate f2 3321921 0 2024-02-16T22:56:23.855 INFO:tasks.workunit.client.1.smithi192.stdout:9/682: write d7/d15/f5d [4514911,95820] 0 2024-02-16T22:56:23.858 INFO:tasks.workunit.client.1.smithi192.stdout:9/683: read - d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/feb zero size 2024-02-16T22:56:23.859 INFO:tasks.workunit.client.0.smithi064.stdout:2/536: rmdir d7/d14/d1e/d34/d73/d8c 0 2024-02-16T22:56:23.859 INFO:tasks.workunit.client.1.smithi192.stdout:6/564: write da/df/d12/d15/f5c [1146490,123687] 0 2024-02-16T22:56:23.862 INFO:tasks.workunit.client.1.smithi192.stdout:5/527: dwrite d6/d12/d2d/f43 [0,4194304] 0 2024-02-16T22:56:23.863 INFO:tasks.workunit.client.1.smithi192.stdout:4/624: sync 2024-02-16T22:56:23.864 INFO:tasks.workunit.client.1.smithi192.stdout:9/684: read d7/d19/d5e/d6f/de1/f96 [900111,60480] 0 2024-02-16T22:56:23.864 INFO:tasks.workunit.client.1.smithi192.stdout:4/625: fdatasync d1/d26/d66/d89/f97 0 2024-02-16T22:56:23.864 INFO:tasks.workunit.client.1.smithi192.stdout:3/552: creat da/d11/d20/d39/fc4 x:0 0 0 2024-02-16T22:56:23.866 INFO:tasks.workunit.client.0.smithi064.stdout:8/440: mkdir df/d31/d61/d94 0 2024-02-16T22:56:23.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/553: dread da/d11/d63/fa6 [0,4194304] 0 2024-02-16T22:56:23.868 INFO:tasks.workunit.client.0.smithi064.stdout:7/435: mkdir d5/d33/d45/d73/d88 0 2024-02-16T22:56:23.869 INFO:tasks.workunit.client.1.smithi192.stdout:8/546: write d5/db/d47/f78 [29455,55756] 0 2024-02-16T22:56:23.870 INFO:tasks.workunit.client.0.smithi064.stdout:7/436: write d5/d6/f1d [843740,26153] 0 2024-02-16T22:56:23.871 INFO:tasks.workunit.client.1.smithi192.stdout:7/637: write d1/d2/f25 [1826901,70104] 0 2024-02-16T22:56:23.872 INFO:tasks.workunit.client.0.smithi064.stdout:6/507: mknod d0/d6/d10/c9c 0 2024-02-16T22:56:23.874 INFO:tasks.workunit.client.1.smithi192.stdout:1/521: truncate d3/d5b/d77/f88 183144 0 2024-02-16T22:56:23.874 INFO:tasks.workunit.client.0.smithi064.stdout:0/397: link da/f1b da/dc/de/d12/d2c/f7f 0 2024-02-16T22:56:23.875 INFO:tasks.workunit.client.0.smithi064.stdout:4/477: dread dc/dd/d32/f1f [0,4194304] 0 2024-02-16T22:56:23.876 INFO:tasks.workunit.client.1.smithi192.stdout:2/590: creat d22/d24/d39/d42/d85/d98/db0/fc1 x:0 0 0 2024-02-16T22:56:23.876 INFO:tasks.workunit.client.1.smithi192.stdout:2/591: fdatasync f4 0 2024-02-16T22:56:23.877 INFO:tasks.workunit.client.1.smithi192.stdout:2/592: readlink d22/d24/d39/d42/d65/d69/d4f/l6d 0 2024-02-16T22:56:23.878 INFO:tasks.workunit.client.0.smithi064.stdout:0/398: write da/dc/de/d12/d1d/f34 [2427142,125825] 0 2024-02-16T22:56:23.878 INFO:tasks.workunit.client.0.smithi064.stdout:3/557: rename d4/d1f/d75 to d4/d1f/d5b/d51/dc1 0 2024-02-16T22:56:23.879 INFO:tasks.workunit.client.1.smithi192.stdout:2/593: fdatasync d22/d24/d39/d42/d52/f60 0 2024-02-16T22:56:23.879 INFO:tasks.workunit.client.0.smithi064.stdout:7/437: dwrite d5/d6/f30 [8388608,4194304] 0 2024-02-16T22:56:23.885 INFO:tasks.workunit.client.0.smithi064.stdout:1/416: symlink d2/df/d11/l95 0 2024-02-16T22:56:23.885 INFO:tasks.workunit.client.0.smithi064.stdout:3/558: dread d4/d13/d4d/d5c/f6c [0,4194304] 0 2024-02-16T22:56:23.885 INFO:tasks.workunit.client.0.smithi064.stdout:5/379: creat d6/d25/d48/d67/d80/f81 x:0 0 0 2024-02-16T22:56:23.885 INFO:tasks.workunit.client.1.smithi192.stdout:6/565: write da/df/d18/f9f [1400306,98149] 0 2024-02-16T22:56:23.886 INFO:tasks.workunit.client.1.smithi192.stdout:6/566: readlink da/lb4 0 2024-02-16T22:56:23.887 INFO:tasks.workunit.client.1.smithi192.stdout:6/567: write da/df/d12/d15/d1b/f8a [1991641,18182] 0 2024-02-16T22:56:23.888 INFO:tasks.workunit.client.0.smithi064.stdout:7/438: dwrite d5/d33/f3d [8388608,4194304] 0 2024-02-16T22:56:23.889 INFO:tasks.workunit.client.1.smithi192.stdout:4/626: symlink d1/d26/d66/d89/d11/lda 0 2024-02-16T22:56:23.891 INFO:tasks.workunit.client.1.smithi192.stdout:9/685: readlink d7/d19/d37/d39/l85 0 2024-02-16T22:56:23.891 INFO:tasks.workunit.client.1.smithi192.stdout:6/568: dread da/df/d12/f75 [0,4194304] 0 2024-02-16T22:56:23.892 INFO:tasks.workunit.client.1.smithi192.stdout:6/569: dread - da/df/d6b/d74/f9d zero size 2024-02-16T22:56:23.892 INFO:tasks.workunit.client.1.smithi192.stdout:0/560: sync 2024-02-16T22:56:23.893 INFO:tasks.workunit.client.0.smithi064.stdout:9/437: symlink d3/d9/d1e/d42/d7b/d86/l90 0 2024-02-16T22:56:23.894 INFO:tasks.workunit.client.1.smithi192.stdout:0/561: chown d1/d3/f12 1862083 1 2024-02-16T22:56:23.894 INFO:tasks.workunit.client.1.smithi192.stdout:3/554: symlink da/d11/d63/d6b/d4e/lc5 0 2024-02-16T22:56:23.895 INFO:tasks.workunit.client.0.smithi064.stdout:6/508: unlink d0/d6/d10/c9c 0 2024-02-16T22:56:23.896 INFO:tasks.workunit.client.0.smithi064.stdout:4/478: mkdir dc/d4a/d92 0 2024-02-16T22:56:23.897 INFO:tasks.workunit.client.0.smithi064.stdout:4/479: chown dc/dd/d32/d46/d5a/f83 8905 1 2024-02-16T22:56:23.898 INFO:tasks.workunit.client.1.smithi192.stdout:6/570: dread da/df/d12/d57/f35 [0,4194304] 0 2024-02-16T22:56:23.899 INFO:tasks.workunit.client.1.smithi192.stdout:1/522: truncate d3/de/d27/d34/d52/f56 579650 0 2024-02-16T22:56:23.900 INFO:tasks.workunit.client.1.smithi192.stdout:1/523: write d3/de/d27/d34/d52/d5a/fa2 [768363,63855] 0 2024-02-16T22:56:23.901 INFO:tasks.workunit.client.0.smithi064.stdout:0/399: creat da/dc/de/d12/d2c/d2e/d38/f80 x:0 0 0 2024-02-16T22:56:23.901 INFO:tasks.workunit.client.1.smithi192.stdout:1/524: fdatasync d3/f22 0 2024-02-16T22:56:23.903 INFO:tasks.workunit.client.0.smithi064.stdout:1/417: write d2/df/d11/f6d [1989246,75165] 0 2024-02-16T22:56:23.908 INFO:tasks.workunit.client.0.smithi064.stdout:3/559: rmdir d4/d13/d4d 39 2024-02-16T22:56:23.912 INFO:tasks.workunit.client.0.smithi064.stdout:8/441: fsync df/f15 0 2024-02-16T22:56:23.913 INFO:tasks.workunit.client.0.smithi064.stdout:8/442: dread - df/d31/d2f/d4f/f8d zero size 2024-02-16T22:56:23.914 INFO:tasks.workunit.client.1.smithi192.stdout:4/627: fsync d1/d26/d66/d89/f99 0 2024-02-16T22:56:23.914 INFO:tasks.workunit.client.0.smithi064.stdout:8/443: chown df/d31/d2f/l25 7186413 1 2024-02-16T22:56:23.914 INFO:tasks.workunit.client.0.smithi064.stdout:8/444: chown f3 175023385 1 2024-02-16T22:56:23.916 INFO:tasks.workunit.client.0.smithi064.stdout:8/445: write df/d31/d2f/d4f/f66 [5110010,28245] 0 2024-02-16T22:56:23.917 INFO:tasks.workunit.client.1.smithi192.stdout:9/686: creat d7/d19/d28/d43/fee x:0 0 0 2024-02-16T22:56:23.917 INFO:tasks.workunit.client.1.smithi192.stdout:9/687: read - d7/d19/d37/fb5 zero size 2024-02-16T22:56:23.921 INFO:tasks.workunit.client.0.smithi064.stdout:4/480: creat dc/dd/d32/d22/f93 x:0 0 0 2024-02-16T22:56:23.926 INFO:tasks.workunit.client.1.smithi192.stdout:8/547: unlink d5/d26/d8f/l9a 0 2024-02-16T22:56:23.927 INFO:tasks.workunit.client.0.smithi064.stdout:2/537: creat d7/de/d95/fa3 x:0 0 0 2024-02-16T22:56:23.928 INFO:tasks.workunit.client.1.smithi192.stdout:8/548: write d5/d26/d8f/d4d/f4b [4360513,91844] 0 2024-02-16T22:56:23.929 INFO:tasks.workunit.client.1.smithi192.stdout:7/638: mknod d1/d2/de/d2c/d49/d89/dbd/ce4 0 2024-02-16T22:56:23.929 INFO:tasks.workunit.client.0.smithi064.stdout:5/380: symlink d6/d1c/d43/d7d/l82 0 2024-02-16T22:56:23.931 INFO:tasks.workunit.client.1.smithi192.stdout:7/639: write d1/de1/f3c [1925380,4722] 0 2024-02-16T22:56:23.931 INFO:tasks.workunit.client.0.smithi064.stdout:2/538: dread d7/de/f1f [0,4194304] 0 2024-02-16T22:56:23.932 INFO:tasks.workunit.client.1.smithi192.stdout:2/594: creat d22/d24/d39/d42/d85/d98/db0/db6/fc2 x:0 0 0 2024-02-16T22:56:23.932 INFO:tasks.workunit.client.1.smithi192.stdout:5/528: dwrite d6/f33 [0,4194304] 0 2024-02-16T22:56:23.934 INFO:tasks.workunit.client.1.smithi192.stdout:5/529: dread - d6/d12/d4e/d46/d4f/f62 zero size 2024-02-16T22:56:23.934 INFO:tasks.workunit.client.1.smithi192.stdout:2/595: write d22/d24/d39/d42/d65/d66/f83 [3637176,7139] 0 2024-02-16T22:56:23.935 INFO:tasks.workunit.client.1.smithi192.stdout:1/525: mkdir d3/de/d27/d34/d52/db7 0 2024-02-16T22:56:23.936 INFO:tasks.workunit.client.1.smithi192.stdout:2/596: write d22/f3c [1912235,35122] 0 2024-02-16T22:56:23.937 INFO:tasks.workunit.client.0.smithi064.stdout:7/439: mknod d5/d6/d54/d7b/c89 0 2024-02-16T22:56:23.943 INFO:tasks.workunit.client.0.smithi064.stdout:0/400: dwrite da/f10 [8388608,4194304] 0 2024-02-16T22:56:23.944 INFO:tasks.workunit.client.0.smithi064.stdout:9/438: mkdir d3/d9/d91 0 2024-02-16T22:56:23.944 INFO:tasks.workunit.client.1.smithi192.stdout:2/597: dwrite d22/d24/d39/d42/d65/d69/d4f/f97 [0,4194304] 0 2024-02-16T22:56:23.945 INFO:tasks.workunit.client.1.smithi192.stdout:9/688: mkdir d7/d15/d36/d38/d3f/d8a/def 0 2024-02-16T22:56:23.945 INFO:tasks.workunit.client.0.smithi064.stdout:8/446: symlink df/d2a/d5c/l95 0 2024-02-16T22:56:23.947 INFO:tasks.workunit.client.0.smithi064.stdout:4/481: rmdir dc/dd/d16/d7c 39 2024-02-16T22:56:23.949 INFO:tasks.workunit.client.0.smithi064.stdout:9/439: dwrite d3/f63 [4194304,4194304] 0 2024-02-16T22:56:23.957 INFO:tasks.workunit.client.0.smithi064.stdout:2/539: creat d7/d14/d1e/fa4 x:0 0 0 2024-02-16T22:56:23.958 INFO:tasks.workunit.client.0.smithi064.stdout:7/440: creat d5/d33/f8a x:0 0 0 2024-02-16T22:56:23.960 INFO:tasks.workunit.client.0.smithi064.stdout:0/401: chown da/dc/de/d12/d1d/c6a 273 1 2024-02-16T22:56:23.963 INFO:tasks.workunit.client.0.smithi064.stdout:6/509: creat d0/d6/d10/f9d x:0 0 0 2024-02-16T22:56:23.963 INFO:tasks.workunit.client.1.smithi192.stdout:5/530: rmdir d6/d20 39 2024-02-16T22:56:23.965 INFO:tasks.workunit.client.0.smithi064.stdout:4/482: rmdir dc/dd/d32/d58 39 2024-02-16T22:56:23.965 INFO:tasks.workunit.client.0.smithi064.stdout:6/510: dread - d0/d5/df/d17/d18/d5f/d76/f9a zero size 2024-02-16T22:56:23.966 INFO:tasks.workunit.client.0.smithi064.stdout:7/441: dwrite d5/d6/f30 [4194304,4194304] 0 2024-02-16T22:56:23.968 INFO:tasks.workunit.client.0.smithi064.stdout:4/483: write dc/dd/d16/f3a [4227543,34411] 0 2024-02-16T22:56:23.968 INFO:tasks.workunit.client.0.smithi064.stdout:6/511: dread - d0/d5/d54/d60/f82 zero size 2024-02-16T22:56:23.968 INFO:tasks.workunit.client.0.smithi064.stdout:1/418: rename d2/f42 to d2/df/d14/f96 0 2024-02-16T22:56:23.972 INFO:tasks.workunit.client.0.smithi064.stdout:9/440: dread - d3/d11/d15/f51 zero size 2024-02-16T22:56:23.973 INFO:tasks.workunit.client.0.smithi064.stdout:1/419: dread - d2/df/d14/d2c/d62/d91/f65 zero size 2024-02-16T22:56:23.973 INFO:tasks.workunit.client.0.smithi064.stdout:0/402: dread da/dc/de/d12/d2c/d2e/f3e [0,4194304] 0 2024-02-16T22:56:23.973 INFO:tasks.workunit.client.1.smithi192.stdout:1/526: mknod d3/de/d27/d34/cb8 0 2024-02-16T22:56:23.974 INFO:tasks.workunit.client.0.smithi064.stdout:1/420: chown d2/df 42490 1 2024-02-16T22:56:23.974 INFO:tasks.workunit.client.1.smithi192.stdout:4/628: mkdir d1/d26/d34/db9/ddb 0 2024-02-16T22:56:23.975 INFO:tasks.workunit.client.1.smithi192.stdout:4/629: truncate d1/d26/f77 1823448 0 2024-02-16T22:56:23.976 INFO:tasks.workunit.client.1.smithi192.stdout:4/630: dread - d1/d26/d34/dca/fd1 zero size 2024-02-16T22:56:23.977 INFO:tasks.workunit.client.0.smithi064.stdout:0/403: readlink da/dc/de/d12/d1d/l57 0 2024-02-16T22:56:23.984 INFO:tasks.workunit.client.0.smithi064.stdout:6/512: dwrite d0/d6/d10/f40 [4194304,4194304] 0 2024-02-16T22:56:23.992 INFO:tasks.workunit.client.1.smithi192.stdout:6/571: creat da/df/fba x:0 0 0 2024-02-16T22:56:23.995 INFO:tasks.workunit.client.0.smithi064.stdout:8/447: rename df/d31/f7e to df/d31/d2f/d68/f96 0 2024-02-16T22:56:23.996 INFO:tasks.workunit.client.1.smithi192.stdout:7/640: creat d1/d2/d2d/ddf/fe5 x:0 0 0 2024-02-16T22:56:23.996 INFO:tasks.workunit.client.1.smithi192.stdout:8/549: write d5/d26/d83/d86/d5f/f5e [1046356,110051] 0 2024-02-16T22:56:23.997 INFO:tasks.workunit.client.0.smithi064.stdout:2/540: mknod d7/ca5 0 2024-02-16T22:56:23.998 INFO:tasks.workunit.client.1.smithi192.stdout:7/641: dread d1/de1/f87 [0,4194304] 0 2024-02-16T22:56:24.000 INFO:tasks.workunit.client.1.smithi192.stdout:1/527: mknod d3/d40/cb9 0 2024-02-16T22:56:24.001 INFO:tasks.workunit.client.1.smithi192.stdout:7/642: dread d1/d2/de/d2c/d49/f6f [0,4194304] 0 2024-02-16T22:56:24.003 INFO:tasks.workunit.client.0.smithi064.stdout:1/421: chown d2/df/d14/d2c/d62/d91/d56/d5b 9484073 1 2024-02-16T22:56:24.004 INFO:tasks.workunit.client.1.smithi192.stdout:0/562: link d1/c20 d1/d3/d29/d43/caf 0 2024-02-16T22:56:24.009 INFO:tasks.workunit.client.1.smithi192.stdout:2/598: dwrite d22/d24/d39/d3d/f40 [0,4194304] 0 2024-02-16T22:56:24.012 INFO:tasks.workunit.client.1.smithi192.stdout:3/555: truncate da/f3d 1397117 0 2024-02-16T22:56:24.015 INFO:tasks.workunit.client.1.smithi192.stdout:7/643: mkdir d1/d2/d2d/d61/d8a/de6 0 2024-02-16T22:56:24.016 INFO:tasks.workunit.client.1.smithi192.stdout:2/599: dwrite d22/d24/fb9 [0,4194304] 0 2024-02-16T22:56:24.017 INFO:tasks.workunit.client.1.smithi192.stdout:0/563: mkdir d1/d3/de/d22/d37/d67/d75/db0 0 2024-02-16T22:56:24.020 INFO:tasks.workunit.client.1.smithi192.stdout:9/689: rename d7/d19/d5e/d6f/de1 to d7/d19/d28/d43/d70/df0 0 2024-02-16T22:56:24.027 INFO:tasks.workunit.client.1.smithi192.stdout:9/690: fsync d7/d15/d36/d38/d3f/fc8 0 2024-02-16T22:56:24.027 INFO:tasks.workunit.client.1.smithi192.stdout:7/644: dwrite d1/d69/fce [0,4194304] 0 2024-02-16T22:56:24.027 INFO:tasks.workunit.client.1.smithi192.stdout:9/691: chown d7/d15/d36/d77/c82 214858247 1 2024-02-16T22:56:24.027 INFO:tasks.workunit.client.1.smithi192.stdout:3/556: readlink da/df/lc0 0 2024-02-16T22:56:24.028 INFO:tasks.workunit.client.1.smithi192.stdout:0/564: dwrite d1/f3e [0,4194304] 0 2024-02-16T22:56:24.036 INFO:tasks.workunit.client.1.smithi192.stdout:5/531: dwrite d6/f2b [0,4194304] 0 2024-02-16T22:56:24.038 INFO:tasks.workunit.client.1.smithi192.stdout:6/572: creat da/fbb x:0 0 0 2024-02-16T22:56:24.040 INFO:tasks.workunit.client.1.smithi192.stdout:6/573: chown da/df/d12/d15/c7a 11 1 2024-02-16T22:56:24.040 INFO:tasks.workunit.client.1.smithi192.stdout:6/574: stat da/c8f 0 2024-02-16T22:56:24.041 INFO:tasks.workunit.client.1.smithi192.stdout:7/645: fdatasync d1/d2/de/d2c/d49/da4/f82 0 2024-02-16T22:56:24.041 INFO:tasks.workunit.client.1.smithi192.stdout:7/646: chown d1/d2/d2d/d92 7821 1 2024-02-16T22:56:24.042 INFO:tasks.workunit.client.1.smithi192.stdout:7/647: fsync d1/d2/de/d2c/d49/da4/f82 0 2024-02-16T22:56:24.043 INFO:tasks.workunit.client.1.smithi192.stdout:7/648: read - d1/d2/de/d83/d9f/fa2 zero size 2024-02-16T22:56:24.044 INFO:tasks.workunit.client.1.smithi192.stdout:9/692: creat d7/d15/d36/d77/ff1 x:0 0 0 2024-02-16T22:56:24.045 INFO:tasks.workunit.client.0.smithi064.stdout:6/513: creat d0/d1d/f9e x:0 0 0 2024-02-16T22:56:24.051 INFO:tasks.workunit.client.1.smithi192.stdout:4/631: rename d1/d26/d66/d89/c38 to d1/d26/d34/db9/cdc 0 2024-02-16T22:56:24.052 INFO:tasks.workunit.client.1.smithi192.stdout:4/632: chown d1/d26/d5e/cb3 66815403 1 2024-02-16T22:56:24.053 INFO:tasks.workunit.client.0.smithi064.stdout:5/381: rename d6/d1c/d4b/f6e to d6/d18/d39/f83 0 2024-02-16T22:56:24.054 INFO:tasks.workunit.client.1.smithi192.stdout:5/532: mknod d6/d8/d86/cb1 0 2024-02-16T22:56:24.057 INFO:tasks.workunit.client.0.smithi064.stdout:0/404: symlink da/l81 0 2024-02-16T22:56:24.058 INFO:tasks.workunit.client.0.smithi064.stdout:0/405: write da/d26/d3f/f75 [4539,73709] 0 2024-02-16T22:56:24.058 INFO:tasks.workunit.client.1.smithi192.stdout:6/575: creat da/d40/fbc x:0 0 0 2024-02-16T22:56:24.059 INFO:tasks.workunit.client.1.smithi192.stdout:6/576: fsync da/df/d12/d15/d1b/f60 0 2024-02-16T22:56:24.060 INFO:tasks.workunit.client.0.smithi064.stdout:9/441: mknod d3/d11/d15/d5a/d2a/d2e/d31/d3f/d80/c92 0 2024-02-16T22:56:24.061 INFO:tasks.workunit.client.1.smithi192.stdout:3/557: creat da/d6a/fc6 x:0 0 0 2024-02-16T22:56:24.062 INFO:tasks.workunit.client.0.smithi064.stdout:1/422: truncate d2/df/d11/f3b 432820 0 2024-02-16T22:56:24.063 INFO:tasks.workunit.client.0.smithi064.stdout:7/442: creat d5/f8b x:0 0 0 2024-02-16T22:56:24.067 INFO:tasks.workunit.client.1.smithi192.stdout:8/550: rename d5/db/d47/f78 to d5/db/d47/d8c/d9d/fa8 0 2024-02-16T22:56:24.069 INFO:tasks.workunit.client.1.smithi192.stdout:4/633: creat d1/d26/d34/db9/fdd x:0 0 0 2024-02-16T22:56:24.076 INFO:tasks.workunit.client.0.smithi064.stdout:9/442: mknod d3/d11/d15/d5a/d2a/d2e/d31/d3f/c93 0 2024-02-16T22:56:24.076 INFO:tasks.workunit.client.0.smithi064.stdout:4/484: write dc/dd/d32/d58/f14 [2944331,24761] 0 2024-02-16T22:56:24.076 INFO:tasks.workunit.client.0.smithi064.stdout:9/443: dread - d3/d11/d15/f7f zero size 2024-02-16T22:56:24.077 INFO:tasks.workunit.client.1.smithi192.stdout:6/577: creat da/df/d12/d57/fbd x:0 0 0 2024-02-16T22:56:24.080 INFO:tasks.workunit.client.0.smithi064.stdout:7/443: symlink d5/d33/d42/l8c 0 2024-02-16T22:56:24.084 INFO:tasks.workunit.client.1.smithi192.stdout:7/649: truncate d1/d2/de/d2c/f8b 455998 0 2024-02-16T22:56:24.087 INFO:tasks.workunit.client.0.smithi064.stdout:7/444: dwrite d5/d33/d45/d64/d56/d5d/f80 [0,4194304] 0 2024-02-16T22:56:24.087 INFO:tasks.workunit.client.0.smithi064.stdout:8/448: link df/d31/d2f/d38/d48/l56 df/d31/d2f/l97 0 2024-02-16T22:56:24.096 INFO:tasks.workunit.client.0.smithi064.stdout:0/406: write da/d26/d3f/f6b [67395,16613] 0 2024-02-16T22:56:24.099 INFO:tasks.workunit.client.1.smithi192.stdout:0/565: rename d1/d3/de/d22/c5c to d1/d3/d39/cb1 0 2024-02-16T22:56:24.099 INFO:tasks.workunit.client.1.smithi192.stdout:0/566: dread - d1/d3/de/d22/d37/d67/d34/f56 zero size 2024-02-16T22:56:24.100 INFO:tasks.workunit.client.0.smithi064.stdout:6/514: creat d0/d6/d10/f9f x:0 0 0 2024-02-16T22:56:24.101 INFO:tasks.workunit.client.1.smithi192.stdout:4/634: truncate d1/f85 155583 0 2024-02-16T22:56:24.101 INFO:tasks.workunit.client.1.smithi192.stdout:4/635: readlink d1/d26/d66/d89/d11/lda 0 2024-02-16T22:56:24.102 INFO:tasks.workunit.client.1.smithi192.stdout:5/533: creat d6/d12/d4e/d5b/daf/fb2 x:0 0 0 2024-02-16T22:56:24.105 INFO:tasks.workunit.client.0.smithi064.stdout:2/541: rename d7/d14/d25/c68 to d7/ca6 0 2024-02-16T22:56:24.106 INFO:tasks.workunit.client.1.smithi192.stdout:1/528: link d3/de/l58 d3/de/d27/d34/d36/d47/d5d/lba 0 2024-02-16T22:56:24.107 INFO:tasks.workunit.client.0.smithi064.stdout:1/423: dwrite d2/df/d14/d2c/f5f [0,4194304] 0 2024-02-16T22:56:24.111 INFO:tasks.workunit.client.0.smithi064.stdout:7/445: chown d5/cc 31 1 2024-02-16T22:56:24.112 INFO:tasks.workunit.client.1.smithi192.stdout:1/529: dwrite d3/f21 [0,4194304] 0 2024-02-16T22:56:24.117 INFO:tasks.workunit.client.0.smithi064.stdout:9/444: dread d3/d9/f41 [0,4194304] 0 2024-02-16T22:56:24.118 INFO:tasks.workunit.client.0.smithi064.stdout:0/407: creat da/d26/f82 x:0 0 0 2024-02-16T22:56:24.119 INFO:tasks.workunit.client.0.smithi064.stdout:9/445: chown d3/d11/f1f 19252897 1 2024-02-16T22:56:24.120 INFO:tasks.workunit.client.1.smithi192.stdout:7/650: creat d1/d2/d37/fe7 x:0 0 0 2024-02-16T22:56:24.121 INFO:tasks.workunit.client.1.smithi192.stdout:7/651: fdatasync d1/d2/f29 0 2024-02-16T22:56:24.123 INFO:tasks.workunit.client.1.smithi192.stdout:7/652: write d1/d2/de/d2c/d49/d54/d6a/f95 [8485919,62979] 0 2024-02-16T22:56:24.123 INFO:tasks.workunit.client.1.smithi192.stdout:9/693: creat d7/d19/d37/ff2 x:0 0 0 2024-02-16T22:56:24.125 INFO:tasks.workunit.client.1.smithi192.stdout:2/600: getdents d22/d24/d39/d42/d84 0 2024-02-16T22:56:24.125 INFO:tasks.workunit.client.1.smithi192.stdout:2/601: chown d22/d24/d39/d42/d65/d69/d4f/c8f 257283 1 2024-02-16T22:56:24.128 INFO:tasks.workunit.client.1.smithi192.stdout:3/558: write da/d11/d63/d6b/d80/f83 [533955,67638] 0 2024-02-16T22:56:24.132 INFO:tasks.workunit.client.0.smithi064.stdout:8/449: symlink df/d31/d2f/d3e/d5f/d93/l98 0 2024-02-16T22:56:24.133 INFO:tasks.workunit.client.1.smithi192.stdout:2/602: dwrite d22/d24/d39/d42/d65/d69/d4f/fae [0,4194304] 0 2024-02-16T22:56:24.133 INFO:tasks.workunit.client.0.smithi064.stdout:4/485: link dc/de/d28/c88 dc/dd/d32/d46/d5a/d7e/c94 0 2024-02-16T22:56:24.134 INFO:tasks.workunit.client.1.smithi192.stdout:2/603: readlink d22/d24/d39/l53 0 2024-02-16T22:56:24.135 INFO:tasks.workunit.client.1.smithi192.stdout:6/578: creat da/df/d58/d72/fbe x:0 0 0 2024-02-16T22:56:24.135 INFO:tasks.workunit.client.1.smithi192.stdout:6/579: stat da/d40 0 2024-02-16T22:56:24.138 INFO:tasks.workunit.client.0.smithi064.stdout:4/486: chown dc/dd/d32/d46/d5a/l7f 3 1 2024-02-16T22:56:24.138 INFO:tasks.workunit.client.0.smithi064.stdout:2/542: dread d7/f1a [0,4194304] 0 2024-02-16T22:56:24.138 INFO:tasks.workunit.client.0.smithi064.stdout:6/515: write d0/d1b/f4c [4141515,63485] 0 2024-02-16T22:56:24.144 INFO:tasks.workunit.client.1.smithi192.stdout:1/530: dread d3/f22 [4194304,4194304] 0 2024-02-16T22:56:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:8/450: dwrite df/d31/d2f/f7d [0,4194304] 0 2024-02-16T22:56:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:8/451: read - df/d31/d2f/d4f/f8d zero size 2024-02-16T22:56:24.147 INFO:tasks.workunit.client.1.smithi192.stdout:0/567: creat d1/d3/de/d22/d37/d67/d75/db0/fb2 x:0 0 0 2024-02-16T22:56:24.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/559: rmdir da/df 39 2024-02-16T22:56:24.152 INFO:tasks.workunit.client.1.smithi192.stdout:4/636: rename d1/d26/d66/d89/c25 to d1/d15/d48/d8d/cde 0 2024-02-16T22:56:24.153 INFO:tasks.workunit.client.1.smithi192.stdout:8/551: dwrite d5/d9/f17 [0,4194304] 0 2024-02-16T22:56:24.155 INFO:tasks.workunit.client.0.smithi064.stdout:4/487: creat dc/d1d/f95 x:0 0 0 2024-02-16T22:56:24.157 INFO:tasks.workunit.client.0.smithi064.stdout:7/446: truncate d5/d6/f5b 638716 0 2024-02-16T22:56:24.158 INFO:tasks.workunit.client.0.smithi064.stdout:2/543: mknod d7/d14/d25/d46/ca7 0 2024-02-16T22:56:24.159 INFO:tasks.workunit.client.0.smithi064.stdout:6/516: unlink d0/d1d/f88 0 2024-02-16T22:56:24.162 INFO:tasks.workunit.client.1.smithi192.stdout:7/653: creat d1/d2/de/d2c/d49/d54/dd8/fe8 x:0 0 0 2024-02-16T22:56:24.163 INFO:tasks.workunit.client.0.smithi064.stdout:8/452: write fa [2318371,78412] 0 2024-02-16T22:56:24.163 INFO:tasks.workunit.client.1.smithi192.stdout:1/531: mknod d3/d5b/cbb 0 2024-02-16T22:56:24.166 INFO:tasks.workunit.client.0.smithi064.stdout:9/446: dread d3/d11/d15/d5a/d2a/d2e/f39 [0,4194304] 0 2024-02-16T22:56:24.166 INFO:tasks.workunit.client.1.smithi192.stdout:0/568: unlink d1/d3/d29/d43/c59 0 2024-02-16T22:56:24.167 INFO:tasks.workunit.client.0.smithi064.stdout:5/382: rename d6/c7 to d6/d1c/c84 0 2024-02-16T22:56:24.168 INFO:tasks.workunit.client.0.smithi064.stdout:4/488: symlink dc/d4a/l96 0 2024-02-16T22:56:24.169 INFO:tasks.workunit.client.0.smithi064.stdout:3/560: sync 2024-02-16T22:56:24.169 INFO:tasks.workunit.client.0.smithi064.stdout:4/489: write dc/dd/d32/d46/d5a/f65 [763563,49679] 0 2024-02-16T22:56:24.170 INFO:tasks.workunit.client.0.smithi064.stdout:3/561: readlink d4/d13/d1b/d48/l70 0 2024-02-16T22:56:24.170 INFO:tasks.workunit.client.0.smithi064.stdout:4/490: fsync dc/d1d/f69 0 2024-02-16T22:56:24.171 INFO:tasks.workunit.client.1.smithi192.stdout:9/694: rename d7/d19/d37/d39/f4e to d7/d15/d36/ff3 0 2024-02-16T22:56:24.174 INFO:tasks.workunit.client.0.smithi064.stdout:1/424: link d2/df/d14/d18/d48/d84/l90 d2/df/d11/d71/l97 0 2024-02-16T22:56:24.176 INFO:tasks.workunit.client.1.smithi192.stdout:5/534: rmdir d6/d12/d4e/d5b/d67/d8b 0 2024-02-16T22:56:24.177 INFO:tasks.workunit.client.0.smithi064.stdout:8/453: creat df/d31/d2f/d4f/f99 x:0 0 0 2024-02-16T22:56:24.181 INFO:tasks.workunit.client.0.smithi064.stdout:0/408: getdents da 0 2024-02-16T22:56:24.182 INFO:tasks.workunit.client.1.smithi192.stdout:2/604: write d22/d24/d2b/f45 [238811,85780] 0 2024-02-16T22:56:24.183 INFO:tasks.workunit.client.1.smithi192.stdout:2/605: write d22/d24/d39/d3d/fbb [482265,48497] 0 2024-02-16T22:56:24.185 INFO:tasks.workunit.client.0.smithi064.stdout:8/454: dwrite df/d31/d2f/d3e/f83 [0,4194304] 0 2024-02-16T22:56:24.185 INFO:tasks.workunit.client.0.smithi064.stdout:0/409: truncate da/dc/de/d12/d2c/d2e/f3e 2614595 0 2024-02-16T22:56:24.187 INFO:tasks.workunit.client.0.smithi064.stdout:0/410: truncate da/dc/de/f76 10201 0 2024-02-16T22:56:24.187 INFO:tasks.workunit.client.0.smithi064.stdout:5/383: mkdir d6/d18/d19/d1a/d85 0 2024-02-16T22:56:24.188 INFO:tasks.workunit.client.0.smithi064.stdout:5/384: write d6/d18/d19/d1a/f60 [651280,50079] 0 2024-02-16T22:56:24.188 INFO:tasks.workunit.client.1.smithi192.stdout:0/569: mknod d1/d3/d29/d43/cb3 0 2024-02-16T22:56:24.189 INFO:tasks.workunit.client.1.smithi192.stdout:0/570: readlink d1/d3/de/l40 0 2024-02-16T22:56:24.190 INFO:tasks.workunit.client.1.smithi192.stdout:0/571: stat d1/d3/de/d22/da7 0 2024-02-16T22:56:24.191 INFO:tasks.workunit.client.0.smithi064.stdout:7/447: creat d5/d6/d65/d7f/f8d x:0 0 0 2024-02-16T22:56:24.193 INFO:tasks.workunit.client.0.smithi064.stdout:7/448: write d5/d6/f1d [2678568,87780] 0 2024-02-16T22:56:24.195 INFO:tasks.workunit.client.0.smithi064.stdout:0/411: dwrite da/dc/f4b [0,4194304] 0 2024-02-16T22:56:24.196 INFO:tasks.workunit.client.1.smithi192.stdout:6/580: rename da/fbb to da/df/d12/d15/d1b/fbf 0 2024-02-16T22:56:24.198 INFO:tasks.workunit.client.1.smithi192.stdout:6/581: fdatasync da/df/d12/d57/d81/fb7 0 2024-02-16T22:56:24.199 INFO:tasks.workunit.client.1.smithi192.stdout:6/582: write da/f5f [1341053,6699] 0 2024-02-16T22:56:24.200 INFO:tasks.workunit.client.0.smithi064.stdout:2/544: write d7/de/d95/d44/f49 [141136,13988] 0 2024-02-16T22:56:24.202 INFO:tasks.workunit.client.0.smithi064.stdout:2/545: stat d7/de/c53 0 2024-02-16T22:56:24.203 INFO:tasks.workunit.client.0.smithi064.stdout:6/517: symlink d0/d6/d10/d33/d72/la0 0 2024-02-16T22:56:24.206 INFO:tasks.workunit.client.0.smithi064.stdout:0/412: dwrite da/d26/d3f/f75 [0,4194304] 0 2024-02-16T22:56:24.208 INFO:tasks.workunit.client.0.smithi064.stdout:9/447: write d3/d11/d15/d5a/d2a/d2e/d31/d3f/f5b [918355,109831] 0 2024-02-16T22:56:24.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/535: fsync d6/d12/d1d/f34 0 2024-02-16T22:56:24.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/536: write d6/d8/d77/f85 [676870,68599] 0 2024-02-16T22:56:24.210 INFO:tasks.workunit.client.0.smithi064.stdout:0/413: fsync da/fb 0 2024-02-16T22:56:24.213 INFO:tasks.workunit.client.1.smithi192.stdout:2/606: dread d22/d24/d39/d3d/f80 [0,4194304] 0 2024-02-16T22:56:24.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/532: mknod d3/de/d27/d34/d52/d63/d7a/cbc 0 2024-02-16T22:56:24.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/572: mknod d1/d3/de/d22/d37/d67/d34/cb4 0 2024-02-16T22:56:24.216 INFO:tasks.workunit.client.1.smithi192.stdout:3/560: rename da/d11/d2b/d43 to da/d11/d20/d39/d53/dc7 0 2024-02-16T22:56:24.217 INFO:tasks.workunit.client.0.smithi064.stdout:5/385: mknod d6/d25/c86 0 2024-02-16T22:56:24.221 INFO:tasks.workunit.client.1.smithi192.stdout:9/695: write d7/d15/d36/d38/f49 [445824,103487] 0 2024-02-16T22:56:24.222 INFO:tasks.workunit.client.1.smithi192.stdout:0/573: dwrite d1/d3/d39/f5f [0,4194304] 0 2024-02-16T22:56:24.223 INFO:tasks.workunit.client.1.smithi192.stdout:1/533: dread d3/de/ff [0,4194304] 0 2024-02-16T22:56:24.226 INFO:tasks.workunit.client.1.smithi192.stdout:4/637: creat d1/d15/d29/fdf x:0 0 0 2024-02-16T22:56:24.229 INFO:tasks.workunit.client.0.smithi064.stdout:6/518: write d0/d1d/f61 [3658776,64129] 0 2024-02-16T22:56:24.230 INFO:tasks.workunit.client.0.smithi064.stdout:6/519: dread - d0/d5/df/f79 zero size 2024-02-16T22:56:24.230 INFO:tasks.workunit.client.0.smithi064.stdout:2/546: mknod d7/d14/d25/d46/d55/d57/ca8 0 2024-02-16T22:56:24.232 INFO:tasks.workunit.client.0.smithi064.stdout:2/547: write d7/f86 [1029620,21557] 0 2024-02-16T22:56:24.233 INFO:tasks.workunit.client.1.smithi192.stdout:5/537: creat d6/d12/d4e/d46/d4f/fb3 x:0 0 0 2024-02-16T22:56:24.234 INFO:tasks.workunit.client.1.smithi192.stdout:9/696: dwrite d7/d19/d5e/d81/fcc [0,4194304] 0 2024-02-16T22:56:24.235 INFO:tasks.workunit.client.1.smithi192.stdout:8/552: dwrite d5/d16/f18 [0,4194304] 0 2024-02-16T22:56:24.237 INFO:tasks.workunit.client.1.smithi192.stdout:9/697: chown d7/d19/ccb 74052805 1 2024-02-16T22:56:24.238 INFO:tasks.workunit.client.1.smithi192.stdout:2/607: fdatasync d22/d24/d39/d42/d84/f8c 0 2024-02-16T22:56:24.238 INFO:tasks.workunit.client.1.smithi192.stdout:1/534: dwrite d3/de/d27/d34/d36/d47/faf [0,4194304] 0 2024-02-16T22:56:24.240 INFO:tasks.workunit.client.1.smithi192.stdout:2/608: write d22/d24/d39/fb7 [1039111,73769] 0 2024-02-16T22:56:24.242 INFO:tasks.workunit.client.1.smithi192.stdout:1/535: chown d3/de/d27/d34/d52/d63/l92 371 1 2024-02-16T22:56:24.245 INFO:tasks.workunit.client.0.smithi064.stdout:7/449: write d5/d6/f18 [4347104,110872] 0 2024-02-16T22:56:24.246 INFO:tasks.workunit.client.1.smithi192.stdout:3/561: fsync da/d11/d20/d39/f5d 0 2024-02-16T22:56:24.247 INFO:tasks.workunit.client.0.smithi064.stdout:7/450: chown d5 4922407 1 2024-02-16T22:56:24.248 INFO:tasks.workunit.client.0.smithi064.stdout:4/491: write dc/dd/f59 [140655,66042] 0 2024-02-16T22:56:24.250 INFO:tasks.workunit.client.1.smithi192.stdout:3/562: truncate da/d11/d63/d6b/d80/fb5 430859 0 2024-02-16T22:56:24.253 INFO:tasks.workunit.client.1.smithi192.stdout:1/536: dwrite d3/f50 [0,4194304] 0 2024-02-16T22:56:24.257 INFO:tasks.workunit.client.0.smithi064.stdout:4/492: dwrite dc/dd/d32/f6a [0,4194304] 0 2024-02-16T22:56:24.259 INFO:tasks.workunit.client.1.smithi192.stdout:4/638: dread d1/d15/d23/f52 [0,4194304] 0 2024-02-16T22:56:24.260 INFO:tasks.workunit.client.0.smithi064.stdout:4/493: dread - dc/dd/d32/d46/d5a/f83 zero size 2024-02-16T22:56:24.261 INFO:tasks.workunit.client.1.smithi192.stdout:1/537: dwrite d3/de/d27/d34/d52/d5a/fa2 [0,4194304] 0 2024-02-16T22:56:24.265 INFO:tasks.workunit.client.1.smithi192.stdout:7/654: getdents d1/d2/de/d2c/d49 0 2024-02-16T22:56:24.266 INFO:tasks.workunit.client.0.smithi064.stdout:1/425: dwrite d2/df/d14/d2c/d62/f72 [0,4194304] 0 2024-02-16T22:56:24.266 INFO:tasks.workunit.client.0.smithi064.stdout:6/520: mkdir d0/d5/df/d17/da1 0 2024-02-16T22:56:24.268 INFO:tasks.workunit.client.0.smithi064.stdout:2/548: symlink d7/d14/d25/la9 0 2024-02-16T22:56:24.271 INFO:tasks.workunit.client.1.smithi192.stdout:5/538: creat d6/d12/d4e/d5b/d67/d78/fb4 x:0 0 0 2024-02-16T22:56:24.272 INFO:tasks.workunit.client.1.smithi192.stdout:9/698: mknod d7/d15/d36/d77/cf4 0 2024-02-16T22:56:24.273 INFO:tasks.workunit.client.1.smithi192.stdout:9/699: write d7/d19/d28/fe2 [500853,19120] 0 2024-02-16T22:56:24.274 INFO:tasks.workunit.client.1.smithi192.stdout:2/609: unlink d22/d24/d2b/f45 0 2024-02-16T22:56:24.276 INFO:tasks.workunit.client.0.smithi064.stdout:8/455: link df/d31/d2f/l25 df/d31/d61/d94/l9a 0 2024-02-16T22:56:24.278 INFO:tasks.workunit.client.0.smithi064.stdout:8/456: truncate df/d31/d61/f65 4743823 0 2024-02-16T22:56:24.279 INFO:tasks.workunit.client.1.smithi192.stdout:6/583: creat da/df/d18/fc0 x:0 0 0 2024-02-16T22:56:24.281 INFO:tasks.workunit.client.0.smithi064.stdout:3/562: link d4/d13/d35/fac d4/d14/fc2 0 2024-02-16T22:56:24.285 INFO:tasks.workunit.client.0.smithi064.stdout:4/494: symlink dc/dd/d5b/l97 0 2024-02-16T22:56:24.289 INFO:tasks.workunit.client.0.smithi064.stdout:1/426: mknod d2/df/d14/d18/d48/c98 0 2024-02-16T22:56:24.290 INFO:tasks.workunit.client.1.smithi192.stdout:4/639: mkdir d1/d26/d6f/de0 0 2024-02-16T22:56:24.290 INFO:tasks.workunit.client.0.smithi064.stdout:2/549: read - d7/de/d95/f3a zero size 2024-02-16T22:56:24.295 INFO:tasks.workunit.client.0.smithi064.stdout:7/451: dread d5/f27 [0,4194304] 0 2024-02-16T22:56:24.295 INFO:tasks.workunit.client.1.smithi192.stdout:1/538: dread d3/de/f65 [0,4194304] 0 2024-02-16T22:56:24.296 INFO:tasks.workunit.client.0.smithi064.stdout:3/563: rename d4/d13/d1b/d27/d99/c73 to d4/d13/d1b/d27/d99/d7d/cc3 0 2024-02-16T22:56:24.296 INFO:tasks.workunit.client.1.smithi192.stdout:4/640: dwrite d1/d15/d48/d8d/f9a [0,4194304] 0 2024-02-16T22:56:24.300 INFO:tasks.workunit.client.0.smithi064.stdout:6/521: mkdir d0/d6/da/da2 0 2024-02-16T22:56:24.302 INFO:tasks.workunit.client.1.smithi192.stdout:2/610: mknod d22/cc3 0 2024-02-16T22:56:24.305 INFO:tasks.workunit.client.0.smithi064.stdout:2/550: creat d7/d14/d25/d46/d55/d7f/faa x:0 0 0 2024-02-16T22:56:24.306 INFO:tasks.workunit.client.0.smithi064.stdout:5/386: dwrite d6/d1c/f20 [0,4194304] 0 2024-02-16T22:56:24.308 INFO:tasks.workunit.client.1.smithi192.stdout:3/563: rename da/d11/d63/d6b/d66/f67 to da/d11/d63/fc8 0 2024-02-16T22:56:24.312 INFO:tasks.workunit.client.0.smithi064.stdout:0/414: truncate da/dc/de/d12/d2c/f31 1472808 0 2024-02-16T22:56:24.313 INFO:tasks.workunit.client.0.smithi064.stdout:4/495: symlink dc/dd/d16/d7c/l98 0 2024-02-16T22:56:24.315 INFO:tasks.workunit.client.1.smithi192.stdout:7/655: creat d1/d2/d2d/d61/d8a/de6/fe9 x:0 0 0 2024-02-16T22:56:24.316 INFO:tasks.workunit.client.0.smithi064.stdout:6/522: symlink d0/d12/d7b/la3 0 2024-02-16T22:56:24.316 INFO:tasks.workunit.client.0.smithi064.stdout:6/523: chown d0/d5/lb 0 1 2024-02-16T22:56:24.319 INFO:tasks.workunit.client.0.smithi064.stdout:1/427: creat d2/df/d11/d71/f99 x:0 0 0 2024-02-16T22:56:24.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/641: symlink d1/d15/d23/le1 0 2024-02-16T22:56:24.322 INFO:tasks.workunit.client.1.smithi192.stdout:0/574: truncate d1/d3/f12 2122101 0 2024-02-16T22:56:24.324 INFO:tasks.workunit.client.0.smithi064.stdout:5/387: fdatasync d6/fc 0 2024-02-16T22:56:24.324 INFO:tasks.workunit.client.0.smithi064.stdout:5/388: readlink d6/d1c/d43/d7d/l82 0 2024-02-16T22:56:24.325 INFO:tasks.workunit.client.0.smithi064.stdout:5/389: chown d6/d25/d48/d67/l6f 3313786 1 2024-02-16T22:56:24.325 INFO:tasks.workunit.client.1.smithi192.stdout:2/611: mkdir d22/d24/d39/d42/d85/d98/dc4 0 2024-02-16T22:56:24.326 INFO:tasks.workunit.client.0.smithi064.stdout:5/390: write d6/d25/d48/d67/f7f [410142,90390] 0 2024-02-16T22:56:24.329 INFO:tasks.workunit.client.0.smithi064.stdout:7/452: truncate d5/d6/d54/f40 4159949 0 2024-02-16T22:56:24.330 INFO:tasks.workunit.client.0.smithi064.stdout:7/453: chown d5/d6/d54/f78 4 1 2024-02-16T22:56:24.331 INFO:tasks.workunit.client.1.smithi192.stdout:6/584: symlink da/df/d18/lc1 0 2024-02-16T22:56:24.334 INFO:tasks.workunit.client.0.smithi064.stdout:3/564: rename d4/d13/d1b/l85 to d4/d13/d1b/d27/d82/db0/lc4 0 2024-02-16T22:56:24.334 INFO:tasks.workunit.client.0.smithi064.stdout:3/565: stat d4/d13/f28 0 2024-02-16T22:56:24.335 INFO:tasks.workunit.client.1.smithi192.stdout:0/575: dread d1/f51 [0,4194304] 0 2024-02-16T22:56:24.335 INFO:tasks.workunit.client.0.smithi064.stdout:8/457: write df/d31/d2f/d4f/f8d [739929,84972] 0 2024-02-16T22:56:24.335 INFO:tasks.workunit.client.1.smithi192.stdout:3/564: write da/d11/d20/f24 [2533177,55125] 0 2024-02-16T22:56:24.336 INFO:tasks.workunit.client.1.smithi192.stdout:3/565: chown da/d11/d20/f24 254 1 2024-02-16T22:56:24.336 INFO:tasks.workunit.client.0.smithi064.stdout:4/496: chown dc/de/d6d/l90 366 1 2024-02-16T22:56:24.337 INFO:tasks.workunit.client.0.smithi064.stdout:4/497: stat dc/dd/d32/d46/d5a/f65 0 2024-02-16T22:56:24.338 INFO:tasks.workunit.client.1.smithi192.stdout:7/656: mknod d1/d2/d2d/d61/d8a/cea 0 2024-02-16T22:56:24.343 INFO:tasks.workunit.client.1.smithi192.stdout:5/539: creat d6/d12/d1d/fb5 x:0 0 0 2024-02-16T22:56:24.351 INFO:tasks.workunit.client.1.smithi192.stdout:2/612: creat d22/d24/d39/d42/d84/fc5 x:0 0 0 2024-02-16T22:56:24.351 INFO:tasks.workunit.client.1.smithi192.stdout:2/613: truncate d22/d24/d39/d42/d65/d66/fa4 163148 0 2024-02-16T22:56:24.354 INFO:tasks.workunit.client.1.smithi192.stdout:9/700: write d7/d19/d37/f3d [7061827,10794] 0 2024-02-16T22:56:24.355 INFO:tasks.workunit.client.0.smithi064.stdout:0/415: rename da/dc to da/dc/de/d12/d2c/d2e/d83 22 2024-02-16T22:56:24.357 INFO:tasks.workunit.client.0.smithi064.stdout:0/416: truncate da/d26/f33 4950311 0 2024-02-16T22:56:24.357 INFO:tasks.workunit.client.1.smithi192.stdout:9/701: fdatasync d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/da2/fd9 0 2024-02-16T22:56:24.357 INFO:tasks.workunit.client.1.smithi192.stdout:5/540: creat d6/d12/d1d/d56/fb6 x:0 0 0 2024-02-16T22:56:24.358 INFO:tasks.workunit.client.0.smithi064.stdout:3/566: creat d4/d1f/d5b/d51/fc5 x:0 0 0 2024-02-16T22:56:24.359 INFO:tasks.workunit.client.1.smithi192.stdout:5/541: write d6/d12/d2d/f43 [1426829,61069] 0 2024-02-16T22:56:24.359 INFO:tasks.workunit.client.1.smithi192.stdout:5/542: readlink d6/d8/l31 0 2024-02-16T22:56:24.360 INFO:tasks.workunit.client.1.smithi192.stdout:5/543: write d6/d12/d4e/d5b/d67/d78/fb4 [996221,60252] 0 2024-02-16T22:56:24.360 INFO:tasks.workunit.client.0.smithi064.stdout:8/458: creat df/d31/d61/d79/f9b x:0 0 0 2024-02-16T22:56:24.364 INFO:tasks.workunit.client.1.smithi192.stdout:1/539: write d3/de/d27/d34/f4e [265906,84956] 0 2024-02-16T22:56:24.364 INFO:tasks.workunit.client.1.smithi192.stdout:2/614: chown c9 52 1 2024-02-16T22:56:24.365 INFO:tasks.workunit.client.1.smithi192.stdout:1/540: fsync d3/f50 0 2024-02-16T22:56:24.367 INFO:tasks.workunit.client.1.smithi192.stdout:2/615: write d22/d24/d39/d42/d65/d69/d4f/fae [1014680,42044] 0 2024-02-16T22:56:24.368 INFO:tasks.workunit.client.0.smithi064.stdout:2/551: creat d7/de/fab x:0 0 0 2024-02-16T22:56:24.368 INFO:tasks.workunit.client.0.smithi064.stdout:6/524: dwrite d0/d6/f15 [0,4194304] 0 2024-02-16T22:56:24.371 INFO:tasks.workunit.client.1.smithi192.stdout:4/642: dwrite d1/d26/d66/f9e [0,4194304] 0 2024-02-16T22:56:24.375 INFO:tasks.workunit.client.1.smithi192.stdout:5/544: read d6/d8/d86/f89 [2613053,10452] 0 2024-02-16T22:56:24.383 INFO:tasks.workunit.client.1.smithi192.stdout:1/541: fdatasync d3/de/d27/d34/d36/d47/f4b 0 2024-02-16T22:56:24.384 INFO:tasks.workunit.client.1.smithi192.stdout:9/702: write d7/d15/f27 [2394912,38637] 0 2024-02-16T22:56:24.385 INFO:tasks.workunit.client.1.smithi192.stdout:2/616: symlink d22/lc6 0 2024-02-16T22:56:24.385 INFO:tasks.workunit.client.0.smithi064.stdout:1/428: truncate d2/df/d14/d2c/d62/d91/f64 3667418 0 2024-02-16T22:56:24.387 INFO:tasks.workunit.client.1.smithi192.stdout:1/542: chown d3/de/d27/d34/d52/d72/ca3 115 1 2024-02-16T22:56:24.388 INFO:tasks.workunit.client.0.smithi064.stdout:1/429: write d2/df/d11/d1a/d1d/f24 [1792045,104679] 0 2024-02-16T22:56:24.388 INFO:tasks.workunit.client.1.smithi192.stdout:7/657: dwrite d1/d2/d37/f68 [0,4194304] 0 2024-02-16T22:56:24.390 INFO:tasks.workunit.client.1.smithi192.stdout:4/643: mknod d1/d26/d66/d89/da8/da9/ce2 0 2024-02-16T22:56:24.392 INFO:tasks.workunit.client.1.smithi192.stdout:5/545: write d6/d20/f4d [910004,76727] 0 2024-02-16T22:56:24.392 INFO:tasks.workunit.client.0.smithi064.stdout:7/454: link d5/d33/d53/c71 d5/d33/d45/d73/d88/c8e 0 2024-02-16T22:56:24.395 INFO:tasks.workunit.client.1.smithi192.stdout:5/546: chown d6/d8/d75 1 1 2024-02-16T22:56:24.397 INFO:tasks.workunit.client.1.smithi192.stdout:2/617: creat d22/fc7 x:0 0 0 2024-02-16T22:56:24.400 INFO:tasks.workunit.client.0.smithi064.stdout:4/498: creat dc/de/f99 x:0 0 0 2024-02-16T22:56:24.403 INFO:tasks.workunit.client.0.smithi064.stdout:4/499: truncate dc/dd/d32/d22/f93 817555 0 2024-02-16T22:56:24.408 INFO:tasks.workunit.client.0.smithi064.stdout:0/417: dwrite da/dc/de/d12/d1d/f2f [0,4194304] 0 2024-02-16T22:56:24.410 INFO:tasks.workunit.client.1.smithi192.stdout:9/703: fdatasync d7/d19/f94 0 2024-02-16T22:56:24.411 INFO:tasks.workunit.client.1.smithi192.stdout:9/704: fsync d7/d15/d36/d38/d3f/d8a/d98/f22 0 2024-02-16T22:56:24.412 INFO:tasks.workunit.client.0.smithi064.stdout:1/430: chown d2/df/l94 352 1 2024-02-16T22:56:24.413 INFO:tasks.workunit.client.1.smithi192.stdout:9/705: write d7/d19/d37/f8c [85901,22535] 0 2024-02-16T22:56:24.414 INFO:tasks.workunit.client.0.smithi064.stdout:9/448: sync 2024-02-16T22:56:24.414 INFO:tasks.workunit.client.1.smithi192.stdout:5/547: creat d6/d12/d4e/d58/fb7 x:0 0 0 2024-02-16T22:56:24.415 INFO:tasks.workunit.client.0.smithi064.stdout:9/449: truncate d3/d11/d15/f7f 119705 0 2024-02-16T22:56:24.417 INFO:tasks.workunit.client.0.smithi064.stdout:6/525: unlink d0/d6/c4d 0 2024-02-16T22:56:24.419 INFO:tasks.workunit.client.0.smithi064.stdout:5/391: getdents d6/d18/d39 0 2024-02-16T22:56:24.420 INFO:tasks.workunit.client.0.smithi064.stdout:5/392: write d6/d25/d48/f4a [2997118,796] 0 2024-02-16T22:56:24.421 INFO:tasks.workunit.client.1.smithi192.stdout:1/543: creat d3/de/d27/d34/d69/d85/fbd x:0 0 0 2024-02-16T22:56:24.422 INFO:tasks.workunit.client.0.smithi064.stdout:7/455: unlink d5/d33/f3d 0 2024-02-16T22:56:24.425 INFO:tasks.workunit.client.1.smithi192.stdout:7/658: unlink d1/d2/d2d/d92/d9b/lcb 0 2024-02-16T22:56:24.429 INFO:tasks.workunit.client.0.smithi064.stdout:2/552: rename d7/d14/d4b/d63/f9c to d7/de/d95/fac 0 2024-02-16T22:56:24.430 INFO:tasks.workunit.client.0.smithi064.stdout:2/553: fdatasync d7/d14/d3e/f5d 0 2024-02-16T22:56:24.430 INFO:tasks.workunit.client.1.smithi192.stdout:7/659: dwrite d1/d2/d2d/d61/d88/d90/d4c/f6b [4194304,4194304] 0 2024-02-16T22:56:24.431 INFO:tasks.workunit.client.0.smithi064.stdout:2/554: read d7/d14/d1e/d34/f61 [2273509,13369] 0 2024-02-16T22:56:24.431 INFO:tasks.workunit.client.0.smithi064.stdout:8/459: link df/d2a/f34 df/d31/d2f/d3e/f9c 0 2024-02-16T22:56:24.434 INFO:tasks.workunit.client.0.smithi064.stdout:9/450: chown d3/d9/f21 48 1 2024-02-16T22:56:24.436 INFO:tasks.workunit.client.1.smithi192.stdout:9/706: unlink d7/d15/l5f 0 2024-02-16T22:56:24.436 INFO:tasks.workunit.client.0.smithi064.stdout:6/526: mkdir d0/d6/d10/d33/da4 0 2024-02-16T22:56:24.438 INFO:tasks.workunit.client.1.smithi192.stdout:5/548: unlink d6/d12/c79 0 2024-02-16T22:56:24.439 INFO:tasks.workunit.client.1.smithi192.stdout:5/549: fdatasync d6/d8/d80/f40 0 2024-02-16T22:56:24.441 INFO:tasks.workunit.client.1.smithi192.stdout:2/618: truncate f4 2074076 0 2024-02-16T22:56:24.441 INFO:tasks.workunit.client.1.smithi192.stdout:8/553: sync 2024-02-16T22:56:24.442 INFO:tasks.workunit.client.1.smithi192.stdout:2/619: write fe [6383240,106305] 0 2024-02-16T22:56:24.443 INFO:tasks.workunit.client.0.smithi064.stdout:7/456: unlink d5/d6/f18 0 2024-02-16T22:56:24.445 INFO:tasks.workunit.client.1.smithi192.stdout:1/544: creat d3/de/d27/d34/d69/d85/d89/fbe x:0 0 0 2024-02-16T22:56:24.447 INFO:tasks.workunit.client.0.smithi064.stdout:3/567: getdents d4/d13/d1b 0 2024-02-16T22:56:24.451 INFO:tasks.workunit.client.1.smithi192.stdout:4/644: rmdir d1/d26/d6f/da3 0 2024-02-16T22:56:24.452 INFO:tasks.workunit.client.1.smithi192.stdout:7/660: creat d1/d2/de/d83/d9f/feb x:0 0 0 2024-02-16T22:56:24.457 INFO:tasks.workunit.client.1.smithi192.stdout:9/707: unlink d7/d19/d28/d43/d70/f8d 0 2024-02-16T22:56:24.461 INFO:tasks.workunit.client.0.smithi064.stdout:0/418: mknod da/dc/de/d12/d2c/d2e/c84 0 2024-02-16T22:56:24.463 INFO:tasks.workunit.client.0.smithi064.stdout:1/431: creat d2/df/d14/d18/d48/d84/f9a x:0 0 0 2024-02-16T22:56:24.465 INFO:tasks.workunit.client.0.smithi064.stdout:8/460: symlink df/d31/d2f/d3e/d5f/l9d 0 2024-02-16T22:56:24.466 INFO:tasks.workunit.client.0.smithi064.stdout:8/461: dread - df/d31/d2f/d38/f46 zero size 2024-02-16T22:56:24.471 INFO:tasks.workunit.client.1.smithi192.stdout:2/620: write d22/d24/d2b/d2c/f37 [106295,114213] 0 2024-02-16T22:56:24.472 INFO:tasks.workunit.client.0.smithi064.stdout:6/527: read d0/d5/d54/d60/f7c [308392,95173] 0 2024-02-16T22:56:24.475 INFO:tasks.workunit.client.0.smithi064.stdout:5/393: truncate d6/fc 2333772 0 2024-02-16T22:56:24.476 INFO:tasks.workunit.client.0.smithi064.stdout:5/394: truncate d6/d1c/d43/f58 798689 0 2024-02-16T22:56:24.477 INFO:tasks.workunit.client.0.smithi064.stdout:5/395: read - d6/d1c/f53 zero size 2024-02-16T22:56:24.478 INFO:tasks.workunit.client.1.smithi192.stdout:4/645: rename d1/d26/d66/d89/da8/da9/fce to d1/d26/d6f/fe3 0 2024-02-16T22:56:24.480 INFO:tasks.workunit.client.1.smithi192.stdout:7/661: mknod d1/d2/d2d/ddf/cec 0 2024-02-16T22:56:24.485 INFO:tasks.workunit.client.0.smithi064.stdout:7/457: dwrite d5/d6/d54/d41/f4b [0,4194304] 0 2024-02-16T22:56:24.486 INFO:tasks.workunit.client.0.smithi064.stdout:9/451: write d3/d11/d15/d5a/f85 [5777372,6493] 0 2024-02-16T22:56:24.488 INFO:tasks.workunit.client.0.smithi064.stdout:2/555: mkdir d7/d14/d1e/d34/dad 0 2024-02-16T22:56:24.489 INFO:tasks.workunit.client.0.smithi064.stdout:9/452: write d3/d11/d15/d23/f6e [82079,83019] 0 2024-02-16T22:56:24.492 INFO:tasks.workunit.client.0.smithi064.stdout:0/419: fdatasync f3 0 2024-02-16T22:56:24.493 INFO:tasks.workunit.client.0.smithi064.stdout:0/420: chown da/dc/de 78859791 1 2024-02-16T22:56:24.495 INFO:tasks.workunit.client.0.smithi064.stdout:1/432: rename d2/lb to d2/df/d14/d18/d48/d6b/d75/l9b 0 2024-02-16T22:56:24.499 INFO:tasks.workunit.client.1.smithi192.stdout:8/554: creat d5/db/d47/d43/d90/fa9 x:0 0 0 2024-02-16T22:56:24.501 INFO:tasks.workunit.client.1.smithi192.stdout:2/621: creat d22/d24/d39/d42/d85/d98/fc8 x:0 0 0 2024-02-16T22:56:24.502 INFO:tasks.workunit.client.1.smithi192.stdout:2/622: stat d22/d24/d39/d3d/la3 0 2024-02-16T22:56:24.504 INFO:tasks.workunit.client.0.smithi064.stdout:6/528: dread - d0/d1d/d21/f66 zero size 2024-02-16T22:56:24.506 INFO:tasks.workunit.client.0.smithi064.stdout:5/396: creat d6/d25/d48/d67/f87 x:0 0 0 2024-02-16T22:56:24.508 INFO:tasks.workunit.client.0.smithi064.stdout:7/458: creat d5/d33/d42/f8f x:0 0 0 2024-02-16T22:56:24.509 INFO:tasks.workunit.client.0.smithi064.stdout:7/459: dread - d5/d6/d54/f5e zero size 2024-02-16T22:56:24.512 INFO:tasks.workunit.client.1.smithi192.stdout:1/545: dwrite d3/d40/fa1 [0,4194304] 0 2024-02-16T22:56:24.512 INFO:tasks.workunit.client.0.smithi064.stdout:6/529: dwrite d0/d1d/f9e [0,4194304] 0 2024-02-16T22:56:24.513 INFO:tasks.workunit.client.0.smithi064.stdout:2/556: mknod d7/d14/d1e/cae 0 2024-02-16T22:56:24.516 INFO:tasks.workunit.client.0.smithi064.stdout:6/530: dread d0/d6/da/f6a [0,4194304] 0 2024-02-16T22:56:24.518 INFO:tasks.workunit.client.0.smithi064.stdout:0/421: symlink da/l85 0 2024-02-16T22:56:24.523 INFO:tasks.workunit.client.0.smithi064.stdout:5/397: creat d6/d1c/d43/d7d/f88 x:0 0 0 2024-02-16T22:56:24.537 INFO:tasks.workunit.client.0.smithi064.stdout:2/557: dread d7/d14/d4b/f88 [0,4194304] 0 2024-02-16T22:56:24.547 INFO:tasks.workunit.client.0.smithi064.stdout:9/453: write d3/f18 [3334030,31803] 0 2024-02-16T22:56:24.551 INFO:tasks.workunit.client.0.smithi064.stdout:2/558: rmdir d7/d14/d4b 39 2024-02-16T22:56:24.554 INFO:tasks.workunit.client.0.smithi064.stdout:8/462: getdents df/d31/d8f/d80 0 2024-02-16T22:56:24.554 INFO:tasks.workunit.client.0.smithi064.stdout:8/463: stat df/d31/d61/d79 0 2024-02-16T22:56:24.555 INFO:tasks.workunit.client.0.smithi064.stdout:5/398: dread d6/f37 [0,4194304] 0 2024-02-16T22:56:24.558 INFO:tasks.workunit.client.0.smithi064.stdout:1/433: truncate d2/df/d14/f2d 2645161 0 2024-02-16T22:56:24.560 INFO:tasks.workunit.client.0.smithi064.stdout:6/531: mkdir d0/d6/d10/d33/da5 0 2024-02-16T22:56:24.566 INFO:tasks.workunit.client.0.smithi064.stdout:2/559: dread d7/de/f15 [0,4194304] 0 2024-02-16T22:56:24.570 INFO:tasks.workunit.client.0.smithi064.stdout:5/399: rename d6/d18/d19/d1a/f30 to d6/d25/f89 0 2024-02-16T22:56:24.570 INFO:tasks.workunit.client.0.smithi064.stdout:7/460: dwrite d5/d33/d45/d64/f85 [0,4194304] 0 2024-02-16T22:56:24.572 INFO:tasks.workunit.client.0.smithi064.stdout:1/434: creat d2/df/d14/d18/d48/d84/f9c x:0 0 0 2024-02-16T22:56:24.572 INFO:tasks.workunit.client.0.smithi064.stdout:7/461: write d5/d6/d54/d41/f87 [295165,49387] 0 2024-02-16T22:56:24.574 INFO:tasks.workunit.client.0.smithi064.stdout:1/435: fsync d2/df/d14/d18/d40/f68 0 2024-02-16T22:56:24.574 INFO:tasks.workunit.client.1.smithi192.stdout:9/708: creat d7/d19/d37/d5a/ff5 x:0 0 0 2024-02-16T22:56:24.576 INFO:tasks.workunit.client.0.smithi064.stdout:6/532: mknod d0/d5/df/d17/d4a/ca6 0 2024-02-16T22:56:24.577 INFO:tasks.workunit.client.0.smithi064.stdout:1/436: fdatasync d2/df/d11/d1a/d30/f63 0 2024-02-16T22:56:24.578 INFO:tasks.workunit.client.1.smithi192.stdout:5/550: creat d6/d12/d1d/fb8 x:0 0 0 2024-02-16T22:56:24.579 INFO:tasks.workunit.client.0.smithi064.stdout:6/533: write d0/d5/df/d17/f62 [204050,113160] 0 2024-02-16T22:56:24.581 INFO:tasks.workunit.client.0.smithi064.stdout:2/560: mkdir d7/d14/d25/daf 0 2024-02-16T22:56:24.582 INFO:tasks.workunit.client.0.smithi064.stdout:5/400: dwrite d6/d18/d19/d1a/f3b [4194304,4194304] 0 2024-02-16T22:56:24.584 INFO:tasks.workunit.client.0.smithi064.stdout:2/561: write d7/de/d95/d44/f6d [102537,11298] 0 2024-02-16T22:56:24.584 INFO:tasks.workunit.client.0.smithi064.stdout:7/462: dread d5/d6/d54/d41/f87 [0,4194304] 0 2024-02-16T22:56:24.587 INFO:tasks.workunit.client.0.smithi064.stdout:9/454: dread d3/d9/f3c [0,4194304] 0 2024-02-16T22:56:24.587 INFO:tasks.workunit.client.0.smithi064.stdout:7/463: write d5/d6/d54/f5e [62779,38660] 0 2024-02-16T22:56:24.590 INFO:tasks.workunit.client.0.smithi064.stdout:2/562: write d7/d14/d1e/d34/d73/f77 [595798,104823] 0 2024-02-16T22:56:24.593 INFO:tasks.workunit.client.1.smithi192.stdout:1/546: rmdir d3/de/d27/d34/d52/d5a 39 2024-02-16T22:56:24.600 INFO:tasks.workunit.client.1.smithi192.stdout:1/547: dwrite d3/de/d27/d34/d52/fb4 [0,4194304] 0 2024-02-16T22:56:24.600 INFO:tasks.workunit.client.1.smithi192.stdout:8/555: write d5/fa7 [1043956,128421] 0 2024-02-16T22:56:24.601 INFO:tasks.workunit.client.0.smithi064.stdout:0/422: rename da/dc/de/f20 to da/dc/de/f86 0 2024-02-16T22:56:24.602 INFO:tasks.workunit.client.1.smithi192.stdout:2/623: mkdir d22/d24/d2b/d2c/dc9 0 2024-02-16T22:56:24.604 INFO:tasks.workunit.client.0.smithi064.stdout:5/401: creat d6/d18/f8a x:0 0 0 2024-02-16T22:56:24.612 INFO:tasks.workunit.client.0.smithi064.stdout:8/464: write df/d31/d2f/d38/d48/f54 [679672,37056] 0 2024-02-16T22:56:24.613 INFO:tasks.workunit.client.0.smithi064.stdout:2/563: write d7/d14/d25/f69 [629900,41074] 0 2024-02-16T22:56:24.615 INFO:tasks.workunit.client.0.smithi064.stdout:8/465: write df/d31/d2f/d38/d48/f54 [439366,21510] 0 2024-02-16T22:56:24.616 INFO:tasks.workunit.client.0.smithi064.stdout:8/466: chown df/d31/d2f/d4f/f8d 0 1 2024-02-16T22:56:24.618 INFO:tasks.workunit.client.0.smithi064.stdout:1/437: symlink d2/df/d11/d25/l9d 0 2024-02-16T22:56:24.619 INFO:tasks.workunit.client.1.smithi192.stdout:2/624: mkdir d22/d47/dca 0 2024-02-16T22:56:24.620 INFO:tasks.workunit.client.0.smithi064.stdout:0/423: mkdir da/dc/de/d12/d2c/d2e/d38/d87 0 2024-02-16T22:56:24.620 INFO:tasks.workunit.client.1.smithi192.stdout:4/646: dwrite d1/d26/d66/d89/f78 [0,4194304] 0 2024-02-16T22:56:24.620 INFO:tasks.workunit.client.1.smithi192.stdout:6/585: sync 2024-02-16T22:56:24.621 INFO:tasks.workunit.client.1.smithi192.stdout:7/662: getdents d1/d2/de/d2c/d49/d89/dbd 0 2024-02-16T22:56:24.623 INFO:tasks.workunit.client.0.smithi064.stdout:5/402: rmdir d6/d1c/d43 39 2024-02-16T22:56:24.626 INFO:tasks.workunit.client.1.smithi192.stdout:8/556: unlink d5/l22 0 2024-02-16T22:56:24.629 INFO:tasks.workunit.client.0.smithi064.stdout:6/534: link d0/d85/f92 d0/d12/d7b/fa7 0 2024-02-16T22:56:24.629 INFO:tasks.workunit.client.1.smithi192.stdout:9/709: dwrite d7/d15/f20 [0,4194304] 0 2024-02-16T22:56:24.629 INFO:tasks.workunit.client.1.smithi192.stdout:5/551: write d6/d12/f44 [3852363,21567] 0 2024-02-16T22:56:24.631 INFO:tasks.workunit.client.1.smithi192.stdout:5/552: write d6/d8/d77/f85 [1010596,51895] 0 2024-02-16T22:56:24.632 INFO:tasks.workunit.client.0.smithi064.stdout:9/455: rename d3/d9/f60 to d3/d11/d15/f94 0 2024-02-16T22:56:24.636 INFO:tasks.workunit.client.0.smithi064.stdout:4/500: sync 2024-02-16T22:56:24.636 INFO:tasks.workunit.client.0.smithi064.stdout:3/568: sync 2024-02-16T22:56:24.639 INFO:tasks.workunit.client.1.smithi192.stdout:2/625: rename d22/d24/d2b/d2c to d22/d47/dcb 0 2024-02-16T22:56:24.640 INFO:tasks.workunit.client.0.smithi064.stdout:7/464: creat d5/d33/d45/d73/d88/f90 x:0 0 0 2024-02-16T22:56:24.642 INFO:tasks.workunit.client.1.smithi192.stdout:1/548: write d3/d5b/f8a [897318,79242] 0 2024-02-16T22:56:24.643 INFO:tasks.workunit.client.1.smithi192.stdout:6/586: mknod da/df/d6b/d83/cc2 0 2024-02-16T22:56:24.644 INFO:tasks.workunit.client.0.smithi064.stdout:8/467: mknod df/d31/d2f/d4f/c9e 0 2024-02-16T22:56:24.646 INFO:tasks.workunit.client.0.smithi064.stdout:8/468: write df/d31/d61/d79/d81/f82 [1493958,109501] 0 2024-02-16T22:56:24.646 INFO:tasks.workunit.client.1.smithi192.stdout:7/663: creat d1/d2/de/d2c/fed x:0 0 0 2024-02-16T22:56:24.649 INFO:tasks.workunit.client.1.smithi192.stdout:1/549: dwrite d3/de/d27/f6a [0,4194304] 0 2024-02-16T22:56:24.652 INFO:tasks.workunit.client.0.smithi064.stdout:1/438: read d2/df/d14/d18/f26 [186685,59752] 0 2024-02-16T22:56:24.654 INFO:tasks.workunit.client.1.smithi192.stdout:9/710: stat d7/d19/d37/f55 0 2024-02-16T22:56:24.655 INFO:tasks.workunit.client.0.smithi064.stdout:1/439: truncate d2/df/d11/d1a/f34 1766452 0 2024-02-16T22:56:24.656 INFO:tasks.workunit.client.0.smithi064.stdout:0/424: dwrite da/f10 [8388608,4194304] 0 2024-02-16T22:56:24.658 INFO:tasks.workunit.client.0.smithi064.stdout:6/535: rename d0/d5/d54 to d0/d1d/da8 0 2024-02-16T22:56:24.660 INFO:tasks.workunit.client.1.smithi192.stdout:4/647: symlink d1/d26/d34/dc7/le4 0 2024-02-16T22:56:24.660 INFO:tasks.workunit.client.0.smithi064.stdout:9/456: mkdir d3/d11/d15/d5a/d2a/d2e/d31/d95 0 2024-02-16T22:56:24.662 INFO:tasks.workunit.client.0.smithi064.stdout:9/457: readlink d3/d11/d15/d5a/d2a/d2e/d31/d62/l75 0 2024-02-16T22:56:24.662 INFO:tasks.workunit.client.0.smithi064.stdout:3/569: symlink d4/d13/d1b/d48/lc6 0 2024-02-16T22:56:24.664 INFO:tasks.workunit.client.1.smithi192.stdout:6/587: symlink da/d76/lc3 0 2024-02-16T22:56:24.665 INFO:tasks.workunit.client.0.smithi064.stdout:2/564: mknod d7/d14/d1e/d34/d3f/d9a/cb0 0 2024-02-16T22:56:24.668 INFO:tasks.workunit.client.1.smithi192.stdout:4/648: dwrite d1/d26/d66/d89/d11/f98 [0,4194304] 0 2024-02-16T22:56:24.671 INFO:tasks.workunit.client.0.smithi064.stdout:2/565: dread d7/de/f8d [0,4194304] 0 2024-02-16T22:56:24.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/557: unlink d5/db/f64 0 2024-02-16T22:56:24.674 INFO:tasks.workunit.client.0.smithi064.stdout:2/566: truncate d7/d14/d25/d46/f97 70893 0 2024-02-16T22:56:24.676 INFO:tasks.workunit.client.0.smithi064.stdout:8/469: dread df/d31/d2f/d3e/f9c [0,4194304] 0 2024-02-16T22:56:24.678 INFO:tasks.workunit.client.0.smithi064.stdout:5/403: creat d6/d18/d19/d1a/d85/f8b x:0 0 0 2024-02-16T22:56:24.680 INFO:tasks.workunit.client.0.smithi064.stdout:0/425: mkdir da/dc/de/d37/d88 0 2024-02-16T22:56:24.681 INFO:tasks.workunit.client.1.smithi192.stdout:2/626: dread d22/d24/d39/d42/d65/d69/f68 [0,4194304] 0 2024-02-16T22:56:24.682 INFO:tasks.workunit.client.1.smithi192.stdout:2/627: readlink d22/d47/dcb/l62 0 2024-02-16T22:56:24.687 INFO:tasks.workunit.client.1.smithi192.stdout:8/558: chown d5/d9/f5a 76461230 1 2024-02-16T22:56:24.694 INFO:tasks.workunit.client.1.smithi192.stdout:5/553: rename d6/d12/d1d/d56/fb6 to d6/d8/d80/fb9 0 2024-02-16T22:56:24.698 INFO:tasks.workunit.client.1.smithi192.stdout:7/664: mknod d1/d2/d2d/d61/d8a/cee 0 2024-02-16T22:56:24.700 INFO:tasks.workunit.client.1.smithi192.stdout:7/665: read d1/d2/d2d/d61/d8a/fde [1865104,107152] 0 2024-02-16T22:56:24.700 INFO:tasks.workunit.client.1.smithi192.stdout:6/588: write da/df/d12/d15/f2d [39376,68202] 0 2024-02-16T22:56:24.701 INFO:tasks.workunit.client.0.smithi064.stdout:1/440: symlink d2/df/d14/d2c/d62/d91/d56/l9e 0 2024-02-16T22:56:24.701 INFO:tasks.workunit.client.1.smithi192.stdout:7/666: write d1/f11 [1470474,4016] 0 2024-02-16T22:56:24.701 INFO:tasks.workunit.client.1.smithi192.stdout:6/589: stat da/df/d12/d15/l67 0 2024-02-16T22:56:24.702 INFO:tasks.workunit.client.0.smithi064.stdout:2/567: write d7/f54 [364039,55496] 0 2024-02-16T22:56:24.703 INFO:tasks.workunit.client.0.smithi064.stdout:2/568: dread - d7/de/d95/fac zero size 2024-02-16T22:56:24.703 INFO:tasks.workunit.client.1.smithi192.stdout:6/590: chown da/df/d58/f9a 6609 1 2024-02-16T22:56:24.705 INFO:tasks.workunit.client.0.smithi064.stdout:7/465: rename d5/d6/f34 to d5/d33/d45/d64/f91 0 2024-02-16T22:56:24.706 INFO:tasks.workunit.client.0.smithi064.stdout:7/466: write d5/d6/fb [4284874,37208] 0 2024-02-16T22:56:24.707 INFO:tasks.workunit.client.1.smithi192.stdout:1/550: dwrite d3/de/d27/f42 [0,4194304] 0 2024-02-16T22:56:24.707 INFO:tasks.workunit.client.0.smithi064.stdout:7/467: fdatasync d5/d33/d45/f52 0 2024-02-16T22:56:24.711 INFO:tasks.workunit.client.1.smithi192.stdout:7/667: dwrite d1/d2/de/d2c/d49/f71 [0,4194304] 0 2024-02-16T22:56:24.716 INFO:tasks.workunit.client.1.smithi192.stdout:5/554: write d6/d12/d4e/d46/d4f/f6e [1516563,28303] 0 2024-02-16T22:56:24.717 INFO:tasks.workunit.client.1.smithi192.stdout:5/555: read d6/f2b [3960942,121183] 0 2024-02-16T22:56:24.718 INFO:tasks.workunit.client.1.smithi192.stdout:8/559: creat d5/db/faa x:0 0 0 2024-02-16T22:56:24.720 INFO:tasks.workunit.client.0.smithi064.stdout:0/426: mkdir da/dc/de/d12/d2c/d2e/d38/d7e/d89 0 2024-02-16T22:56:24.725 INFO:tasks.workunit.client.0.smithi064.stdout:6/536: rename d0/d5/df/d17/d18/d5f to d0/d3c/da9 0 2024-02-16T22:56:24.727 INFO:tasks.workunit.client.1.smithi192.stdout:6/591: symlink da/df/d6b/d7d/lc4 0 2024-02-16T22:56:24.727 INFO:tasks.workunit.client.0.smithi064.stdout:8/470: dwrite df/d2a/f70 [0,4194304] 0 2024-02-16T22:56:24.729 INFO:tasks.workunit.client.0.smithi064.stdout:3/570: rmdir d4/d1f/d5b/d51/dc1/da1 0 2024-02-16T22:56:24.730 INFO:tasks.workunit.client.0.smithi064.stdout:7/468: symlink d5/d33/d45/d64/d56/d5d/l92 0 2024-02-16T22:56:24.730 INFO:tasks.workunit.client.0.smithi064.stdout:7/469: chown f0 503233 1 2024-02-16T22:56:24.732 INFO:tasks.workunit.client.1.smithi192.stdout:8/560: fsync d5/d26/d8f/d4d/f4b 0 2024-02-16T22:56:24.733 INFO:tasks.workunit.client.0.smithi064.stdout:9/458: link d3/d9/d1e/d3e/d76/c78 d3/d11/d15/d5a/d2a/d2e/d31/d95/c96 0 2024-02-16T22:56:24.734 INFO:tasks.workunit.client.0.smithi064.stdout:9/459: chown d3/d11/d15/d23/c3a 7391312 1 2024-02-16T22:56:24.735 INFO:tasks.workunit.client.1.smithi192.stdout:5/556: unlink d6/d8/d80/fb9 0 2024-02-16T22:56:24.741 INFO:tasks.workunit.client.1.smithi192.stdout:7/668: creat d1/de1/dd0/fef x:0 0 0 2024-02-16T22:56:24.742 INFO:tasks.workunit.client.0.smithi064.stdout:0/427: symlink da/dc/de/d12/d2c/d2e/d38/l8a 0 2024-02-16T22:56:24.743 INFO:tasks.workunit.client.1.smithi192.stdout:6/592: symlink da/df/d6b/d74/lc5 0 2024-02-16T22:56:24.743 INFO:tasks.workunit.client.1.smithi192.stdout:6/593: stat da/df/f84 0 2024-02-16T22:56:24.743 INFO:tasks.workunit.client.0.smithi064.stdout:1/441: mknod d2/c9f 0 2024-02-16T22:56:24.743 INFO:tasks.workunit.client.0.smithi064.stdout:9/460: dwrite d3/d11/d15/d5a/d2a/d2e/d31/f84 [0,4194304] 0 2024-02-16T22:56:24.744 INFO:tasks.workunit.client.0.smithi064.stdout:0/428: fsync da/dc/de/d37/f4e 0 2024-02-16T22:56:24.747 INFO:tasks.workunit.client.1.smithi192.stdout:3/566: sync 2024-02-16T22:56:24.747 INFO:tasks.workunit.client.1.smithi192.stdout:0/576: sync 2024-02-16T22:56:24.748 INFO:tasks.workunit.client.0.smithi064.stdout:6/537: dread - d0/d5/df/f63 zero size 2024-02-16T22:56:24.749 INFO:tasks.workunit.client.0.smithi064.stdout:6/538: read d0/d1b/f7a [413097,43219] 0 2024-02-16T22:56:24.752 INFO:tasks.workunit.client.0.smithi064.stdout:3/571: unlink d4/d13/d1b/c91 0 2024-02-16T22:56:24.752 INFO:tasks.workunit.client.1.smithi192.stdout:1/551: write d3/de/d27/d34/f45 [2055800,106887] 0 2024-02-16T22:56:24.755 INFO:tasks.workunit.client.1.smithi192.stdout:7/669: creat d1/d2/de/d83/d9f/ff0 x:0 0 0 2024-02-16T22:56:24.756 INFO:tasks.workunit.client.0.smithi064.stdout:5/404: link d6/d25/d48/d67/c71 d6/d1c/c8c 0 2024-02-16T22:56:24.759 INFO:tasks.workunit.client.1.smithi192.stdout:6/594: creat da/df/d12/d57/fc6 x:0 0 0 2024-02-16T22:56:24.761 INFO:tasks.workunit.client.1.smithi192.stdout:8/561: rename d5/f1e to d5/db/d47/d8c/fab 0 2024-02-16T22:56:24.763 INFO:tasks.workunit.client.1.smithi192.stdout:5/557: write d6/d12/d4e/f2e [424018,102987] 0 2024-02-16T22:56:24.767 INFO:tasks.workunit.client.1.smithi192.stdout:6/595: dread da/df/d12/d15/f5c [0,4194304] 0 2024-02-16T22:56:24.769 INFO:tasks.workunit.client.0.smithi064.stdout:1/442: dread d2/df/d11/d25/f80 [0,4194304] 0 2024-02-16T22:56:24.771 INFO:tasks.workunit.client.1.smithi192.stdout:0/577: write d1/d3/de/d22/d37/d47/f48 [3849578,84866] 0 2024-02-16T22:56:24.776 INFO:tasks.workunit.client.0.smithi064.stdout:7/470: creat d5/d7d/f93 x:0 0 0 2024-02-16T22:56:24.776 INFO:tasks.workunit.client.1.smithi192.stdout:7/670: dwrite d1/d2/de/d2c/d49/da4/f85 [0,4194304] 0 2024-02-16T22:56:24.777 INFO:tasks.workunit.client.1.smithi192.stdout:5/558: fsync d6/d12/d4e/d46/d4f/f62 0 2024-02-16T22:56:24.777 INFO:tasks.workunit.client.1.smithi192.stdout:5/559: chown d6/d20 77826747 1 2024-02-16T22:56:24.778 INFO:tasks.workunit.client.1.smithi192.stdout:7/671: chown d1/d2/d2d/d61/d88/d90/db2 121634 1 2024-02-16T22:56:24.779 INFO:tasks.workunit.client.1.smithi192.stdout:5/560: fdatasync d6/d12/d4e/d46/f5d 0 2024-02-16T22:56:24.779 INFO:tasks.workunit.client.0.smithi064.stdout:9/461: mknod d3/d9/c97 0 2024-02-16T22:56:24.779 INFO:tasks.workunit.client.1.smithi192.stdout:7/672: stat d1/d2/d2d/d61/d88/d90/d4c/f6b 0 2024-02-16T22:56:24.780 INFO:tasks.workunit.client.1.smithi192.stdout:7/673: fsync d1/d2/de/d2c/d49/f80 0 2024-02-16T22:56:24.781 INFO:tasks.workunit.client.0.smithi064.stdout:9/462: readlink d3/d11/d15/d23/l2d 0 2024-02-16T22:56:24.782 INFO:tasks.workunit.client.1.smithi192.stdout:5/561: read d6/d12/d4e/d46/f5d [1116168,36441] 0 2024-02-16T22:56:24.782 INFO:tasks.workunit.client.0.smithi064.stdout:2/569: rename d7/d14/d25/d46/d55/f7e to d7/fb1 0 2024-02-16T22:56:24.783 INFO:tasks.workunit.client.1.smithi192.stdout:7/674: write d1/d2/de/d2c/d49/da1/fc8 [182055,90212] 0 2024-02-16T22:56:24.783 INFO:tasks.workunit.client.1.smithi192.stdout:5/562: write d6/d8/fa3 [450248,114688] 0 2024-02-16T22:56:24.786 INFO:tasks.workunit.client.0.smithi064.stdout:1/443: creat d2/df/d14/d18/d40/fa0 x:0 0 0 2024-02-16T22:56:24.790 INFO:tasks.workunit.client.0.smithi064.stdout:7/471: creat d5/d33/d42/d81/f94 x:0 0 0 2024-02-16T22:56:24.791 INFO:tasks.workunit.client.0.smithi064.stdout:7/472: dread - d5/d33/d53/f70 zero size 2024-02-16T22:56:24.792 INFO:tasks.workunit.client.0.smithi064.stdout:5/405: fsync d6/f24 0 2024-02-16T22:56:24.795 INFO:tasks.workunit.client.1.smithi192.stdout:5/563: creat d6/d20/d93/fba x:0 0 0 2024-02-16T22:56:24.796 INFO:tasks.workunit.client.0.smithi064.stdout:9/463: dread d3/d11/d15/d5a/f85 [4194304,4194304] 0 2024-02-16T22:56:24.797 INFO:tasks.workunit.client.0.smithi064.stdout:7/473: dread d5/d6/fb [0,4194304] 0 2024-02-16T22:56:24.800 INFO:tasks.workunit.client.0.smithi064.stdout:2/570: mknod d7/d14/d3e/cb2 0 2024-02-16T22:56:24.801 INFO:tasks.workunit.client.0.smithi064.stdout:6/539: creat d0/d6/d10/d33/da5/faa x:0 0 0 2024-02-16T22:56:24.803 INFO:tasks.workunit.client.0.smithi064.stdout:6/540: dread - d0/d1d/d21/f66 zero size 2024-02-16T22:56:24.803 INFO:tasks.workunit.client.1.smithi192.stdout:3/567: rename da/df/c44 to da/d11/d63/d6b/d66/cc9 0 2024-02-16T22:56:24.804 INFO:tasks.workunit.client.1.smithi192.stdout:3/568: readlink da/d11/lc1 0 2024-02-16T22:56:24.805 INFO:tasks.workunit.client.0.smithi064.stdout:1/444: symlink d2/df/d14/d18/d48/la1 0 2024-02-16T22:56:24.808 INFO:tasks.workunit.client.0.smithi064.stdout:3/572: rename d4/fb9 to d4/d13/d1b/d27/d44/fc7 0 2024-02-16T22:56:24.808 INFO:tasks.workunit.client.0.smithi064.stdout:1/445: write d2/df/d11/d1a/f34 [204449,36798] 0 2024-02-16T22:56:24.811 INFO:tasks.workunit.client.1.smithi192.stdout:7/675: symlink d1/d2/d2d/d92/d9b/lf1 0 2024-02-16T22:56:24.811 INFO:tasks.workunit.client.0.smithi064.stdout:6/541: dwrite d0/d1d/f9e [0,4194304] 0 2024-02-16T22:56:24.812 INFO:tasks.workunit.client.1.smithi192.stdout:8/562: truncate d5/f25 1572487 0 2024-02-16T22:56:24.812 INFO:tasks.workunit.client.1.smithi192.stdout:7/676: write d1/d2/d37/f68 [4709066,101584] 0 2024-02-16T22:56:24.817 INFO:tasks.workunit.client.1.smithi192.stdout:6/596: rename da/df/d12/d15/d1b/c65 to da/df/d71/dab/cc7 0 2024-02-16T22:56:24.817 INFO:tasks.workunit.client.0.smithi064.stdout:0/429: link da/dc/c4f da/d26/c8b 0 2024-02-16T22:56:24.818 INFO:tasks.workunit.client.1.smithi192.stdout:6/597: stat da/df/d12/d15/d1b/cb3 0 2024-02-16T22:56:24.818 INFO:tasks.workunit.client.1.smithi192.stdout:6/598: dread - da/df/d6b/d74/f8b zero size 2024-02-16T22:56:24.819 INFO:tasks.workunit.client.1.smithi192.stdout:3/569: mkdir da/d11/d20/d39/dca 0 2024-02-16T22:56:24.819 INFO:tasks.workunit.client.0.smithi064.stdout:3/573: dread d4/d13/d1b/d27/d44/f4b [0,4194304] 0 2024-02-16T22:56:24.820 INFO:tasks.workunit.client.0.smithi064.stdout:5/406: dread d6/d18/d19/f47 [0,4194304] 0 2024-02-16T22:56:24.820 INFO:tasks.workunit.client.1.smithi192.stdout:6/599: write da/df/d12/f51 [1273088,5566] 0 2024-02-16T22:56:24.821 INFO:tasks.workunit.client.1.smithi192.stdout:0/578: getdents d1/d3/de/d22 0 2024-02-16T22:56:24.822 INFO:tasks.workunit.client.1.smithi192.stdout:6/600: chown da/df/d12/d57/f2c 2 1 2024-02-16T22:56:24.823 INFO:tasks.workunit.client.1.smithi192.stdout:6/601: readlink da/df/d12/d15/l67 0 2024-02-16T22:56:24.825 INFO:tasks.workunit.client.0.smithi064.stdout:7/474: rmdir d5/d33 39 2024-02-16T22:56:24.828 INFO:tasks.workunit.client.1.smithi192.stdout:8/563: fdatasync d5/fa7 0 2024-02-16T22:56:24.829 INFO:tasks.workunit.client.1.smithi192.stdout:8/564: write d5/db/faa [428079,43194] 0 2024-02-16T22:56:24.836 INFO:tasks.workunit.client.1.smithi192.stdout:9/711: sync 2024-02-16T22:56:24.836 INFO:tasks.workunit.client.1.smithi192.stdout:9/712: chown d7/d15/d36/d77/db1 7267287 1 2024-02-16T22:56:24.837 INFO:tasks.workunit.client.1.smithi192.stdout:9/713: write d7/d19/d28/fb9 [479549,118976] 0 2024-02-16T22:56:24.841 INFO:tasks.workunit.client.1.smithi192.stdout:0/579: creat d1/d3/de/d22/da7/fb5 x:0 0 0 2024-02-16T22:56:24.843 INFO:tasks.workunit.client.1.smithi192.stdout:5/564: truncate d6/f6f 585975 0 2024-02-16T22:56:24.849 INFO:tasks.workunit.client.1.smithi192.stdout:3/570: write da/d11/d63/d6b/d71/f77 [492700,101854] 0 2024-02-16T22:56:24.851 INFO:tasks.workunit.client.0.smithi064.stdout:1/446: dread d2/df/d14/d2c/d62/d91/f5a [0,4194304] 0 2024-02-16T22:56:24.853 INFO:tasks.workunit.client.0.smithi064.stdout:7/475: fsync d5/d33/d42/f8f 0 2024-02-16T22:56:24.857 INFO:tasks.workunit.client.0.smithi064.stdout:7/476: write d5/d6/d54/f63 [2580599,17773] 0 2024-02-16T22:56:24.857 INFO:tasks.workunit.client.0.smithi064.stdout:3/574: dwrite d4/d13/d1b/d4f/f6f [0,4194304] 0 2024-02-16T22:56:24.862 INFO:tasks.workunit.client.0.smithi064.stdout:1/447: dread d2/df/d14/f96 [0,4194304] 0 2024-02-16T22:56:24.862 INFO:tasks.workunit.client.0.smithi064.stdout:7/477: dread d5/d6/f30 [8388608,4194304] 0 2024-02-16T22:56:24.863 INFO:tasks.workunit.client.0.smithi064.stdout:7/478: stat d5/d6/l69 0 2024-02-16T22:56:24.868 INFO:tasks.workunit.client.0.smithi064.stdout:7/479: dwrite d5/f10 [0,4194304] 0 2024-02-16T22:56:24.918 INFO:tasks.workunit.client.1.smithi192.stdout:7/677: write d1/d2/d2d/f40 [3409435,51775] 0 2024-02-16T22:56:24.919 INFO:tasks.workunit.client.1.smithi192.stdout:7/678: readlink d1/d2/d2d/d61/d88/d90/db2/ld5 0 2024-02-16T22:56:24.920 INFO:tasks.workunit.client.1.smithi192.stdout:6/602: write da/d40/f4c [2508520,80612] 0 2024-02-16T22:56:24.920 INFO:tasks.workunit.client.0.smithi064.stdout:0/430: mknod da/dc/d3b/d72/d7c/c8c 0 2024-02-16T22:56:24.926 INFO:tasks.workunit.client.1.smithi192.stdout:6/603: dwrite da/df/d18/fc0 [0,4194304] 0 2024-02-16T22:56:24.936 INFO:tasks.workunit.client.1.smithi192.stdout:4/649: sync 2024-02-16T22:56:24.936 INFO:tasks.workunit.client.1.smithi192.stdout:2/628: sync 2024-02-16T22:56:24.936 INFO:tasks.workunit.client.1.smithi192.stdout:1/552: sync 2024-02-16T22:56:24.938 INFO:tasks.workunit.client.1.smithi192.stdout:9/714: mkdir d7/d15/d36/d38/d3f/d8a/def/df6 0 2024-02-16T22:56:24.938 INFO:tasks.workunit.client.1.smithi192.stdout:9/715: stat d7/d19/d37/d5a 0 2024-02-16T22:56:24.940 INFO:tasks.workunit.client.1.smithi192.stdout:0/580: symlink d1/lb6 0 2024-02-16T22:56:24.941 INFO:tasks.workunit.client.1.smithi192.stdout:0/581: chown d1/d3/d39/f79 689017 1 2024-02-16T22:56:24.942 INFO:tasks.workunit.client.0.smithi064.stdout:5/407: truncate d6/d18/d19/f47 2255389 0 2024-02-16T22:56:24.943 INFO:tasks.workunit.client.1.smithi192.stdout:8/565: dwrite d5/db/d47/f3b [0,4194304] 0 2024-02-16T22:56:24.944 INFO:tasks.workunit.client.1.smithi192.stdout:8/566: write d5/db/de/f80 [1239159,69073] 0 2024-02-16T22:56:24.945 INFO:tasks.workunit.client.1.smithi192.stdout:5/565: link d6/d8/d75/f7b d6/d12/d4e/d58/fbb 0 2024-02-16T22:56:24.945 INFO:tasks.workunit.client.1.smithi192.stdout:5/566: chown d6/d8/d77/la6 3732 1 2024-02-16T22:56:24.946 INFO:tasks.workunit.client.1.smithi192.stdout:5/567: fdatasync d6/d12/d1d/fb0 0 2024-02-16T22:56:24.947 INFO:tasks.workunit.client.0.smithi064.stdout:1/448: rmdir d2/df/d11/d1a/d30 39 2024-02-16T22:56:24.948 INFO:tasks.workunit.client.1.smithi192.stdout:7/679: stat d1/c98 0 2024-02-16T22:56:24.948 INFO:tasks.workunit.client.1.smithi192.stdout:7/680: dread - d1/fc1 zero size 2024-02-16T22:56:24.949 INFO:tasks.workunit.client.0.smithi064.stdout:1/449: readlink d2/df/d14/d2c/d62/d91/d56/l6f 0 2024-02-16T22:56:24.951 INFO:tasks.workunit.client.0.smithi064.stdout:1/450: write d2/df/d14/f16 [5868633,121382] 0 2024-02-16T22:56:24.954 INFO:tasks.workunit.client.0.smithi064.stdout:7/480: rename d5/d33/d45/d64/c59 to d5/d33/d42/d81/c95 0 2024-02-16T22:56:24.957 INFO:tasks.workunit.client.0.smithi064.stdout:6/542: creat d0/d5/fab x:0 0 0 2024-02-16T22:56:24.958 INFO:tasks.workunit.client.1.smithi192.stdout:6/604: mkdir da/d76/dc8 0 2024-02-16T22:56:24.963 INFO:tasks.workunit.client.0.smithi064.stdout:3/575: write d4/d1f/d5b/d51/d7c/f49 [4990686,95541] 0 2024-02-16T22:56:24.963 INFO:tasks.workunit.client.0.smithi064.stdout:2/571: getdents d7/d14/d25/d46/d55/d57 0 2024-02-16T22:56:24.965 INFO:tasks.workunit.client.1.smithi192.stdout:3/571: sync 2024-02-16T22:56:24.966 INFO:tasks.workunit.client.1.smithi192.stdout:2/629: mkdir d22/d24/d39/d42/d85/d98/dcc 0 2024-02-16T22:56:24.967 INFO:tasks.workunit.client.1.smithi192.stdout:2/630: chown d22/d24/d39/d42/d52/d7c/cab 55493787 1 2024-02-16T22:56:24.967 INFO:tasks.workunit.client.1.smithi192.stdout:4/650: dread d1/d26/d66/d89/f68 [0,4194304] 0 2024-02-16T22:56:24.969 INFO:tasks.workunit.client.1.smithi192.stdout:2/631: write d22/d24/d39/d42/f5f [1462052,53843] 0 2024-02-16T22:56:24.969 INFO:tasks.workunit.client.1.smithi192.stdout:2/632: chown c11 3 1 2024-02-16T22:56:24.970 INFO:tasks.workunit.client.1.smithi192.stdout:2/633: write d22/d24/d2b/f38 [2610744,108571] 0 2024-02-16T22:56:24.971 INFO:tasks.workunit.client.1.smithi192.stdout:0/582: truncate d1/d3/d29/f73 1023836 0 2024-02-16T22:56:24.973 INFO:tasks.workunit.client.0.smithi064.stdout:7/481: mknod d5/d6/d54/d7b/c96 0 2024-02-16T22:56:24.974 INFO:tasks.workunit.client.1.smithi192.stdout:8/567: chown d5/f84 6222 1 2024-02-16T22:56:24.975 INFO:tasks.workunit.client.0.smithi064.stdout:6/543: rmdir d0/d1d 39 2024-02-16T22:56:24.978 INFO:tasks.workunit.client.0.smithi064.stdout:0/431: creat da/d71/f8d x:0 0 0 2024-02-16T22:56:24.979 INFO:tasks.workunit.client.1.smithi192.stdout:7/681: unlink d1/d2/d37/l42 0 2024-02-16T22:56:24.983 INFO:tasks.workunit.client.1.smithi192.stdout:6/605: mknod da/df/d6b/d74/cc9 0 2024-02-16T22:56:24.984 INFO:tasks.workunit.client.1.smithi192.stdout:6/606: truncate da/df/d18/f63 4887138 0 2024-02-16T22:56:24.984 INFO:tasks.workunit.client.1.smithi192.stdout:6/607: chown da 284087 1 2024-02-16T22:56:24.986 INFO:tasks.workunit.client.1.smithi192.stdout:1/553: mkdir d3/de/d27/d34/dbf 0 2024-02-16T22:56:24.988 INFO:tasks.workunit.client.1.smithi192.stdout:1/554: write d3/d5b/f84 [461739,41960] 0 2024-02-16T22:56:24.988 INFO:tasks.workunit.client.0.smithi064.stdout:2/572: rmdir d7/d14/d1e 39 2024-02-16T22:56:24.989 INFO:tasks.workunit.client.0.smithi064.stdout:5/408: mknod d6/d1c/c8d 0 2024-02-16T22:56:24.989 INFO:tasks.workunit.client.1.smithi192.stdout:3/572: mknod da/d11/d20/d39/ccb 0 2024-02-16T22:56:24.990 INFO:tasks.workunit.client.0.smithi064.stdout:5/409: dread - d6/d18/d19/d1a/d85/f8b zero size 2024-02-16T22:56:24.993 INFO:tasks.workunit.client.0.smithi064.stdout:7/482: creat d5/d33/d45/d64/d56/d5d/f97 x:0 0 0 2024-02-16T22:56:24.994 INFO:tasks.workunit.client.0.smithi064.stdout:6/544: write d0/f91 [1067446,90757] 0 2024-02-16T22:56:25.000 INFO:tasks.workunit.client.0.smithi064.stdout:3/576: truncate d4/d13/d35/fac 1432039 0 2024-02-16T22:56:25.001 INFO:tasks.workunit.client.0.smithi064.stdout:3/577: dread - d4/d13/d35/f3e zero size 2024-02-16T22:56:25.002 INFO:tasks.workunit.client.1.smithi192.stdout:2/634: mkdir d22/d24/d39/d42/d85/d98/db0/dcd 0 2024-02-16T22:56:25.003 INFO:tasks.workunit.client.1.smithi192.stdout:8/568: creat d5/d26/d8f/d4d/fac x:0 0 0 2024-02-16T22:56:25.003 INFO:tasks.workunit.client.0.smithi064.stdout:6/545: dwrite d0/d5/df/f8d [0,4194304] 0 2024-02-16T22:56:25.003 INFO:tasks.workunit.client.0.smithi064.stdout:7/483: dread d5/d6/f19 [0,4194304] 0 2024-02-16T22:56:25.005 INFO:tasks.workunit.client.0.smithi064.stdout:6/546: chown d0/c5d 316 1 2024-02-16T22:56:25.006 INFO:tasks.workunit.client.1.smithi192.stdout:5/568: mkdir d6/d12/d4e/d5b/d67/d78/dae/dbc 0 2024-02-16T22:56:25.007 INFO:tasks.workunit.client.1.smithi192.stdout:5/569: dread - d6/d12/d4e/d46/d4f/fb3 zero size 2024-02-16T22:56:25.007 INFO:tasks.workunit.client.1.smithi192.stdout:5/570: dread - d6/f9d zero size 2024-02-16T22:56:25.008 INFO:tasks.workunit.client.1.smithi192.stdout:5/571: dread - d6/d12/d4e/faa zero size 2024-02-16T22:56:25.009 INFO:tasks.workunit.client.1.smithi192.stdout:5/572: chown d6/d20/d93/fba 665393773 1 2024-02-16T22:56:25.009 INFO:tasks.workunit.client.1.smithi192.stdout:7/682: rmdir d1/d2/de/d2c 39 2024-02-16T22:56:25.010 INFO:tasks.workunit.client.1.smithi192.stdout:7/683: chown d1/d2/d2d/d92/la0 31331929 1 2024-02-16T22:56:25.011 INFO:tasks.workunit.client.1.smithi192.stdout:5/573: truncate d6/d12/d4e/faa 892821 0 2024-02-16T22:56:25.011 INFO:tasks.workunit.client.1.smithi192.stdout:6/608: symlink da/df/d12/d57/d81/lca 0 2024-02-16T22:56:25.014 INFO:tasks.workunit.client.1.smithi192.stdout:1/555: creat d3/d5b/fc0 x:0 0 0 2024-02-16T22:56:25.016 INFO:tasks.workunit.client.0.smithi064.stdout:0/432: dread da/fb [0,4194304] 0 2024-02-16T22:56:25.025 INFO:tasks.workunit.client.1.smithi192.stdout:8/569: rename d5/d26/d8f/d4d/da1 to d5/d26/dad 0 2024-02-16T22:56:25.025 INFO:tasks.workunit.client.1.smithi192.stdout:0/583: write d1/d3/d29/f1a [824993,64104] 0 2024-02-16T22:56:25.027 INFO:tasks.workunit.client.1.smithi192.stdout:2/635: dwrite d22/f4a [4194304,4194304] 0 2024-02-16T22:56:25.031 INFO:tasks.workunit.client.0.smithi064.stdout:2/573: mknod d7/d14/d3e/cb3 0 2024-02-16T22:56:25.033 INFO:tasks.workunit.client.0.smithi064.stdout:5/410: mkdir d6/d18/d19/d1a/d8e 0 2024-02-16T22:56:25.035 INFO:tasks.workunit.client.0.smithi064.stdout:1/451: link d2/df/f54 d2/df/fa2 0 2024-02-16T22:56:25.036 INFO:tasks.workunit.client.1.smithi192.stdout:9/716: getdents d7/d19/d37/d39 0 2024-02-16T22:56:25.038 INFO:tasks.workunit.client.1.smithi192.stdout:4/651: link d1/d26/c32 d1/d26/d34/db9/ce5 0 2024-02-16T22:56:25.039 INFO:tasks.workunit.client.0.smithi064.stdout:1/452: dread d2/df/d14/d2c/f5f [0,4194304] 0 2024-02-16T22:56:25.039 INFO:tasks.workunit.client.1.smithi192.stdout:4/652: fdatasync d1/d26/d66/f9e 0 2024-02-16T22:56:25.040 INFO:tasks.workunit.client.0.smithi064.stdout:1/453: write d2/df/d14/f8d [451584,67997] 0 2024-02-16T22:56:25.046 INFO:tasks.workunit.client.1.smithi192.stdout:3/573: dwrite da/d11/d35/f70 [0,4194304] 0 2024-02-16T22:56:25.047 INFO:tasks.workunit.client.1.smithi192.stdout:3/574: readlink da/d11/d63/d6b/l7d 0 2024-02-16T22:56:25.050 INFO:tasks.workunit.client.1.smithi192.stdout:6/609: truncate da/df/d12/f90 1067986 0 2024-02-16T22:56:25.051 INFO:tasks.workunit.client.0.smithi064.stdout:5/411: write d6/d18/d19/d1a/f45 [84550,60178] 0 2024-02-16T22:56:25.055 INFO:tasks.workunit.client.1.smithi192.stdout:9/717: rmdir d7/d19/d28/d43/d70/df0 39 2024-02-16T22:56:25.058 INFO:tasks.workunit.client.1.smithi192.stdout:6/610: dwrite da/df/d12/d15/d1b/f97 [0,4194304] 0 2024-02-16T22:56:25.059 INFO:tasks.workunit.client.1.smithi192.stdout:9/718: write d7/d19/d28/d43/fee [205301,36249] 0 2024-02-16T22:56:25.059 INFO:tasks.workunit.client.1.smithi192.stdout:1/556: dwrite d3/de/d27/d34/d36/d47/f5e [0,4194304] 0 2024-02-16T22:56:25.060 INFO:tasks.workunit.client.0.smithi064.stdout:1/454: rmdir d2/df/d14/d2c/d62/d91/d56/d5b 39 2024-02-16T22:56:25.060 INFO:tasks.workunit.client.1.smithi192.stdout:4/653: mknod d1/d26/d6f/ce6 0 2024-02-16T22:56:25.063 INFO:tasks.workunit.client.0.smithi064.stdout:6/547: rename d0/d12/f49 to d0/d1d/da8/fac 0 2024-02-16T22:56:25.065 INFO:tasks.workunit.client.1.smithi192.stdout:0/584: unlink d1/f61 0 2024-02-16T22:56:25.067 INFO:tasks.workunit.client.1.smithi192.stdout:7/684: symlink d1/d2/de/d2c/d49/d54/d9d/ddd/lf2 0 2024-02-16T22:56:25.070 INFO:tasks.workunit.client.1.smithi192.stdout:6/611: dwrite da/df/d12/d15/d1b/fa3 [0,4194304] 0 2024-02-16T22:56:25.070 INFO:tasks.workunit.client.0.smithi064.stdout:2/574: mknod d7/d14/d1e/d34/d73/cb4 0 2024-02-16T22:56:25.071 INFO:tasks.workunit.client.1.smithi192.stdout:3/575: stat da/d11/f4b 0 2024-02-16T22:56:25.072 INFO:tasks.workunit.client.1.smithi192.stdout:2/636: link d22/f4a d22/d24/d39/d42/d85/d98/db0/db6/fce 0 2024-02-16T22:56:25.073 INFO:tasks.workunit.client.0.smithi064.stdout:5/412: fsync d6/d25/d48/f55 0 2024-02-16T22:56:25.074 INFO:tasks.workunit.client.0.smithi064.stdout:5/413: fsync d6/d18/d19/d1a/f60 0 2024-02-16T22:56:25.076 INFO:tasks.workunit.client.0.smithi064.stdout:5/414: chown d6/d18/d19/d1a/f76 17154 1 2024-02-16T22:56:25.076 INFO:tasks.workunit.client.0.smithi064.stdout:4/501: sync 2024-02-16T22:56:25.078 INFO:tasks.workunit.client.0.smithi064.stdout:5/415: write d6/d25/d48/f4a [4434845,95265] 0 2024-02-16T22:56:25.078 INFO:tasks.workunit.client.1.smithi192.stdout:0/585: dwrite d1/d3/de/d22/f30 [0,4194304] 0 2024-02-16T22:56:25.078 INFO:tasks.workunit.client.0.smithi064.stdout:7/484: creat d5/d33/d45/f98 x:0 0 0 2024-02-16T22:56:25.079 INFO:tasks.workunit.client.0.smithi064.stdout:5/416: chown d6/d1c/c8c 203093431 1 2024-02-16T22:56:25.080 INFO:tasks.workunit.client.1.smithi192.stdout:9/719: mknod d7/cf7 0 2024-02-16T22:56:25.080 INFO:tasks.workunit.client.1.smithi192.stdout:0/586: write d1/d3/d29/f8d [147958,109538] 0 2024-02-16T22:56:25.081 INFO:tasks.workunit.client.0.smithi064.stdout:7/485: chown d5/d6/d54/d7b/c96 8034719 1 2024-02-16T22:56:25.081 INFO:tasks.workunit.client.0.smithi064.stdout:1/455: symlink d2/df/d14/d2c/la3 0 2024-02-16T22:56:25.082 INFO:tasks.workunit.client.1.smithi192.stdout:2/637: dread d22/d24/d39/d42/d65/d69/f68 [0,4194304] 0 2024-02-16T22:56:25.083 INFO:tasks.workunit.client.1.smithi192.stdout:8/570: creat d5/fae x:0 0 0 2024-02-16T22:56:25.084 INFO:tasks.workunit.client.1.smithi192.stdout:2/638: write d22/d24/d39/d42/d85/d98/db0/fc1 [861532,17258] 0 2024-02-16T22:56:25.086 INFO:tasks.workunit.client.1.smithi192.stdout:8/571: write d5/d26/d83/d86/d5f/f5e [1525312,126351] 0 2024-02-16T22:56:25.087 INFO:tasks.workunit.client.1.smithi192.stdout:8/572: chown d5/db/d47/l76 1 1 2024-02-16T22:56:25.089 INFO:tasks.workunit.client.0.smithi064.stdout:6/548: symlink d0/d5/df/d17/d4a/lad 0 2024-02-16T22:56:25.090 INFO:tasks.workunit.client.0.smithi064.stdout:0/433: creat da/dc/d3b/f8e x:0 0 0 2024-02-16T22:56:25.092 INFO:tasks.workunit.client.0.smithi064.stdout:2/575: mkdir d7/d14/d1e/d34/d73/db5 0 2024-02-16T22:56:25.095 INFO:tasks.workunit.client.1.smithi192.stdout:6/612: creat da/d40/fcb x:0 0 0 2024-02-16T22:56:25.096 INFO:tasks.workunit.client.1.smithi192.stdout:0/587: dread d1/f8 [0,4194304] 0 2024-02-16T22:56:25.097 INFO:tasks.workunit.client.1.smithi192.stdout:6/613: write da/d40/fcb [563288,91534] 0 2024-02-16T22:56:25.098 INFO:tasks.workunit.client.1.smithi192.stdout:6/614: write da/df/d6b/f88 [630211,26651] 0 2024-02-16T22:56:25.099 INFO:tasks.workunit.client.1.smithi192.stdout:3/576: write da/d11/d20/d39/d53/f8a [404309,106083] 0 2024-02-16T22:56:25.102 INFO:tasks.workunit.client.1.smithi192.stdout:0/588: fsync d1/d3/de/d22/f30 0 2024-02-16T22:56:25.102 INFO:tasks.workunit.client.0.smithi064.stdout:5/417: truncate d6/d1c/d43/f4c 797519 0 2024-02-16T22:56:25.103 INFO:tasks.workunit.client.0.smithi064.stdout:5/418: dread - d6/d25/d48/d67/f6c zero size 2024-02-16T22:56:25.104 INFO:tasks.workunit.client.0.smithi064.stdout:4/502: unlink dc/dd/d16/d7c/l98 0 2024-02-16T22:56:25.104 INFO:tasks.workunit.client.0.smithi064.stdout:4/503: fdatasync dc/d1d/f95 0 2024-02-16T22:56:25.105 INFO:tasks.workunit.client.1.smithi192.stdout:9/720: readlink d7/d19/d28/d43/d70/d8e/l9b 0 2024-02-16T22:56:25.106 INFO:tasks.workunit.client.0.smithi064.stdout:7/486: creat d5/d33/d53/f99 x:0 0 0 2024-02-16T22:56:25.107 INFO:tasks.workunit.client.0.smithi064.stdout:7/487: read - d5/d33/d42/f8f zero size 2024-02-16T22:56:25.108 INFO:tasks.workunit.client.1.smithi192.stdout:5/574: sync 2024-02-16T22:56:25.108 INFO:tasks.workunit.client.0.smithi064.stdout:1/456: truncate d2/f43 2287129 0 2024-02-16T22:56:25.110 INFO:tasks.workunit.client.0.smithi064.stdout:3/578: getdents d4/d13/d41 0 2024-02-16T22:56:25.112 INFO:tasks.workunit.client.1.smithi192.stdout:1/557: truncate d3/de/d27/d34/d52/f56 1491312 0 2024-02-16T22:56:25.114 INFO:tasks.workunit.client.0.smithi064.stdout:6/549: rename d0/f22 to d0/d89/fae 0 2024-02-16T22:56:25.115 INFO:tasks.workunit.client.0.smithi064.stdout:7/488: dwrite d5/d6/d54/d41/f4b [0,4194304] 0 2024-02-16T22:56:25.116 INFO:tasks.workunit.client.1.smithi192.stdout:5/575: dwrite d6/d20/f4d [4194304,4194304] 0 2024-02-16T22:56:25.117 INFO:tasks.workunit.client.0.smithi064.stdout:0/434: mkdir da/dc/d3b/d5a/d8f 0 2024-02-16T22:56:25.117 INFO:tasks.workunit.client.1.smithi192.stdout:4/654: dwrite d1/d26/d66/d89/f9 [0,4194304] 0 2024-02-16T22:56:25.119 INFO:tasks.workunit.client.0.smithi064.stdout:6/550: dread d0/d5/df/d17/f62 [0,4194304] 0 2024-02-16T22:56:25.119 INFO:tasks.workunit.client.0.smithi064.stdout:0/435: dread - da/dc/de/d12/d2c/d2e/d38/f80 zero size 2024-02-16T22:56:25.119 INFO:tasks.workunit.client.0.smithi064.stdout:6/551: readlink d0/d6/da/l2e 0 2024-02-16T22:56:25.121 INFO:tasks.workunit.client.0.smithi064.stdout:4/504: dread dc/dd/d32/d22/f56 [0,4194304] 0 2024-02-16T22:56:25.121 INFO:tasks.workunit.client.0.smithi064.stdout:0/436: write da/dc/d3b/f5f [614710,16105] 0 2024-02-16T22:56:25.122 INFO:tasks.workunit.client.1.smithi192.stdout:7/685: creat d1/d2/de/d2c/d49/dba/ff3 x:0 0 0 2024-02-16T22:56:25.123 INFO:tasks.workunit.client.0.smithi064.stdout:5/419: chown d6/f34 5302856 1 2024-02-16T22:56:25.124 INFO:tasks.workunit.client.1.smithi192.stdout:7/686: chown d1/d2/de/f34 52552 1 2024-02-16T22:56:25.128 INFO:tasks.workunit.client.0.smithi064.stdout:1/457: creat d2/df/d14/d18/d48/d84/fa4 x:0 0 0 2024-02-16T22:56:25.130 INFO:tasks.workunit.client.1.smithi192.stdout:8/573: dwrite d5/db/d47/d8c/d9d/fa8 [0,4194304] 0 2024-02-16T22:56:25.132 INFO:tasks.workunit.client.0.smithi064.stdout:3/579: creat d4/d13/d1b/d27/d99/d7d/fc8 x:0 0 0 2024-02-16T22:56:25.132 INFO:tasks.workunit.client.0.smithi064.stdout:1/458: chown d2/df/d14/d2c/d62/d91/d56/l9e 49343 1 2024-02-16T22:56:25.133 INFO:tasks.workunit.client.0.smithi064.stdout:1/459: write d2/df/d14/f88 [899404,122047] 0 2024-02-16T22:56:25.134 INFO:tasks.workunit.client.1.smithi192.stdout:0/589: creat d1/d3/de/d22/d37/d67/fb7 x:0 0 0 2024-02-16T22:56:25.135 INFO:tasks.workunit.client.0.smithi064.stdout:1/460: chown d2/df/f13 6540377 1 2024-02-16T22:56:25.136 INFO:tasks.workunit.client.1.smithi192.stdout:9/721: creat d7/d15/d36/d77/db1/ff8 x:0 0 0 2024-02-16T22:56:25.136 INFO:tasks.workunit.client.0.smithi064.stdout:1/461: chown d2/c79 97403 1 2024-02-16T22:56:25.137 INFO:tasks.workunit.client.1.smithi192.stdout:9/722: truncate d7/d19/d5e/d81/f95 323098 0 2024-02-16T22:56:25.137 INFO:tasks.workunit.client.0.smithi064.stdout:6/552: mknod d0/d5/df/caf 0 2024-02-16T22:56:25.138 INFO:tasks.workunit.client.0.smithi064.stdout:7/489: read d5/d6/f5b [56408,101339] 0 2024-02-16T22:56:25.139 INFO:tasks.workunit.client.0.smithi064.stdout:2/576: mknod d7/d14/d1e/d34/d73/db5/cb6 0 2024-02-16T22:56:25.140 INFO:tasks.workunit.client.1.smithi192.stdout:2/639: creat d22/d47/dcb/dc9/fcf x:0 0 0 2024-02-16T22:56:25.140 INFO:tasks.workunit.client.0.smithi064.stdout:7/490: stat d5/d33/d45/d73/d88 0 2024-02-16T22:56:25.142 INFO:tasks.workunit.client.1.smithi192.stdout:1/558: mknod d3/de/da4/cc1 0 2024-02-16T22:56:25.142 INFO:tasks.workunit.client.1.smithi192.stdout:2/640: fsync d22/d24/d39/d42/d65/d66/f83 0 2024-02-16T22:56:25.145 INFO:tasks.workunit.client.1.smithi192.stdout:9/723: dwrite d7/d19/d37/d5a/ff5 [0,4194304] 0 2024-02-16T22:56:25.145 INFO:tasks.workunit.client.0.smithi064.stdout:5/420: creat d6/d25/d48/d67/f8f x:0 0 0 2024-02-16T22:56:25.149 INFO:tasks.workunit.client.0.smithi064.stdout:3/580: symlink d4/d13/d1b/d27/d44/lc9 0 2024-02-16T22:56:25.150 INFO:tasks.workunit.client.1.smithi192.stdout:7/687: creat d1/d2/de/d2c/d49/dba/ff4 x:0 0 0 2024-02-16T22:56:25.150 INFO:tasks.workunit.client.0.smithi064.stdout:1/462: fdatasync d2/df/d11/d1a/d1d/f58 0 2024-02-16T22:56:25.153 INFO:tasks.workunit.client.0.smithi064.stdout:2/577: symlink d7/d14/d25/lb7 0 2024-02-16T22:56:25.155 INFO:tasks.workunit.client.1.smithi192.stdout:0/590: mknod d1/d3/de/d22/d37/d67/d75/cb8 0 2024-02-16T22:56:25.162 INFO:tasks.workunit.client.0.smithi064.stdout:9/464: sync 2024-02-16T22:56:25.162 INFO:tasks.workunit.client.0.smithi064.stdout:8/471: sync 2024-02-16T22:56:25.167 INFO:tasks.workunit.client.1.smithi192.stdout:9/724: chown d7/d19/d28/d43/d70/df0 94850913 1 2024-02-16T22:56:25.168 INFO:tasks.workunit.client.1.smithi192.stdout:6/615: link da/d76/lac da/d40/db6/lcc 0 2024-02-16T22:56:25.169 INFO:tasks.workunit.client.0.smithi064.stdout:1/463: mknod d2/df/d14/d2c/d62/ca5 0 2024-02-16T22:56:25.170 INFO:tasks.workunit.client.1.smithi192.stdout:7/688: mknod d1/d2/d2d/d92/dcd/cf5 0 2024-02-16T22:56:25.172 INFO:tasks.workunit.client.1.smithi192.stdout:7/689: dread - d1/d2/de/d2c/d49/fd7 zero size 2024-02-16T22:56:25.174 INFO:tasks.workunit.client.1.smithi192.stdout:7/690: chown d1/d2/d37/f9c 1268893 1 2024-02-16T22:56:25.174 INFO:tasks.workunit.client.1.smithi192.stdout:0/591: creat d1/d3/de/d22/d37/d67/d34/da1/fb9 x:0 0 0 2024-02-16T22:56:25.176 INFO:tasks.workunit.client.1.smithi192.stdout:6/616: dwrite da/df/fba [0,4194304] 0 2024-02-16T22:56:25.177 INFO:tasks.workunit.client.1.smithi192.stdout:0/592: write d1/f7a [611047,52956] 0 2024-02-16T22:56:25.180 INFO:tasks.workunit.client.1.smithi192.stdout:6/617: write da/df/d12/d15/f2d [890722,38963] 0 2024-02-16T22:56:25.183 INFO:tasks.workunit.client.1.smithi192.stdout:3/577: dwrite da/df/f1f [0,4194304] 0 2024-02-16T22:56:25.185 INFO:tasks.workunit.client.1.smithi192.stdout:1/559: dread d3/de/d27/f30 [4194304,4194304] 0 2024-02-16T22:56:25.185 INFO:tasks.workunit.client.0.smithi064.stdout:4/505: write dc/d1d/f36 [1466725,125173] 0 2024-02-16T22:56:25.186 INFO:tasks.workunit.client.1.smithi192.stdout:1/560: readlink d3/de/d27/d34/d52/d72/l98 0 2024-02-16T22:56:25.187 INFO:tasks.workunit.client.0.smithi064.stdout:4/506: chown dc/dd/d32/d58/f38 5518 1 2024-02-16T22:56:25.188 INFO:tasks.workunit.client.0.smithi064.stdout:4/507: stat dc/d1d/c68 0 2024-02-16T22:56:25.190 INFO:tasks.workunit.client.0.smithi064.stdout:0/437: write da/dc/de/f17 [2658213,54219] 0 2024-02-16T22:56:25.191 INFO:tasks.workunit.client.0.smithi064.stdout:0/438: stat da/dc/de/d12/d1d/f34 0 2024-02-16T22:56:25.192 INFO:tasks.workunit.client.1.smithi192.stdout:4/655: write d1/d26/d6f/f8f [540228,104082] 0 2024-02-16T22:56:25.192 INFO:tasks.workunit.client.1.smithi192.stdout:5/576: link d6/lf d6/d20/d93/lbd 0 2024-02-16T22:56:25.195 INFO:tasks.workunit.client.0.smithi064.stdout:9/465: rename d3/c5 to d3/d11/d15/d5a/d38/c98 0 2024-02-16T22:56:25.197 INFO:tasks.workunit.client.1.smithi192.stdout:7/691: unlink d1/d2/de/lb9 0 2024-02-16T22:56:25.199 INFO:tasks.workunit.client.0.smithi064.stdout:6/553: link d0/d1b/l7e d0/d5/lb0 0 2024-02-16T22:56:25.201 INFO:tasks.workunit.client.0.smithi064.stdout:1/464: truncate d2/df/d14/d2c/f3c 421277 0 2024-02-16T22:56:25.202 INFO:tasks.workunit.client.0.smithi064.stdout:7/491: write d5/d6/d54/d41/f87 [1274390,36866] 0 2024-02-16T22:56:25.204 INFO:tasks.workunit.client.1.smithi192.stdout:7/692: dwrite d1/d2/d37/fe7 [0,4194304] 0 2024-02-16T22:56:25.205 INFO:tasks.workunit.client.1.smithi192.stdout:6/618: unlink da/df/d12/d15/f3e 0 2024-02-16T22:56:25.207 INFO:tasks.workunit.client.0.smithi064.stdout:0/439: stat da/c77 0 2024-02-16T22:56:25.207 INFO:tasks.workunit.client.0.smithi064.stdout:5/421: write d6/d25/d48/f55 [410468,99898] 0 2024-02-16T22:56:25.208 INFO:tasks.workunit.client.1.smithi192.stdout:6/619: chown da/df/d6b/f88 0 1 2024-02-16T22:56:25.213 INFO:tasks.workunit.client.1.smithi192.stdout:8/574: link d5/l10 d5/d26/d83/d86/d5f/laf 0 2024-02-16T22:56:25.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/561: dread d3/de/d27/f30 [0,4194304] 0 2024-02-16T22:56:25.214 INFO:tasks.workunit.client.0.smithi064.stdout:3/581: getdents d4/d13/d1b/d27/d99 0 2024-02-16T22:56:25.218 INFO:tasks.workunit.client.1.smithi192.stdout:1/562: dread d3/f21 [0,4194304] 0 2024-02-16T22:56:25.222 INFO:tasks.workunit.client.0.smithi064.stdout:6/554: dread d0/d5/df/d17/d18/f3f [0,4194304] 0 2024-02-16T22:56:25.222 INFO:tasks.workunit.client.0.smithi064.stdout:8/472: write f3 [4558438,41547] 0 2024-02-16T22:56:25.224 INFO:tasks.workunit.client.0.smithi064.stdout:6/555: dread - d0/d6/da/f95 zero size 2024-02-16T22:56:25.224 INFO:tasks.workunit.client.0.smithi064.stdout:2/578: dwrite d7/d14/d1e/f21 [0,4194304] 0 2024-02-16T22:56:25.226 INFO:tasks.workunit.client.0.smithi064.stdout:0/440: mkdir da/dc/de/d12/d2c/d2e/d38/d90 0 2024-02-16T22:56:25.234 INFO:tasks.workunit.client.0.smithi064.stdout:8/473: dwrite df/d31/d2f/d4f/f66 [0,4194304] 0 2024-02-16T22:56:25.236 INFO:tasks.workunit.client.0.smithi064.stdout:3/582: chown d4/d13/d1b/f68 62 1 2024-02-16T22:56:25.242 INFO:tasks.workunit.client.0.smithi064.stdout:9/466: write d3/d9/f21 [3825290,65755] 0 2024-02-16T22:56:25.243 INFO:tasks.workunit.client.0.smithi064.stdout:5/422: dread d6/d1c/f54 [0,4194304] 0 2024-02-16T22:56:25.243 INFO:tasks.workunit.client.0.smithi064.stdout:6/556: chown d0/d1b/l75 84194192 1 2024-02-16T22:56:25.244 INFO:tasks.workunit.client.0.smithi064.stdout:4/508: truncate dc/dd/d32/d58/f12 3848140 0 2024-02-16T22:56:25.245 INFO:tasks.workunit.client.0.smithi064.stdout:4/509: stat dc/dd/d32/d58/f70 0 2024-02-16T22:56:25.246 INFO:tasks.workunit.client.0.smithi064.stdout:4/510: fdatasync dc/d1d/f69 0 2024-02-16T22:56:25.246 INFO:tasks.workunit.client.1.smithi192.stdout:0/593: write d1/d3/de/d22/d37/d47/d53/f65 [277188,64007] 0 2024-02-16T22:56:25.246 INFO:tasks.workunit.client.0.smithi064.stdout:4/511: chown dc/dd/d32/d58/l30 100 1 2024-02-16T22:56:25.247 INFO:tasks.workunit.client.1.smithi192.stdout:8/575: creat d5/d26/d8f/fb0 x:0 0 0 2024-02-16T22:56:25.248 INFO:tasks.workunit.client.0.smithi064.stdout:2/579: write d7/d14/d1e/d34/f61 [2354780,70694] 0 2024-02-16T22:56:25.249 INFO:tasks.workunit.client.1.smithi192.stdout:3/578: dwrite da/d11/d63/d6b/f87 [0,4194304] 0 2024-02-16T22:56:25.250 INFO:tasks.workunit.client.0.smithi064.stdout:8/474: symlink df/d31/d2f/d68/l9f 0 2024-02-16T22:56:25.251 INFO:tasks.workunit.client.1.smithi192.stdout:1/563: mknod d3/de/d27/d34/d52/d63/d7a/cc2 0 2024-02-16T22:56:25.252 INFO:tasks.workunit.client.0.smithi064.stdout:2/580: write d7/de/d95/fa3 [961309,77701] 0 2024-02-16T22:56:25.252 INFO:tasks.workunit.client.1.smithi192.stdout:4/656: dwrite d1/d26/d66/d89/f1e [0,4194304] 0 2024-02-16T22:56:25.253 INFO:tasks.workunit.client.1.smithi192.stdout:1/564: stat d3/d5b/f84 0 2024-02-16T22:56:25.253 INFO:tasks.workunit.client.0.smithi064.stdout:8/475: chown df/d31/d61/d79 367 1 2024-02-16T22:56:25.254 INFO:tasks.workunit.client.1.smithi192.stdout:9/725: getdents d7/d15/d36/d38/d3f/d8a/d98/d6e 0 2024-02-16T22:56:25.255 INFO:tasks.workunit.client.0.smithi064.stdout:1/465: creat d2/fa6 x:0 0 0 2024-02-16T22:56:25.256 INFO:tasks.workunit.client.1.smithi192.stdout:7/693: symlink d1/de1/lf6 0 2024-02-16T22:56:25.256 INFO:tasks.workunit.client.1.smithi192.stdout:0/594: dread d1/d3/de/d22/f30 [0,4194304] 0 2024-02-16T22:56:25.256 INFO:tasks.workunit.client.0.smithi064.stdout:8/476: dread - df/d31/d2f/d38/f62 zero size 2024-02-16T22:56:25.258 INFO:tasks.workunit.client.1.smithi192.stdout:0/595: chown d1/d3/de/d22/d37/d67/f9b 23 1 2024-02-16T22:56:25.258 INFO:tasks.workunit.client.0.smithi064.stdout:4/512: dwrite dc/d1d/f69 [0,4194304] 0 2024-02-16T22:56:25.259 INFO:tasks.workunit.client.1.smithi192.stdout:6/620: truncate da/df/d12/d15/d1b/f1e 39370 0 2024-02-16T22:56:25.260 INFO:tasks.workunit.client.0.smithi064.stdout:5/423: mkdir d6/d18/d90 0 2024-02-16T22:56:25.261 INFO:tasks.workunit.client.1.smithi192.stdout:6/621: fdatasync da/df/d12/d15/d1b/f37 0 2024-02-16T22:56:25.264 INFO:tasks.workunit.client.0.smithi064.stdout:0/441: symlink da/dc/d3b/d5a/d8f/l91 0 2024-02-16T22:56:25.265 INFO:tasks.workunit.client.0.smithi064.stdout:4/513: truncate dc/dd/d16/f3a 4522996 0 2024-02-16T22:56:25.266 INFO:tasks.workunit.client.0.smithi064.stdout:4/514: stat dc/dd/d16/d6e/f7b 0 2024-02-16T22:56:25.267 INFO:tasks.workunit.client.1.smithi192.stdout:2/641: sync 2024-02-16T22:56:25.267 INFO:tasks.workunit.client.0.smithi064.stdout:2/581: mkdir d7/d14/d3e/db8 0 2024-02-16T22:56:25.269 INFO:tasks.workunit.client.0.smithi064.stdout:2/582: write d7/de/d95/d44/f6d [902038,28953] 0 2024-02-16T22:56:25.271 INFO:tasks.workunit.client.1.smithi192.stdout:6/622: dwrite da/df/d58/d72/fbe [0,4194304] 0 2024-02-16T22:56:25.273 INFO:tasks.workunit.client.0.smithi064.stdout:5/424: mkdir d6/d25/d48/d67/d91 0 2024-02-16T22:56:25.275 INFO:tasks.workunit.client.1.smithi192.stdout:4/657: symlink d1/d26/d5e/le7 0 2024-02-16T22:56:25.275 INFO:tasks.workunit.client.1.smithi192.stdout:1/565: read - d3/f7e zero size 2024-02-16T22:56:25.276 INFO:tasks.workunit.client.0.smithi064.stdout:8/477: mknod df/d31/d2f/d3e/ca0 0 2024-02-16T22:56:25.277 INFO:tasks.workunit.client.1.smithi192.stdout:9/726: rename d7/d15/d36/d38/d3f/c75 to d7/d15/d36/d77/db1/cf9 0 2024-02-16T22:56:25.279 INFO:tasks.workunit.client.1.smithi192.stdout:7/694: fsync d1/d2/d2d/ddf/fe5 0 2024-02-16T22:56:25.284 INFO:tasks.workunit.client.1.smithi192.stdout:5/577: getdents d6/d12/d4e/d5b/d67 0 2024-02-16T22:56:25.285 INFO:tasks.workunit.client.1.smithi192.stdout:9/727: dwrite d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f9a [4194304,4194304] 0 2024-02-16T22:56:25.289 INFO:tasks.workunit.client.0.smithi064.stdout:1/466: dread d2/df/d14/d18/f26 [0,4194304] 0 2024-02-16T22:56:25.289 INFO:tasks.workunit.client.1.smithi192.stdout:8/576: mknod d5/d68/cb1 0 2024-02-16T22:56:25.294 INFO:tasks.workunit.client.1.smithi192.stdout:5/578: dwrite d6/d20/d93/fba [0,4194304] 0 2024-02-16T22:56:25.295 INFO:tasks.workunit.client.1.smithi192.stdout:2/642: mkdir d22/d47/dcb/d3f/dd0 0 2024-02-16T22:56:25.295 INFO:tasks.workunit.client.0.smithi064.stdout:6/557: write d0/d5/f42 [3157478,105661] 0 2024-02-16T22:56:25.297 INFO:tasks.workunit.client.0.smithi064.stdout:8/478: mknod df/d31/d8f/d80/ca1 0 2024-02-16T22:56:25.300 INFO:tasks.workunit.client.0.smithi064.stdout:0/442: link da/dc/de/d37/f46 da/dc/de/d12/d2c/f92 0 2024-02-16T22:56:25.302 INFO:tasks.workunit.client.0.smithi064.stdout:4/515: mkdir dc/dd/d32/d46/d5a/d7e/d9a 0 2024-02-16T22:56:25.305 INFO:tasks.workunit.client.0.smithi064.stdout:3/583: link d4/d13/d4d/l65 d4/d13/d4d/d5c/lca 0 2024-02-16T22:56:25.307 INFO:tasks.workunit.client.1.smithi192.stdout:0/596: creat d1/d3/dac/fba x:0 0 0 2024-02-16T22:56:25.309 INFO:tasks.workunit.client.0.smithi064.stdout:9/467: write d3/d9/f12 [2327824,116435] 0 2024-02-16T22:56:25.312 INFO:tasks.workunit.client.1.smithi192.stdout:9/728: symlink d7/d15/d36/d38/d3f/d8a/d98/d6e/lfa 0 2024-02-16T22:56:25.314 INFO:tasks.workunit.client.0.smithi064.stdout:4/516: dread dc/dd/d32/f1f [4194304,4194304] 0 2024-02-16T22:56:25.316 INFO:tasks.workunit.client.0.smithi064.stdout:4/517: write dc/dd/d32/d46/d5a/f84 [776869,79019] 0 2024-02-16T22:56:25.317 INFO:tasks.workunit.client.0.smithi064.stdout:4/518: chown dc/dd/d32/d46/d79 26121 1 2024-02-16T22:56:25.318 INFO:tasks.workunit.client.0.smithi064.stdout:6/558: mkdir d0/d1d/db1 0 2024-02-16T22:56:25.320 INFO:tasks.workunit.client.1.smithi192.stdout:8/577: mkdir d5/d26/d83/d86/d5f/db2 0 2024-02-16T22:56:25.323 INFO:tasks.workunit.client.1.smithi192.stdout:3/579: creat da/fcc x:0 0 0 2024-02-16T22:56:25.325 INFO:tasks.workunit.client.0.smithi064.stdout:0/443: dread da/f10 [0,4194304] 0 2024-02-16T22:56:25.325 INFO:tasks.workunit.client.0.smithi064.stdout:4/519: dwrite dc/dd/d32/d46/d5a/f65 [0,4194304] 0 2024-02-16T22:56:25.326 INFO:tasks.workunit.client.1.smithi192.stdout:5/579: creat d6/d12/d4e/d5b/daf/fbe x:0 0 0 2024-02-16T22:56:25.327 INFO:tasks.workunit.client.0.smithi064.stdout:0/444: dread - da/dc/de/d12/f6c zero size 2024-02-16T22:56:25.328 INFO:tasks.workunit.client.0.smithi064.stdout:0/445: chown da/dc/de/d37/l79 123469997 1 2024-02-16T22:56:25.330 INFO:tasks.workunit.client.0.smithi064.stdout:2/583: write d7/d14/d1e/f92 [1156310,87080] 0 2024-02-16T22:56:25.332 INFO:tasks.workunit.client.0.smithi064.stdout:2/584: write d7/f13 [1261445,88998] 0 2024-02-16T22:56:25.333 INFO:tasks.workunit.client.1.smithi192.stdout:2/643: creat d22/d24/d39/d42/d65/d69/d4f/fd1 x:0 0 0 2024-02-16T22:56:25.335 INFO:tasks.workunit.client.0.smithi064.stdout:9/468: symlink d3/d9/d1e/d42/l99 0 2024-02-16T22:56:25.335 INFO:tasks.workunit.client.1.smithi192.stdout:2/644: write d22/d24/d39/d42/d52/f60 [958971,101949] 0 2024-02-16T22:56:25.339 INFO:tasks.workunit.client.0.smithi064.stdout:2/585: dwrite d7/d14/d1e/d34/d3f/f4e [0,4194304] 0 2024-02-16T22:56:25.340 INFO:tasks.workunit.client.1.smithi192.stdout:4/658: mknod d1/d15/d48/db5/dd5/ce8 0 2024-02-16T22:56:25.340 INFO:tasks.workunit.client.0.smithi064.stdout:2/586: dread - d7/de/d95/d44/f80 zero size 2024-02-16T22:56:25.342 INFO:tasks.workunit.client.1.smithi192.stdout:1/566: dwrite d3/f75 [0,4194304] 0 2024-02-16T22:56:25.343 INFO:tasks.workunit.client.1.smithi192.stdout:0/597: unlink d1/f5 0 2024-02-16T22:56:25.344 INFO:tasks.workunit.client.1.smithi192.stdout:2/645: dwrite fa [0,4194304] 0 2024-02-16T22:56:25.346 INFO:tasks.workunit.client.0.smithi064.stdout:1/467: write d2/df/d14/d18/f26 [532799,90396] 0 2024-02-16T22:56:25.351 INFO:tasks.workunit.client.1.smithi192.stdout:4/659: dread d1/d26/d66/d89/da8/fb8 [0,4194304] 0 2024-02-16T22:56:25.352 INFO:tasks.workunit.client.0.smithi064.stdout:2/587: dwrite d7/d14/d1e/fa4 [0,4194304] 0 2024-02-16T22:56:25.354 INFO:tasks.workunit.client.1.smithi192.stdout:8/578: creat d5/db/de/df/fb3 x:0 0 0 2024-02-16T22:56:25.357 INFO:tasks.workunit.client.1.smithi192.stdout:7/695: dwrite d1/d2/d2d/d61/d88/d90/d4c/f70 [0,4194304] 0 2024-02-16T22:56:25.359 INFO:tasks.workunit.client.0.smithi064.stdout:4/520: rename dc/dd/d32/f1f to dc/d1d/d43/d7a/f9b 0 2024-02-16T22:56:25.362 INFO:tasks.workunit.client.0.smithi064.stdout:3/584: truncate d4/f1e 1109916 0 2024-02-16T22:56:25.363 INFO:tasks.workunit.client.0.smithi064.stdout:3/585: stat d4/d13/d1b/d27/d44/lb6 0 2024-02-16T22:56:25.364 INFO:tasks.workunit.client.1.smithi192.stdout:2/646: dread f19 [4194304,4194304] 0 2024-02-16T22:56:25.364 INFO:tasks.workunit.client.0.smithi064.stdout:3/586: stat d4/d13/d1b/d27/d99/f89 0 2024-02-16T22:56:25.371 INFO:tasks.workunit.client.1.smithi192.stdout:6/623: dwrite da/df/f1c [4194304,4194304] 0 2024-02-16T22:56:25.372 INFO:tasks.workunit.client.0.smithi064.stdout:5/425: getdents d6 0 2024-02-16T22:56:25.373 INFO:tasks.workunit.client.0.smithi064.stdout:1/468: dread d2/df/d14/d2c/d62/d91/d56/f57 [0,4194304] 0 2024-02-16T22:56:25.374 INFO:tasks.workunit.client.1.smithi192.stdout:9/729: mknod d7/d19/d37/ddf/cfb 0 2024-02-16T22:56:25.376 INFO:tasks.workunit.client.0.smithi064.stdout:3/587: dwrite d4/f17 [8388608,4194304] 0 2024-02-16T22:56:25.376 INFO:tasks.workunit.client.0.smithi064.stdout:6/559: truncate d0/d6/d10/d33/f55 331566 0 2024-02-16T22:56:25.378 INFO:tasks.workunit.client.0.smithi064.stdout:6/560: fsync d0/d1d/f9e 0 2024-02-16T22:56:25.379 INFO:tasks.workunit.client.0.smithi064.stdout:2/588: mkdir d7/d14/d1e/d34/d73/db5/db9 0 2024-02-16T22:56:25.379 INFO:tasks.workunit.client.0.smithi064.stdout:6/561: write d0/d5/f42 [2620969,23101] 0 2024-02-16T22:56:25.380 INFO:tasks.workunit.client.0.smithi064.stdout:2/589: readlink d7/d14/d1e/l2b 0 2024-02-16T22:56:25.381 INFO:tasks.workunit.client.0.smithi064.stdout:3/588: truncate d4/d13/d1b/d4f/fb3 993770 0 2024-02-16T22:56:25.382 INFO:tasks.workunit.client.0.smithi064.stdout:6/562: write d0/d12/f74 [1470016,88804] 0 2024-02-16T22:56:25.386 INFO:tasks.workunit.client.0.smithi064.stdout:4/521: unlink dc/dd/d32/l26 0 2024-02-16T22:56:25.387 INFO:tasks.workunit.client.0.smithi064.stdout:4/522: fdatasync dc/f6c 0 2024-02-16T22:56:25.387 INFO:tasks.workunit.client.1.smithi192.stdout:2/647: rmdir d22/d24/d39/d42/d85/d98 39 2024-02-16T22:56:25.388 INFO:tasks.workunit.client.0.smithi064.stdout:0/446: mkdir da/dc/de/d12/d2c/d2e/d38/d7e/d89/d93 0 2024-02-16T22:56:25.390 INFO:tasks.workunit.client.0.smithi064.stdout:4/523: dread dc/dd/d32/d46/d5a/f66 [0,4194304] 0 2024-02-16T22:56:25.390 INFO:tasks.workunit.client.0.smithi064.stdout:0/447: chown da/dc/de/d12/d2c/d2e/l5b 428773 1 2024-02-16T22:56:25.390 INFO:tasks.workunit.client.0.smithi064.stdout:9/469: mkdir d3/d89/d9a 0 2024-02-16T22:56:25.391 INFO:tasks.workunit.client.0.smithi064.stdout:4/524: write dc/d1d/f8b [693604,100289] 0 2024-02-16T22:56:25.392 INFO:tasks.workunit.client.0.smithi064.stdout:4/525: chown l9 0 1 2024-02-16T22:56:25.394 INFO:tasks.workunit.client.0.smithi064.stdout:5/426: rename d6/c9 to d6/d18/d19/d1a/d85/c92 0 2024-02-16T22:56:25.400 INFO:tasks.workunit.client.0.smithi064.stdout:8/479: dwrite f9 [0,4194304] 0 2024-02-16T22:56:25.402 INFO:tasks.workunit.client.1.smithi192.stdout:0/598: dread d1/d3/de/f10 [0,4194304] 0 2024-02-16T22:56:25.403 INFO:tasks.workunit.client.0.smithi064.stdout:8/480: readlink df/d31/d47/l86 0 2024-02-16T22:56:25.404 INFO:tasks.workunit.client.1.smithi192.stdout:0/599: dread - d1/d3/d29/d43/d9c/fad zero size 2024-02-16T22:56:25.406 INFO:tasks.workunit.client.1.smithi192.stdout:0/600: fdatasync d1/f3e 0 2024-02-16T22:56:25.406 INFO:tasks.workunit.client.1.smithi192.stdout:5/580: dwrite d6/d8/d75/f7b [0,4194304] 0 2024-02-16T22:56:25.407 INFO:tasks.workunit.client.1.smithi192.stdout:3/580: creat da/d11/d63/d6b/d66/fcd x:0 0 0 2024-02-16T22:56:25.408 INFO:tasks.workunit.client.1.smithi192.stdout:3/581: dread - da/fcc zero size 2024-02-16T22:56:25.412 INFO:tasks.workunit.client.1.smithi192.stdout:2/648: dread - d22/d24/d39/d42/d85/d98/fc8 zero size 2024-02-16T22:56:25.418 INFO:tasks.workunit.client.0.smithi064.stdout:0/448: unlink da/d71/l78 0 2024-02-16T22:56:25.422 INFO:tasks.workunit.client.0.smithi064.stdout:9/470: truncate d3/d11/d15/f51 436682 0 2024-02-16T22:56:25.426 INFO:tasks.workunit.client.0.smithi064.stdout:0/449: dwrite da/dc/de/f76 [0,4194304] 0 2024-02-16T22:56:25.431 INFO:tasks.workunit.client.0.smithi064.stdout:7/492: sync 2024-02-16T22:56:25.431 INFO:tasks.workunit.client.0.smithi064.stdout:7/493: truncate d5/d6/d65/d7f/f8d 416508 0 2024-02-16T22:56:25.432 INFO:tasks.workunit.client.0.smithi064.stdout:2/590: rename d7/d14/d1e/d34/d65/f72 to d7/d14/d1e/d34/d73/fba 0 2024-02-16T22:56:25.433 INFO:tasks.workunit.client.1.smithi192.stdout:6/624: truncate da/df/d12/f90 1008393 0 2024-02-16T22:56:25.435 INFO:tasks.workunit.client.1.smithi192.stdout:0/601: mknod d1/d3/d29/d43/d9c/cbb 0 2024-02-16T22:56:25.437 INFO:tasks.workunit.client.0.smithi064.stdout:8/481: mkdir df/d31/d2f/d38/d48/da2 0 2024-02-16T22:56:25.437 INFO:tasks.workunit.client.0.smithi064.stdout:8/482: write df/d2a/f87 [300452,23251] 0 2024-02-16T22:56:25.439 INFO:tasks.workunit.client.1.smithi192.stdout:7/696: link d1/d2/de/d2c/d49/d54/de3/c7b d1/d2/de/d83/cf7 0 2024-02-16T22:56:25.440 INFO:tasks.workunit.client.0.smithi064.stdout:3/589: read d4/d13/d1b/d27/d61/d63/f71 [670092,43375] 0 2024-02-16T22:56:25.441 INFO:tasks.workunit.client.0.smithi064.stdout:6/563: mkdir d0/d6/d10/db2 0 2024-02-16T22:56:25.442 INFO:tasks.workunit.client.1.smithi192.stdout:2/649: read - d22/d24/d2b/f8a zero size 2024-02-16T22:56:25.444 INFO:tasks.workunit.client.1.smithi192.stdout:8/579: dwrite d5/f61 [0,4194304] 0 2024-02-16T22:56:25.444 INFO:tasks.workunit.client.1.smithi192.stdout:1/567: link d3/de/d27/d34/l3f d3/de/da4/lc3 0 2024-02-16T22:56:25.446 INFO:tasks.workunit.client.1.smithi192.stdout:1/568: stat d3/de/d27/d34/d36/d47/f5e 0 2024-02-16T22:56:25.446 INFO:tasks.workunit.client.0.smithi064.stdout:3/590: read d4/d13/d1b/d27/f7f [3128742,88248] 0 2024-02-16T22:56:25.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/602: dwrite d1/d3/f7c [0,4194304] 0 2024-02-16T22:56:25.447 INFO:tasks.workunit.client.1.smithi192.stdout:8/580: write d5/d26/d83/d86/d5f/f5e [264381,55058] 0 2024-02-16T22:56:25.448 INFO:tasks.workunit.client.0.smithi064.stdout:3/591: chown d4/d14/d2b/laa 4248 1 2024-02-16T22:56:25.451 INFO:tasks.workunit.client.0.smithi064.stdout:4/526: mkdir dc/dd/d16/d9c 0 2024-02-16T22:56:25.452 INFO:tasks.workunit.client.1.smithi192.stdout:4/660: link d1/d15/d48/d8d/cde d1/d15/d29/d3c/ce9 0 2024-02-16T22:56:25.453 INFO:tasks.workunit.client.1.smithi192.stdout:3/582: dread da/d11/d63/d6b/d4e/f56 [0,4194304] 0 2024-02-16T22:56:25.453 INFO:tasks.workunit.client.1.smithi192.stdout:6/625: unlink da/f34 0 2024-02-16T22:56:25.454 INFO:tasks.workunit.client.1.smithi192.stdout:9/730: dwrite d7/d19/d28/f52 [0,4194304] 0 2024-02-16T22:56:25.455 INFO:tasks.workunit.client.0.smithi064.stdout:9/471: truncate d3/d11/d15/d5a/d2a/d2e/f39 5166696 0 2024-02-16T22:56:25.455 INFO:tasks.workunit.client.0.smithi064.stdout:9/472: chown d3/d9/d1e 3 1 2024-02-16T22:56:25.456 INFO:tasks.workunit.client.1.smithi192.stdout:9/731: chown d7/d15/d36/d77/dbd 128680332 1 2024-02-16T22:56:25.457 INFO:tasks.workunit.client.0.smithi064.stdout:9/473: write d3/d11/d15/d5a/f26 [3445345,108559] 0 2024-02-16T22:56:25.458 INFO:tasks.workunit.client.0.smithi064.stdout:0/450: mknod da/dc/d3b/d72/c94 0 2024-02-16T22:56:25.460 INFO:tasks.workunit.client.1.smithi192.stdout:2/650: fsync f19 0 2024-02-16T22:56:25.462 INFO:tasks.workunit.client.1.smithi192.stdout:5/581: write d6/d12/d1d/f3a [136803,38404] 0 2024-02-16T22:56:25.462 INFO:tasks.workunit.client.1.smithi192.stdout:7/697: mknod d1/d2/de/d2c/d49/d54/dd8/cf8 0 2024-02-16T22:56:25.466 INFO:tasks.workunit.client.1.smithi192.stdout:4/661: dread d1/d26/f77 [0,4194304] 0 2024-02-16T22:56:25.468 INFO:tasks.workunit.client.0.smithi064.stdout:2/591: unlink d7/de/d95/fa3 0 2024-02-16T22:56:25.469 INFO:tasks.workunit.client.0.smithi064.stdout:1/469: creat d2/df/d14/fa7 x:0 0 0 2024-02-16T22:56:25.470 INFO:tasks.workunit.client.0.smithi064.stdout:1/470: stat d2/c79 0 2024-02-16T22:56:25.470 INFO:tasks.workunit.client.0.smithi064.stdout:7/494: dread d5/f23 [0,4194304] 0 2024-02-16T22:56:25.471 INFO:tasks.workunit.client.0.smithi064.stdout:1/471: read - d2/df/d11/d25/f85 zero size 2024-02-16T22:56:25.472 INFO:tasks.workunit.client.1.smithi192.stdout:2/651: dwrite d22/d24/d39/d42/d65/d69/d4f/fae [4194304,4194304] 0 2024-02-16T22:56:25.474 INFO:tasks.workunit.client.1.smithi192.stdout:2/652: chown d22/d24/c26 98806450 1 2024-02-16T22:56:25.474 INFO:tasks.workunit.client.1.smithi192.stdout:2/653: stat f21 0 2024-02-16T22:56:25.476 INFO:tasks.workunit.client.1.smithi192.stdout:0/603: creat d1/d3/de/d22/d37/d67/d34/fbc x:0 0 0 2024-02-16T22:56:25.482 INFO:tasks.workunit.client.1.smithi192.stdout:8/581: rename d5/d26/d8f/d4d to d5/d26/d83/db4 0 2024-02-16T22:56:25.483 INFO:tasks.workunit.client.1.smithi192.stdout:8/582: read - d5/d26/f88 zero size 2024-02-16T22:56:25.484 INFO:tasks.workunit.client.1.smithi192.stdout:6/626: fsync da/df/d12/f9c 0 2024-02-16T22:56:25.485 INFO:tasks.workunit.client.1.smithi192.stdout:2/654: dwrite d22/d24/d39/fb7 [0,4194304] 0 2024-02-16T22:56:25.485 INFO:tasks.workunit.client.0.smithi064.stdout:6/564: creat d0/d5/df/d17/d18/fb3 x:0 0 0 2024-02-16T22:56:25.488 INFO:tasks.workunit.client.0.smithi064.stdout:4/527: unlink dc/d1d/f69 0 2024-02-16T22:56:25.493 INFO:tasks.workunit.client.0.smithi064.stdout:0/451: symlink da/dc/de/d12/d1d/l95 0 2024-02-16T22:56:25.496 INFO:tasks.workunit.client.0.smithi064.stdout:5/427: dread d6/d18/d19/d1a/f45 [0,4194304] 0 2024-02-16T22:56:25.496 INFO:tasks.workunit.client.1.smithi192.stdout:5/582: creat d6/d12/d4e/d5b/fbf x:0 0 0 2024-02-16T22:56:25.498 INFO:tasks.workunit.client.1.smithi192.stdout:1/569: mkdir d3/de/d27/d34/dbf/dc4 0 2024-02-16T22:56:25.499 INFO:tasks.workunit.client.1.smithi192.stdout:3/583: truncate da/f1a 1103946 0 2024-02-16T22:56:25.500 INFO:tasks.workunit.client.1.smithi192.stdout:4/662: dread d1/d15/d48/d8d/f7b [0,4194304] 0 2024-02-16T22:56:25.503 INFO:tasks.workunit.client.0.smithi064.stdout:7/495: dread d5/d6/f3b [0,4194304] 0 2024-02-16T22:56:25.503 INFO:tasks.workunit.client.0.smithi064.stdout:2/592: mknod d7/d14/d1e/d34/d65/cbb 0 2024-02-16T22:56:25.504 INFO:tasks.workunit.client.1.smithi192.stdout:0/604: mknod d1/d3/de/d22/cbd 0 2024-02-16T22:56:25.505 INFO:tasks.workunit.client.1.smithi192.stdout:3/584: dwrite da/d11/d63/d6b/d80/fb5 [0,4194304] 0 2024-02-16T22:56:25.505 INFO:tasks.workunit.client.0.smithi064.stdout:7/496: dwrite d5/d6/d65/f6d [0,4194304] 0 2024-02-16T22:56:25.506 INFO:tasks.workunit.client.1.smithi192.stdout:0/605: dread d1/d3/d29/f8d [0,4194304] 0 2024-02-16T22:56:25.506 INFO:tasks.workunit.client.0.smithi064.stdout:1/472: mkdir d2/df/da8 0 2024-02-16T22:56:25.510 INFO:tasks.workunit.client.0.smithi064.stdout:3/592: write d4/d13/d1b/d27/d99/f89 [803533,100300] 0 2024-02-16T22:56:25.515 INFO:tasks.workunit.client.0.smithi064.stdout:6/565: mkdir d0/d5/df/d17/d4a/db4 0 2024-02-16T22:56:25.518 INFO:tasks.workunit.client.0.smithi064.stdout:4/528: creat dc/dd/d32/d22/f9d x:0 0 0 2024-02-16T22:56:25.519 INFO:tasks.workunit.client.0.smithi064.stdout:9/474: creat d3/d89/d9a/f9b x:0 0 0 2024-02-16T22:56:25.524 INFO:tasks.workunit.client.0.smithi064.stdout:2/593: rmdir d7/d14/d1e/d34/d3f 39 2024-02-16T22:56:25.525 INFO:tasks.workunit.client.0.smithi064.stdout:2/594: chown d7/d14/d25/l94 19972 1 2024-02-16T22:56:25.526 INFO:tasks.workunit.client.0.smithi064.stdout:2/595: readlink d7/d14/d1e/l2b 0 2024-02-16T22:56:25.530 INFO:tasks.workunit.client.0.smithi064.stdout:1/473: read d2/df/d14/d2c/f5f [1209299,48590] 0 2024-02-16T22:56:25.532 INFO:tasks.workunit.client.0.smithi064.stdout:3/593: chown d4/d13/d1b/d27/d82/db0/lc4 51817168 1 2024-02-16T22:56:25.534 INFO:tasks.workunit.client.1.smithi192.stdout:8/583: dread d5/f38 [0,4194304] 0 2024-02-16T22:56:25.535 INFO:tasks.workunit.client.0.smithi064.stdout:8/483: rmdir df/d31/d2f/d4f/d6a/d92 0 2024-02-16T22:56:25.535 INFO:tasks.workunit.client.1.smithi192.stdout:8/584: write d5/d26/fa0 [284620,106558] 0 2024-02-16T22:56:25.537 INFO:tasks.workunit.client.0.smithi064.stdout:6/566: mknod d0/d3c/cb5 0 2024-02-16T22:56:25.541 INFO:tasks.workunit.client.0.smithi064.stdout:0/452: mkdir da/dc/de/d12/d2c/d2e/d96 0 2024-02-16T22:56:25.541 INFO:tasks.workunit.client.0.smithi064.stdout:0/453: chown da/dc/de/d12/d2c/d2e/l5b 1 1 2024-02-16T22:56:25.542 INFO:tasks.workunit.client.0.smithi064.stdout:9/475: dread - d3/d9/d1e/d3e/f65 zero size 2024-02-16T22:56:25.542 INFO:tasks.workunit.client.0.smithi064.stdout:0/454: read - da/d26/f82 zero size 2024-02-16T22:56:25.544 INFO:tasks.workunit.client.0.smithi064.stdout:0/455: write da/dc/de/d12/d2c/d2e/d38/f67 [486367,45163] 0 2024-02-16T22:56:25.546 INFO:tasks.workunit.client.0.smithi064.stdout:0/456: fdatasync da/dc/de/d12/d2c/f6e 0 2024-02-16T22:56:25.548 INFO:tasks.workunit.client.0.smithi064.stdout:2/596: creat d7/de/d95/d44/fbc x:0 0 0 2024-02-16T22:56:25.548 INFO:tasks.workunit.client.0.smithi064.stdout:0/457: write da/dc/de/f17 [2849218,103562] 0 2024-02-16T22:56:25.548 INFO:tasks.workunit.client.0.smithi064.stdout:2/597: write d7/f8 [411970,110075] 0 2024-02-16T22:56:25.553 INFO:tasks.workunit.client.0.smithi064.stdout:6/567: creat d0/d89/fb6 x:0 0 0 2024-02-16T22:56:25.553 INFO:tasks.workunit.client.0.smithi064.stdout:6/568: chown d0/d5/f42 19 1 2024-02-16T22:56:25.554 INFO:tasks.workunit.client.0.smithi064.stdout:6/569: fdatasync d0/d5/df/f86 0 2024-02-16T22:56:25.556 INFO:tasks.workunit.client.0.smithi064.stdout:4/529: link f3 dc/dd/d32/d46/d82/f9e 0 2024-02-16T22:56:25.561 INFO:tasks.workunit.client.1.smithi192.stdout:6/627: creat da/df/d71/dab/fcd x:0 0 0 2024-02-16T22:56:25.565 INFO:tasks.workunit.client.1.smithi192.stdout:5/583: fdatasync d6/d12/d4e/d5b/f5e 0 2024-02-16T22:56:25.567 INFO:tasks.workunit.client.0.smithi064.stdout:0/458: mknod da/dc/de/d12/d2c/d2e/d38/d7e/c97 0 2024-02-16T22:56:25.568 INFO:tasks.workunit.client.0.smithi064.stdout:0/459: read - da/dc/d3b/d5a/f73 zero size 2024-02-16T22:56:25.569 INFO:tasks.workunit.client.1.smithi192.stdout:5/584: dwrite d6/f9d [0,4194304] 0 2024-02-16T22:56:25.570 INFO:tasks.workunit.client.0.smithi064.stdout:2/598: creat d7/d14/d1e/d34/d73/db5/fbd x:0 0 0 2024-02-16T22:56:25.570 INFO:tasks.workunit.client.1.smithi192.stdout:5/585: stat d6/d8/d75/d8c 0 2024-02-16T22:56:25.571 INFO:tasks.workunit.client.1.smithi192.stdout:1/570: mknod d3/de/d27/d34/d69/d85/d89/cc5 0 2024-02-16T22:56:25.581 INFO:tasks.workunit.client.0.smithi064.stdout:3/594: write d4/d13/d1b/f68 [857586,76091] 0 2024-02-16T22:56:25.581 INFO:tasks.workunit.client.0.smithi064.stdout:6/570: write d0/d1d/da8/fac [2117657,21852] 0 2024-02-16T22:56:25.583 INFO:tasks.workunit.client.1.smithi192.stdout:0/606: unlink d1/d3/de/d22/d37/d67/d34/f56 0 2024-02-16T22:56:25.584 INFO:tasks.workunit.client.0.smithi064.stdout:5/428: creat d6/d1c/f93 x:0 0 0 2024-02-16T22:56:25.585 INFO:tasks.workunit.client.1.smithi192.stdout:5/586: dread d6/d12/f1e [4194304,4194304] 0 2024-02-16T22:56:25.590 INFO:tasks.workunit.client.0.smithi064.stdout:9/476: write d3/d9/d1e/d42/f58 [876752,84087] 0 2024-02-16T22:56:25.590 INFO:tasks.workunit.client.0.smithi064.stdout:0/460: symlink da/dc/d3b/d5a/l98 0 2024-02-16T22:56:25.594 INFO:tasks.workunit.client.0.smithi064.stdout:2/599: mkdir d7/d14/d25/d46/d55/d57/dbe 0 2024-02-16T22:56:25.594 INFO:tasks.workunit.client.0.smithi064.stdout:5/429: dwrite d6/d1c/f53 [0,4194304] 0 2024-02-16T22:56:25.594 INFO:tasks.workunit.client.1.smithi192.stdout:8/585: write d5/f84 [2831767,57630] 0 2024-02-16T22:56:25.594 INFO:tasks.workunit.client.1.smithi192.stdout:7/698: write d1/d2/de/d2c/f8b [915449,38671] 0 2024-02-16T22:56:25.596 INFO:tasks.workunit.client.1.smithi192.stdout:8/586: readlink d5/d26/d83/db4/l7b 0 2024-02-16T22:56:25.597 INFO:tasks.workunit.client.0.smithi064.stdout:4/530: link dc/d4a/f7d dc/de/d6d/f9f 0 2024-02-16T22:56:25.598 INFO:tasks.workunit.client.1.smithi192.stdout:8/587: write d5/d26/d83/d86/d5f/f9f [266583,58777] 0 2024-02-16T22:56:25.598 INFO:tasks.workunit.client.0.smithi064.stdout:9/477: dwrite d3/d11/d15/d5a/f29 [0,4194304] 0 2024-02-16T22:56:25.599 INFO:tasks.workunit.client.1.smithi192.stdout:8/588: truncate d5/d26/fa0 840111 0 2024-02-16T22:56:25.601 INFO:tasks.workunit.client.0.smithi064.stdout:3/595: write d4/f1e [175043,6368] 0 2024-02-16T22:56:25.603 INFO:tasks.workunit.client.0.smithi064.stdout:6/571: creat d0/d3c/fb7 x:0 0 0 2024-02-16T22:56:25.605 INFO:tasks.workunit.client.0.smithi064.stdout:7/497: link d5/d6/d54/f40 d5/d33/d45/f9a 0 2024-02-16T22:56:25.607 INFO:tasks.workunit.client.0.smithi064.stdout:7/498: write d5/d33/d53/f6e [1417733,40159] 0 2024-02-16T22:56:25.608 INFO:tasks.workunit.client.0.smithi064.stdout:7/499: chown d5/d33/f67 0 1 2024-02-16T22:56:25.613 INFO:tasks.workunit.client.1.smithi192.stdout:4/663: dwrite d1/d26/d34/f60 [0,4194304] 0 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: pgmap v17: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 70 MiB/s rd, 307 MiB/s wr, 514 op/s 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:25.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:56:25.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:25 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:56:25.619 INFO:tasks.workunit.client.1.smithi192.stdout:7/699: dread d1/d2/d2d/d61/d88/d90/d4c/f6b [0,4194304] 0 2024-02-16T22:56:25.637 INFO:tasks.workunit.client.0.smithi064.stdout:5/430: read d6/d25/d48/f4a [3010207,116193] 0 2024-02-16T22:56:25.642 INFO:tasks.workunit.client.0.smithi064.stdout:4/531: rmdir dc/dd/d32/d4b 39 2024-02-16T22:56:25.643 INFO:tasks.workunit.client.0.smithi064.stdout:2/600: dread d7/d14/d1e/d34/d3f/f4e [0,4194304] 0 2024-02-16T22:56:25.643 INFO:tasks.workunit.client.0.smithi064.stdout:4/532: chown dc/dd/d16/d6e/f7b 1 1 2024-02-16T22:56:25.645 INFO:tasks.workunit.client.1.smithi192.stdout:5/587: creat d6/d12/d2d/fc0 x:0 0 0 2024-02-16T22:56:25.645 INFO:tasks.workunit.client.0.smithi064.stdout:4/533: write dc/dd/d32/d22/f24 [4236406,106841] 0 2024-02-16T22:56:25.646 INFO:tasks.workunit.client.0.smithi064.stdout:9/478: mknod d3/d11/d15/d5a/d2a/d2e/d31/d3f/d80/c9c 0 2024-02-16T22:56:25.647 INFO:tasks.workunit.client.1.smithi192.stdout:9/732: getdents d7/d15/d36/d38/d3f/d8a/d98 0 2024-02-16T22:56:25.650 INFO:tasks.workunit.client.0.smithi064.stdout:6/572: rename d0/d1d/d21/c35 to d0/d1d/d21/cb8 0 2024-02-16T22:56:25.652 INFO:tasks.workunit.client.1.smithi192.stdout:2/655: rename d22/d24/d39/d42/d52/d7c/cb8 to d22/d24/d2b/cd2 0 2024-02-16T22:56:25.653 INFO:tasks.workunit.client.0.smithi064.stdout:7/500: rmdir d5/d33/d45/d73 39 2024-02-16T22:56:25.659 INFO:tasks.workunit.client.0.smithi064.stdout:9/479: dread d3/d11/d15/d5a/f26 [0,4194304] 0 2024-02-16T22:56:25.662 INFO:tasks.workunit.client.0.smithi064.stdout:5/431: creat d6/d1c/d4b/f94 x:0 0 0 2024-02-16T22:56:25.667 INFO:tasks.workunit.client.0.smithi064.stdout:5/432: truncate d6/d25/d48/d67/d80/f81 693111 0 2024-02-16T22:56:25.668 INFO:tasks.workunit.client.1.smithi192.stdout:7/700: creat d1/d2/d37/d86/ff9 x:0 0 0 2024-02-16T22:56:25.668 INFO:tasks.workunit.client.0.smithi064.stdout:9/480: dwrite d3/d9/d1e/d42/f58 [0,4194304] 0 2024-02-16T22:56:25.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/589: dread d5/d16/f45 [0,4194304] 0 2024-02-16T22:56:25.674 INFO:tasks.workunit.client.0.smithi064.stdout:4/534: rename dc/f33 to dc/dd/d4f/fa0 0 2024-02-16T22:56:25.675 INFO:tasks.workunit.client.1.smithi192.stdout:7/701: dwrite d1/d2/d2d/d61/d88/d90/f1d [8388608,4194304] 0 2024-02-16T22:56:25.675 INFO:tasks.workunit.client.0.smithi064.stdout:4/535: dread - dc/dd/d32/d22/f9d zero size 2024-02-16T22:56:25.676 INFO:tasks.workunit.client.0.smithi064.stdout:5/433: dwrite d6/d18/d19/f2e [0,4194304] 0 2024-02-16T22:56:25.679 INFO:tasks.workunit.client.1.smithi192.stdout:0/607: truncate d1/f3e 2998435 0 2024-02-16T22:56:25.679 INFO:tasks.workunit.client.0.smithi064.stdout:5/434: truncate d6/d25/d48/d67/f6c 536238 0 2024-02-16T22:56:25.681 INFO:tasks.workunit.client.1.smithi192.stdout:9/733: rmdir d7/d19/d5b 39 2024-02-16T22:56:25.682 INFO:tasks.workunit.client.0.smithi064.stdout:7/501: mknod d5/d33/c9b 0 2024-02-16T22:56:25.686 INFO:tasks.workunit.client.1.smithi192.stdout:3/585: rename da/d11/d63/d6b/d71/f98 to da/d11/d20/d39/d53/fce 0 2024-02-16T22:56:25.689 INFO:tasks.workunit.client.1.smithi192.stdout:2/656: creat d22/d24/d2b/d67/fd3 x:0 0 0 2024-02-16T22:56:25.691 INFO:tasks.workunit.client.0.smithi064.stdout:0/461: link da/dc/de/d12/d2c/d2e/f36 da/dc/d3b/d72/f99 0 2024-02-16T22:56:25.691 INFO:tasks.workunit.client.1.smithi192.stdout:2/657: dread d22/d24/d39/d42/d65/d66/fa4 [0,4194304] 0 2024-02-16T22:56:25.693 INFO:tasks.workunit.client.1.smithi192.stdout:6/628: creat da/df/fce x:0 0 0 2024-02-16T22:56:25.695 INFO:tasks.workunit.client.1.smithi192.stdout:8/590: rmdir d5/db/de/df 39 2024-02-16T22:56:25.698 INFO:tasks.workunit.client.0.smithi064.stdout:3/596: creat d4/d13/d1b/fcb x:0 0 0 2024-02-16T22:56:25.700 INFO:tasks.workunit.client.1.smithi192.stdout:5/588: mkdir d6/d12/d4e/d5b/d67/d78/dae/dbc/dc1 0 2024-02-16T22:56:25.700 INFO:tasks.workunit.client.0.smithi064.stdout:5/435: mkdir d6/d18/d95 0 2024-02-16T22:56:25.701 INFO:tasks.workunit.client.0.smithi064.stdout:5/436: chown d6/d25/d48/f5e 244166 1 2024-02-16T22:56:25.702 INFO:tasks.workunit.client.1.smithi192.stdout:7/702: dread d1/d2/f7e [0,4194304] 0 2024-02-16T22:56:25.703 INFO:tasks.workunit.client.1.smithi192.stdout:0/608: rmdir d1/d3/de/d22/d37/d67 39 2024-02-16T22:56:25.705 INFO:tasks.workunit.client.1.smithi192.stdout:9/734: dread - d7/d19/d28/f9f zero size 2024-02-16T22:56:25.708 INFO:tasks.workunit.client.0.smithi064.stdout:7/502: creat d5/d6/d65/d7f/f9c x:0 0 0 2024-02-16T22:56:25.709 INFO:tasks.workunit.client.0.smithi064.stdout:5/437: dwrite d6/d18/d39/f77 [0,4194304] 0 2024-02-16T22:56:25.711 INFO:tasks.workunit.client.0.smithi064.stdout:0/462: readlink da/l81 0 2024-02-16T22:56:25.716 INFO:tasks.workunit.client.1.smithi192.stdout:3/586: stat da/d11/c1b 0 2024-02-16T22:56:25.716 INFO:tasks.workunit.client.0.smithi064.stdout:2/601: creat d7/fbf x:0 0 0 2024-02-16T22:56:25.718 INFO:tasks.workunit.client.0.smithi064.stdout:4/536: mkdir dc/dd/d32/d46/d82/da1 0 2024-02-16T22:56:25.719 INFO:tasks.workunit.client.0.smithi064.stdout:4/537: fsync dc/dd/d16/f5e 0 2024-02-16T22:56:25.720 INFO:tasks.workunit.client.0.smithi064.stdout:0/463: dread da/dc/de/d37/f4e [0,4194304] 0 2024-02-16T22:56:25.721 INFO:tasks.workunit.client.0.smithi064.stdout:0/464: readlink da/dc/l23 0 2024-02-16T22:56:25.721 INFO:tasks.workunit.client.0.smithi064.stdout:0/465: fsync f9 0 2024-02-16T22:56:25.722 INFO:tasks.workunit.client.1.smithi192.stdout:6/629: rmdir da/df/d12/d57 39 2024-02-16T22:56:25.722 INFO:tasks.workunit.client.0.smithi064.stdout:0/466: chown da/d26/d3f/f47 298 1 2024-02-16T22:56:25.722 INFO:tasks.workunit.client.1.smithi192.stdout:6/630: chown da/df/d18/f63 6486014 1 2024-02-16T22:56:25.725 INFO:tasks.workunit.client.1.smithi192.stdout:5/589: rmdir d6/d8/d86 39 2024-02-16T22:56:25.728 INFO:tasks.workunit.client.0.smithi064.stdout:7/503: symlink d5/d6/d54/d41/l9d 0 2024-02-16T22:56:25.729 INFO:tasks.workunit.client.0.smithi064.stdout:6/573: dwrite d0/d1d/f84 [0,4194304] 0 2024-02-16T22:56:25.730 INFO:tasks.workunit.client.1.smithi192.stdout:7/703: creat d1/d2/de/d2c/d49/d89/ffa x:0 0 0 2024-02-16T22:56:25.732 INFO:tasks.workunit.client.1.smithi192.stdout:0/609: unlink d1/d3/de/d22/d37/d47/d53/f65 0 2024-02-16T22:56:25.733 INFO:tasks.workunit.client.1.smithi192.stdout:0/610: write d1/d3/de/f3c [353143,9010] 0 2024-02-16T22:56:25.736 INFO:tasks.workunit.client.1.smithi192.stdout:9/735: creat d7/d15/d36/d77/dbd/ffc x:0 0 0 2024-02-16T22:56:25.736 INFO:tasks.workunit.client.0.smithi064.stdout:4/538: rmdir dc/dd/d32/d58 39 2024-02-16T22:56:25.738 INFO:tasks.workunit.client.0.smithi064.stdout:4/539: read dc/dd/d4f/f5c [119530,28487] 0 2024-02-16T22:56:25.740 INFO:tasks.workunit.client.0.smithi064.stdout:0/467: rename da/d26/l68 to da/d26/d3f/l9a 0 2024-02-16T22:56:25.743 INFO:tasks.workunit.client.0.smithi064.stdout:5/438: mknod d6/c96 0 2024-02-16T22:56:25.744 INFO:tasks.workunit.client.0.smithi064.stdout:0/468: dread da/f10 [8388608,4194304] 0 2024-02-16T22:56:25.745 INFO:tasks.workunit.client.1.smithi192.stdout:4/664: getdents d1 0 2024-02-16T22:56:25.745 INFO:tasks.workunit.client.1.smithi192.stdout:4/665: chown d1/d26/d34/db9/fdd 26433744 1 2024-02-16T22:56:25.747 INFO:tasks.workunit.client.0.smithi064.stdout:2/602: mknod d7/d14/d25/d46/d55/d57/dbe/cc0 0 2024-02-16T22:56:25.749 INFO:tasks.workunit.client.0.smithi064.stdout:3/597: creat d4/d13/d1b/fcc x:0 0 0 2024-02-16T22:56:25.750 INFO:tasks.workunit.client.1.smithi192.stdout:7/704: mkdir d1/d2/de/d83/d9f/dfb 0 2024-02-16T22:56:25.754 INFO:tasks.workunit.client.0.smithi064.stdout:7/504: rename d5/d33/d45/d64/d56/d5d/f97 to d5/d33/d45/d64/f9e 0 2024-02-16T22:56:25.759 INFO:tasks.workunit.client.1.smithi192.stdout:1/571: rename d3/de/d27/d34/f45 to d3/de/fc6 0 2024-02-16T22:56:25.760 INFO:tasks.workunit.client.1.smithi192.stdout:1/572: dread - d3/de/d87/fa9 zero size 2024-02-16T22:56:25.760 INFO:tasks.workunit.client.1.smithi192.stdout:1/573: fdatasync d3/de/d27/f4a 0 2024-02-16T22:56:25.761 INFO:tasks.workunit.client.1.smithi192.stdout:9/736: symlink d7/d15/d36/d38/d3f/d8a/lfd 0 2024-02-16T22:56:25.762 INFO:tasks.workunit.client.0.smithi064.stdout:3/598: dread - d4/f84 zero size 2024-02-16T22:56:25.764 INFO:tasks.workunit.client.0.smithi064.stdout:4/540: mkdir dc/dd/d32/d46/d5a/d7e/d9a/da2 0 2024-02-16T22:56:25.765 INFO:tasks.workunit.client.1.smithi192.stdout:6/631: truncate da/df/d12/f90 861512 0 2024-02-16T22:56:25.768 INFO:tasks.workunit.client.0.smithi064.stdout:6/574: write d0/d1d/da8/d60/f6d [4314793,43768] 0 2024-02-16T22:56:25.769 INFO:tasks.workunit.client.0.smithi064.stdout:7/505: creat d5/d33/d53/f9f x:0 0 0 2024-02-16T22:56:25.770 INFO:tasks.workunit.client.1.smithi192.stdout:5/590: truncate d6/d8/d80/f40 754273 0 2024-02-16T22:56:25.771 INFO:tasks.workunit.client.0.smithi064.stdout:2/603: write d7/de/f56 [1227112,126210] 0 2024-02-16T22:56:25.772 INFO:tasks.workunit.client.0.smithi064.stdout:5/439: mknod d6/d1c/d43/d7d/c97 0 2024-02-16T22:56:25.773 INFO:tasks.workunit.client.1.smithi192.stdout:3/587: write da/d11/f2f [2249035,36532] 0 2024-02-16T22:56:25.778 INFO:tasks.workunit.client.0.smithi064.stdout:0/469: creat da/dc/de/d37/d88/f9b x:0 0 0 2024-02-16T22:56:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: pgmap v17: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 70 MiB/s rd, 307 MiB/s wr, 514 op/s 2024-02-16T22:56:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:56:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:25 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:56:25.786 INFO:tasks.workunit.client.0.smithi064.stdout:4/541: symlink dc/de/d6d/la3 0 2024-02-16T22:56:25.786 INFO:tasks.workunit.client.1.smithi192.stdout:8/591: rename d5/d9/l3c to d5/db/d47/d8c/d9d/lb5 0 2024-02-16T22:56:25.788 INFO:tasks.workunit.client.1.smithi192.stdout:1/574: fdatasync d3/de/d27/d34/d36/d47/f79 0 2024-02-16T22:56:25.791 INFO:tasks.workunit.client.1.smithi192.stdout:5/591: mkdir d6/d12/d4e/d58/dc2 0 2024-02-16T22:56:25.791 INFO:tasks.workunit.client.1.smithi192.stdout:5/592: chown d6/d12 0 1 2024-02-16T22:56:25.795 INFO:tasks.workunit.client.1.smithi192.stdout:4/666: link d1/d26/d66/d89/f97 d1/d15/d23/fea 0 2024-02-16T22:56:25.798 INFO:tasks.workunit.client.0.smithi064.stdout:3/599: creat d4/d13/d1b/d27/d82/db0/fcd x:0 0 0 2024-02-16T22:56:25.798 INFO:tasks.workunit.client.0.smithi064.stdout:6/575: write d0/d1b/f7a [1627791,112902] 0 2024-02-16T22:56:25.799 INFO:tasks.workunit.client.0.smithi064.stdout:6/576: dread - d0/d89/f93 zero size 2024-02-16T22:56:25.801 INFO:tasks.workunit.client.0.smithi064.stdout:3/600: truncate d4/d13/d1b/d27/f7f 5059760 0 2024-02-16T22:56:25.804 INFO:tasks.workunit.client.0.smithi064.stdout:7/506: dwrite d5/d6/fb [0,4194304] 0 2024-02-16T22:56:25.805 INFO:tasks.workunit.client.0.smithi064.stdout:5/440: rmdir d6/d25 39 2024-02-16T22:56:25.809 INFO:tasks.workunit.client.0.smithi064.stdout:0/470: truncate da/fb 53681 0 2024-02-16T22:56:25.816 INFO:tasks.workunit.client.0.smithi064.stdout:5/441: write d6/d18/d19/d1a/f69 [1546263,13855] 0 2024-02-16T22:56:25.819 INFO:tasks.workunit.client.0.smithi064.stdout:6/577: mkdir d0/d6/da/d56/db9 0 2024-02-16T22:56:25.821 INFO:tasks.workunit.client.0.smithi064.stdout:7/507: dread d5/d6/f1f [0,4194304] 0 2024-02-16T22:56:25.823 INFO:tasks.workunit.client.0.smithi064.stdout:2/604: getdents d7/d14/d25 0 2024-02-16T22:56:25.823 INFO:tasks.workunit.client.0.smithi064.stdout:3/601: dread d4/d13/d1b/f1c [0,4194304] 0 2024-02-16T22:56:25.824 INFO:tasks.workunit.client.0.smithi064.stdout:2/605: chown d7/de/d95 21666 1 2024-02-16T22:56:25.826 INFO:tasks.workunit.client.0.smithi064.stdout:4/542: getdents dc 0 2024-02-16T22:56:25.826 INFO:tasks.workunit.client.0.smithi064.stdout:3/602: read d4/d14/fc2 [336971,128007] 0 2024-02-16T22:56:25.826 INFO:tasks.workunit.client.0.smithi064.stdout:2/606: chown d7/d14/d1e/d34/c59 135 1 2024-02-16T22:56:25.832 INFO:tasks.workunit.client.0.smithi064.stdout:7/508: fdatasync d5/d6/d54/f78 0 2024-02-16T22:56:25.839 INFO:tasks.workunit.client.0.smithi064.stdout:3/603: write d4/d13/d1b/d27/f58 [1159801,94009] 0 2024-02-16T22:56:25.841 INFO:tasks.workunit.client.0.smithi064.stdout:6/578: truncate d0/d6/f15 4158102 0 2024-02-16T22:56:25.842 INFO:tasks.workunit.client.0.smithi064.stdout:3/604: write d4/d13/d1b/d27/d99/d7d/fc8 [179011,71837] 0 2024-02-16T22:56:25.842 INFO:tasks.workunit.client.0.smithi064.stdout:7/509: fdatasync d5/d33/d45/d64/f9e 0 2024-02-16T22:56:25.843 INFO:tasks.workunit.client.0.smithi064.stdout:2/607: creat d7/d14/d1e/d34/dad/fc1 x:0 0 0 2024-02-16T22:56:25.846 INFO:tasks.workunit.client.0.smithi064.stdout:5/442: rmdir d6/d18/d5d 0 2024-02-16T22:56:25.847 INFO:tasks.workunit.client.0.smithi064.stdout:2/608: chown d7/d14/d1e/d34/d3f/f79 185501 1 2024-02-16T22:56:25.850 INFO:tasks.workunit.client.0.smithi064.stdout:0/471: getdents da/dc/de/d12/d2c 0 2024-02-16T22:56:25.851 INFO:tasks.workunit.client.0.smithi064.stdout:4/543: link dc/dd/d4f/f5c dc/dd/d32/d4b/fa4 0 2024-02-16T22:56:25.855 INFO:tasks.workunit.client.0.smithi064.stdout:6/579: dwrite d0/d5/f42 [4194304,4194304] 0 2024-02-16T22:56:25.858 INFO:tasks.workunit.client.0.smithi064.stdout:3/605: dwrite d4/d13/d4d/fb8 [0,4194304] 0 2024-02-16T22:56:25.859 INFO:tasks.workunit.client.1.smithi192.stdout:7/705: link d1/de1/dd0/fd2 d1/d2/d2d/d92/d9b/ffc 0 2024-02-16T22:56:25.860 INFO:tasks.workunit.client.1.smithi192.stdout:8/592: mknod d5/d26/d83/db4/cb6 0 2024-02-16T22:56:25.860 INFO:tasks.workunit.client.1.smithi192.stdout:4/667: stat d1/d15/d23/cc9 0 2024-02-16T22:56:25.860 INFO:tasks.workunit.client.0.smithi064.stdout:0/472: readlink da/dc/de/d12/d2c/l58 0 2024-02-16T22:56:25.860 INFO:tasks.workunit.client.1.smithi192.stdout:5/593: creat d6/d20/d93/fc3 x:0 0 0 2024-02-16T22:56:25.861 INFO:tasks.workunit.client.1.smithi192.stdout:4/668: chown d1/d15/d29/c3f 871601 1 2024-02-16T22:56:25.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/669: stat d1/d26/d66/d89/d11/lda 0 2024-02-16T22:56:25.867 INFO:tasks.workunit.client.0.smithi064.stdout:5/443: creat d6/d25/d48/f98 x:0 0 0 2024-02-16T22:56:25.868 INFO:tasks.workunit.client.0.smithi064.stdout:6/580: dwrite d0/d89/fb6 [0,4194304] 0 2024-02-16T22:56:25.869 INFO:tasks.workunit.client.0.smithi064.stdout:1/474: sync 2024-02-16T22:56:25.870 INFO:tasks.workunit.client.1.smithi192.stdout:3/588: mknod da/d11/d63/d6b/d71/da1/db9/ccf 0 2024-02-16T22:56:25.870 INFO:tasks.workunit.client.0.smithi064.stdout:5/444: chown d6/d1c/f59 3 1 2024-02-16T22:56:25.875 INFO:tasks.workunit.client.0.smithi064.stdout:2/609: rename d7/d14/d25/d46/d55/d7f/l8e to d7/d14/d25/d46/d55/lc2 0 2024-02-16T22:56:25.875 INFO:tasks.workunit.client.0.smithi064.stdout:3/606: dwrite d4/d1f/d34/f45 [0,4194304] 0 2024-02-16T22:56:25.884 INFO:tasks.workunit.client.0.smithi064.stdout:7/510: dwrite d5/d6/f22 [0,4194304] 0 2024-02-16T22:56:25.884 INFO:tasks.workunit.client.1.smithi192.stdout:7/706: stat d1/de1/f7c 0 2024-02-16T22:56:25.886 INFO:tasks.workunit.client.0.smithi064.stdout:5/445: symlink d6/d25/d48/d67/l99 0 2024-02-16T22:56:25.891 INFO:tasks.workunit.client.1.smithi192.stdout:7/707: dwrite d1/d2/de/d83/d9f/feb [0,4194304] 0 2024-02-16T22:56:25.892 INFO:tasks.workunit.client.1.smithi192.stdout:5/594: dread d6/f29 [0,4194304] 0 2024-02-16T22:56:25.892 INFO:tasks.workunit.client.1.smithi192.stdout:4/670: rename d1/d26/d34/dca to d1/d15/d48/dc3/deb 0 2024-02-16T22:56:25.893 INFO:tasks.workunit.client.0.smithi064.stdout:0/473: rename da/dc/de/d37/f46 to da/dc/de/d37/d88/f9c 0 2024-02-16T22:56:25.896 INFO:tasks.workunit.client.1.smithi192.stdout:2/658: sync 2024-02-16T22:56:25.897 INFO:tasks.workunit.client.1.smithi192.stdout:0/611: link d1/c74 d1/d3/de/d22/d37/d67/d75/db0/cbe 0 2024-02-16T22:56:25.897 INFO:tasks.workunit.client.1.smithi192.stdout:2/659: readlink d22/l6f 0 2024-02-16T22:56:25.900 INFO:tasks.workunit.client.0.smithi064.stdout:4/544: creat dc/dd/d32/fa5 x:0 0 0 2024-02-16T22:56:25.901 INFO:tasks.workunit.client.0.smithi064.stdout:7/511: creat d5/d7d/fa0 x:0 0 0 2024-02-16T22:56:25.903 INFO:tasks.workunit.client.1.smithi192.stdout:8/593: creat d5/db/de/df/fb7 x:0 0 0 2024-02-16T22:56:25.903 INFO:tasks.workunit.client.0.smithi064.stdout:1/475: truncate d2/df/d14/d2c/d62/d91/f64 3497432 0 2024-02-16T22:56:25.904 INFO:tasks.workunit.client.1.smithi192.stdout:5/595: write d6/f29 [328064,4114] 0 2024-02-16T22:56:25.906 INFO:tasks.workunit.client.1.smithi192.stdout:8/594: write d5/db/de/f80 [368710,116901] 0 2024-02-16T22:56:25.907 INFO:tasks.workunit.client.1.smithi192.stdout:4/671: rmdir d1/d26/d66/d89/d11 39 2024-02-16T22:56:25.907 INFO:tasks.workunit.client.1.smithi192.stdout:3/589: fsync da/d11/d63/d6b/d71/f7b 0 2024-02-16T22:56:25.909 INFO:tasks.workunit.client.0.smithi064.stdout:2/610: creat d7/d14/d4b/d63/fc3 x:0 0 0 2024-02-16T22:56:25.910 INFO:tasks.workunit.client.1.smithi192.stdout:7/708: creat d1/d2/d2d/d61/d88/d90/d4c/d60/ffd x:0 0 0 2024-02-16T22:56:25.910 INFO:tasks.workunit.client.0.smithi064.stdout:2/611: dread d7/de/f8d [0,4194304] 0 2024-02-16T22:56:25.911 INFO:tasks.workunit.client.1.smithi192.stdout:2/660: chown d22/d24/c8b 25750323 1 2024-02-16T22:56:25.911 INFO:tasks.workunit.client.1.smithi192.stdout:5/596: dwrite d6/d12/d4e/d5b/d67/d78/fb4 [0,4194304] 0 2024-02-16T22:56:25.912 INFO:tasks.workunit.client.1.smithi192.stdout:0/612: symlink d1/d3/d39/lbf 0 2024-02-16T22:56:25.914 INFO:tasks.workunit.client.1.smithi192.stdout:2/661: write d22/d24/d39/d42/d52/f60 [1953746,106479] 0 2024-02-16T22:56:25.915 INFO:tasks.workunit.client.0.smithi064.stdout:6/581: link d0/d12/d7b/f99 d0/d6/da/fba 0 2024-02-16T22:56:25.918 INFO:tasks.workunit.client.0.smithi064.stdout:1/476: mknod d2/df/d14/d18/d48/d84/ca9 0 2024-02-16T22:56:25.926 INFO:tasks.workunit.client.0.smithi064.stdout:2/612: fsync d7/d14/d1e/d34/f50 0 2024-02-16T22:56:25.927 INFO:tasks.workunit.client.1.smithi192.stdout:3/590: truncate da/d11/d63/f9e 1017145 0 2024-02-16T22:56:25.927 INFO:tasks.workunit.client.1.smithi192.stdout:3/591: stat da/d11/d35/cbb 0 2024-02-16T22:56:25.928 INFO:tasks.workunit.client.1.smithi192.stdout:3/592: dread - da/d11/d20/d39/d53/dc7/f88 zero size 2024-02-16T22:56:25.931 INFO:tasks.workunit.client.1.smithi192.stdout:7/709: unlink d1/d2/de/d2c/d49/da1/lb1 0 2024-02-16T22:56:25.932 INFO:tasks.workunit.client.1.smithi192.stdout:7/710: chown d1/d2/d2d/d61/d8a/fde 481980 1 2024-02-16T22:56:25.933 INFO:tasks.workunit.client.0.smithi064.stdout:2/613: dwrite d7/d14/d1e/d34/f61 [0,4194304] 0 2024-02-16T22:56:25.937 INFO:tasks.workunit.client.1.smithi192.stdout:5/597: rename d6/d12/d4e/da5 to d6/d8/d75/d8c/dc4 0 2024-02-16T22:56:25.938 INFO:tasks.workunit.client.0.smithi064.stdout:3/607: rename d4/f18 to d4/d14/fce 0 2024-02-16T22:56:25.939 INFO:tasks.workunit.client.0.smithi064.stdout:3/608: dread d4/d13/d4d/d5c/f6c [0,4194304] 0 2024-02-16T22:56:25.939 INFO:tasks.workunit.client.1.smithi192.stdout:0/613: creat d1/d3/de/d22/fc0 x:0 0 0 2024-02-16T22:56:25.940 INFO:tasks.workunit.client.0.smithi064.stdout:4/545: creat dc/dd/d16/fa6 x:0 0 0 2024-02-16T22:56:25.941 INFO:tasks.workunit.client.1.smithi192.stdout:2/662: creat d22/d24/d39/d42/d84/fd4 x:0 0 0 2024-02-16T22:56:25.942 INFO:tasks.workunit.client.0.smithi064.stdout:4/546: chown dc/dd/d32/d46/d82/f9e 82 1 2024-02-16T22:56:25.943 INFO:tasks.workunit.client.1.smithi192.stdout:8/595: symlink d5/db/lb8 0 2024-02-16T22:56:25.946 INFO:tasks.workunit.client.0.smithi064.stdout:3/609: dwrite d4/d13/d1b/d27/d99/d7d/fc8 [0,4194304] 0 2024-02-16T22:56:25.950 INFO:tasks.workunit.client.0.smithi064.stdout:2/614: rmdir d7/de 39 2024-02-16T22:56:25.951 INFO:tasks.workunit.client.0.smithi064.stdout:7/512: dwrite d5/d33/d45/f9a [0,4194304] 0 2024-02-16T22:56:25.953 INFO:tasks.workunit.client.0.smithi064.stdout:7/513: fdatasync d5/d33/f60 0 2024-02-16T22:56:25.954 INFO:tasks.workunit.client.0.smithi064.stdout:7/514: fsync d5/d6/d54/d41/f7e 0 2024-02-16T22:56:25.956 INFO:tasks.workunit.client.1.smithi192.stdout:0/614: creat d1/d3/de/d22/d37/d67/d34/fc1 x:0 0 0 2024-02-16T22:56:25.960 INFO:tasks.workunit.client.1.smithi192.stdout:2/663: mknod d22/d47/dcb/d3f/dac/cd5 0 2024-02-16T22:56:25.962 INFO:tasks.workunit.client.1.smithi192.stdout:8/596: rename d5/d16/f45 to d5/d26/d83/fb9 0 2024-02-16T22:56:25.963 INFO:tasks.workunit.client.1.smithi192.stdout:8/597: write d5/d26/d83/d86/f41 [5229730,97995] 0 2024-02-16T22:56:25.965 INFO:tasks.workunit.client.1.smithi192.stdout:0/615: mknod d1/d3/de/d22/cc2 0 2024-02-16T22:56:25.967 INFO:tasks.workunit.client.0.smithi064.stdout:2/615: creat d7/d14/d1e/d34/d73/fc4 x:0 0 0 2024-02-16T22:56:25.969 INFO:tasks.workunit.client.1.smithi192.stdout:7/711: dwrite d1/d2/d37/f9c [0,4194304] 0 2024-02-16T22:56:25.970 INFO:tasks.workunit.client.1.smithi192.stdout:8/598: truncate d5/d9/f6b 331312 0 2024-02-16T22:56:25.970 INFO:tasks.workunit.client.1.smithi192.stdout:7/712: fsync d1/d2/d37/d86/ff9 0 2024-02-16T22:56:25.972 INFO:tasks.workunit.client.1.smithi192.stdout:7/713: write d1/d2/f46 [3293351,130684] 0 2024-02-16T22:56:25.973 INFO:tasks.workunit.client.0.smithi064.stdout:7/515: creat d5/d33/d45/d64/d56/d66/fa1 x:0 0 0 2024-02-16T22:56:25.973 INFO:tasks.workunit.client.0.smithi064.stdout:7/516: stat d5/d33/d45/d64/c62 0 2024-02-16T22:56:25.976 INFO:tasks.workunit.client.1.smithi192.stdout:5/598: dread d6/d12/f18 [4194304,4194304] 0 2024-02-16T22:56:25.981 INFO:tasks.workunit.client.1.smithi192.stdout:8/599: rename d5/d26/d83/d86/d5f/f8d to d5/d26/d83/db4/fba 0 2024-02-16T22:56:25.987 INFO:tasks.workunit.client.1.smithi192.stdout:0/616: truncate d1/fa 3850745 0 2024-02-16T22:56:25.987 INFO:tasks.workunit.client.0.smithi064.stdout:2/616: symlink d7/d14/d1e/lc5 0 2024-02-16T22:56:25.988 INFO:tasks.workunit.client.1.smithi192.stdout:5/599: creat d6/d20/fc5 x:0 0 0 2024-02-16T22:56:25.988 INFO:tasks.workunit.client.0.smithi064.stdout:2/617: write d7/d14/d25/f7c [176065,25739] 0 2024-02-16T22:56:25.994 INFO:tasks.workunit.client.0.smithi064.stdout:1/477: getdents d2/df/d11/d1a 0 2024-02-16T22:56:25.994 INFO:tasks.workunit.client.0.smithi064.stdout:7/517: mkdir d5/d6/d54/d41/da2 0 2024-02-16T22:56:25.994 INFO:tasks.workunit.client.1.smithi192.stdout:2/664: dread d22/d24/d39/d42/d65/d69/d4f/fae [0,4194304] 0 2024-02-16T22:56:25.995 INFO:tasks.workunit.client.0.smithi064.stdout:7/518: dread - d5/d33/d45/d64/f9e zero size 2024-02-16T22:56:25.995 INFO:tasks.workunit.client.1.smithi192.stdout:7/714: rename d1/d2/d37/d86/ldb to d1/d2/de/d2c/d49/d54/dd8/lfe 0 2024-02-16T22:56:25.996 INFO:tasks.workunit.client.0.smithi064.stdout:7/519: dread - d5/d33/d53/f70 zero size 2024-02-16T22:56:25.996 INFO:tasks.workunit.client.1.smithi192.stdout:7/715: dread - d1/d2/de/d2c/d49/da4/fcf zero size 2024-02-16T22:56:25.997 INFO:tasks.workunit.client.1.smithi192.stdout:8/600: creat d5/d68/fbb x:0 0 0 2024-02-16T22:56:25.997 INFO:tasks.workunit.client.1.smithi192.stdout:7/716: read - d1/d2/de/d2c/d49/d54/de3/fd3 zero size 2024-02-16T22:56:25.999 INFO:tasks.workunit.client.1.smithi192.stdout:5/600: mknod d6/d8/d77/cc6 0 2024-02-16T22:56:26.004 INFO:tasks.workunit.client.1.smithi192.stdout:8/601: write d5/d9/f69 [3141872,16457] 0 2024-02-16T22:56:26.013 INFO:tasks.workunit.client.1.smithi192.stdout:7/717: symlink d1/d2/de/d2c/d49/d89/dbd/lff 0 2024-02-16T22:56:26.017 INFO:tasks.workunit.client.1.smithi192.stdout:5/601: creat d6/d12/fc7 x:0 0 0 2024-02-16T22:56:26.018 INFO:tasks.workunit.client.1.smithi192.stdout:5/602: write d6/d12/d2d/f96 [100488,84876] 0 2024-02-16T22:56:26.018 INFO:tasks.workunit.client.1.smithi192.stdout:5/603: stat d6/d8/l31 0 2024-02-16T22:56:26.026 INFO:tasks.workunit.client.1.smithi192.stdout:8/602: fdatasync d5/db/de/f37 0 2024-02-16T22:56:26.027 INFO:tasks.workunit.client.1.smithi192.stdout:7/718: creat d1/d2/de/d2c/d49/d54/de3/f100 x:0 0 0 2024-02-16T22:56:26.032 INFO:tasks.workunit.client.1.smithi192.stdout:0/617: truncate d1/d3/de/d22/d37/d67/d75/f80 1494590 0 2024-02-16T22:56:26.033 INFO:tasks.workunit.client.1.smithi192.stdout:2/665: rmdir d22/d47/dcb/d3f/dd0 0 2024-02-16T22:56:26.041 INFO:tasks.workunit.client.1.smithi192.stdout:7/719: unlink d1/d2/de/d2c/d49/da4/c65 0 2024-02-16T22:56:26.043 INFO:tasks.workunit.client.1.smithi192.stdout:5/604: rename d6/d8/d86/f89 to d6/d12/d1d/d56/fc8 0 2024-02-16T22:56:26.045 INFO:tasks.workunit.client.1.smithi192.stdout:0/618: mknod d1/d3/de/d22/d37/d67/cc3 0 2024-02-16T22:56:26.045 INFO:tasks.workunit.client.0.smithi064.stdout:8/484: sync 2024-02-16T22:56:26.046 INFO:tasks.workunit.client.0.smithi064.stdout:9/481: sync 2024-02-16T22:56:26.046 INFO:tasks.workunit.client.0.smithi064.stdout:9/482: fdatasync d3/d9/d1e/f8d 0 2024-02-16T22:56:26.049 INFO:tasks.workunit.client.1.smithi192.stdout:7/720: fsync d1/de1/fb8 0 2024-02-16T22:56:26.054 INFO:tasks.workunit.client.0.smithi064.stdout:0/474: rename da/c24 to da/dc/d3b/c9d 0 2024-02-16T22:56:26.056 INFO:tasks.workunit.client.0.smithi064.stdout:7/520: mkdir d5/d33/d53/da3 0 2024-02-16T22:56:26.057 INFO:tasks.workunit.client.0.smithi064.stdout:7/521: dread - d5/d33/d45/d64/f9e zero size 2024-02-16T22:56:26.058 INFO:tasks.workunit.client.0.smithi064.stdout:7/522: chown d5/d33/c9b 0 1 2024-02-16T22:56:26.059 INFO:tasks.workunit.client.0.smithi064.stdout:4/547: getdents dc/d1d 0 2024-02-16T22:56:26.060 INFO:tasks.workunit.client.1.smithi192.stdout:2/666: unlink d22/d47/dcb/d3f/l55 0 2024-02-16T22:56:26.061 INFO:tasks.workunit.client.0.smithi064.stdout:3/610: getdents d4/d1f/d5b/d51/d7c 0 2024-02-16T22:56:26.062 INFO:tasks.workunit.client.0.smithi064.stdout:3/611: chown d4/d14/f2a 942 1 2024-02-16T22:56:26.063 INFO:tasks.workunit.client.1.smithi192.stdout:9/737: sync 2024-02-16T22:56:26.065 INFO:tasks.workunit.client.0.smithi064.stdout:2/618: read d7/de/f11 [825220,60649] 0 2024-02-16T22:56:26.070 INFO:tasks.workunit.client.0.smithi064.stdout:3/612: dwrite d4/f17 [0,4194304] 0 2024-02-16T22:56:26.074 INFO:tasks.workunit.client.0.smithi064.stdout:2/619: dwrite d7/f54 [0,4194304] 0 2024-02-16T22:56:26.075 INFO:tasks.workunit.client.0.smithi064.stdout:7/523: rmdir d5/d33/d42 39 2024-02-16T22:56:26.075 INFO:tasks.workunit.client.1.smithi192.stdout:5/605: dread d6/d12/d4e/d58/d8f/fa9 [0,4194304] 0 2024-02-16T22:56:26.076 INFO:tasks.workunit.client.0.smithi064.stdout:7/524: readlink d5/d33/d45/d64/l5f 0 2024-02-16T22:56:26.080 INFO:tasks.workunit.client.0.smithi064.stdout:7/525: chown d5/d33/d45/f98 4 1 2024-02-16T22:56:26.080 INFO:tasks.workunit.client.1.smithi192.stdout:8/603: dwrite d5/d9/f73 [0,4194304] 0 2024-02-16T22:56:26.081 INFO:tasks.workunit.client.0.smithi064.stdout:4/548: creat dc/dd/d4f/fa7 x:0 0 0 2024-02-16T22:56:26.084 INFO:tasks.workunit.client.1.smithi192.stdout:9/738: rename d7/d19/d37/f3d to d7/d15/d36/dbe/ffe 0 2024-02-16T22:56:26.087 INFO:tasks.workunit.client.1.smithi192.stdout:2/667: getdents d22/d24/d39/d42/d85/d98/db0/dcd 0 2024-02-16T22:56:26.089 INFO:tasks.workunit.client.1.smithi192.stdout:8/604: dwrite d5/db/de/df/f62 [0,4194304] 0 2024-02-16T22:56:26.094 INFO:tasks.workunit.client.0.smithi064.stdout:4/549: dwrite dc/d1d/f8b [0,4194304] 0 2024-02-16T22:56:26.094 INFO:tasks.workunit.client.1.smithi192.stdout:5/606: mknod d6/d12/d4e/d58/d8f/cc9 0 2024-02-16T22:56:26.094 INFO:tasks.workunit.client.1.smithi192.stdout:5/607: stat d6/d12/d2d/l5a 0 2024-02-16T22:56:26.097 INFO:tasks.workunit.client.0.smithi064.stdout:3/613: readlink d4/d13/d4d/d5c/lca 0 2024-02-16T22:56:26.099 INFO:tasks.workunit.client.0.smithi064.stdout:6/582: dread d0/d6/d10/f1c [0,4194304] 0 2024-02-16T22:56:26.099 INFO:tasks.workunit.client.0.smithi064.stdout:2/620: creat d7/d14/d4b/fc6 x:0 0 0 2024-02-16T22:56:26.101 INFO:tasks.workunit.client.0.smithi064.stdout:7/526: write d5/d33/d42/d81/f94 [932899,101145] 0 2024-02-16T22:56:26.104 INFO:tasks.workunit.client.1.smithi192.stdout:0/619: rename d1/f8 to d1/d3/d29/d43/d9c/fc4 0 2024-02-16T22:56:26.107 INFO:tasks.workunit.client.1.smithi192.stdout:9/739: mkdir d7/d15/d36/d77/dff 0 2024-02-16T22:56:26.109 INFO:tasks.workunit.client.0.smithi064.stdout:9/483: creat d3/d11/d15/d5a/d2a/d2e/f9d x:0 0 0 2024-02-16T22:56:26.111 INFO:tasks.workunit.client.0.smithi064.stdout:1/478: dread d2/df/d11/d1a/d30/f63 [0,4194304] 0 2024-02-16T22:56:26.115 INFO:tasks.workunit.client.0.smithi064.stdout:8/485: rename df/d31/d5e/f64 to df/d2a/fa3 0 2024-02-16T22:56:26.116 INFO:tasks.workunit.client.0.smithi064.stdout:8/486: fdatasync df/f1a 0 2024-02-16T22:56:26.120 INFO:tasks.workunit.client.1.smithi192.stdout:8/605: unlink d5/db/d47/d43/d67/f2f 0 2024-02-16T22:56:26.122 INFO:tasks.workunit.client.0.smithi064.stdout:3/614: unlink d4/d1f/d5b/d51/l80 0 2024-02-16T22:56:26.122 INFO:tasks.workunit.client.1.smithi192.stdout:5/608: unlink d6/d12/d2d/d5c/f65 0 2024-02-16T22:56:26.123 INFO:tasks.workunit.client.0.smithi064.stdout:6/583: symlink d0/d5/df/lbb 0 2024-02-16T22:56:26.124 INFO:tasks.workunit.client.1.smithi192.stdout:5/609: truncate d6/d12/fc7 846125 0 2024-02-16T22:56:26.125 INFO:tasks.workunit.client.1.smithi192.stdout:2/668: rename d22/f2a to d22/d24/d39/d42/d65/d66/fd6 0 2024-02-16T22:56:26.126 INFO:tasks.workunit.client.0.smithi064.stdout:5/446: sync 2024-02-16T22:56:26.129 INFO:tasks.workunit.client.1.smithi192.stdout:5/610: dwrite d6/d12/d2d/d5c/f99 [0,4194304] 0 2024-02-16T22:56:26.129 INFO:tasks.workunit.client.1.smithi192.stdout:9/740: mkdir d7/d15/d36/d38/d3f/d8a/def/d100 0 2024-02-16T22:56:26.131 INFO:tasks.workunit.client.0.smithi064.stdout:9/484: mkdir d3/d11/d9e 0 2024-02-16T22:56:26.136 INFO:tasks.workunit.client.1.smithi192.stdout:5/611: dwrite d6/f33 [4194304,4194304] 0 2024-02-16T22:56:26.137 INFO:tasks.workunit.client.0.smithi064.stdout:1/479: mknod d2/df/d11/caa 0 2024-02-16T22:56:26.137 INFO:tasks.workunit.client.0.smithi064.stdout:8/487: dread f3 [0,4194304] 0 2024-02-16T22:56:26.137 INFO:tasks.workunit.client.1.smithi192.stdout:5/612: dread - d6/d8/d75/fac zero size 2024-02-16T22:56:26.138 INFO:tasks.workunit.client.0.smithi064.stdout:8/488: chown df/l18 15426956 1 2024-02-16T22:56:26.138 INFO:tasks.workunit.client.1.smithi192.stdout:7/721: getdents d1/d2 0 2024-02-16T22:56:26.139 INFO:tasks.workunit.client.0.smithi064.stdout:8/489: truncate df/d31/d2f/f59 1313752 0 2024-02-16T22:56:26.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/632: sync 2024-02-16T22:56:26.144 INFO:tasks.workunit.client.1.smithi192.stdout:8/606: mknod d5/d9/cbc 0 2024-02-16T22:56:26.147 INFO:tasks.workunit.client.0.smithi064.stdout:2/621: rename d7/d14/d3e/f5d to d7/d14/d4b/fc7 0 2024-02-16T22:56:26.151 INFO:tasks.workunit.client.1.smithi192.stdout:2/669: dread d22/d47/dcb/f43 [0,4194304] 0 2024-02-16T22:56:26.153 INFO:tasks.workunit.client.1.smithi192.stdout:2/670: fdatasync d22/d24/d39/d42/d65/d69/d4f/f97 0 2024-02-16T22:56:26.153 INFO:tasks.workunit.client.0.smithi064.stdout:2/622: dread d7/de/d95/d44/f6d [0,4194304] 0 2024-02-16T22:56:26.153 INFO:tasks.workunit.client.1.smithi192.stdout:2/671: chown d22/d24/d39/d42 520246318 1 2024-02-16T22:56:26.155 INFO:tasks.workunit.client.0.smithi064.stdout:4/550: mkdir dc/dd/d32/d46/d79/da8 0 2024-02-16T22:56:26.156 INFO:tasks.workunit.client.0.smithi064.stdout:4/551: chown dc/dd/d32/d22/f52 4744978 1 2024-02-16T22:56:26.158 INFO:tasks.workunit.client.0.smithi064.stdout:3/615: creat d4/d13/d4d/d5c/fcf x:0 0 0 2024-02-16T22:56:26.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/672: dwrite d22/d24/d39/d42/d65/d69/d4f/fbf [0,4194304] 0 2024-02-16T22:56:26.159 INFO:tasks.workunit.client.1.smithi192.stdout:9/741: mkdir d7/d15/d36/d77/dbd/d101 0 2024-02-16T22:56:26.162 INFO:tasks.workunit.client.0.smithi064.stdout:6/584: mkdir d0/d1d/dbc 0 2024-02-16T22:56:26.166 INFO:tasks.workunit.client.0.smithi064.stdout:7/527: creat d5/d33/d45/d73/fa4 x:0 0 0 2024-02-16T22:56:26.166 INFO:tasks.workunit.client.1.smithi192.stdout:7/722: dread d1/f12 [0,4194304] 0 2024-02-16T22:56:26.168 INFO:tasks.workunit.client.1.smithi192.stdout:0/620: fdatasync d1/d3/d29/d43/d9c/fc4 0 2024-02-16T22:56:26.169 INFO:tasks.workunit.client.0.smithi064.stdout:3/616: dwrite d4/d13/d1b/d27/f58 [0,4194304] 0 2024-02-16T22:56:26.173 INFO:tasks.workunit.client.0.smithi064.stdout:0/475: mkdir da/dc/de/d12/d2c/d2e/d38/d7e/d89/d9e 0 2024-02-16T22:56:26.173 INFO:tasks.workunit.client.1.smithi192.stdout:0/621: truncate d1/d3/de/d22/d37/d67/d34/f7f 489406 0 2024-02-16T22:56:26.174 INFO:tasks.workunit.client.1.smithi192.stdout:6/633: rmdir da/df/d18/d52 39 2024-02-16T22:56:26.174 INFO:tasks.workunit.client.1.smithi192.stdout:7/723: dwrite d1/d2/de/d2c/d49/d54/de3/fd3 [0,4194304] 0 2024-02-16T22:56:26.176 INFO:tasks.workunit.client.0.smithi064.stdout:3/617: dwrite d4/d13/d1b/d27/d99/f87 [0,4194304] 0 2024-02-16T22:56:26.177 INFO:tasks.workunit.client.1.smithi192.stdout:7/724: dread - d1/d2/de/d83/d9f/fa2 zero size 2024-02-16T22:56:26.181 INFO:tasks.workunit.client.0.smithi064.stdout:8/490: mknod df/d31/d2f/ca4 0 2024-02-16T22:56:26.194 INFO:tasks.workunit.client.1.smithi192.stdout:9/742: fsync f2 0 2024-02-16T22:56:26.198 INFO:tasks.workunit.client.0.smithi064.stdout:4/552: dread - dc/dd/d32/d58/f8e zero size 2024-02-16T22:56:26.201 INFO:tasks.workunit.client.0.smithi064.stdout:6/585: mknod d0/d12/d7b/cbd 0 2024-02-16T22:56:26.202 INFO:tasks.workunit.client.0.smithi064.stdout:7/528: symlink d5/d6/d65/d7f/la5 0 2024-02-16T22:56:26.203 INFO:tasks.workunit.client.1.smithi192.stdout:0/622: chown d1/f24 2088738 1 2024-02-16T22:56:26.203 INFO:tasks.workunit.client.0.smithi064.stdout:9/485: dwrite d3/d9/d1e/d3e/f65 [0,4194304] 0 2024-02-16T22:56:26.205 INFO:tasks.workunit.client.0.smithi064.stdout:5/447: symlink d6/d25/d48/d67/d91/l9a 0 2024-02-16T22:56:26.205 INFO:tasks.workunit.client.0.smithi064.stdout:6/586: write d0/d6/d10/f40 [6466947,92859] 0 2024-02-16T22:56:26.207 INFO:tasks.workunit.client.1.smithi192.stdout:7/725: dread d1/d2/de/d2c/d49/da4/fa3 [0,4194304] 0 2024-02-16T22:56:26.209 INFO:tasks.workunit.client.1.smithi192.stdout:4/672: sync 2024-02-16T22:56:26.209 INFO:tasks.workunit.client.1.smithi192.stdout:1/575: sync 2024-02-16T22:56:26.209 INFO:tasks.workunit.client.1.smithi192.stdout:3/593: sync 2024-02-16T22:56:26.217 INFO:tasks.workunit.client.1.smithi192.stdout:4/673: dwrite d1/d15/d29/fdf [0,4194304] 0 2024-02-16T22:56:26.218 INFO:tasks.workunit.client.0.smithi064.stdout:0/476: dwrite da/d26/f33 [0,4194304] 0 2024-02-16T22:56:26.222 INFO:tasks.workunit.client.0.smithi064.stdout:2/623: dread d7/de/f1f [0,4194304] 0 2024-02-16T22:56:26.222 INFO:tasks.workunit.client.0.smithi064.stdout:8/491: fsync df/d31/d2f/d38/f39 0 2024-02-16T22:56:26.228 INFO:tasks.workunit.client.0.smithi064.stdout:7/529: dread d5/d33/d53/f6e [0,4194304] 0 2024-02-16T22:56:26.228 INFO:tasks.workunit.client.0.smithi064.stdout:7/530: stat d5/d6/l69 0 2024-02-16T22:56:26.231 INFO:tasks.workunit.client.1.smithi192.stdout:8/607: write d5/db/d47/d8c/fab [9146407,47243] 0 2024-02-16T22:56:26.234 INFO:tasks.workunit.client.0.smithi064.stdout:4/553: rename dc/de/d6d/l90 to dc/dd/d16/d7c/la9 0 2024-02-16T22:56:26.237 INFO:tasks.workunit.client.1.smithi192.stdout:0/623: dread d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:26.237 INFO:tasks.workunit.client.0.smithi064.stdout:8/492: dread df/d31/f29 [0,4194304] 0 2024-02-16T22:56:26.237 INFO:tasks.workunit.client.0.smithi064.stdout:7/531: dwrite d5/d33/d45/d73/d88/f90 [0,4194304] 0 2024-02-16T22:56:26.238 INFO:tasks.workunit.client.1.smithi192.stdout:2/673: write d22/d24/d39/d42/d65/d66/fd6 [470409,89795] 0 2024-02-16T22:56:26.239 INFO:tasks.workunit.client.1.smithi192.stdout:2/674: chown f10 15127713 1 2024-02-16T22:56:26.239 INFO:tasks.workunit.client.0.smithi064.stdout:7/532: fsync d5/d33/d45/d64/f9e 0 2024-02-16T22:56:26.244 INFO:tasks.workunit.client.0.smithi064.stdout:5/448: chown d6/d18/d19/l57 396 1 2024-02-16T22:56:26.245 INFO:tasks.workunit.client.0.smithi064.stdout:9/486: creat d3/d9/d1e/d42/f9f x:0 0 0 2024-02-16T22:56:26.245 INFO:tasks.workunit.client.1.smithi192.stdout:1/576: truncate d3/de/f1a 501884 0 2024-02-16T22:56:26.246 INFO:tasks.workunit.client.0.smithi064.stdout:9/487: write d3/d9/f12 [2259843,60733] 0 2024-02-16T22:56:26.247 INFO:tasks.workunit.client.0.smithi064.stdout:6/587: creat d0/d5/df/d17/fbe x:0 0 0 2024-02-16T22:56:26.248 INFO:tasks.workunit.client.1.smithi192.stdout:7/726: dread d1/de1/fb8 [0,4194304] 0 2024-02-16T22:56:26.248 INFO:tasks.workunit.client.1.smithi192.stdout:7/727: write d1/d2/d37/d86/ff9 [176234,3248] 0 2024-02-16T22:56:26.254 INFO:tasks.workunit.client.1.smithi192.stdout:7/728: dread d1/d2/de/f34 [0,4194304] 0 2024-02-16T22:56:26.259 INFO:tasks.workunit.client.1.smithi192.stdout:5/613: getdents d6/d12/d4e/d58/d8f 0 2024-02-16T22:56:26.261 INFO:tasks.workunit.client.1.smithi192.stdout:9/743: dwrite d7/d19/d28/f2c [0,4194304] 0 2024-02-16T22:56:26.263 INFO:tasks.workunit.client.1.smithi192.stdout:8/608: rename d5/d26/d8f/ca3 to d5/d26/d83/cbd 0 2024-02-16T22:56:26.264 INFO:tasks.workunit.client.0.smithi064.stdout:2/624: mknod d7/d14/d4b/cc8 0 2024-02-16T22:56:26.265 INFO:tasks.workunit.client.0.smithi064.stdout:2/625: chown d7/d14/d1e/d34/f61 6631674 1 2024-02-16T22:56:26.266 INFO:tasks.workunit.client.1.smithi192.stdout:6/634: mkdir da/df/d12/d57/daf/dcf 0 2024-02-16T22:56:26.267 INFO:tasks.workunit.client.0.smithi064.stdout:6/588: dread d0/d1d/f9e [0,4194304] 0 2024-02-16T22:56:26.267 INFO:tasks.workunit.client.1.smithi192.stdout:9/744: dwrite d7/d15/d36/d38/f49 [0,4194304] 0 2024-02-16T22:56:26.270 INFO:tasks.workunit.client.1.smithi192.stdout:0/624: creat d1/d3/de/d22/d37/d67/d75/fc5 x:0 0 0 2024-02-16T22:56:26.271 INFO:tasks.workunit.client.0.smithi064.stdout:4/554: creat dc/d1d/faa x:0 0 0 2024-02-16T22:56:26.273 INFO:tasks.workunit.client.1.smithi192.stdout:2/675: mknod d22/d24/d39/d42/d85/d98/cd7 0 2024-02-16T22:56:26.279 INFO:tasks.workunit.client.1.smithi192.stdout:1/577: truncate d3/de/d27/f44 3574944 0 2024-02-16T22:56:26.284 INFO:tasks.workunit.client.0.smithi064.stdout:1/480: getdents d2 0 2024-02-16T22:56:26.286 INFO:tasks.workunit.client.1.smithi192.stdout:7/729: mknod d1/d2/d2d/d61/d8a/de6/c101 0 2024-02-16T22:56:26.287 INFO:tasks.workunit.client.1.smithi192.stdout:7/730: chown d1/d2/de/l16 518900653 1 2024-02-16T22:56:26.288 INFO:tasks.workunit.client.0.smithi064.stdout:3/618: rmdir d4/d1f/d34/da6 0 2024-02-16T22:56:26.288 INFO:tasks.workunit.client.1.smithi192.stdout:5/614: rmdir d6/d12/d4e/d5b/d67 39 2024-02-16T22:56:26.294 INFO:tasks.workunit.client.1.smithi192.stdout:3/594: truncate da/d11/d20/f69 2005772 0 2024-02-16T22:56:26.298 INFO:tasks.workunit.client.0.smithi064.stdout:6/589: mkdir d0/d1d/da8/d60/dbf 0 2024-02-16T22:56:26.298 INFO:tasks.workunit.client.0.smithi064.stdout:6/590: chown d0/d6/da/d56/db9 447442 1 2024-02-16T22:56:26.299 INFO:tasks.workunit.client.1.smithi192.stdout:6/635: creat da/df/d71/fd0 x:0 0 0 2024-02-16T22:56:26.300 INFO:tasks.workunit.client.1.smithi192.stdout:4/674: write d1/d26/d66/d89/fc [1592652,126416] 0 2024-02-16T22:56:26.300 INFO:tasks.workunit.client.1.smithi192.stdout:6/636: dread - da/df/f6e zero size 2024-02-16T22:56:26.304 INFO:tasks.workunit.client.1.smithi192.stdout:0/625: creat d1/d3/de/d22/d37/d67/d75/db0/fc6 x:0 0 0 2024-02-16T22:56:26.306 INFO:tasks.workunit.client.0.smithi064.stdout:0/477: write da/dc/de/d12/d1d/f3d [565921,44820] 0 2024-02-16T22:56:26.307 INFO:tasks.workunit.client.0.smithi064.stdout:5/449: symlink d6/d18/d19/d1a/d8e/l9b 0 2024-02-16T22:56:26.311 INFO:tasks.workunit.client.0.smithi064.stdout:9/488: symlink d3/d88/la0 0 2024-02-16T22:56:26.312 INFO:tasks.workunit.client.0.smithi064.stdout:9/489: fsync d3/f63 0 2024-02-16T22:56:26.315 INFO:tasks.workunit.client.0.smithi064.stdout:6/591: dread d0/d1d/d21/f2d [0,4194304] 0 2024-02-16T22:56:26.316 INFO:tasks.workunit.client.1.smithi192.stdout:5/615: creat d6/d12/d4e/d46/fca x:0 0 0 2024-02-16T22:56:26.317 INFO:tasks.workunit.client.1.smithi192.stdout:5/616: dread - d6/d8/f9e zero size 2024-02-16T22:56:26.318 INFO:tasks.workunit.client.1.smithi192.stdout:5/617: fdatasync d6/d12/d4e/faa 0 2024-02-16T22:56:26.319 INFO:tasks.workunit.client.0.smithi064.stdout:2/626: mknod d7/cc9 0 2024-02-16T22:56:26.323 INFO:tasks.workunit.client.1.smithi192.stdout:8/609: link d5/d26/d83/d86/d5f/f5e d5/db/d47/d43/d90/fbe 0 2024-02-16T22:56:26.325 INFO:tasks.workunit.client.1.smithi192.stdout:4/675: symlink d1/d26/d5e/lec 0 2024-02-16T22:56:26.326 INFO:tasks.workunit.client.0.smithi064.stdout:8/493: rename df/l6c to df/d31/la5 0 2024-02-16T22:56:26.327 INFO:tasks.workunit.client.1.smithi192.stdout:6/637: fsync da/df/d12/d57/f2c 0 2024-02-16T22:56:26.328 INFO:tasks.workunit.client.0.smithi064.stdout:7/533: link d5/d33/d45/d64/c5c d5/d33/d42/d81/d82/ca6 0 2024-02-16T22:56:26.328 INFO:tasks.workunit.client.1.smithi192.stdout:4/676: dread d1/d15/d48/d8d/f7b [0,4194304] 0 2024-02-16T22:56:26.328 INFO:tasks.workunit.client.0.smithi064.stdout:7/534: dread - d5/d33/d42/f8f zero size 2024-02-16T22:56:26.329 INFO:tasks.workunit.client.0.smithi064.stdout:7/535: fdatasync d5/d33/d45/d64/d56/d66/fa1 0 2024-02-16T22:56:26.331 INFO:tasks.workunit.client.0.smithi064.stdout:5/450: mknod d6/d18/c9c 0 2024-02-16T22:56:26.335 INFO:tasks.workunit.client.1.smithi192.stdout:1/578: rename d3/de/d27/d34/c62 to d3/d5b/db1/db6/cc7 0 2024-02-16T22:56:26.336 INFO:tasks.workunit.client.0.smithi064.stdout:7/536: dwrite d5/d33/d45/f98 [0,4194304] 0 2024-02-16T22:56:26.337 INFO:tasks.workunit.client.0.smithi064.stdout:7/537: dread - d5/d6/d65/d7f/f9c zero size 2024-02-16T22:56:26.338 INFO:tasks.workunit.client.0.smithi064.stdout:4/555: truncate dc/d1d/f2b 1273379 0 2024-02-16T22:56:26.348 INFO:tasks.workunit.client.1.smithi192.stdout:2/676: dwrite f4 [0,4194304] 0 2024-02-16T22:56:26.349 INFO:tasks.workunit.client.1.smithi192.stdout:2/677: chown d22/d47/f7d 976734809 1 2024-02-16T22:56:26.352 INFO:tasks.workunit.client.0.smithi064.stdout:6/592: symlink d0/d5/df/lc0 0 2024-02-16T22:56:26.354 INFO:tasks.workunit.client.0.smithi064.stdout:1/481: write d2/df/d14/d2c/f3c [569397,29391] 0 2024-02-16T22:56:26.357 INFO:tasks.workunit.client.0.smithi064.stdout:9/490: write d3/d11/d15/d5a/d2a/d2e/f59 [3306542,48748] 0 2024-02-16T22:56:26.360 INFO:tasks.workunit.client.0.smithi064.stdout:5/451: creat d6/d1c/d43/d7d/f9d x:0 0 0 2024-02-16T22:56:26.361 INFO:tasks.workunit.client.0.smithi064.stdout:5/452: fdatasync d6/d1c/d43/d7d/f88 0 2024-02-16T22:56:26.366 INFO:tasks.workunit.client.1.smithi192.stdout:6/638: rmdir da/df 39 2024-02-16T22:56:26.368 INFO:tasks.workunit.client.0.smithi064.stdout:4/556: creat dc/dd/d16/d7c/fab x:0 0 0 2024-02-16T22:56:26.368 INFO:tasks.workunit.client.0.smithi064.stdout:0/478: dread da/dc/d3b/f3c [0,4194304] 0 2024-02-16T22:56:26.369 INFO:tasks.workunit.client.1.smithi192.stdout:8/610: write d5/db/f51 [2313350,43683] 0 2024-02-16T22:56:26.374 INFO:tasks.workunit.client.0.smithi064.stdout:2/627: creat d7/de/d60/fca x:0 0 0 2024-02-16T22:56:26.374 INFO:tasks.workunit.client.0.smithi064.stdout:2/628: chown d7/d14/d3e 782374904 1 2024-02-16T22:56:26.375 INFO:tasks.workunit.client.1.smithi192.stdout:1/579: creat d3/de/d27/d34/d69/d85/d89/daa/fc8 x:0 0 0 2024-02-16T22:56:26.377 INFO:tasks.workunit.client.0.smithi064.stdout:4/557: dread dc/f19 [4194304,4194304] 0 2024-02-16T22:56:26.377 INFO:tasks.workunit.client.1.smithi192.stdout:7/731: rmdir d1/d2/d2d/d61/d88/dd9 0 2024-02-16T22:56:26.378 INFO:tasks.workunit.client.0.smithi064.stdout:1/482: symlink d2/df/d14/d2a/lab 0 2024-02-16T22:56:26.380 INFO:tasks.workunit.client.0.smithi064.stdout:8/494: creat df/d31/d61/d94/fa6 x:0 0 0 2024-02-16T22:56:26.382 INFO:tasks.workunit.client.0.smithi064.stdout:9/491: mkdir d3/d89/da1 0 2024-02-16T22:56:26.388 INFO:tasks.workunit.client.1.smithi192.stdout:9/745: link d7/lb d7/d19/d28/d43/d70/l102 0 2024-02-16T22:56:26.389 INFO:tasks.workunit.client.0.smithi064.stdout:3/619: getdents d4/d1f/d5b 0 2024-02-16T22:56:26.390 INFO:tasks.workunit.client.1.smithi192.stdout:6/639: write da/df/d12/d57/d81/fb2 [72372,1069] 0 2024-02-16T22:56:26.392 INFO:tasks.workunit.client.1.smithi192.stdout:4/677: symlink d1/d15/d48/led 0 2024-02-16T22:56:26.395 INFO:tasks.workunit.client.1.smithi192.stdout:8/611: mkdir d5/d26/d83/d86/d5f/dbf 0 2024-02-16T22:56:26.396 INFO:tasks.workunit.client.1.smithi192.stdout:8/612: chown d5/d26/fa0 15316 1 2024-02-16T22:56:26.396 INFO:tasks.workunit.client.1.smithi192.stdout:1/580: unlink d3/d40/cb9 0 2024-02-16T22:56:26.398 INFO:tasks.workunit.client.0.smithi064.stdout:1/483: creat d2/df/d14/d18/d48/d6b/d75/fac x:0 0 0 2024-02-16T22:56:26.398 INFO:tasks.workunit.client.0.smithi064.stdout:6/593: truncate d0/d5/df/d17/f7d 846512 0 2024-02-16T22:56:26.399 INFO:tasks.workunit.client.1.smithi192.stdout:7/732: creat d1/d69/f102 x:0 0 0 2024-02-16T22:56:26.400 INFO:tasks.workunit.client.0.smithi064.stdout:1/484: read d2/df/fa2 [472930,113676] 0 2024-02-16T22:56:26.402 INFO:tasks.workunit.client.0.smithi064.stdout:8/495: unlink df/d31/d2f/d38/f46 0 2024-02-16T22:56:26.402 INFO:tasks.workunit.client.1.smithi192.stdout:3/595: getdents da/d11/d35 0 2024-02-16T22:56:26.404 INFO:tasks.workunit.client.1.smithi192.stdout:2/678: creat d22/d24/d39/d42/d85/d98/db0/dcd/fd8 x:0 0 0 2024-02-16T22:56:26.407 INFO:tasks.workunit.client.1.smithi192.stdout:5/618: rmdir d6/d12/d4e/d58/dc2 0 2024-02-16T22:56:26.408 INFO:tasks.workunit.client.0.smithi064.stdout:4/558: dread dc/dd/d16/f5d [0,4194304] 0 2024-02-16T22:56:26.409 INFO:tasks.workunit.client.0.smithi064.stdout:5/453: link d6/d18/d19/d1a/f76 d6/d25/d48/d67/f9e 0 2024-02-16T22:56:26.413 INFO:tasks.workunit.client.1.smithi192.stdout:6/640: unlink da/f5f 0 2024-02-16T22:56:26.415 INFO:tasks.workunit.client.0.smithi064.stdout:5/454: dwrite d6/d1c/d4b/f94 [0,4194304] 0 2024-02-16T22:56:26.417 INFO:tasks.workunit.client.0.smithi064.stdout:0/479: creat da/dc/de/d12/d2c/d2e/d38/d87/f9f x:0 0 0 2024-02-16T22:56:26.417 INFO:tasks.workunit.client.0.smithi064.stdout:5/455: chown d6/d1c/f62 1034590 1 2024-02-16T22:56:26.418 INFO:tasks.workunit.client.0.smithi064.stdout:0/480: chown da/dc/d3b/f8e 499930 1 2024-02-16T22:56:26.422 INFO:tasks.workunit.client.1.smithi192.stdout:9/746: dread d7/d19/d28/f74 [4194304,4194304] 0 2024-02-16T22:56:26.422 INFO:tasks.workunit.client.0.smithi064.stdout:3/620: unlink d4/d13/d41/f3b 0 2024-02-16T22:56:26.423 INFO:tasks.workunit.client.0.smithi064.stdout:3/621: chown d4/d14/c5a 19512896 1 2024-02-16T22:56:26.424 INFO:tasks.workunit.client.1.smithi192.stdout:8/613: chown d5/d26/d83/cbd 84 1 2024-02-16T22:56:26.425 INFO:tasks.workunit.client.0.smithi064.stdout:3/622: chown d4/d1f/d5b/d51/d7c/l4a 123 1 2024-02-16T22:56:26.425 INFO:tasks.workunit.client.1.smithi192.stdout:8/614: read d5/d26/d83/d86/f6e [588820,104022] 0 2024-02-16T22:56:26.428 INFO:tasks.workunit.client.1.smithi192.stdout:0/626: rename d1/d3/d29/d43 to d1/dc7 0 2024-02-16T22:56:26.432 INFO:tasks.workunit.client.0.smithi064.stdout:6/594: creat d0/d6/d10/d33/da5/fc1 x:0 0 0 2024-02-16T22:56:26.434 INFO:tasks.workunit.client.1.smithi192.stdout:3/596: creat da/d11/d63/d6b/d80/fd0 x:0 0 0 2024-02-16T22:56:26.436 INFO:tasks.workunit.client.1.smithi192.stdout:5/619: rmdir d6/d12/d4e/d58 39 2024-02-16T22:56:26.439 INFO:tasks.workunit.client.0.smithi064.stdout:7/538: getdents d5/d6/d54 0 2024-02-16T22:56:26.441 INFO:tasks.workunit.client.0.smithi064.stdout:4/559: mknod dc/d1d/cac 0 2024-02-16T22:56:26.446 INFO:tasks.workunit.client.0.smithi064.stdout:5/456: creat d6/d18/d19/d1a/d85/f9f x:0 0 0 2024-02-16T22:56:26.450 INFO:tasks.workunit.client.1.smithi192.stdout:8/615: mkdir d5/d26/d8f/dc0 0 2024-02-16T22:56:26.451 INFO:tasks.workunit.client.1.smithi192.stdout:4/678: dwrite d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:26.451 INFO:tasks.workunit.client.0.smithi064.stdout:3/623: mknod d4/d13/d1b/d27/d99/d7d/cd0 0 2024-02-16T22:56:26.456 INFO:tasks.workunit.client.0.smithi064.stdout:6/595: creat d0/d5/df/fc2 x:0 0 0 2024-02-16T22:56:26.459 INFO:tasks.workunit.client.0.smithi064.stdout:3/624: dwrite d4/d1f/d34/f45 [0,4194304] 0 2024-02-16T22:56:26.461 INFO:tasks.workunit.client.1.smithi192.stdout:2/679: link d22/d47/dcb/f37 d22/d24/d2b/d67/fd9 0 2024-02-16T22:56:26.462 INFO:tasks.workunit.client.0.smithi064.stdout:3/625: fsync d4/d13/d41/f8d 0 2024-02-16T22:56:26.464 INFO:tasks.workunit.client.0.smithi064.stdout:6/596: dwrite d0/d6/d10/f40 [4194304,4194304] 0 2024-02-16T22:56:26.464 INFO:tasks.workunit.client.0.smithi064.stdout:9/492: rename d3/d11/d15/d5a/d38/f6f to d3/d9/fa2 0 2024-02-16T22:56:26.466 INFO:tasks.workunit.client.1.smithi192.stdout:3/597: truncate da/d11/d63/d6b/d66/d75/f76 4042134 0 2024-02-16T22:56:26.468 INFO:tasks.workunit.client.0.smithi064.stdout:7/539: mkdir d5/d6/d65/da7 0 2024-02-16T22:56:26.469 INFO:tasks.workunit.client.0.smithi064.stdout:7/540: readlink d5/d6/d54/d41/l9d 0 2024-02-16T22:56:26.470 INFO:tasks.workunit.client.0.smithi064.stdout:7/541: write d5/d6/d65/f6d [4545443,25697] 0 2024-02-16T22:56:26.472 INFO:tasks.workunit.client.1.smithi192.stdout:1/581: truncate d3/d5b/f84 489220 0 2024-02-16T22:56:26.472 INFO:tasks.workunit.client.1.smithi192.stdout:6/641: mknod da/df/d12/d57/daf/dcf/cd1 0 2024-02-16T22:56:26.473 INFO:tasks.workunit.client.0.smithi064.stdout:0/481: mkdir da/da0 0 2024-02-16T22:56:26.473 INFO:tasks.workunit.client.1.smithi192.stdout:6/642: stat da/df/d12/d57/daf 0 2024-02-16T22:56:26.474 INFO:tasks.workunit.client.1.smithi192.stdout:9/747: symlink d7/d15/d36/d77/dbd/de0/l103 0 2024-02-16T22:56:26.474 INFO:tasks.workunit.client.1.smithi192.stdout:6/643: fsync da/df/d12/f47 0 2024-02-16T22:56:26.474 INFO:tasks.workunit.client.1.smithi192.stdout:1/582: write d3/de/d27/d34/d36/d47/f5e [3714654,98923] 0 2024-02-16T22:56:26.475 INFO:tasks.workunit.client.0.smithi064.stdout:5/457: readlink d6/l11 0 2024-02-16T22:56:26.476 INFO:tasks.workunit.client.1.smithi192.stdout:9/748: write d7/d19/d37/d5a/ff5 [4079171,55532] 0 2024-02-16T22:56:26.477 INFO:tasks.workunit.client.0.smithi064.stdout:2/629: getdents d7/d14/d1e/d34/d73/db5 0 2024-02-16T22:56:26.478 INFO:tasks.workunit.client.0.smithi064.stdout:2/630: dread - d7/d14/d25/d46/d55/d7f/f99 zero size 2024-02-16T22:56:26.478 INFO:tasks.workunit.client.1.smithi192.stdout:4/679: read d1/d26/d66/d89/f64 [531237,95503] 0 2024-02-16T22:56:26.479 INFO:tasks.workunit.client.0.smithi064.stdout:1/485: getdents d2 0 2024-02-16T22:56:26.482 INFO:tasks.workunit.client.0.smithi064.stdout:1/486: truncate d2/df/d14/d18/d48/d6b/d75/fac 977671 0 2024-02-16T22:56:26.484 INFO:tasks.workunit.client.1.smithi192.stdout:6/644: dwrite da/df/d12/d57/fbd [0,4194304] 0 2024-02-16T22:56:26.485 INFO:tasks.workunit.client.1.smithi192.stdout:9/749: dwrite d7/d19/fe5 [0,4194304] 0 2024-02-16T22:56:26.487 INFO:tasks.workunit.client.1.smithi192.stdout:6/645: write da/df/d12/d15/f2d [1000807,4364] 0 2024-02-16T22:56:26.488 INFO:tasks.workunit.client.1.smithi192.stdout:6/646: chown da/df/d12/d57 0 1 2024-02-16T22:56:26.496 INFO:tasks.workunit.client.0.smithi064.stdout:3/626: dwrite d4/d13/d1b/d4f/f5d [4194304,4194304] 0 2024-02-16T22:56:26.504 INFO:tasks.workunit.client.0.smithi064.stdout:3/627: dwrite d4/fd [0,4194304] 0 2024-02-16T22:56:26.507 INFO:tasks.workunit.client.1.smithi192.stdout:8/616: write d5/db/d47/d43/d67/f85 [833372,115099] 0 2024-02-16T22:56:26.507 INFO:tasks.workunit.client.1.smithi192.stdout:0/627: link d1/d3/de/d22/d37/d47/l4a d1/dc7/d9c/lc8 0 2024-02-16T22:56:26.508 INFO:tasks.workunit.client.0.smithi064.stdout:4/560: creat dc/dd/d16/d9c/fad x:0 0 0 2024-02-16T22:56:26.509 INFO:tasks.workunit.client.1.smithi192.stdout:7/733: getdents d1/d2/de/d2c/d49/d89 0 2024-02-16T22:56:26.516 INFO:tasks.workunit.client.0.smithi064.stdout:2/631: rename d7/d14/d4b/d63/l83 to d7/d14/d4b/d63/lcb 0 2024-02-16T22:56:26.516 INFO:tasks.workunit.client.1.smithi192.stdout:3/598: write da/d11/d63/fa6 [513491,96863] 0 2024-02-16T22:56:26.517 INFO:tasks.workunit.client.1.smithi192.stdout:2/680: mkdir d22/d47/dcb/dda 0 2024-02-16T22:56:26.518 INFO:tasks.workunit.client.0.smithi064.stdout:2/632: truncate d7/d14/d4b/fc6 996098 0 2024-02-16T22:56:26.520 INFO:tasks.workunit.client.0.smithi064.stdout:8/496: link df/f1b df/d31/d2f/d4f/fa7 0 2024-02-16T22:56:26.522 INFO:tasks.workunit.client.0.smithi064.stdout:6/597: symlink d0/d6/da/lc3 0 2024-02-16T22:56:26.523 INFO:tasks.workunit.client.1.smithi192.stdout:9/750: mkdir d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/d104 0 2024-02-16T22:56:26.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/751: chown f1 106514 1 2024-02-16T22:56:26.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/752: chown d7/d15/d36/d77/db1/lbb 79 1 2024-02-16T22:56:26.525 INFO:tasks.workunit.client.0.smithi064.stdout:9/493: mkdir d3/d11/d15/d5a/d2a/d2e/da3 0 2024-02-16T22:56:26.526 INFO:tasks.workunit.client.1.smithi192.stdout:6/647: mknod da/df/d58/d72/cd2 0 2024-02-16T22:56:26.529 INFO:tasks.workunit.client.0.smithi064.stdout:0/482: creat da/dc/d3b/d5a/fa1 x:0 0 0 2024-02-16T22:56:26.529 INFO:tasks.workunit.client.1.smithi192.stdout:9/753: dwrite f1 [0,4194304] 0 2024-02-16T22:56:26.530 INFO:tasks.workunit.client.0.smithi064.stdout:3/628: read d4/d13/f28 [1530036,74405] 0 2024-02-16T22:56:26.538 INFO:tasks.workunit.client.0.smithi064.stdout:8/497: creat df/d31/d61/fa8 x:0 0 0 2024-02-16T22:56:26.540 INFO:tasks.workunit.client.0.smithi064.stdout:6/598: creat d0/d12/fc4 x:0 0 0 2024-02-16T22:56:26.542 INFO:tasks.workunit.client.0.smithi064.stdout:9/494: chown d3/d9/d6a/l74 830992 1 2024-02-16T22:56:26.542 INFO:tasks.workunit.client.0.smithi064.stdout:9/495: chown d3/d9/d6a 14877 1 2024-02-16T22:56:26.543 INFO:tasks.workunit.client.1.smithi192.stdout:9/754: dread d7/d15/f27 [0,4194304] 0 2024-02-16T22:56:26.546 INFO:tasks.workunit.client.0.smithi064.stdout:4/561: link dc/dd/d32/d46/f55 dc/d1d/d43/fae 0 2024-02-16T22:56:26.549 INFO:tasks.workunit.client.1.smithi192.stdout:9/755: dread d7/f9d [0,4194304] 0 2024-02-16T22:56:26.549 INFO:tasks.workunit.client.0.smithi064.stdout:5/458: link d6/d1c/l75 d6/d1c/d43/d7d/la0 0 2024-02-16T22:56:26.556 INFO:tasks.workunit.client.0.smithi064.stdout:6/599: mkdir d0/d3c/dc5 0 2024-02-16T22:56:26.558 INFO:tasks.workunit.client.0.smithi064.stdout:9/496: rmdir d3/d11/d15/d5a/d2a 39 2024-02-16T22:56:26.560 INFO:tasks.workunit.client.0.smithi064.stdout:7/542: link d5/d33/d45/d64/d56/d5d/c7c d5/d7d/ca8 0 2024-02-16T22:56:26.561 INFO:tasks.workunit.client.1.smithi192.stdout:9/756: dread d7/d19/d37/d5a/ff5 [0,4194304] 0 2024-02-16T22:56:26.562 INFO:tasks.workunit.client.0.smithi064.stdout:7/543: chown d5/d6/d54/f5e 7 1 2024-02-16T22:56:26.569 INFO:tasks.workunit.client.0.smithi064.stdout:1/487: getdents d2/df/d14/d18 0 2024-02-16T22:56:26.570 INFO:tasks.workunit.client.0.smithi064.stdout:1/488: truncate d2/df/d11/f6d 4551476 0 2024-02-16T22:56:26.571 INFO:tasks.workunit.client.0.smithi064.stdout:2/633: rename d7/d14/d1e/d34/d3f/c93 to d7/d14/d1e/d34/ccc 0 2024-02-16T22:56:26.577 INFO:tasks.workunit.client.0.smithi064.stdout:9/497: symlink d3/d9/d1e/d42/la4 0 2024-02-16T22:56:26.579 INFO:tasks.workunit.client.0.smithi064.stdout:7/544: symlink d5/d6/d54/la9 0 2024-02-16T22:56:26.582 INFO:tasks.workunit.client.0.smithi064.stdout:4/562: link dc/d1d/d43/fae dc/dd/d5b/faf 0 2024-02-16T22:56:26.586 INFO:tasks.workunit.client.1.smithi192.stdout:7/734: mknod d1/d2/d37/d86/c103 0 2024-02-16T22:56:26.587 INFO:tasks.workunit.client.1.smithi192.stdout:4/680: symlink d1/d26/d66/d89/d11/d58/lee 0 2024-02-16T22:56:26.588 INFO:tasks.workunit.client.1.smithi192.stdout:4/681: truncate d1/d15/d48/d8d/d9b/fb0 7441470 0 2024-02-16T22:56:26.588 INFO:tasks.workunit.client.0.smithi064.stdout:1/489: fsync d2/f36 0 2024-02-16T22:56:26.591 INFO:tasks.workunit.client.0.smithi064.stdout:5/459: write f4 [1238110,17885] 0 2024-02-16T22:56:26.592 INFO:tasks.workunit.client.0.smithi064.stdout:8/498: link df/d31/d8f/l6e df/d31/d47/la9 0 2024-02-16T22:56:26.594 INFO:tasks.workunit.client.0.smithi064.stdout:6/600: mknod d0/d6/d10/d33/d72/cc6 0 2024-02-16T22:56:26.601 INFO:tasks.workunit.client.0.smithi064.stdout:7/545: creat d5/d33/d45/d64/d56/faa x:0 0 0 2024-02-16T22:56:26.602 INFO:tasks.workunit.client.1.smithi192.stdout:5/620: sync 2024-02-16T22:56:26.602 INFO:tasks.workunit.client.1.smithi192.stdout:6/648: creat da/df/d12/d15/d1b/d70/fd3 x:0 0 0 2024-02-16T22:56:26.603 INFO:tasks.workunit.client.0.smithi064.stdout:4/563: creat dc/de/d6d/fb0 x:0 0 0 2024-02-16T22:56:26.606 INFO:tasks.workunit.client.0.smithi064.stdout:3/629: rmdir d4/d13/d1b/d48/dc0 0 2024-02-16T22:56:26.606 INFO:tasks.workunit.client.0.smithi064.stdout:7/546: dread d5/d6/f5b [0,4194304] 0 2024-02-16T22:56:26.607 INFO:tasks.workunit.client.1.smithi192.stdout:1/583: truncate d3/de/d27/f3b 4284242 0 2024-02-16T22:56:26.611 INFO:tasks.workunit.client.1.smithi192.stdout:8/617: write d5/d9/f5a [303926,113263] 0 2024-02-16T22:56:26.612 INFO:tasks.workunit.client.0.smithi064.stdout:7/547: dread d5/d33/d45/d73/d88/f90 [0,4194304] 0 2024-02-16T22:56:26.615 INFO:tasks.workunit.client.0.smithi064.stdout:1/490: rmdir d2/df/d14/d2c 39 2024-02-16T22:56:26.616 INFO:tasks.workunit.client.1.smithi192.stdout:9/757: creat d7/d19/d28/d43/d70/d8e/f105 x:0 0 0 2024-02-16T22:56:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Adjusting osd_memory_target on smithi192 to 2581M 2024-02-16T22:56:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:56:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:56:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:56:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:56:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: pgmap v18: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 63 MiB/s rd, 217 MiB/s wr, 379 op/s 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Reconfiguring node-exporter.smithi064 (dependencies changed)... 2024-02-16T22:56:26.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:26 smithi064 ceph-mon[29226]: Reconfiguring daemon node-exporter.smithi064 on smithi064 2024-02-16T22:56:26.620 INFO:tasks.workunit.client.0.smithi064.stdout:4/564: dread dc/dd/d32/d58/f14 [0,4194304] 0 2024-02-16T22:56:26.620 INFO:tasks.workunit.client.0.smithi064.stdout:4/565: read - dc/dd/d32/d58/f8e zero size 2024-02-16T22:56:26.620 INFO:tasks.workunit.client.0.smithi064.stdout:5/460: creat d6/d18/d19/d1a/fa1 x:0 0 0 2024-02-16T22:56:26.621 INFO:tasks.workunit.client.0.smithi064.stdout:5/461: chown d6/d18/d19/l22 28552307 1 2024-02-16T22:56:26.622 INFO:tasks.workunit.client.0.smithi064.stdout:8/499: symlink df/d31/d47/laa 0 2024-02-16T22:56:26.622 INFO:tasks.workunit.client.0.smithi064.stdout:5/462: fsync d6/d25/d48/d67/f7f 0 2024-02-16T22:56:26.623 INFO:tasks.workunit.client.0.smithi064.stdout:8/500: readlink df/d31/d47/l4e 0 2024-02-16T22:56:26.626 INFO:tasks.workunit.client.1.smithi192.stdout:4/682: rmdir d1/d26/dbe 39 2024-02-16T22:56:26.629 INFO:tasks.workunit.client.1.smithi192.stdout:3/599: creat da/d6a/fd1 x:0 0 0 2024-02-16T22:56:26.634 INFO:tasks.workunit.client.1.smithi192.stdout:2/681: truncate d22/f41 142795 0 2024-02-16T22:56:26.636 INFO:tasks.workunit.client.1.smithi192.stdout:5/621: symlink d6/d12/d2d/lcb 0 2024-02-16T22:56:26.638 INFO:tasks.workunit.client.0.smithi064.stdout:3/630: rename d4/d13/d1b/d27/d44/lc9 to d4/d1f/ld1 0 2024-02-16T22:56:26.639 INFO:tasks.workunit.client.1.smithi192.stdout:6/649: symlink da/df/d58/d72/ld4 0 2024-02-16T22:56:26.641 INFO:tasks.workunit.client.1.smithi192.stdout:7/735: write d1/d2/de/f24 [961748,87853] 0 2024-02-16T22:56:26.642 INFO:tasks.workunit.client.0.smithi064.stdout:1/491: symlink d2/df/d14/d18/d40/lad 0 2024-02-16T22:56:26.642 INFO:tasks.workunit.client.0.smithi064.stdout:9/498: write d3/d11/d15/f51 [812611,23637] 0 2024-02-16T22:56:26.645 INFO:tasks.workunit.client.0.smithi064.stdout:2/634: creat d7/de/fcd x:0 0 0 2024-02-16T22:56:26.650 INFO:tasks.workunit.client.0.smithi064.stdout:5/463: creat d6/d1c/d43/d7d/fa2 x:0 0 0 2024-02-16T22:56:26.652 INFO:tasks.workunit.client.0.smithi064.stdout:8/501: truncate f3 880389 0 2024-02-16T22:56:26.652 INFO:tasks.workunit.client.0.smithi064.stdout:8/502: fdatasync df/d31/d2f/d3e/f83 0 2024-02-16T22:56:26.653 INFO:tasks.workunit.client.1.smithi192.stdout:9/758: mknod d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/c106 0 2024-02-16T22:56:26.654 INFO:tasks.workunit.client.1.smithi192.stdout:9/759: chown d7/d19/d37/cbc 176177146 1 2024-02-16T22:56:26.654 INFO:tasks.workunit.client.1.smithi192.stdout:9/760: write d7/d19/d5e/fcd [1885795,536] 0 2024-02-16T22:56:26.659 INFO:tasks.workunit.client.1.smithi192.stdout:4/683: mkdir d1/d15/d48/d8d/def 0 2024-02-16T22:56:26.660 INFO:tasks.workunit.client.1.smithi192.stdout:1/584: dwrite d3/de/f82 [0,4194304] 0 2024-02-16T22:56:26.660 INFO:tasks.workunit.client.0.smithi064.stdout:7/548: dwrite d5/d33/d42/f4f [0,4194304] 0 2024-02-16T22:56:26.661 INFO:tasks.workunit.client.1.smithi192.stdout:1/585: chown d3/de/da4/cc1 1547 1 2024-02-16T22:56:26.661 INFO:tasks.workunit.client.1.smithi192.stdout:3/600: truncate da/d11/d63/d6b/d66/f92 1257536 0 2024-02-16T22:56:26.666 INFO:tasks.workunit.client.1.smithi192.stdout:2/682: creat d22/d24/d39/d42/d85/d98/fdb x:0 0 0 2024-02-16T22:56:26.667 INFO:tasks.workunit.client.1.smithi192.stdout:2/683: chown d22/d24/d39/d42/d65/d69/db1/lbd 28790 1 2024-02-16T22:56:26.669 INFO:tasks.workunit.client.0.smithi064.stdout:1/492: stat d2/df/d14/d2c/f49 0 2024-02-16T22:56:26.670 INFO:tasks.workunit.client.1.smithi192.stdout:6/650: mkdir da/d76/dd5 0 2024-02-16T22:56:26.672 INFO:tasks.workunit.client.1.smithi192.stdout:6/651: write da/df/f1c [5157447,76017] 0 2024-02-16T22:56:26.672 INFO:tasks.workunit.client.0.smithi064.stdout:2/635: rename d7/d14/d1e/d34/d73/db5/db9 to d7/de/d60/dce 0 2024-02-16T22:56:26.675 INFO:tasks.workunit.client.1.smithi192.stdout:8/618: symlink d5/d26/d83/d86/d5f/db2/lc1 0 2024-02-16T22:56:26.677 INFO:tasks.workunit.client.0.smithi064.stdout:6/601: link d0/d12/f74 d0/d5/fc7 0 2024-02-16T22:56:26.678 INFO:tasks.workunit.client.0.smithi064.stdout:6/602: chown d0/d5/fab 21 1 2024-02-16T22:56:26.679 INFO:tasks.workunit.client.1.smithi192.stdout:9/761: symlink d7/d19/d37/d5a/l107 0 2024-02-16T22:56:26.680 INFO:tasks.workunit.client.1.smithi192.stdout:0/628: getdents d1/d3/de/d22/d37 0 2024-02-16T22:56:26.680 INFO:tasks.workunit.client.1.smithi192.stdout:8/619: dwrite d5/d16/f18 [0,4194304] 0 2024-02-16T22:56:26.681 INFO:tasks.workunit.client.0.smithi064.stdout:7/549: chown d5/d33/d42/f74 13 1 2024-02-16T22:56:26.684 INFO:tasks.workunit.client.0.smithi064.stdout:3/631: mknod d4/d13/d1b/d27/cd2 0 2024-02-16T22:56:26.685 INFO:tasks.workunit.client.1.smithi192.stdout:1/586: chown d3/de/d27/d34/d52/d5a/fa2 1 1 2024-02-16T22:56:26.686 INFO:tasks.workunit.client.0.smithi064.stdout:1/493: mknod d2/df/d14/d18/d48/d6b/cae 0 2024-02-16T22:56:26.689 INFO:tasks.workunit.client.0.smithi064.stdout:9/499: mkdir d3/d11/d9e/da5 0 2024-02-16T22:56:26.695 INFO:tasks.workunit.client.0.smithi064.stdout:1/494: dread d2/df/d11/d1a/f34 [0,4194304] 0 2024-02-16T22:56:26.695 INFO:tasks.workunit.client.1.smithi192.stdout:5/622: symlink d6/d12/d4e/d5b/d67/d78/dae/dbc/lcc 0 2024-02-16T22:56:26.696 INFO:tasks.workunit.client.1.smithi192.stdout:5/623: write d6/d20/d93/fc3 [548293,109469] 0 2024-02-16T22:56:26.697 INFO:tasks.workunit.client.0.smithi064.stdout:8/503: dread df/d31/d2f/d38/d48/f49 [0,4194304] 0 2024-02-16T22:56:26.700 INFO:tasks.workunit.client.0.smithi064.stdout:2/636: rmdir d7/de/d95/d44 39 2024-02-16T22:56:26.702 INFO:tasks.workunit.client.1.smithi192.stdout:5/624: dwrite d6/d12/d2d/f96 [0,4194304] 0 2024-02-16T22:56:26.705 INFO:tasks.workunit.client.0.smithi064.stdout:1/495: dread d2/df/d11/d1a/d1d/f58 [0,4194304] 0 2024-02-16T22:56:26.705 INFO:tasks.workunit.client.0.smithi064.stdout:6/603: dread d0/d5/df/d17/f7d [0,4194304] 0 2024-02-16T22:56:26.706 INFO:tasks.workunit.client.1.smithi192.stdout:5/625: dread d6/d8/d75/f7b [0,4194304] 0 2024-02-16T22:56:26.707 INFO:tasks.workunit.client.0.smithi064.stdout:7/550: fdatasync d5/f27 0 2024-02-16T22:56:26.710 INFO:tasks.workunit.client.0.smithi064.stdout:7/551: stat d5/d33/d42/c55 0 2024-02-16T22:56:26.711 INFO:tasks.workunit.client.0.smithi064.stdout:3/632: write d4/d14/fce [1902388,19774] 0 2024-02-16T22:56:26.712 INFO:tasks.workunit.client.1.smithi192.stdout:5/626: dwrite d6/d12/d2d/d5c/f99 [0,4194304] 0 2024-02-16T22:56:26.712 INFO:tasks.workunit.client.0.smithi064.stdout:3/633: stat d4/d13/d1b/fcb 0 2024-02-16T22:56:26.715 INFO:tasks.workunit.client.0.smithi064.stdout:9/500: creat d3/d11/d15/d5a/d38/fa6 x:0 0 0 2024-02-16T22:56:26.717 INFO:tasks.workunit.client.1.smithi192.stdout:5/627: read d6/d12/f44 [3287039,105052] 0 2024-02-16T22:56:26.717 INFO:tasks.workunit.client.1.smithi192.stdout:8/620: truncate d5/d26/d83/d86/f1c 8943076 0 2024-02-16T22:56:26.719 INFO:tasks.workunit.client.1.smithi192.stdout:5/628: stat d6/d12/d4e/d46/c49 0 2024-02-16T22:56:26.720 INFO:tasks.workunit.client.0.smithi064.stdout:8/504: mkdir df/d31/d8f/dab 0 2024-02-16T22:56:26.721 INFO:tasks.workunit.client.0.smithi064.stdout:8/505: chown df/d2a/d5c/l95 19540 1 2024-02-16T22:56:26.723 INFO:tasks.workunit.client.1.smithi192.stdout:4/684: dwrite d1/d6b/fa1 [0,4194304] 0 2024-02-16T22:56:26.724 INFO:tasks.workunit.client.1.smithi192.stdout:4/685: chown d1/d26/d34/f50 15001 1 2024-02-16T22:56:26.727 INFO:tasks.workunit.client.0.smithi064.stdout:2/637: rmdir d7/d14/d4b 39 2024-02-16T22:56:26.727 INFO:tasks.workunit.client.1.smithi192.stdout:0/629: dread d1/d3/de/d22/d37/d67/d75/f8e [0,4194304] 0 2024-02-16T22:56:26.728 INFO:tasks.workunit.client.0.smithi064.stdout:5/464: write d6/f31 [1003020,812] 0 2024-02-16T22:56:26.729 INFO:tasks.workunit.client.1.smithi192.stdout:0/630: dread - d1/d3/de/d22/d37/d67/d34/fc1 zero size 2024-02-16T22:56:26.732 INFO:tasks.workunit.client.0.smithi064.stdout:8/506: dwrite df/d31/d61/d79/f9b [0,4194304] 0 2024-02-16T22:56:26.734 INFO:tasks.workunit.client.0.smithi064.stdout:8/507: chown df/c16 2069143 1 2024-02-16T22:56:26.736 INFO:tasks.workunit.client.0.smithi064.stdout:8/508: fsync df/d31/d61/fa8 0 2024-02-16T22:56:26.739 INFO:tasks.workunit.client.0.smithi064.stdout:1/496: mknod d2/df/d11/d71/caf 0 2024-02-16T22:56:26.740 INFO:tasks.workunit.client.0.smithi064.stdout:7/552: mkdir d5/d33/d45/d73/dab 0 2024-02-16T22:56:26.748 INFO:tasks.workunit.client.0.smithi064.stdout:7/553: dwrite d5/d33/d45/d64/f85 [0,4194304] 0 2024-02-16T22:56:26.749 INFO:tasks.workunit.client.1.smithi192.stdout:5/629: dread d6/d12/d1d/f3a [0,4194304] 0 2024-02-16T22:56:26.750 INFO:tasks.workunit.client.0.smithi064.stdout:7/554: fdatasync d5/d33/f60 0 2024-02-16T22:56:26.751 INFO:tasks.workunit.client.0.smithi064.stdout:7/555: readlink d5/l2b 0 2024-02-16T22:56:26.754 INFO:tasks.workunit.client.0.smithi064.stdout:5/465: unlink d6/d25/d48/d67/f6c 0 2024-02-16T22:56:26.755 INFO:tasks.workunit.client.0.smithi064.stdout:5/466: chown d6/d25/d48/c49 53565534 1 2024-02-16T22:56:26.755 INFO:tasks.workunit.client.1.smithi192.stdout:1/587: dwrite d3/de/f35 [4194304,4194304] 0 2024-02-16T22:56:26.756 INFO:tasks.workunit.client.1.smithi192.stdout:3/601: link da/d11/d20/fab da/d11/d63/d6b/d66/fd2 0 2024-02-16T22:56:26.758 INFO:tasks.workunit.client.0.smithi064.stdout:8/509: mknod df/d31/d61/d79/d81/cac 0 2024-02-16T22:56:26.759 INFO:tasks.workunit.client.0.smithi064.stdout:8/510: write df/d31/f29 [2176179,84137] 0 2024-02-16T22:56:26.762 INFO:tasks.workunit.client.1.smithi192.stdout:6/652: write f9 [414518,96898] 0 2024-02-16T22:56:26.763 INFO:tasks.workunit.client.1.smithi192.stdout:7/736: getdents d1/d2/de/d2c/d49/da4 0 2024-02-16T22:56:26.764 INFO:tasks.workunit.client.1.smithi192.stdout:6/653: write da/df/d6b/f88 [3283126,70696] 0 2024-02-16T22:56:26.766 INFO:tasks.workunit.client.1.smithi192.stdout:4/686: dread d1/d26/d66/d89/f1e [0,4194304] 0 2024-02-16T22:56:26.766 INFO:tasks.workunit.client.1.smithi192.stdout:6/654: write da/df/d12/d15/d1b/f8a [502257,82825] 0 2024-02-16T22:56:26.771 INFO:tasks.workunit.client.0.smithi064.stdout:9/501: fsync d3/d11/d15/d5a/d2a/d2e/f5f 0 2024-02-16T22:56:26.773 INFO:tasks.workunit.client.0.smithi064.stdout:9/502: dread - d3/d11/d15/d5a/d2a/d2e/f5f zero size 2024-02-16T22:56:26.774 INFO:tasks.workunit.client.0.smithi064.stdout:3/634: dwrite d4/d13/d35/fac [0,4194304] 0 2024-02-16T22:56:26.776 INFO:tasks.workunit.client.0.smithi064.stdout:9/503: stat d3/d9/d1e/d3e/d76 0 2024-02-16T22:56:26.779 INFO:tasks.workunit.client.0.smithi064.stdout:7/556: rename d5/d33/d45/f9a to d5/d33/d42/fac 0 2024-02-16T22:56:26.781 INFO:tasks.workunit.client.1.smithi192.stdout:9/762: rename d7/f12 to d7/d19/d28/f108 0 2024-02-16T22:56:26.783 INFO:tasks.workunit.client.0.smithi064.stdout:6/604: creat d0/d1b/fc8 x:0 0 0 2024-02-16T22:56:26.784 INFO:tasks.workunit.client.0.smithi064.stdout:6/605: chown d0/d5/df/d17/d4a/l57 1870 1 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Adjusting osd_memory_target on smithi192 to 2581M 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: pgmap v18: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 63 MiB/s rd, 217 MiB/s wr, 379 op/s 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Reconfiguring node-exporter.smithi064 (dependencies changed)... 2024-02-16T22:56:26.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:26 smithi192 ceph-mon[36254]: Reconfiguring daemon node-exporter.smithi064 on smithi064 2024-02-16T22:56:26.786 INFO:tasks.workunit.client.0.smithi064.stdout:8/511: fdatasync df/d2a/f34 0 2024-02-16T22:56:26.787 INFO:tasks.workunit.client.1.smithi192.stdout:9/763: dwrite d7/d15/d36/d38/d3f/fc8 [0,4194304] 0 2024-02-16T22:56:26.787 INFO:tasks.workunit.client.1.smithi192.stdout:8/621: creat d5/d26/d83/d86/d5f/dbf/fc2 x:0 0 0 2024-02-16T22:56:26.789 INFO:tasks.workunit.client.0.smithi064.stdout:8/512: fsync df/d31/d2f/f7d 0 2024-02-16T22:56:26.792 INFO:tasks.workunit.client.1.smithi192.stdout:3/602: creat da/d11/d20/fd3 x:0 0 0 2024-02-16T22:56:26.796 INFO:tasks.workunit.client.0.smithi064.stdout:7/557: dwrite d5/d6/d54/f5e [0,4194304] 0 2024-02-16T22:56:26.797 INFO:tasks.workunit.client.1.smithi192.stdout:7/737: mkdir d1/d2/de/d2c/d49/da4/d104 0 2024-02-16T22:56:26.798 INFO:tasks.workunit.client.0.smithi064.stdout:1/497: dwrite d2/df/fa2 [0,4194304] 0 2024-02-16T22:56:26.801 INFO:tasks.workunit.client.0.smithi064.stdout:9/504: symlink d3/d11/d15/d5a/d38/la7 0 2024-02-16T22:56:26.801 INFO:tasks.workunit.client.0.smithi064.stdout:5/467: rename d6/d1c/d43/f79 to d6/d18/d19/d1a/d85/fa3 0 2024-02-16T22:56:26.811 INFO:tasks.workunit.client.1.smithi192.stdout:9/764: symlink d7/d19/d5e/l109 0 2024-02-16T22:56:26.812 INFO:tasks.workunit.client.1.smithi192.stdout:1/588: dread d3/de/d27/d34/d52/f57 [0,4194304] 0 2024-02-16T22:56:26.820 INFO:tasks.workunit.client.1.smithi192.stdout:6/655: mknod da/cd6 0 2024-02-16T22:56:26.820 INFO:tasks.workunit.client.1.smithi192.stdout:6/656: chown da/df/d6b/d74/lc5 0 1 2024-02-16T22:56:26.823 INFO:tasks.workunit.client.0.smithi064.stdout:8/513: rename df/f12 to df/d31/d47/fad 0 2024-02-16T22:56:26.824 INFO:tasks.workunit.client.0.smithi064.stdout:1/498: fdatasync d2/df/f54 0 2024-02-16T22:56:26.826 INFO:tasks.workunit.client.1.smithi192.stdout:9/765: unlink d7/d19/d37/fb5 0 2024-02-16T22:56:26.828 INFO:tasks.workunit.client.1.smithi192.stdout:2/684: sync 2024-02-16T22:56:26.828 INFO:tasks.workunit.client.0.smithi064.stdout:5/468: mkdir d6/d25/da4 0 2024-02-16T22:56:26.829 INFO:tasks.workunit.client.1.smithi192.stdout:2/685: stat d22/d24/d39/d42/d52/fb5 0 2024-02-16T22:56:26.829 INFO:tasks.workunit.client.1.smithi192.stdout:1/589: rmdir d3/de/d27/d34/d52/d72 39 2024-02-16T22:56:26.830 INFO:tasks.workunit.client.0.smithi064.stdout:6/606: creat d0/d1d/dbc/fc9 x:0 0 0 2024-02-16T22:56:26.831 INFO:tasks.workunit.client.1.smithi192.stdout:4/687: dwrite d1/d26/d66/d89/f68 [0,4194304] 0 2024-02-16T22:56:26.832 INFO:tasks.workunit.client.1.smithi192.stdout:8/622: mkdir d5/d26/d8f/dc0/dc3 0 2024-02-16T22:56:26.833 INFO:tasks.workunit.client.0.smithi064.stdout:7/558: mkdir d5/d6/d65/da7/dad 0 2024-02-16T22:56:26.835 INFO:tasks.workunit.client.1.smithi192.stdout:5/630: rmdir d6/d12/d4e/d5b/d67/d78/dae/dbc/dc1 0 2024-02-16T22:56:26.836 INFO:tasks.workunit.client.1.smithi192.stdout:6/657: dread da/df/d6b/f88 [0,4194304] 0 2024-02-16T22:56:26.837 INFO:tasks.workunit.client.1.smithi192.stdout:3/603: mkdir da/d11/dd4 0 2024-02-16T22:56:26.838 INFO:tasks.workunit.client.1.smithi192.stdout:4/688: dwrite d1/d26/d66/d89/d11/d58/f5d [0,4194304] 0 2024-02-16T22:56:26.838 INFO:tasks.workunit.client.1.smithi192.stdout:4/689: chown d1/f4 53 1 2024-02-16T22:56:26.841 INFO:tasks.workunit.client.0.smithi064.stdout:9/505: truncate d3/f10 4652191 0 2024-02-16T22:56:26.842 INFO:tasks.workunit.client.0.smithi064.stdout:8/514: dread df/d31/d2f/f55 [0,4194304] 0 2024-02-16T22:56:26.842 INFO:tasks.workunit.client.0.smithi064.stdout:9/506: readlink d3/d9/d1e/l67 0 2024-02-16T22:56:26.848 INFO:tasks.workunit.client.0.smithi064.stdout:7/559: rmdir d5/d33/d42 39 2024-02-16T22:56:26.849 INFO:tasks.workunit.client.1.smithi192.stdout:1/590: mkdir d3/de/d27/d34/d52/d63/d7a/dc9 0 2024-02-16T22:56:26.852 INFO:tasks.workunit.client.0.smithi064.stdout:9/507: creat d3/d9/d1e/d42/d7b/fa8 x:0 0 0 2024-02-16T22:56:26.852 INFO:tasks.workunit.client.0.smithi064.stdout:7/560: read d5/d6/d54/d41/f4b [2236967,127600] 0 2024-02-16T22:56:26.853 INFO:tasks.workunit.client.1.smithi192.stdout:5/631: creat d6/d12/d4e/d5b/daf/fcd x:0 0 0 2024-02-16T22:56:26.853 INFO:tasks.workunit.client.0.smithi064.stdout:9/508: write d3/f8a [819215,50038] 0 2024-02-16T22:56:26.854 INFO:tasks.workunit.client.1.smithi192.stdout:6/658: dread da/df/d12/d15/d1b/d70/fb1 [0,4194304] 0 2024-02-16T22:56:26.855 INFO:tasks.workunit.client.0.smithi064.stdout:6/607: mknod d0/d3c/dc5/cca 0 2024-02-16T22:56:26.857 INFO:tasks.workunit.client.0.smithi064.stdout:5/469: truncate d6/f13 9609 0 2024-02-16T22:56:26.858 INFO:tasks.workunit.client.1.smithi192.stdout:7/738: creat d1/d2/de/d2c/d49/d54/f105 x:0 0 0 2024-02-16T22:56:26.860 INFO:tasks.workunit.client.0.smithi064.stdout:9/509: dread d3/d11/d15/f51 [0,4194304] 0 2024-02-16T22:56:26.861 INFO:tasks.workunit.client.1.smithi192.stdout:4/690: symlink d1/d26/d6f/lf0 0 2024-02-16T22:56:26.862 INFO:tasks.workunit.client.0.smithi064.stdout:5/470: dread d6/d18/d19/f2e [0,4194304] 0 2024-02-16T22:56:26.863 INFO:tasks.workunit.client.0.smithi064.stdout:3/635: getdents d4/d13/d1b/d27/d81 0 2024-02-16T22:56:26.865 INFO:tasks.workunit.client.0.smithi064.stdout:8/515: mkdir df/d31/d8f/dab/dae 0 2024-02-16T22:56:26.865 INFO:tasks.workunit.client.0.smithi064.stdout:8/516: write df/f37 [5061952,107053] 0 2024-02-16T22:56:26.869 INFO:tasks.workunit.client.1.smithi192.stdout:2/686: symlink d22/d24/ldc 0 2024-02-16T22:56:26.873 INFO:tasks.workunit.client.1.smithi192.stdout:8/623: truncate d5/db/d47/d8c/f9c 938597 0 2024-02-16T22:56:26.878 INFO:tasks.workunit.client.1.smithi192.stdout:6/659: mknod da/df/d12/d57/cd7 0 2024-02-16T22:56:26.879 INFO:tasks.workunit.client.1.smithi192.stdout:6/660: fsync da/df/d18/f29 0 2024-02-16T22:56:26.879 INFO:tasks.workunit.client.0.smithi064.stdout:1/499: dwrite d2/df/d14/d2c/f49 [0,4194304] 0 2024-02-16T22:56:26.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/661: fdatasync da/df/d12/d15/d1b/f60 0 2024-02-16T22:56:26.882 INFO:tasks.workunit.client.0.smithi064.stdout:9/510: mkdir d3/d11/d9e/da9 0 2024-02-16T22:56:26.886 INFO:tasks.workunit.client.1.smithi192.stdout:7/739: mknod d1/d2/de/d2c/c106 0 2024-02-16T22:56:26.886 INFO:tasks.workunit.client.0.smithi064.stdout:1/500: dread d2/df/f13 [4194304,4194304] 0 2024-02-16T22:56:26.887 INFO:tasks.workunit.client.1.smithi192.stdout:7/740: stat d1/d2/d2d/d92/dcd/fe0 0 2024-02-16T22:56:26.888 INFO:tasks.workunit.client.0.smithi064.stdout:5/471: mkdir d6/d25/d48/d67/d91/da5 0 2024-02-16T22:56:26.888 INFO:tasks.workunit.client.1.smithi192.stdout:4/691: creat d1/d15/d48/d8d/d9b/ff1 x:0 0 0 2024-02-16T22:56:26.888 INFO:tasks.workunit.client.1.smithi192.stdout:7/741: dread - d1/fc1 zero size 2024-02-16T22:56:26.889 INFO:tasks.workunit.client.0.smithi064.stdout:5/472: write d6/d18/d19/d1a/f60 [547683,42804] 0 2024-02-16T22:56:26.891 INFO:tasks.workunit.client.1.smithi192.stdout:3/604: dwrite da/df/f79 [0,4194304] 0 2024-02-16T22:56:26.891 INFO:tasks.workunit.client.0.smithi064.stdout:3/636: rename d4/d13/d1b/d4f/fb3 to d4/d13/d1b/d27/d61/d63/fd3 0 2024-02-16T22:56:26.891 INFO:tasks.workunit.client.0.smithi064.stdout:5/473: dread - d6/d18/f8a zero size 2024-02-16T22:56:26.892 INFO:tasks.workunit.client.1.smithi192.stdout:9/766: creat d7/d19/d28/d43/d70/f10a x:0 0 0 2024-02-16T22:56:26.893 INFO:tasks.workunit.client.1.smithi192.stdout:9/767: chown d7/d19/d5e/l109 243975 1 2024-02-16T22:56:26.895 INFO:tasks.workunit.client.1.smithi192.stdout:2/687: creat d22/d24/d39/d42/d65/d69/db1/fdd x:0 0 0 2024-02-16T22:56:26.896 INFO:tasks.workunit.client.1.smithi192.stdout:1/591: symlink d3/de/d27/d34/d52/lca 0 2024-02-16T22:56:26.896 INFO:tasks.workunit.client.1.smithi192.stdout:2/688: fsync d22/d24/d39/d42/d85/d98/db0/fc1 0 2024-02-16T22:56:26.902 INFO:tasks.workunit.client.1.smithi192.stdout:0/631: sync 2024-02-16T22:56:26.907 INFO:tasks.workunit.client.0.smithi064.stdout:8/517: rename df/d2a/f60 to df/d31/d2f/d3e/faf 0 2024-02-16T22:56:26.909 INFO:tasks.workunit.client.1.smithi192.stdout:9/768: mkdir d7/d19/d37/dbf/d10b 0 2024-02-16T22:56:26.910 INFO:tasks.workunit.client.0.smithi064.stdout:5/474: dread d6/d25/d48/f5e [0,4194304] 0 2024-02-16T22:56:26.910 INFO:tasks.workunit.client.1.smithi192.stdout:9/769: fdatasync d7/d19/d28/fb9 0 2024-02-16T22:56:26.911 INFO:tasks.workunit.client.1.smithi192.stdout:9/770: chown d7/cf7 3004 1 2024-02-16T22:56:26.913 INFO:tasks.workunit.client.1.smithi192.stdout:1/592: rmdir d3/d40/d9f 39 2024-02-16T22:56:26.914 INFO:tasks.workunit.client.1.smithi192.stdout:1/593: chown d3/de/d27/d34/d52/d5a/f90 51 1 2024-02-16T22:56:26.915 INFO:tasks.workunit.client.1.smithi192.stdout:1/594: chown d3/de/ff 777936276 1 2024-02-16T22:56:26.915 INFO:tasks.workunit.client.0.smithi064.stdout:6/608: creat d0/d1b/fcb x:0 0 0 2024-02-16T22:56:26.916 INFO:tasks.workunit.client.1.smithi192.stdout:7/742: dread d1/f11 [4194304,4194304] 0 2024-02-16T22:56:26.916 INFO:tasks.workunit.client.1.smithi192.stdout:7/743: chown d1/de1/lf6 5984368 1 2024-02-16T22:56:26.930 INFO:tasks.workunit.client.0.smithi064.stdout:7/561: rename d5/d33/d42/fac to d5/d33/d42/d81/fae 0 2024-02-16T22:56:26.931 INFO:tasks.workunit.client.1.smithi192.stdout:4/692: rmdir d1/d26/dbe 39 2024-02-16T22:56:26.932 INFO:tasks.workunit.client.0.smithi064.stdout:5/475: write d6/d1c/f20 [3497514,96595] 0 2024-02-16T22:56:26.938 INFO:tasks.workunit.client.1.smithi192.stdout:9/771: creat d7/d19/d37/d39/f10c x:0 0 0 2024-02-16T22:56:26.941 INFO:tasks.workunit.client.0.smithi064.stdout:9/511: link d3/fc d3/d11/d15/d5a/d2a/d2e/d31/d3f/faa 0 2024-02-16T22:56:26.943 INFO:tasks.workunit.client.1.smithi192.stdout:1/595: dread d3/de/d27/d34/d36/d47/f79 [0,4194304] 0 2024-02-16T22:56:26.944 INFO:tasks.workunit.client.0.smithi064.stdout:7/562: unlink d5/d6/d54/l51 0 2024-02-16T22:56:26.944 INFO:tasks.workunit.client.0.smithi064.stdout:7/563: fdatasync d5/d33/d45/d73/fa4 0 2024-02-16T22:56:26.946 INFO:tasks.workunit.client.0.smithi064.stdout:3/637: getdents d4/d13/d1b/d27/d99 0 2024-02-16T22:56:26.947 INFO:tasks.workunit.client.1.smithi192.stdout:5/632: getdents d6/d12/d4e/d5b/daf 0 2024-02-16T22:56:26.948 INFO:tasks.workunit.client.0.smithi064.stdout:3/638: write d4/d13/d1b/d27/d99/d7d/fc8 [2331151,80083] 0 2024-02-16T22:56:26.948 INFO:tasks.workunit.client.0.smithi064.stdout:6/609: symlink d0/d1d/db1/lcc 0 2024-02-16T22:56:26.949 INFO:tasks.workunit.client.1.smithi192.stdout:2/689: write d22/d24/d39/d42/d85/d98/db0/db6/fce [4931129,6293] 0 2024-02-16T22:56:26.950 INFO:tasks.workunit.client.0.smithi064.stdout:6/610: dread - d0/d89/f93 zero size 2024-02-16T22:56:26.950 INFO:tasks.workunit.client.1.smithi192.stdout:0/632: fdatasync d1/f2a 0 2024-02-16T22:56:26.951 INFO:tasks.workunit.client.0.smithi064.stdout:6/611: read d0/d5/df/d17/f7d [326353,64544] 0 2024-02-16T22:56:26.952 INFO:tasks.workunit.client.1.smithi192.stdout:8/624: dwrite d5/d9/f54 [0,4194304] 0 2024-02-16T22:56:26.952 INFO:tasks.workunit.client.1.smithi192.stdout:2/690: chown d22/d24/d39/d42/d85/d98/cd7 1 1 2024-02-16T22:56:26.953 INFO:tasks.workunit.client.1.smithi192.stdout:8/625: fdatasync d5/d16/f18 0 2024-02-16T22:56:26.954 INFO:tasks.workunit.client.1.smithi192.stdout:6/662: mkdir da/df/d18/d52/da9/dd8 0 2024-02-16T22:56:26.954 INFO:tasks.workunit.client.0.smithi064.stdout:1/501: getdents d2/df 0 2024-02-16T22:56:26.957 INFO:tasks.workunit.client.0.smithi064.stdout:9/512: mkdir d3/d11/d15/dab 0 2024-02-16T22:56:26.962 INFO:tasks.workunit.client.1.smithi192.stdout:3/605: link da/d11/d63/d6b/d4e/l7f da/d11/d35/ld5 0 2024-02-16T22:56:26.963 INFO:tasks.workunit.client.0.smithi064.stdout:5/476: fdatasync d6/fc 0 2024-02-16T22:56:26.963 INFO:tasks.workunit.client.0.smithi064.stdout:1/502: dwrite d2/df/d14/d18/d40/f68 [0,4194304] 0 2024-02-16T22:56:26.966 INFO:tasks.workunit.client.1.smithi192.stdout:9/772: dread d7/d15/d36/ff3 [0,4194304] 0 2024-02-16T22:56:26.970 INFO:tasks.workunit.client.1.smithi192.stdout:3/606: dwrite da/d11/d63/d6b/f87 [0,4194304] 0 2024-02-16T22:56:26.973 INFO:tasks.workunit.client.1.smithi192.stdout:5/633: mknod d6/d8/d75/d8c/dc4/cce 0 2024-02-16T22:56:26.974 INFO:tasks.workunit.client.1.smithi192.stdout:5/634: fsync d6/d12/d2d/f43 0 2024-02-16T22:56:26.975 INFO:tasks.workunit.client.1.smithi192.stdout:5/635: chown d6/d12/d1d/d56 18 1 2024-02-16T22:56:26.977 INFO:tasks.workunit.client.1.smithi192.stdout:9/773: dwrite d7/d19/d37/ff2 [0,4194304] 0 2024-02-16T22:56:26.977 INFO:tasks.workunit.client.1.smithi192.stdout:5/636: write d6/d12/d4e/d5b/daf/fbe [905346,44894] 0 2024-02-16T22:56:26.980 INFO:tasks.workunit.client.1.smithi192.stdout:5/637: write d6/d12/d4e/d46/d4f/f6e [278568,28727] 0 2024-02-16T22:56:26.984 INFO:tasks.workunit.client.1.smithi192.stdout:5/638: write d6/d8/fa3 [906098,75141] 0 2024-02-16T22:56:26.985 INFO:tasks.workunit.client.1.smithi192.stdout:0/633: rmdir d1/d3/de/d22/d37/d47 39 2024-02-16T22:56:26.985 INFO:tasks.workunit.client.1.smithi192.stdout:5/639: fsync d6/fc 0 2024-02-16T22:56:26.986 INFO:tasks.workunit.client.1.smithi192.stdout:4/693: dwrite d1/d26/d66/d89/f3a [0,4194304] 0 2024-02-16T22:56:26.986 INFO:tasks.workunit.client.1.smithi192.stdout:7/744: write d1/de1/f63 [3912512,53957] 0 2024-02-16T22:56:26.987 INFO:tasks.workunit.client.0.smithi064.stdout:7/564: dread d5/d6/f30 [0,4194304] 0 2024-02-16T22:56:26.987 INFO:tasks.workunit.client.1.smithi192.stdout:0/634: chown d1/d3/de/d22/da7 671158 1 2024-02-16T22:56:26.989 INFO:tasks.workunit.client.1.smithi192.stdout:4/694: chown d1/d26/d66/d89/d11/l33 9 1 2024-02-16T22:56:26.992 INFO:tasks.workunit.client.1.smithi192.stdout:8/626: mknod d5/d68/cc4 0 2024-02-16T22:56:26.993 INFO:tasks.workunit.client.1.smithi192.stdout:0/635: dwrite d1/d3/d39/f5f [0,4194304] 0 2024-02-16T22:56:27.002 INFO:tasks.workunit.client.0.smithi064.stdout:1/503: rename d2/df/d14/d2c/d62/d91/d56/f57 to d2/fb0 0 2024-02-16T22:56:27.004 INFO:tasks.workunit.client.0.smithi064.stdout:7/565: dwrite d5/d33/d42/f8f [0,4194304] 0 2024-02-16T22:56:27.006 INFO:tasks.workunit.client.0.smithi064.stdout:7/566: truncate d5/d6/d65/d7f/f9c 303862 0 2024-02-16T22:56:27.007 INFO:tasks.workunit.client.1.smithi192.stdout:6/663: creat da/df/d71/fd9 x:0 0 0 2024-02-16T22:56:27.007 INFO:tasks.workunit.client.0.smithi064.stdout:7/567: fdatasync d5/d33/f8a 0 2024-02-16T22:56:27.008 INFO:tasks.workunit.client.0.smithi064.stdout:9/513: symlink d3/d11/d9e/da9/lac 0 2024-02-16T22:56:27.010 INFO:tasks.workunit.client.0.smithi064.stdout:5/477: symlink d6/d18/d95/la6 0 2024-02-16T22:56:27.017 INFO:tasks.workunit.client.0.smithi064.stdout:3/639: dread d4/d13/d1b/d4f/fa2 [0,4194304] 0 2024-02-16T22:56:27.017 INFO:tasks.workunit.client.0.smithi064.stdout:3/640: chown d4/f1e 20901756 1 2024-02-16T22:56:27.019 INFO:tasks.workunit.client.1.smithi192.stdout:3/607: symlink da/d11/d35/ld6 0 2024-02-16T22:56:27.022 INFO:tasks.workunit.client.0.smithi064.stdout:3/641: dwrite d4/d1f/d5b/d51/fa4 [0,4194304] 0 2024-02-16T22:56:27.032 INFO:tasks.workunit.client.1.smithi192.stdout:5/640: mkdir d6/d12/d1d/d56/dcf 0 2024-02-16T22:56:27.034 INFO:tasks.workunit.client.0.smithi064.stdout:0/483: sync 2024-02-16T22:56:27.044 INFO:tasks.workunit.client.1.smithi192.stdout:2/691: link f4 d22/d47/dcb/d3f/dac/fde 0 2024-02-16T22:56:27.046 INFO:tasks.workunit.client.1.smithi192.stdout:6/664: rmdir da/df/d12/d15/d1b/d70 39 2024-02-16T22:56:27.050 INFO:tasks.workunit.client.1.smithi192.stdout:3/608: mkdir da/d11/d63/d6b/d71/da1/db9/dd7 0 2024-02-16T22:56:27.055 INFO:tasks.workunit.client.1.smithi192.stdout:2/692: dread d22/d24/d39/d42/d65/d69/f7a [0,4194304] 0 2024-02-16T22:56:27.055 INFO:tasks.workunit.client.1.smithi192.stdout:5/641: creat d6/d12/d2d/fd0 x:0 0 0 2024-02-16T22:56:27.056 INFO:tasks.workunit.client.1.smithi192.stdout:2/693: write d22/d24/d39/d42/d65/f8e [1406777,54262] 0 2024-02-16T22:56:27.058 INFO:tasks.workunit.client.1.smithi192.stdout:2/694: truncate d22/d24/d39/d42/d65/d69/d4f/fbf 4895312 0 2024-02-16T22:56:27.058 INFO:tasks.workunit.client.1.smithi192.stdout:2/695: chown d22/d24/d2b/d99 44098531 1 2024-02-16T22:56:27.059 INFO:tasks.workunit.client.0.smithi064.stdout:6/612: rename d0/d5/f11 to d0/d5/fcd 0 2024-02-16T22:56:27.060 INFO:tasks.workunit.client.0.smithi064.stdout:6/613: truncate d0/d5/fab 311699 0 2024-02-16T22:56:27.061 INFO:tasks.workunit.client.0.smithi064.stdout:6/614: chown d0/d5/df/d17/d4a/db4 155077169 1 2024-02-16T22:56:27.062 INFO:tasks.workunit.client.1.smithi192.stdout:9/774: write d7/f8 [323825,68583] 0 2024-02-16T22:56:27.063 INFO:tasks.workunit.client.0.smithi064.stdout:6/615: dread d0/d5/f90 [0,4194304] 0 2024-02-16T22:56:27.069 INFO:tasks.workunit.client.1.smithi192.stdout:5/642: symlink d6/d12/d4e/d5b/daf/ld1 0 2024-02-16T22:56:27.070 INFO:tasks.workunit.client.1.smithi192.stdout:5/643: chown d6/d12/d4e/l45 1471526 1 2024-02-16T22:56:27.072 INFO:tasks.workunit.client.1.smithi192.stdout:5/644: write d6/d8/f9e [446302,98251] 0 2024-02-16T22:56:27.074 INFO:tasks.workunit.client.1.smithi192.stdout:7/745: dwrite d1/d2/de/d2c/f96 [0,4194304] 0 2024-02-16T22:56:27.075 INFO:tasks.workunit.client.1.smithi192.stdout:1/596: sync 2024-02-16T22:56:27.077 INFO:tasks.workunit.client.0.smithi064.stdout:0/484: stat da/d26 0 2024-02-16T22:56:27.077 INFO:tasks.workunit.client.1.smithi192.stdout:4/695: write d1/d15/f4f [5068635,95958] 0 2024-02-16T22:56:27.077 INFO:tasks.workunit.client.1.smithi192.stdout:5/645: dread d6/d12/d1d/d56/fc8 [0,4194304] 0 2024-02-16T22:56:27.079 INFO:tasks.workunit.client.1.smithi192.stdout:6/665: creat da/df/d12/d15/d1b/d70/fda x:0 0 0 2024-02-16T22:56:27.081 INFO:tasks.workunit.client.1.smithi192.stdout:5/646: dread d6/d8/fa1 [0,4194304] 0 2024-02-16T22:56:27.082 INFO:tasks.workunit.client.1.smithi192.stdout:9/775: rmdir d7/d15/d36/d77/dbd 39 2024-02-16T22:56:27.084 INFO:tasks.workunit.client.0.smithi064.stdout:9/514: dread d3/d11/d15/d5a/d2a/d2e/d31/d3f/f5e [0,4194304] 0 2024-02-16T22:56:27.090 INFO:tasks.workunit.client.1.smithi192.stdout:7/746: mknod d1/d2/d2d/d61/d8a/dbb/c107 0 2024-02-16T22:56:27.091 INFO:tasks.workunit.client.1.smithi192.stdout:7/747: stat d1/d2/d2d/d61/d8a/de6/c101 0 2024-02-16T22:56:27.092 INFO:tasks.workunit.client.1.smithi192.stdout:7/748: readlink d1/d2/de/d2c/d49/d89/dbd/lff 0 2024-02-16T22:56:27.092 INFO:tasks.workunit.client.0.smithi064.stdout:0/485: chown da/d71/f8d 8588 1 2024-02-16T22:56:27.094 INFO:tasks.workunit.client.1.smithi192.stdout:1/597: rmdir d3/de/d27/d34/d69/d85/d89/daa 39 2024-02-16T22:56:27.096 INFO:tasks.workunit.client.0.smithi064.stdout:9/515: creat d3/d9/d1e/fad x:0 0 0 2024-02-16T22:56:27.100 INFO:tasks.workunit.client.0.smithi064.stdout:7/568: creat d5/d33/faf x:0 0 0 2024-02-16T22:56:27.103 INFO:tasks.workunit.client.0.smithi064.stdout:1/504: dwrite d2/df/d14/d2a/f87 [0,4194304] 0 2024-02-16T22:56:27.103 INFO:tasks.workunit.client.1.smithi192.stdout:4/696: unlink d1/d15/d29/d3c/lb6 0 2024-02-16T22:56:27.103 INFO:tasks.workunit.client.1.smithi192.stdout:4/697: dread d1/f85 [0,4194304] 0 2024-02-16T22:56:27.104 INFO:tasks.workunit.client.0.smithi064.stdout:7/569: truncate d5/d33/d45/d73/f86 1009658 0 2024-02-16T22:56:27.104 INFO:tasks.workunit.client.0.smithi064.stdout:5/478: link d6/d25/d48/d67/c74 d6/d25/da4/ca7 0 2024-02-16T22:56:27.104 INFO:tasks.workunit.client.0.smithi064.stdout:1/505: fdatasync d2/fa6 0 2024-02-16T22:56:27.105 INFO:tasks.workunit.client.0.smithi064.stdout:5/479: chown d6/d25/d48/d67/c71 3310049 1 2024-02-16T22:56:27.106 INFO:tasks.workunit.client.0.smithi064.stdout:3/642: write d4/d13/d1b/d4f/fa2 [2838457,71298] 0 2024-02-16T22:56:27.108 INFO:tasks.workunit.client.1.smithi192.stdout:2/696: dwrite d22/d24/d2b/f8a [0,4194304] 0 2024-02-16T22:56:27.109 INFO:tasks.workunit.client.0.smithi064.stdout:0/486: rename da/d26/c69 to da/dc/de/d12/d2c/d2e/d62/ca2 0 2024-02-16T22:56:27.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/627: sync 2024-02-16T22:56:27.111 INFO:tasks.workunit.client.1.smithi192.stdout:5/647: write d6/d12/d1d/f3a [2417617,19749] 0 2024-02-16T22:56:27.111 INFO:tasks.workunit.client.1.smithi192.stdout:8/628: stat d5/db/d47/d8c 0 2024-02-16T22:56:27.114 INFO:tasks.workunit.client.1.smithi192.stdout:9/776: fsync d7/d19/d37/d5a/ff5 0 2024-02-16T22:56:27.118 INFO:tasks.workunit.client.1.smithi192.stdout:0/636: dwrite d1/d3/de/d22/d37/d67/d34/f92 [0,4194304] 0 2024-02-16T22:56:27.119 INFO:tasks.workunit.client.0.smithi064.stdout:1/506: symlink d2/df/d11/d1a/d30/d3d/lb1 0 2024-02-16T22:56:27.121 INFO:tasks.workunit.client.1.smithi192.stdout:9/777: dread d7/f8 [0,4194304] 0 2024-02-16T22:56:27.122 INFO:tasks.workunit.client.0.smithi064.stdout:5/480: symlink d6/d25/la8 0 2024-02-16T22:56:27.122 INFO:tasks.workunit.client.0.smithi064.stdout:5/481: readlink d6/l11 0 2024-02-16T22:56:27.122 INFO:tasks.workunit.client.0.smithi064.stdout:5/482: readlink d6/d25/l50 0 2024-02-16T22:56:27.122 INFO:tasks.workunit.client.1.smithi192.stdout:1/598: mknod d3/d5b/d77/ccb 0 2024-02-16T22:56:27.125 INFO:tasks.workunit.client.0.smithi064.stdout:7/570: mknod d5/d33/d45/d64/d56/d5d/cb0 0 2024-02-16T22:56:27.132 INFO:tasks.workunit.client.1.smithi192.stdout:4/698: rename d1/fd to d1/d26/d66/d89/da8/da9/ff2 0 2024-02-16T22:56:27.138 INFO:tasks.workunit.client.0.smithi064.stdout:0/487: rename da/dc/de/d12/f27 to da/dc/de/d12/d2c/d2e/d38/d7e/fa3 0 2024-02-16T22:56:27.140 INFO:tasks.workunit.client.0.smithi064.stdout:1/507: rmdir d2/df/d11/d1a 39 2024-02-16T22:56:27.143 INFO:tasks.workunit.client.1.smithi192.stdout:8/629: write d5/d9/f44 [2545988,10515] 0 2024-02-16T22:56:27.144 INFO:tasks.workunit.client.1.smithi192.stdout:8/630: write d5/d9/f73 [2727116,18880] 0 2024-02-16T22:56:27.145 INFO:tasks.workunit.client.0.smithi064.stdout:7/571: creat d5/d6/d54/d7b/fb1 x:0 0 0 2024-02-16T22:56:27.146 INFO:tasks.workunit.client.0.smithi064.stdout:9/516: link d3/d9/c97 d3/d11/d15/cae 0 2024-02-16T22:56:27.148 INFO:tasks.workunit.client.0.smithi064.stdout:9/517: write d3/d9/d1e/d3e/d76/f79 [732843,103966] 0 2024-02-16T22:56:27.149 INFO:tasks.workunit.client.1.smithi192.stdout:5/648: mknod d6/d12/d1d/d56/cd2 0 2024-02-16T22:56:27.150 INFO:tasks.workunit.client.0.smithi064.stdout:0/488: mkdir da/d71/da4 0 2024-02-16T22:56:27.152 INFO:tasks.workunit.client.0.smithi064.stdout:5/483: dread d6/d18/d19/f47 [0,4194304] 0 2024-02-16T22:56:27.153 INFO:tasks.workunit.client.0.smithi064.stdout:5/484: readlink d6/d18/d39/l66 0 2024-02-16T22:56:27.153 INFO:tasks.workunit.client.1.smithi192.stdout:8/631: dread d5/db/d47/d43/d90/fbe [0,4194304] 0 2024-02-16T22:56:27.154 INFO:tasks.workunit.client.0.smithi064.stdout:5/485: chown d6/d25/d48/d67/d91/da5 25248 1 2024-02-16T22:56:27.157 INFO:tasks.workunit.client.0.smithi064.stdout:7/572: mkdir d5/d6/d65/d7f/db2 0 2024-02-16T22:56:27.158 INFO:tasks.workunit.client.1.smithi192.stdout:3/609: link da/d11/f38 da/d11/d35/fd8 0 2024-02-16T22:56:27.160 INFO:tasks.workunit.client.1.smithi192.stdout:0/637: creat d1/d3/de/d22/da7/fc9 x:0 0 0 2024-02-16T22:56:27.161 INFO:tasks.workunit.client.0.smithi064.stdout:0/489: fsync da/dc/de/d12/f42 0 2024-02-16T22:56:27.163 INFO:tasks.workunit.client.1.smithi192.stdout:2/697: dwrite d22/d47/f7d [0,4194304] 0 2024-02-16T22:56:27.165 INFO:tasks.workunit.client.0.smithi064.stdout:1/508: mknod d2/df/d11/d25/cb2 0 2024-02-16T22:56:27.166 INFO:tasks.workunit.client.0.smithi064.stdout:1/509: chown d2/df/d14/d18/d48/d84 9 1 2024-02-16T22:56:27.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/599: unlink d3/de/d27/d34/d52/d5a/c66 0 2024-02-16T22:56:27.171 INFO:tasks.workunit.client.1.smithi192.stdout:8/632: dread d5/f61 [0,4194304] 0 2024-02-16T22:56:27.176 INFO:tasks.workunit.client.0.smithi064.stdout:0/490: creat da/dc/d3b/d72/d7c/fa5 x:0 0 0 2024-02-16T22:56:27.177 INFO:tasks.workunit.client.0.smithi064.stdout:7/573: rename d5/d6/l17 to d5/d33/d45/d64/d56/d5d/lb3 0 2024-02-16T22:56:27.178 INFO:tasks.workunit.client.0.smithi064.stdout:7/574: read d5/d6/f22 [1893374,1354] 0 2024-02-16T22:56:27.179 INFO:tasks.workunit.client.0.smithi064.stdout:9/518: link d3/f8a d3/d11/d15/d5a/d2a/d2e/d31/d3f/d80/faf 0 2024-02-16T22:56:27.179 INFO:tasks.workunit.client.0.smithi064.stdout:7/575: dread - d5/d33/d53/f75 zero size 2024-02-16T22:56:27.182 INFO:tasks.workunit.client.0.smithi064.stdout:0/491: creat da/dc/de/d37/fa6 x:0 0 0 2024-02-16T22:56:27.186 INFO:tasks.workunit.client.0.smithi064.stdout:1/510: symlink d2/df/d14/d2c/d62/d91/d56/d5b/lb3 0 2024-02-16T22:56:27.192 INFO:tasks.workunit.client.0.smithi064.stdout:0/492: rename da/dc/de/d12/d2c/c43 to da/d71/da4/ca7 0 2024-02-16T22:56:27.196 INFO:tasks.workunit.client.0.smithi064.stdout:1/511: dread - d2/df/d14/d2c/d62/d91/d56/f7b zero size 2024-02-16T22:56:27.197 INFO:tasks.workunit.client.0.smithi064.stdout:7/576: write d5/d33/d42/f74 [1182132,112729] 0 2024-02-16T22:56:27.197 INFO:tasks.workunit.client.1.smithi192.stdout:7/749: link d1/d2/de/fc7 d1/d2/d37/d86/f108 0 2024-02-16T22:56:27.200 INFO:tasks.workunit.client.1.smithi192.stdout:9/778: getdents d7/d19/d37/dbf/d10b 0 2024-02-16T22:56:27.201 INFO:tasks.workunit.client.0.smithi064.stdout:1/512: dread d2/df/d14/d2c/f3c [0,4194304] 0 2024-02-16T22:56:27.202 INFO:tasks.workunit.client.0.smithi064.stdout:1/513: fsync d2/df/d11/d25/f85 0 2024-02-16T22:56:27.202 INFO:tasks.workunit.client.1.smithi192.stdout:1/600: read - d3/de/d27/d34/d69/d85/d89/daa/fb3 zero size 2024-02-16T22:56:27.205 INFO:tasks.workunit.client.1.smithi192.stdout:8/633: mknod d5/db/d47/d43/cc5 0 2024-02-16T22:56:27.205 INFO:tasks.workunit.client.1.smithi192.stdout:0/638: dwrite d1/dc7/d9c/f72 [0,4194304] 0 2024-02-16T22:56:27.208 INFO:tasks.workunit.client.0.smithi064.stdout:7/577: creat d5/d33/d45/d73/fb4 x:0 0 0 2024-02-16T22:56:27.208 INFO:tasks.workunit.client.1.smithi192.stdout:4/699: fsync d1/d26/d66/d89/d11/d58/f7e 0 2024-02-16T22:56:27.209 INFO:tasks.workunit.client.1.smithi192.stdout:6/666: getdents da/df/d18/d46 0 2024-02-16T22:56:27.210 INFO:tasks.workunit.client.0.smithi064.stdout:4/566: sync 2024-02-16T22:56:27.210 INFO:tasks.workunit.client.0.smithi064.stdout:8/518: sync 2024-02-16T22:56:27.210 INFO:tasks.workunit.client.0.smithi064.stdout:2/638: sync 2024-02-16T22:56:27.211 INFO:tasks.workunit.client.0.smithi064.stdout:0/493: dwrite da/dc/de/d12/d2c/f31 [0,4194304] 0 2024-02-16T22:56:27.211 INFO:tasks.workunit.client.1.smithi192.stdout:5/649: creat d6/d12/d4e/d58/fd3 x:0 0 0 2024-02-16T22:56:27.211 INFO:tasks.workunit.client.0.smithi064.stdout:8/519: chown df/d2a 3152208 1 2024-02-16T22:56:27.212 INFO:tasks.workunit.client.1.smithi192.stdout:0/639: dread d1/d3/de/d22/d37/d67/f9b [0,4194304] 0 2024-02-16T22:56:27.213 INFO:tasks.workunit.client.1.smithi192.stdout:0/640: readlink d1/d3/de/d22/d37/d67/d34/l94 0 2024-02-16T22:56:27.215 INFO:tasks.workunit.client.0.smithi064.stdout:1/514: symlink d2/df/d14/d18/lb4 0 2024-02-16T22:56:27.216 INFO:tasks.workunit.client.0.smithi064.stdout:1/515: readlink d2/l7 0 2024-02-16T22:56:27.216 INFO:tasks.workunit.client.0.smithi064.stdout:0/494: dread - da/dc/de/d12/d2c/d2e/d38/d87/f9f zero size 2024-02-16T22:56:27.220 INFO:tasks.workunit.client.0.smithi064.stdout:7/578: readlink d5/l2b 0 2024-02-16T22:56:27.220 INFO:tasks.workunit.client.0.smithi064.stdout:0/495: chown da/dc/ld 34 1 2024-02-16T22:56:27.220 INFO:tasks.workunit.client.0.smithi064.stdout:2/639: dwrite d7/f10 [0,4194304] 0 2024-02-16T22:56:27.221 INFO:tasks.workunit.client.1.smithi192.stdout:5/650: dread d6/d8/f9e [0,4194304] 0 2024-02-16T22:56:27.221 INFO:tasks.workunit.client.1.smithi192.stdout:5/651: chown d6/d8/c74 0 1 2024-02-16T22:56:27.221 INFO:tasks.workunit.client.1.smithi192.stdout:2/698: mknod d22/d24/d39/d42/cdf 0 2024-02-16T22:56:27.221 INFO:tasks.workunit.client.0.smithi064.stdout:9/519: link d3/d11/d15/d5a/f26 d3/d11/d15/d5a/fb0 0 2024-02-16T22:56:27.226 INFO:tasks.workunit.client.0.smithi064.stdout:4/567: fsync dc/dd/d16/f5d 0 2024-02-16T22:56:27.233 INFO:tasks.workunit.client.1.smithi192.stdout:8/634: dread d5/f84 [0,4194304] 0 2024-02-16T22:56:27.234 INFO:tasks.workunit.client.0.smithi064.stdout:0/496: chown da/dc/de/d12/d2c/d2e/d38/l52 821719 1 2024-02-16T22:56:27.235 INFO:tasks.workunit.client.0.smithi064.stdout:0/497: chown da/dc/de/d12/d2c/l59 0 1 2024-02-16T22:56:27.236 INFO:tasks.workunit.client.0.smithi064.stdout:2/640: dread d7/de/f2f [0,4194304] 0 2024-02-16T22:56:27.238 INFO:tasks.workunit.client.0.smithi064.stdout:8/520: dread f8 [0,4194304] 0 2024-02-16T22:56:27.239 INFO:tasks.workunit.client.0.smithi064.stdout:9/520: rename d3/d11/d15/d5a to d3/d9/d6a/db1 0 2024-02-16T22:56:27.239 INFO:tasks.workunit.client.0.smithi064.stdout:8/521: stat df/d31/d47/fad 0 2024-02-16T22:56:27.239 INFO:tasks.workunit.client.1.smithi192.stdout:3/610: write da/f4c [2693638,17438] 0 2024-02-16T22:56:27.240 INFO:tasks.workunit.client.1.smithi192.stdout:6/667: symlink da/df/d6b/d74/ldb 0 2024-02-16T22:56:27.240 INFO:tasks.workunit.client.1.smithi192.stdout:6/668: chown da/d76/dc8 0 1 2024-02-16T22:56:27.241 INFO:tasks.workunit.client.0.smithi064.stdout:8/522: write df/f4c [1049920,99069] 0 2024-02-16T22:56:27.244 INFO:tasks.workunit.client.0.smithi064.stdout:4/568: unlink dc/l4e 0 2024-02-16T22:56:27.247 INFO:tasks.workunit.client.1.smithi192.stdout:0/641: creat d1/d3/de/d22/d37/d67/d75/fca x:0 0 0 2024-02-16T22:56:27.248 INFO:tasks.workunit.client.0.smithi064.stdout:1/516: write d2/f5c [937392,109096] 0 2024-02-16T22:56:27.250 INFO:tasks.workunit.client.1.smithi192.stdout:5/652: unlink d6/d8/d75/f7b 0 2024-02-16T22:56:27.251 INFO:tasks.workunit.client.0.smithi064.stdout:0/498: mkdir da/dc/d3b/d5a/d8f/da8 0 2024-02-16T22:56:27.254 INFO:tasks.workunit.client.0.smithi064.stdout:2/641: mkdir d7/d14/d1e/d34/d3f/d9a/dcf 0 2024-02-16T22:56:27.254 INFO:tasks.workunit.client.1.smithi192.stdout:2/699: truncate f19 1705613 0 2024-02-16T22:56:27.259 INFO:tasks.workunit.client.0.smithi064.stdout:9/521: creat d3/d9/d6a/fb2 x:0 0 0 2024-02-16T22:56:27.260 INFO:tasks.workunit.client.1.smithi192.stdout:9/779: mknod d7/d19/d28/d43/d70/c10d 0 2024-02-16T22:56:27.261 INFO:tasks.workunit.client.1.smithi192.stdout:9/780: truncate d7/d15/d36/fe7 166855 0 2024-02-16T22:56:27.261 INFO:tasks.workunit.client.0.smithi064.stdout:8/523: creat df/d31/d2f/d4f/d6a/fb0 x:0 0 0 2024-02-16T22:56:27.263 INFO:tasks.workunit.client.1.smithi192.stdout:1/601: symlink d3/de/d27/d34/d52/lcc 0 2024-02-16T22:56:27.264 INFO:tasks.workunit.client.0.smithi064.stdout:4/569: mknod dc/dd/d32/d22/cb1 0 2024-02-16T22:56:27.266 INFO:tasks.workunit.client.1.smithi192.stdout:7/750: truncate d1/d2/d2d/d61/d88/d90/f78 2365992 0 2024-02-16T22:56:27.267 INFO:tasks.workunit.client.0.smithi064.stdout:7/579: creat d5/d33/d45/fb5 x:0 0 0 2024-02-16T22:56:27.269 INFO:tasks.workunit.client.0.smithi064.stdout:7/580: read d5/d6/f1f [805669,102141] 0 2024-02-16T22:56:27.269 INFO:tasks.workunit.client.0.smithi064.stdout:1/517: mkdir d2/df/d14/d2c/db5 0 2024-02-16T22:56:27.269 INFO:tasks.workunit.client.0.smithi064.stdout:7/581: stat f0 0 2024-02-16T22:56:27.270 INFO:tasks.workunit.client.1.smithi192.stdout:4/700: readlink d1/d6b/la4 0 2024-02-16T22:56:27.279 INFO:tasks.workunit.client.0.smithi064.stdout:8/524: dread f3 [0,4194304] 0 2024-02-16T22:56:27.281 INFO:tasks.workunit.client.0.smithi064.stdout:4/570: creat dc/dd/d5b/fb2 x:0 0 0 2024-02-16T22:56:27.283 INFO:tasks.workunit.client.1.smithi192.stdout:2/700: unlink d22/d24/d39/d42/f5f 0 2024-02-16T22:56:27.285 INFO:tasks.workunit.client.1.smithi192.stdout:9/781: rmdir d7/d15/d36/d77/db1 39 2024-02-16T22:56:27.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/635: dwrite d5/d16/f27 [0,4194304] 0 2024-02-16T22:56:27.288 INFO:tasks.workunit.client.1.smithi192.stdout:1/602: mkdir d3/de/d27/d34/d36/d47/dcd 0 2024-02-16T22:56:27.288 INFO:tasks.workunit.client.1.smithi192.stdout:1/603: chown d3/de/f32 425415 1 2024-02-16T22:56:27.289 INFO:tasks.workunit.client.1.smithi192.stdout:6/669: write da/df/d6b/f88 [1912116,51042] 0 2024-02-16T22:56:27.293 INFO:tasks.workunit.client.0.smithi064.stdout:2/642: mkdir d7/d14/d25/daf/dd0 0 2024-02-16T22:56:27.294 INFO:tasks.workunit.client.1.smithi192.stdout:7/751: mkdir d1/d2/de/d2c/d49/dba/d109 0 2024-02-16T22:56:27.296 INFO:tasks.workunit.client.1.smithi192.stdout:7/752: chown d1/d2/d2d/d61/d88/d90/d4c/c91 252815 1 2024-02-16T22:56:27.296 INFO:tasks.workunit.client.1.smithi192.stdout:7/753: fsync d1/d2/d37/fe7 0 2024-02-16T22:56:27.297 INFO:tasks.workunit.client.0.smithi064.stdout:9/522: truncate d3/f10 490272 0 2024-02-16T22:56:27.298 INFO:tasks.workunit.client.0.smithi064.stdout:7/582: write f0 [4601334,52067] 0 2024-02-16T22:56:27.298 INFO:tasks.workunit.client.0.smithi064.stdout:9/523: readlink d3/d9/d1e/d42/la4 0 2024-02-16T22:56:27.300 INFO:tasks.workunit.client.0.smithi064.stdout:8/525: stat df/d31/d47/la9 0 2024-02-16T22:56:27.300 INFO:tasks.workunit.client.0.smithi064.stdout:7/583: chown d5/d33/d45/d64/d56/d5d/lb3 134688796 1 2024-02-16T22:56:27.301 INFO:tasks.workunit.client.0.smithi064.stdout:9/524: chown d3/d9/d6a/db1/f29 8102 1 2024-02-16T22:56:27.302 INFO:tasks.workunit.client.0.smithi064.stdout:9/525: truncate d3/fc 4033683 0 2024-02-16T22:56:27.304 INFO:tasks.workunit.client.0.smithi064.stdout:1/518: getdents d2/df/d14/d2c/db5 0 2024-02-16T22:56:27.311 INFO:tasks.workunit.client.0.smithi064.stdout:7/584: creat d5/d33/d45/d64/d56/d5d/fb6 x:0 0 0 2024-02-16T22:56:27.311 INFO:tasks.workunit.client.0.smithi064.stdout:1/519: dwrite d2/df/d11/f8f [0,4194304] 0 2024-02-16T22:56:27.314 INFO:tasks.workunit.client.0.smithi064.stdout:8/526: rename df/d31/d2f/f58 to df/d31/d61/d79/d81/fb1 0 2024-02-16T22:56:27.316 INFO:tasks.workunit.client.0.smithi064.stdout:8/527: truncate df/d31/d2f/d4f/f99 969612 0 2024-02-16T22:56:27.316 INFO:tasks.workunit.client.0.smithi064.stdout:9/526: symlink d3/d9/d1e/d42/lb3 0 2024-02-16T22:56:27.317 INFO:tasks.workunit.client.1.smithi192.stdout:2/701: write d22/d24/d39/d42/d65/d69/fa1 [298237,92627] 0 2024-02-16T22:56:27.317 INFO:tasks.workunit.client.0.smithi064.stdout:8/528: dread df/d31/d2f/f55 [0,4194304] 0 2024-02-16T22:56:27.318 INFO:tasks.workunit.client.0.smithi064.stdout:8/529: chown df/d31/d2f/d4f/fa7 1334502008 1 2024-02-16T22:56:27.318 INFO:tasks.workunit.client.0.smithi064.stdout:8/530: stat f9 0 2024-02-16T22:56:27.319 INFO:tasks.workunit.client.0.smithi064.stdout:8/531: stat df/c16 0 2024-02-16T22:56:27.320 INFO:tasks.workunit.client.0.smithi064.stdout:2/643: creat d7/d14/d1e/d34/d3f/d9a/dcf/fd1 x:0 0 0 2024-02-16T22:56:27.320 INFO:tasks.workunit.client.0.smithi064.stdout:8/532: chown df/d31/d47/fad 169434 1 2024-02-16T22:56:27.326 INFO:tasks.workunit.client.1.smithi192.stdout:4/701: truncate d1/f30 3963164 0 2024-02-16T22:56:27.327 INFO:tasks.workunit.client.0.smithi064.stdout:9/527: symlink d3/d9/d1e/d42/d7b/lb4 0 2024-02-16T22:56:27.328 INFO:tasks.workunit.client.1.smithi192.stdout:0/642: creat d1/fcb x:0 0 0 2024-02-16T22:56:27.329 INFO:tasks.workunit.client.0.smithi064.stdout:0/499: link da/dc/d3b/c9d da/dc/d3b/ca9 0 2024-02-16T22:56:27.330 INFO:tasks.workunit.client.0.smithi064.stdout:0/500: read - da/dc/de/d12/f6c zero size 2024-02-16T22:56:27.330 INFO:tasks.workunit.client.1.smithi192.stdout:0/643: write d1/d3/de/d22/d37/d67/d34/da1/fb9 [765151,41806] 0 2024-02-16T22:56:27.330 INFO:tasks.workunit.client.0.smithi064.stdout:2/644: creat d7/d14/d3e/fd2 x:0 0 0 2024-02-16T22:56:27.331 INFO:tasks.workunit.client.0.smithi064.stdout:0/501: write da/dc/d3b/f5f [179685,128216] 0 2024-02-16T22:56:27.335 INFO:tasks.workunit.client.1.smithi192.stdout:0/644: dwrite d1/d3/d39/f79 [0,4194304] 0 2024-02-16T22:56:27.336 INFO:tasks.workunit.client.1.smithi192.stdout:8/636: creat d5/d16/d81/fc6 x:0 0 0 2024-02-16T22:56:27.336 INFO:tasks.workunit.client.1.smithi192.stdout:1/604: write d3/de/d27/d34/d36/d47/f79 [1427410,16828] 0 2024-02-16T22:56:27.337 INFO:tasks.workunit.client.0.smithi064.stdout:7/585: link f0 d5/d33/d53/da3/fb7 0 2024-02-16T22:56:27.338 INFO:tasks.workunit.client.1.smithi192.stdout:1/605: write d3/de/d27/d34/d52/f54 [4626907,14196] 0 2024-02-16T22:56:27.338 INFO:tasks.workunit.client.1.smithi192.stdout:2/702: truncate f10 3956784 0 2024-02-16T22:56:27.343 INFO:tasks.workunit.client.1.smithi192.stdout:6/670: link da/df/d12/d57/fc6 da/df/d6b/d7d/fdc 0 2024-02-16T22:56:27.344 INFO:tasks.workunit.client.1.smithi192.stdout:6/671: chown da/d40/fcb 31088 1 2024-02-16T22:56:27.344 INFO:tasks.workunit.client.1.smithi192.stdout:8/637: dwrite d5/db/d47/d8c/d9d/fa8 [0,4194304] 0 2024-02-16T22:56:27.348 INFO:tasks.workunit.client.0.smithi064.stdout:0/502: mkdir da/dc/de/d37/daa 0 2024-02-16T22:56:27.349 INFO:tasks.workunit.client.0.smithi064.stdout:8/533: creat df/d31/d8f/dab/dae/fb2 x:0 0 0 2024-02-16T22:56:27.353 INFO:tasks.workunit.client.0.smithi064.stdout:7/586: chown d5/d7d/ca8 4520 1 2024-02-16T22:56:27.358 INFO:tasks.workunit.client.0.smithi064.stdout:9/528: dread d3/d9/d6a/db1/d2a/d2e/f39 [0,4194304] 0 2024-02-16T22:56:27.361 INFO:tasks.workunit.client.0.smithi064.stdout:1/520: creat d2/df/d14/d18/fb6 x:0 0 0 2024-02-16T22:56:27.363 INFO:tasks.workunit.client.0.smithi064.stdout:8/534: truncate df/d31/d47/f6b 7049 0 2024-02-16T22:56:27.365 INFO:tasks.workunit.client.0.smithi064.stdout:7/587: rename d5/d33/d45/d64/d56/d66/l6b to d5/d33/d45/d73/dab/lb8 0 2024-02-16T22:56:27.368 INFO:tasks.workunit.client.0.smithi064.stdout:9/529: mknod d3/d9/d1e/cb5 0 2024-02-16T22:56:27.369 INFO:tasks.workunit.client.0.smithi064.stdout:1/521: dread - d2/df/f76 zero size 2024-02-16T22:56:27.370 INFO:tasks.workunit.client.1.smithi192.stdout:5/653: getdents d6 0 2024-02-16T22:56:27.371 INFO:tasks.workunit.client.1.smithi192.stdout:6/672: read - da/df/d71/fd9 zero size 2024-02-16T22:56:27.371 INFO:tasks.workunit.client.1.smithi192.stdout:2/703: creat d22/d24/d39/d42/d52/d7c/fe0 x:0 0 0 2024-02-16T22:56:27.371 INFO:tasks.workunit.client.1.smithi192.stdout:3/611: sync 2024-02-16T22:56:27.371 INFO:tasks.workunit.client.0.smithi064.stdout:8/535: fsync df/d31/d2f/d68/f71 0 2024-02-16T22:56:27.372 INFO:tasks.workunit.client.1.smithi192.stdout:1/606: rename d3/de/d27/d34/d36/d47/f5e to d3/de/d27/d34/d36/d47/d5d/fce 0 2024-02-16T22:56:27.372 INFO:tasks.workunit.client.1.smithi192.stdout:2/704: chown f5 25544 1 2024-02-16T22:56:27.373 INFO:tasks.workunit.client.0.smithi064.stdout:2/645: getdents d7/d14/d1e/d34/d73 0 2024-02-16T22:56:27.373 INFO:tasks.workunit.client.1.smithi192.stdout:0/645: dread d1/f24 [0,4194304] 0 2024-02-16T22:56:27.373 INFO:tasks.workunit.client.1.smithi192.stdout:6/673: chown da/df/d6b/d83/la2 157556 1 2024-02-16T22:56:27.374 INFO:tasks.workunit.client.1.smithi192.stdout:9/782: getdents d7/d15/d36/d77 0 2024-02-16T22:56:27.375 INFO:tasks.workunit.client.1.smithi192.stdout:9/783: write d7/d19/d5e/d81/f95 [952890,123327] 0 2024-02-16T22:56:27.377 INFO:tasks.workunit.client.1.smithi192.stdout:3/612: chown da/lb 153 1 2024-02-16T22:56:27.378 INFO:tasks.workunit.client.0.smithi064.stdout:7/588: rmdir d5/d33/d45/d64/d56/d5d 39 2024-02-16T22:56:27.381 INFO:tasks.workunit.client.1.smithi192.stdout:5/654: symlink d6/ld4 0 2024-02-16T22:56:27.382 INFO:tasks.workunit.client.0.smithi064.stdout:9/530: write d3/d11/d15/f94 [4096702,27230] 0 2024-02-16T22:56:27.389 INFO:tasks.workunit.client.0.smithi064.stdout:9/531: dwrite d3/d9/d6a/db1/d2a/d2e/d31/d3f/f5b [0,4194304] 0 2024-02-16T22:56:27.392 INFO:tasks.workunit.client.1.smithi192.stdout:2/705: mknod d22/d24/d39/d42/d52/d7c/ce1 0 2024-02-16T22:56:27.394 INFO:tasks.workunit.client.0.smithi064.stdout:6/616: sync 2024-02-16T22:56:27.394 INFO:tasks.workunit.client.0.smithi064.stdout:9/532: write d3/d9/d1e/d3e/f65 [2685092,112108] 0 2024-02-16T22:56:27.394 INFO:tasks.workunit.client.0.smithi064.stdout:8/536: rename df/d31/d2f/f55 to df/d31/d8f/dab/dae/fb3 0 2024-02-16T22:56:27.395 INFO:tasks.workunit.client.0.smithi064.stdout:0/503: getdents da/dc/d3b 0 2024-02-16T22:56:27.396 INFO:tasks.workunit.client.1.smithi192.stdout:1/607: dread d3/f2d [0,4194304] 0 2024-02-16T22:56:27.397 INFO:tasks.workunit.client.0.smithi064.stdout:1/522: rmdir d2/df/d11/d1a/d1d 39 2024-02-16T22:56:27.397 INFO:tasks.workunit.client.0.smithi064.stdout:6/617: fsync d0/d5/df/f79 0 2024-02-16T22:56:27.398 INFO:tasks.workunit.client.1.smithi192.stdout:9/784: dread d7/d15/d36/f99 [0,4194304] 0 2024-02-16T22:56:27.398 INFO:tasks.workunit.client.0.smithi064.stdout:6/618: chown d0/d1b/fc8 77116190 1 2024-02-16T22:56:27.399 INFO:tasks.workunit.client.1.smithi192.stdout:9/785: write d7/d19/d37/d5a/f63 [1487983,116878] 0 2024-02-16T22:56:27.400 INFO:tasks.workunit.client.0.smithi064.stdout:2/646: read d7/d14/d4b/f64 [1374798,28685] 0 2024-02-16T22:56:27.403 INFO:tasks.workunit.client.0.smithi064.stdout:7/589: symlink d5/d6/lb9 0 2024-02-16T22:56:27.403 INFO:tasks.workunit.client.0.smithi064.stdout:7/590: chown d5/d7d/fa0 13 1 2024-02-16T22:56:27.404 INFO:tasks.workunit.client.1.smithi192.stdout:0/646: mknod d1/d3/d29/ccc 0 2024-02-16T22:56:27.404 INFO:tasks.workunit.client.0.smithi064.stdout:7/591: write d5/f10 [989569,75935] 0 2024-02-16T22:56:27.405 INFO:tasks.workunit.client.0.smithi064.stdout:7/592: readlink d5/l2b 0 2024-02-16T22:56:27.406 INFO:tasks.workunit.client.1.smithi192.stdout:2/706: mkdir d22/d24/d39/d42/d85/d98/db0/db6/de2 0 2024-02-16T22:56:27.408 INFO:tasks.workunit.client.1.smithi192.stdout:6/674: creat da/d40/db6/db8/fdd x:0 0 0 2024-02-16T22:56:27.411 INFO:tasks.workunit.client.1.smithi192.stdout:1/608: creat d3/de/d27/d34/dbf/fcf x:0 0 0 2024-02-16T22:56:27.415 INFO:tasks.workunit.client.1.smithi192.stdout:8/638: link d5/d26/d83/db4/l7b d5/d26/d83/d86/d5f/lc7 0 2024-02-16T22:56:27.416 INFO:tasks.workunit.client.0.smithi064.stdout:7/593: creat d5/d6/d65/d7f/fba x:0 0 0 2024-02-16T22:56:27.418 INFO:tasks.workunit.client.1.smithi192.stdout:2/707: stat c16 0 2024-02-16T22:56:27.423 INFO:tasks.workunit.client.0.smithi064.stdout:9/533: write d3/d9/d1e/d42/f54 [1908287,81895] 0 2024-02-16T22:56:27.424 INFO:tasks.workunit.client.0.smithi064.stdout:7/594: symlink d5/d6/d65/da7/lbb 0 2024-02-16T22:56:27.424 INFO:tasks.workunit.client.0.smithi064.stdout:8/537: dwrite df/d31/d2f/d4f/fa7 [0,4194304] 0 2024-02-16T22:56:27.426 INFO:tasks.workunit.client.1.smithi192.stdout:7/754: sync 2024-02-16T22:56:27.427 INFO:tasks.workunit.client.0.smithi064.stdout:0/504: rename da/dc/de/d12/d2c/d2e/c35 to da/dc/de/d12/cab 0 2024-02-16T22:56:27.427 INFO:tasks.workunit.client.1.smithi192.stdout:7/755: chown d1/d2/de/d2c/d49/f71 419 1 2024-02-16T22:56:27.429 INFO:tasks.workunit.client.1.smithi192.stdout:3/613: creat da/d11/fd9 x:0 0 0 2024-02-16T22:56:27.436 INFO:tasks.workunit.client.0.smithi064.stdout:8/538: dread df/d31/d2f/f59 [0,4194304] 0 2024-02-16T22:56:27.442 INFO:tasks.workunit.client.0.smithi064.stdout:9/534: dread d3/d11/d15/d23/f3d [0,4194304] 0 2024-02-16T22:56:27.442 INFO:tasks.workunit.client.0.smithi064.stdout:9/535: stat d3/d9/d6a 0 2024-02-16T22:56:27.453 INFO:tasks.workunit.client.1.smithi192.stdout:3/614: dread da/d11/d20/f24 [0,4194304] 0 2024-02-16T22:56:27.468 INFO:tasks.workunit.client.1.smithi192.stdout:4/702: sync 2024-02-16T22:56:27.474 INFO:tasks.workunit.client.1.smithi192.stdout:4/703: dwrite d1/d26/d66/d89/d11/d58/f7e [0,4194304] 0 2024-02-16T22:56:27.508 INFO:tasks.workunit.client.1.smithi192.stdout:0/647: mkdir d1/d3/d29/dcd 0 2024-02-16T22:56:27.511 INFO:tasks.workunit.client.1.smithi192.stdout:2/708: mknod d22/d24/d39/d42/d85/d98/db0/dcd/ce3 0 2024-02-16T22:56:27.513 INFO:tasks.workunit.client.0.smithi064.stdout:7/595: write d5/d33/d45/d64/d56/d5d/f80 [1909967,114300] 0 2024-02-16T22:56:27.514 INFO:tasks.workunit.client.0.smithi064.stdout:7/596: write d5/d6/f1d [2678391,6025] 0 2024-02-16T22:56:27.519 INFO:tasks.workunit.client.1.smithi192.stdout:5/655: creat d6/d12/d2d/fd5 x:0 0 0 2024-02-16T22:56:27.521 INFO:tasks.workunit.client.0.smithi064.stdout:0/505: fsync da/dc/de/d12/d1d/f2b 0 2024-02-16T22:56:27.522 INFO:tasks.workunit.client.0.smithi064.stdout:0/506: dread - da/dc/d3b/d5a/f73 zero size 2024-02-16T22:56:27.524 INFO:tasks.workunit.client.1.smithi192.stdout:5/656: dread d6/d12/d4e/d5b/daf/fbe [0,4194304] 0 2024-02-16T22:56:27.527 INFO:tasks.workunit.client.1.smithi192.stdout:7/756: mkdir d1/d69/d10a 0 2024-02-16T22:56:27.529 INFO:tasks.workunit.client.0.smithi064.stdout:8/539: fdatasync df/d31/d2f/d3e/f77 0 2024-02-16T22:56:27.529 INFO:tasks.workunit.client.0.smithi064.stdout:8/540: fsync df/d31/d2f/f7d 0 2024-02-16T22:56:27.531 INFO:tasks.workunit.client.0.smithi064.stdout:8/541: dread f3 [0,4194304] 0 2024-02-16T22:56:27.532 INFO:tasks.workunit.client.0.smithi064.stdout:8/542: dread - df/d31/d2f/d68/f96 zero size 2024-02-16T22:56:27.549 INFO:tasks.workunit.client.0.smithi064.stdout:6/619: link d0/d6/f15 d0/d5/fce 0 2024-02-16T22:56:27.555 INFO:tasks.workunit.client.0.smithi064.stdout:9/536: mkdir d3/d11/d15/dab/db6 0 2024-02-16T22:56:27.564 INFO:tasks.workunit.client.0.smithi064.stdout:8/543: symlink df/d2a/lb4 0 2024-02-16T22:56:27.571 INFO:tasks.workunit.client.0.smithi064.stdout:0/507: link da/l70 da/dc/d3b/d5a/lac 0 2024-02-16T22:56:27.576 INFO:tasks.workunit.client.0.smithi064.stdout:0/508: dread da/d26/f28 [0,4194304] 0 2024-02-16T22:56:27.578 INFO:tasks.workunit.client.0.smithi064.stdout:8/544: dread df/d31/f3f [0,4194304] 0 2024-02-16T22:56:27.583 INFO:tasks.workunit.client.0.smithi064.stdout:8/545: chown df/d31/d2f/l97 107 1 2024-02-16T22:56:27.591 INFO:tasks.workunit.client.0.smithi064.stdout:3/643: sync 2024-02-16T22:56:27.591 INFO:tasks.workunit.client.0.smithi064.stdout:5/486: sync 2024-02-16T22:56:27.593 INFO:tasks.workunit.client.0.smithi064.stdout:8/546: unlink df/d31/d61/d79/d81/f82 0 2024-02-16T22:56:27.599 INFO:tasks.workunit.client.0.smithi064.stdout:5/487: dread d6/d18/d19/d1a/f69 [0,4194304] 0 2024-02-16T22:56:27.599 INFO:tasks.workunit.client.0.smithi064.stdout:3/644: dwrite d4/d13/d1b/d27/d61/d63/f71 [4194304,4194304] 0 2024-02-16T22:56:27.600 INFO:tasks.workunit.client.0.smithi064.stdout:5/488: readlink d6/d18/d39/l66 0 2024-02-16T22:56:27.605 INFO:tasks.workunit.client.0.smithi064.stdout:8/547: mkdir df/d31/d2f/d38/d48/db5 0 2024-02-16T22:56:27.616 INFO:tasks.workunit.client.0.smithi064.stdout:5/489: symlink d6/d1c/la9 0 2024-02-16T22:56:27.623 INFO:tasks.workunit.client.0.smithi064.stdout:5/490: dwrite d6/d1c/d43/f4d [0,4194304] 0 2024-02-16T22:56:27.627 INFO:tasks.workunit.client.0.smithi064.stdout:8/548: creat df/d31/fb6 x:0 0 0 2024-02-16T22:56:27.627 INFO:tasks.workunit.client.0.smithi064.stdout:6/620: creat d0/d5/df/d17/d18/fcf x:0 0 0 2024-02-16T22:56:27.628 INFO:tasks.workunit.client.0.smithi064.stdout:5/491: unlink d6/d25/d48/d67/l6f 0 2024-02-16T22:56:27.628 INFO:tasks.workunit.client.0.smithi064.stdout:8/549: truncate df/d31/fb6 712922 0 2024-02-16T22:56:27.634 INFO:tasks.workunit.client.0.smithi064.stdout:5/492: creat d6/d1c/d43/d7d/faa x:0 0 0 2024-02-16T22:56:27.636 INFO:tasks.workunit.client.0.smithi064.stdout:8/550: write df/d31/d2f/d3e/faf [241595,90059] 0 2024-02-16T22:56:27.638 INFO:tasks.workunit.client.0.smithi064.stdout:6/621: symlink d0/d6/da/da2/ld0 0 2024-02-16T22:56:27.640 INFO:tasks.workunit.client.0.smithi064.stdout:5/493: fsync d6/f1b 0 2024-02-16T22:56:27.643 INFO:tasks.workunit.client.0.smithi064.stdout:6/622: dread - d0/d12/f8b zero size 2024-02-16T22:56:27.644 INFO:tasks.workunit.client.1.smithi192.stdout:5/657: sync 2024-02-16T22:56:27.645 INFO:tasks.workunit.client.0.smithi064.stdout:6/623: mknod d0/d5/df/d17/d18/cd1 0 2024-02-16T22:56:27.647 INFO:tasks.workunit.client.1.smithi192.stdout:2/709: creat d22/d47/dcb/d78/fe4 x:0 0 0 2024-02-16T22:56:27.648 INFO:tasks.workunit.client.1.smithi192.stdout:2/710: write d22/d24/d39/d42/d85/d98/db0/db6/fce [1828527,107369] 0 2024-02-16T22:56:27.650 INFO:tasks.workunit.client.1.smithi192.stdout:1/609: rename d3/de/d27/d34/d36/d47/d5d/l68 to d3/de/d27/d34/ld0 0 2024-02-16T22:56:27.654 INFO:tasks.workunit.client.1.smithi192.stdout:4/704: creat d1/d26/d66/d89/da8/dc5/ff3 x:0 0 0 2024-02-16T22:56:27.655 INFO:tasks.workunit.client.1.smithi192.stdout:2/711: dwrite d22/d24/d39/d42/d52/d7c/fe0 [0,4194304] 0 2024-02-16T22:56:27.657 INFO:tasks.workunit.client.1.smithi192.stdout:0/648: mknod d1/d3/d29/cce 0 2024-02-16T22:56:27.660 INFO:tasks.workunit.client.1.smithi192.stdout:0/649: stat d1/d3/de/c38 0 2024-02-16T22:56:27.660 INFO:tasks.workunit.client.1.smithi192.stdout:5/658: creat d6/d12/d4e/d5b/daf/fd6 x:0 0 0 2024-02-16T22:56:27.664 INFO:tasks.workunit.client.1.smithi192.stdout:6/675: write da/df/d12/d15/f33 [258962,99133] 0 2024-02-16T22:56:27.665 INFO:tasks.workunit.client.1.smithi192.stdout:1/610: symlink d3/de/d27/d34/d69/d85/d89/daa/ld1 0 2024-02-16T22:56:27.667 INFO:tasks.workunit.client.1.smithi192.stdout:3/615: link da/d11/d63/d6b/d4e/ca2 da/d11/d63/d6b/d71/da1/db9/cda 0 2024-02-16T22:56:27.669 INFO:tasks.workunit.client.1.smithi192.stdout:9/786: getdents d7/d19/d5b 0 2024-02-16T22:56:27.670 INFO:tasks.workunit.client.1.smithi192.stdout:1/611: read d3/de/d27/d34/d36/d47/faf [1483052,25251] 0 2024-02-16T22:56:27.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/639: getdents d5/d26 0 2024-02-16T22:56:27.673 INFO:tasks.workunit.client.1.smithi192.stdout:6/676: dwrite da/df/d12/d15/d1b/f37 [4194304,4194304] 0 2024-02-16T22:56:27.673 INFO:tasks.workunit.client.1.smithi192.stdout:9/787: truncate d7/d19/d28/d43/fee 440894 0 2024-02-16T22:56:27.674 INFO:tasks.workunit.client.1.smithi192.stdout:7/757: link d1/d2/de/d2c/d49/d54/de3/c7b d1/d2/de/d2c/d49/d89/c10b 0 2024-02-16T22:56:27.676 INFO:tasks.workunit.client.1.smithi192.stdout:7/758: write d1/d69/f102 [613441,75123] 0 2024-02-16T22:56:27.679 INFO:tasks.workunit.client.1.smithi192.stdout:7/759: chown d1/d2/d2d/d61/d8a/de6/fe9 401032 1 2024-02-16T22:56:27.682 INFO:tasks.workunit.client.1.smithi192.stdout:8/640: dwrite d5/d26/d83/d86/f93 [0,4194304] 0 2024-02-16T22:56:27.682 INFO:tasks.workunit.client.1.smithi192.stdout:0/650: mkdir d1/d3/d29/dcd/dcf 0 2024-02-16T22:56:27.682 INFO:tasks.workunit.client.1.smithi192.stdout:1/612: rmdir d3/d5b/db1 39 2024-02-16T22:56:27.683 INFO:tasks.workunit.client.1.smithi192.stdout:7/760: stat d1/d2/d2d/d61/d8a/de6/c101 0 2024-02-16T22:56:27.684 INFO:tasks.workunit.client.1.smithi192.stdout:5/659: fsync d6/f10 0 2024-02-16T22:56:27.684 INFO:tasks.workunit.client.1.smithi192.stdout:6/677: mkdir da/df/d71/dde 0 2024-02-16T22:56:27.687 INFO:tasks.workunit.client.1.smithi192.stdout:3/616: dwrite da/d11/d63/d6b/faa [0,4194304] 0 2024-02-16T22:56:27.688 INFO:tasks.workunit.client.1.smithi192.stdout:1/613: dread - d3/de/d27/d34/dbf/fcf zero size 2024-02-16T22:56:27.691 INFO:tasks.workunit.client.0.smithi064.stdout:3/645: symlink d4/d1f/ld4 0 2024-02-16T22:56:27.693 INFO:tasks.workunit.client.1.smithi192.stdout:3/617: symlink da/d6a/ldb 0 2024-02-16T22:56:27.693 INFO:tasks.workunit.client.1.smithi192.stdout:1/614: stat d3/de/d27/d34/dbf/fcf 0 2024-02-16T22:56:27.695 INFO:tasks.workunit.client.1.smithi192.stdout:7/761: read d1/d2/d2d/d61/d88/d90/d4c/f70 [2354397,63906] 0 2024-02-16T22:56:27.695 INFO:tasks.workunit.client.0.smithi064.stdout:3/646: creat d4/d1f/d5b/fd5 x:0 0 0 2024-02-16T22:56:27.697 INFO:tasks.workunit.client.1.smithi192.stdout:8/641: chown d5/db/de/df/f1b 542 1 2024-02-16T22:56:27.701 INFO:tasks.workunit.client.1.smithi192.stdout:0/651: mkdir d1/d3/de/d22/d37/d47/dd0 0 2024-02-16T22:56:27.702 INFO:tasks.workunit.client.1.smithi192.stdout:5/660: symlink d6/d12/d2d/d5c/ld7 0 2024-02-16T22:56:27.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/615: mkdir d3/d5b/d77/dd2 0 2024-02-16T22:56:27.703 INFO:tasks.workunit.client.1.smithi192.stdout:8/642: dwrite d5/d26/d83/d86/d5f/dbf/fc2 [0,4194304] 0 2024-02-16T22:56:27.704 INFO:tasks.workunit.client.1.smithi192.stdout:1/616: chown d3/f2d 12503 1 2024-02-16T22:56:27.704 INFO:tasks.workunit.client.0.smithi064.stdout:3/647: creat d4/d13/d1b/d27/fd6 x:0 0 0 2024-02-16T22:56:27.705 INFO:tasks.workunit.client.1.smithi192.stdout:8/643: write d5/d9/f44 [4714620,120030] 0 2024-02-16T22:56:27.707 INFO:tasks.workunit.client.1.smithi192.stdout:8/644: chown d5/d26/d83/db4 5597 1 2024-02-16T22:56:27.711 INFO:tasks.workunit.client.1.smithi192.stdout:5/661: readlink d6/d8/la8 0 2024-02-16T22:56:27.712 INFO:tasks.workunit.client.1.smithi192.stdout:2/712: unlink d22/d24/d39/l63 0 2024-02-16T22:56:27.713 INFO:tasks.workunit.client.1.smithi192.stdout:8/645: dwrite d5/db/f51 [0,4194304] 0 2024-02-16T22:56:27.713 INFO:tasks.workunit.client.0.smithi064.stdout:3/648: dwrite d4/d13/d1b/d27/d61/d63/fd3 [0,4194304] 0 2024-02-16T22:56:27.714 INFO:tasks.workunit.client.1.smithi192.stdout:8/646: stat d5/d16/f4f 0 2024-02-16T22:56:27.717 INFO:tasks.workunit.client.1.smithi192.stdout:1/617: creat d3/de/d27/d34/d36/d47/d5d/fd3 x:0 0 0 2024-02-16T22:56:27.721 INFO:tasks.workunit.client.0.smithi064.stdout:3/649: dwrite d4/d13/d41/f8d [0,4194304] 0 2024-02-16T22:56:27.722 INFO:tasks.workunit.client.1.smithi192.stdout:2/713: dwrite d22/d24/d2b/f8a [0,4194304] 0 2024-02-16T22:56:27.722 INFO:tasks.workunit.client.1.smithi192.stdout:6/678: rename da/df/d6b/d83/f8c to da/df/d12/d15/fdf 0 2024-02-16T22:56:27.723 INFO:tasks.workunit.client.1.smithi192.stdout:8/647: dread d5/db/d47/d8c/d9d/fa8 [0,4194304] 0 2024-02-16T22:56:27.724 INFO:tasks.workunit.client.1.smithi192.stdout:6/679: chown da/df/d12/d15/cb5 29248 1 2024-02-16T22:56:27.727 INFO:tasks.workunit.client.1.smithi192.stdout:2/714: chown d22/d24/fb9 11519352 1 2024-02-16T22:56:27.727 INFO:tasks.workunit.client.0.smithi064.stdout:3/650: symlink d4/d1f/d34/d7b/ld7 0 2024-02-16T22:56:27.730 INFO:tasks.workunit.client.1.smithi192.stdout:5/662: write d6/d12/d4e/d58/fd3 [551163,2755] 0 2024-02-16T22:56:27.734 INFO:tasks.workunit.client.0.smithi064.stdout:9/537: write d3/d11/f28 [86689,54162] 0 2024-02-16T22:56:27.738 INFO:tasks.workunit.client.1.smithi192.stdout:2/715: rename d22/d24/d39/d42/d52/d7c/fe0 to d22/d24/d2b/d99/d9b/fe5 0 2024-02-16T22:56:27.738 INFO:tasks.workunit.client.0.smithi064.stdout:4/571: sync 2024-02-16T22:56:27.738 INFO:tasks.workunit.client.0.smithi064.stdout:4/572: chown dc/d1d/d43 21049757 1 2024-02-16T22:56:27.741 INFO:tasks.workunit.client.1.smithi192.stdout:1/618: creat d3/de/fd4 x:0 0 0 2024-02-16T22:56:27.743 INFO:tasks.workunit.client.1.smithi192.stdout:2/716: dread d22/d24/fa0 [0,4194304] 0 2024-02-16T22:56:27.746 INFO:tasks.workunit.client.1.smithi192.stdout:5/663: link d6/d12/d4e/d5b/daf/fd6 d6/d12/d4e/d5b/d67/d78/dae/dbc/fd8 0 2024-02-16T22:56:27.747 INFO:tasks.workunit.client.0.smithi064.stdout:0/509: write da/dc/de/d12/d2c/d2e/f36 [8700825,24526] 0 2024-02-16T22:56:27.749 INFO:tasks.workunit.client.0.smithi064.stdout:4/573: getdents dc/dd/d32/d58 0 2024-02-16T22:56:27.750 INFO:tasks.workunit.client.0.smithi064.stdout:4/574: dread - dc/dd/d16/fa6 zero size 2024-02-16T22:56:27.751 INFO:tasks.workunit.client.0.smithi064.stdout:0/510: creat da/dc/d3b/d72/d7c/fad x:0 0 0 2024-02-16T22:56:27.751 INFO:tasks.workunit.client.0.smithi064.stdout:4/575: chown dc/dd/d32/d58/c37 52 1 2024-02-16T22:56:27.754 INFO:tasks.workunit.client.1.smithi192.stdout:0/652: rmdir d1/d3/d18 39 2024-02-16T22:56:27.755 INFO:tasks.workunit.client.0.smithi064.stdout:0/511: fdatasync da/d26/f28 0 2024-02-16T22:56:27.759 INFO:tasks.workunit.client.1.smithi192.stdout:2/717: mknod d22/d47/dca/ce6 0 2024-02-16T22:56:27.761 INFO:tasks.workunit.client.1.smithi192.stdout:6/680: dread da/df/d12/d57/f2f [0,4194304] 0 2024-02-16T22:56:27.762 INFO:tasks.workunit.client.1.smithi192.stdout:0/653: mkdir d1/d3/d29/dcd/dcf/dd1 0 2024-02-16T22:56:27.763 INFO:tasks.workunit.client.0.smithi064.stdout:4/576: link dc/dd/d32/c25 dc/de/d6d/cb3 0 2024-02-16T22:56:27.763 INFO:tasks.workunit.client.0.smithi064.stdout:4/577: chown dc/dd/d16/d6e/f7b 391432 1 2024-02-16T22:56:27.764 INFO:tasks.workunit.client.1.smithi192.stdout:7/762: unlink d1/d2/de/l10 0 2024-02-16T22:56:27.765 INFO:tasks.workunit.client.1.smithi192.stdout:2/718: truncate d22/d24/d39/d42/d84/f8c 582259 0 2024-02-16T22:56:27.766 INFO:tasks.workunit.client.0.smithi064.stdout:4/578: rmdir dc/dd/d32/d58 39 2024-02-16T22:56:27.766 INFO:tasks.workunit.client.1.smithi192.stdout:0/654: dread - d1/d3/f83 zero size 2024-02-16T22:56:27.769 INFO:tasks.workunit.client.1.smithi192.stdout:6/681: truncate da/df/d12/f9c 4067658 0 2024-02-16T22:56:27.769 INFO:tasks.workunit.client.1.smithi192.stdout:7/763: getdents d1/d2/d2d/d61/d88/d90/db2 0 2024-02-16T22:56:27.771 INFO:tasks.workunit.client.1.smithi192.stdout:0/655: write d1/d3/de/d22/f28 [2611527,57437] 0 2024-02-16T22:56:27.771 INFO:tasks.workunit.client.1.smithi192.stdout:2/719: truncate d22/f41 364187 0 2024-02-16T22:56:27.772 INFO:tasks.workunit.client.1.smithi192.stdout:6/682: fdatasync da/df/d12/d15/d1b/faa 0 2024-02-16T22:56:27.774 INFO:tasks.workunit.client.0.smithi064.stdout:3/651: rmdir d4/d13/d1b/d4f 39 2024-02-16T22:56:27.776 INFO:tasks.workunit.client.0.smithi064.stdout:7/597: sync 2024-02-16T22:56:27.776 INFO:tasks.workunit.client.0.smithi064.stdout:1/523: sync 2024-02-16T22:56:27.776 INFO:tasks.workunit.client.1.smithi192.stdout:6/683: write da/df/d12/d15/f68 [62095,3999] 0 2024-02-16T22:56:27.777 INFO:tasks.workunit.client.0.smithi064.stdout:7/598: chown d5/d6/d65/da7/lbb 12 1 2024-02-16T22:56:27.777 INFO:tasks.workunit.client.0.smithi064.stdout:3/652: mkdir d4/d13/d1b/d27/d61/dd8 0 2024-02-16T22:56:27.778 INFO:tasks.workunit.client.0.smithi064.stdout:7/599: readlink d5/d6/d65/d7f/la5 0 2024-02-16T22:56:27.779 INFO:tasks.workunit.client.1.smithi192.stdout:6/684: dread - da/df/fce zero size 2024-02-16T22:56:27.779 INFO:tasks.workunit.client.1.smithi192.stdout:0/656: dwrite d1/d3/de/d22/d37/d67/d75/fc5 [0,4194304] 0 2024-02-16T22:56:27.779 INFO:tasks.workunit.client.0.smithi064.stdout:1/524: creat d2/df/d14/d2c/d62/fb7 x:0 0 0 2024-02-16T22:56:27.781 INFO:tasks.workunit.client.1.smithi192.stdout:7/764: dwrite d1/d2/d2d/d61/d88/d90/d4c/f70 [0,4194304] 0 2024-02-16T22:56:27.783 INFO:tasks.workunit.client.0.smithi064.stdout:2/647: rename d7/f35 to d7/d14/d25/daf/fd3 0 2024-02-16T22:56:27.783 INFO:tasks.workunit.client.1.smithi192.stdout:7/765: write d1/fc1 [714262,23933] 0 2024-02-16T22:56:27.784 INFO:tasks.workunit.client.1.smithi192.stdout:0/657: rmdir d1/d3/d29 39 2024-02-16T22:56:27.791 INFO:tasks.workunit.client.0.smithi064.stdout:7/600: mknod d5/d6/d54/d41/da2/cbc 0 2024-02-16T22:56:27.793 INFO:tasks.workunit.client.1.smithi192.stdout:7/766: fdatasync d1/d2/d37/fc3 0 2024-02-16T22:56:27.794 INFO:tasks.workunit.client.0.smithi064.stdout:3/653: creat d4/fd9 x:0 0 0 2024-02-16T22:56:27.795 INFO:tasks.workunit.client.1.smithi192.stdout:6/685: dwrite da/df/d12/d57/f35 [4194304,4194304] 0 2024-02-16T22:56:27.797 INFO:tasks.workunit.client.1.smithi192.stdout:0/658: creat d1/d3/de/d22/fd2 x:0 0 0 2024-02-16T22:56:27.800 INFO:tasks.workunit.client.0.smithi064.stdout:6/624: unlink d0/d5/df/f6f 0 2024-02-16T22:56:27.802 INFO:tasks.workunit.client.0.smithi064.stdout:6/625: dread - d0/d6/da/fba zero size 2024-02-16T22:56:27.804 INFO:tasks.workunit.client.0.smithi064.stdout:8/551: rename df/d2a/l30 to df/d31/d2f/d68/lb7 0 2024-02-16T22:56:27.806 INFO:tasks.workunit.client.0.smithi064.stdout:1/525: dwrite d2/df/d14/d18/d48/d84/f9a [0,4194304] 0 2024-02-16T22:56:27.808 INFO:tasks.workunit.client.1.smithi192.stdout:6/686: mkdir da/df/d6b/d74/de0 0 2024-02-16T22:56:27.808 INFO:tasks.workunit.client.0.smithi064.stdout:8/552: read df/d31/d47/fad [213393,114327] 0 2024-02-16T22:56:27.812 INFO:tasks.workunit.client.1.smithi192.stdout:6/687: symlink da/df/d58/d72/le1 0 2024-02-16T22:56:27.813 INFO:tasks.workunit.client.1.smithi192.stdout:6/688: chown da/df/d18 64 1 2024-02-16T22:56:27.814 INFO:tasks.workunit.client.0.smithi064.stdout:5/494: rename d6/d18/d19/d1a/f3b to d6/d18/d90/fab 0 2024-02-16T22:56:27.815 INFO:tasks.workunit.client.0.smithi064.stdout:1/526: creat d2/df/d14/d18/d48/fb8 x:0 0 0 2024-02-16T22:56:27.819 INFO:tasks.workunit.client.1.smithi192.stdout:6/689: mkdir da/d76/dd5/de2 0 2024-02-16T22:56:27.820 INFO:tasks.workunit.client.1.smithi192.stdout:6/690: stat da/df/d12/d15/d1b/fbf 0 2024-02-16T22:56:27.821 INFO:tasks.workunit.client.1.smithi192.stdout:6/691: write da/df/d12/d57/fbd [309281,51532] 0 2024-02-16T22:56:27.822 INFO:tasks.workunit.client.1.smithi192.stdout:6/692: write da/df/d71/dab/fcd [219070,70411] 0 2024-02-16T22:56:27.825 INFO:tasks.workunit.client.0.smithi064.stdout:4/579: rename dc/de/d28/c88 to dc/dd/d16/cb4 0 2024-02-16T22:56:27.826 INFO:tasks.workunit.client.1.smithi192.stdout:6/693: dread - da/df/d58/f9a zero size 2024-02-16T22:56:27.827 INFO:tasks.workunit.client.1.smithi192.stdout:6/694: write da/df/d12/f90 [1669130,101809] 0 2024-02-16T22:56:27.827 INFO:tasks.workunit.client.0.smithi064.stdout:1/527: creat d2/df/d11/d25/fb9 x:0 0 0 2024-02-16T22:56:27.832 INFO:tasks.workunit.client.1.smithi192.stdout:7/767: dread d1/d2/de/d83/f8d [0,4194304] 0 2024-02-16T22:56:27.832 INFO:tasks.workunit.client.1.smithi192.stdout:7/768: read d1/d69/fce [3019755,33437] 0 2024-02-16T22:56:27.832 INFO:tasks.workunit.client.0.smithi064.stdout:4/580: creat dc/d1d/d43/fb5 x:0 0 0 2024-02-16T22:56:27.832 INFO:tasks.workunit.client.0.smithi064.stdout:4/581: write dc/dd/d4f/fa7 [707744,29334] 0 2024-02-16T22:56:27.833 INFO:tasks.workunit.client.1.smithi192.stdout:6/695: unlink da/df/d12/d15/d1b/d70/fb1 0 2024-02-16T22:56:27.833 INFO:tasks.workunit.client.1.smithi192.stdout:7/769: write d1/d2/de/d2c/fed [970833,48585] 0 2024-02-16T22:56:27.834 INFO:tasks.workunit.client.1.smithi192.stdout:7/770: readlink d1/d2/d2d/d61/d88/lbf 0 2024-02-16T22:56:27.834 INFO:tasks.workunit.client.0.smithi064.stdout:2/648: rename d7/d14/d1e/d34/ccc to d7/d14/d4b/cd4 0 2024-02-16T22:56:27.836 INFO:tasks.workunit.client.0.smithi064.stdout:1/528: mkdir d2/df/d11/dba 0 2024-02-16T22:56:27.838 INFO:tasks.workunit.client.1.smithi192.stdout:8/648: creat d5/fc8 x:0 0 0 2024-02-16T22:56:27.841 INFO:tasks.workunit.client.0.smithi064.stdout:7/601: rename d5/d6/f3b to d5/d6/d65/fbd 0 2024-02-16T22:56:27.846 INFO:tasks.workunit.client.1.smithi192.stdout:8/649: creat d5/d9/fc9 x:0 0 0 2024-02-16T22:56:27.847 INFO:tasks.workunit.client.1.smithi192.stdout:8/650: write d5/d26/f88 [841872,122346] 0 2024-02-16T22:56:27.848 INFO:tasks.workunit.client.0.smithi064.stdout:3/654: rename d4/f17 to d4/d1f/d34/fda 0 2024-02-16T22:56:27.850 INFO:tasks.workunit.client.1.smithi192.stdout:7/771: dread d1/de1/f3c [0,4194304] 0 2024-02-16T22:56:27.850 INFO:tasks.workunit.client.1.smithi192.stdout:1/619: rename d3/f75 to d3/de/d27/d34/d52/d63/d7a/db0/fd5 0 2024-02-16T22:56:27.851 INFO:tasks.workunit.client.1.smithi192.stdout:4/705: write d1/d15/d29/f8b [1108826,76424] 0 2024-02-16T22:56:27.851 INFO:tasks.workunit.client.1.smithi192.stdout:7/772: dread - d1/d2/de/d2c/d49/fd7 zero size 2024-02-16T22:56:27.852 INFO:tasks.workunit.client.0.smithi064.stdout:7/602: creat d5/d33/d45/d73/dab/fbe x:0 0 0 2024-02-16T22:56:27.852 INFO:tasks.workunit.client.1.smithi192.stdout:1/620: chown d3/de/d27/d34/d36/d47/d5d 1737833549 1 2024-02-16T22:56:27.852 INFO:tasks.workunit.client.1.smithi192.stdout:1/621: chown d3/de/l58 63 1 2024-02-16T22:56:27.854 INFO:tasks.workunit.client.0.smithi064.stdout:2/649: symlink d7/de/d95/d44/ld5 0 2024-02-16T22:56:27.855 INFO:tasks.workunit.client.1.smithi192.stdout:8/651: dread d5/db/d47/d8c/d9d/fa8 [0,4194304] 0 2024-02-16T22:56:27.856 INFO:tasks.workunit.client.1.smithi192.stdout:9/788: write f2 [5420302,40587] 0 2024-02-16T22:56:27.857 INFO:tasks.workunit.client.1.smithi192.stdout:4/706: mkdir d1/d26/d6f/df4 0 2024-02-16T22:56:27.859 INFO:tasks.workunit.client.0.smithi064.stdout:3/655: creat d4/d13/d1b/d27/d99/fdb x:0 0 0 2024-02-16T22:56:27.859 INFO:tasks.workunit.client.0.smithi064.stdout:2/650: dwrite d7/f86 [0,4194304] 0 2024-02-16T22:56:27.862 INFO:tasks.workunit.client.1.smithi192.stdout:1/622: dread - d3/de/d27/d34/d52/d63/d81/f8d zero size 2024-02-16T22:56:27.862 INFO:tasks.workunit.client.0.smithi064.stdout:2/651: write d7/d14/d25/f90 [1179759,66175] 0 2024-02-16T22:56:27.865 INFO:tasks.workunit.client.0.smithi064.stdout:2/652: truncate d7/de/d95/d44/fbc 927327 0 2024-02-16T22:56:27.866 INFO:tasks.workunit.client.0.smithi064.stdout:3/656: dwrite d4/d13/d35/fac [8388608,4194304] 0 2024-02-16T22:56:27.867 INFO:tasks.workunit.client.1.smithi192.stdout:7/773: dwrite d1/f11 [4194304,4194304] 0 2024-02-16T22:56:27.872 INFO:tasks.workunit.client.1.smithi192.stdout:5/664: rename d6/d20/d93/fa7 to d6/fd9 0 2024-02-16T22:56:27.873 INFO:tasks.workunit.client.1.smithi192.stdout:1/623: dwrite d3/d40/f94 [0,4194304] 0 2024-02-16T22:56:27.874 INFO:tasks.workunit.client.0.smithi064.stdout:2/653: mkdir d7/de/d95/d44/dd6 0 2024-02-16T22:56:27.875 INFO:tasks.workunit.client.0.smithi064.stdout:2/654: chown d7/de/c2a 99132156 1 2024-02-16T22:56:27.876 INFO:tasks.workunit.client.1.smithi192.stdout:4/707: dread d1/d15/d23/f4a [0,4194304] 0 2024-02-16T22:56:27.879 INFO:tasks.workunit.client.1.smithi192.stdout:9/789: mkdir d7/d15/d36/d38/d3f/d8a/def/d100/d10e 0 2024-02-16T22:56:27.880 INFO:tasks.workunit.client.1.smithi192.stdout:3/618: truncate da/d11/d35/f70 797219 0 2024-02-16T22:56:27.885 INFO:tasks.workunit.client.0.smithi064.stdout:2/655: symlink d7/d14/d1e/d34/d73/db5/ld7 0 2024-02-16T22:56:27.885 INFO:tasks.workunit.client.1.smithi192.stdout:9/790: dwrite d7/d19/d37/ff2 [4194304,4194304] 0 2024-02-16T22:56:27.886 INFO:tasks.workunit.client.0.smithi064.stdout:2/656: fsync d7/de/d95/d44/f80 0 2024-02-16T22:56:27.888 INFO:tasks.workunit.client.0.smithi064.stdout:3/657: creat d4/d13/d1b/d27/d44/d97/fdc x:0 0 0 2024-02-16T22:56:27.889 INFO:tasks.workunit.client.1.smithi192.stdout:7/774: mknod d1/d2/de/d2c/d49/d54/d9d/c10c 0 2024-02-16T22:56:27.889 INFO:tasks.workunit.client.0.smithi064.stdout:7/603: getdents d5/d33/d53/da3 0 2024-02-16T22:56:27.890 INFO:tasks.workunit.client.1.smithi192.stdout:7/775: chown d1/d2/d2d/d61/d88/lbf 1294272 1 2024-02-16T22:56:27.893 INFO:tasks.workunit.client.1.smithi192.stdout:0/659: rename d1/d3/de/d22/d37/d67/d75/f8e to d1/d3/de/d22/d37/d47/fd3 0 2024-02-16T22:56:27.895 INFO:tasks.workunit.client.1.smithi192.stdout:0/660: dread - d1/d3/de/d22/da7/fb5 zero size 2024-02-16T22:56:27.896 INFO:tasks.workunit.client.1.smithi192.stdout:7/776: dwrite d1/d2/d2d/d61/d8a/f94 [0,4194304] 0 2024-02-16T22:56:27.898 INFO:tasks.workunit.client.0.smithi064.stdout:9/538: write d3/d9/ff [4204931,67926] 0 2024-02-16T22:56:27.898 INFO:tasks.workunit.client.1.smithi192.stdout:1/624: creat d3/de/d27/d34/dbf/fd6 x:0 0 0 2024-02-16T22:56:27.899 INFO:tasks.workunit.client.0.smithi064.stdout:7/604: fsync d5/d6/f43 0 2024-02-16T22:56:27.901 INFO:tasks.workunit.client.1.smithi192.stdout:1/625: chown d3/de/d27/d34/d52/d5a/c93 12 1 2024-02-16T22:56:27.901 INFO:tasks.workunit.client.1.smithi192.stdout:8/652: link d5/db/de/df/f1b d5/d16/d81/fca 0 2024-02-16T22:56:27.902 INFO:tasks.workunit.client.1.smithi192.stdout:8/653: fdatasync d5/d16/f27 0 2024-02-16T22:56:27.903 INFO:tasks.workunit.client.1.smithi192.stdout:1/626: dread - d3/de/d27/d34/d69/d85/d89/daa/fc8 zero size 2024-02-16T22:56:27.910 INFO:tasks.workunit.client.0.smithi064.stdout:7/605: symlink d5/d6/d65/d7f/lbf 0 2024-02-16T22:56:27.911 INFO:tasks.workunit.client.0.smithi064.stdout:0/512: write da/dc/de/d12/f2a [6593431,111389] 0 2024-02-16T22:56:27.912 INFO:tasks.workunit.client.0.smithi064.stdout:0/513: dread - da/dc/d3b/f7b zero size 2024-02-16T22:56:27.914 INFO:tasks.workunit.client.1.smithi192.stdout:5/665: mkdir d6/dda 0 2024-02-16T22:56:27.918 INFO:tasks.workunit.client.1.smithi192.stdout:0/661: creat d1/d3/de/d22/d37/d47/fd4 x:0 0 0 2024-02-16T22:56:27.919 INFO:tasks.workunit.client.0.smithi064.stdout:0/514: symlink da/dc/de/d12/d2c/d2e/d38/d87/lae 0 2024-02-16T22:56:27.920 INFO:tasks.workunit.client.1.smithi192.stdout:9/791: dread d7/d19/d28/f2c [0,4194304] 0 2024-02-16T22:56:27.920 INFO:tasks.workunit.client.0.smithi064.stdout:0/515: dread - da/d71/f7d zero size 2024-02-16T22:56:27.922 INFO:tasks.workunit.client.1.smithi192.stdout:4/708: creat d1/d15/d29/dd4/ff5 x:0 0 0 2024-02-16T22:56:27.922 INFO:tasks.workunit.client.1.smithi192.stdout:2/720: truncate d22/d24/f96 3966527 0 2024-02-16T22:56:27.926 INFO:tasks.workunit.client.1.smithi192.stdout:3/619: fsync da/f9c 0 2024-02-16T22:56:27.929 INFO:tasks.workunit.client.0.smithi064.stdout:0/516: link da/dc/de/d12/d1d/f3d da/dc/de/d12/d2c/d2e/d38/d90/faf 0 2024-02-16T22:56:27.931 INFO:tasks.workunit.client.0.smithi064.stdout:3/658: symlink d4/d13/d1b/d27/d61/d63/ldd 0 2024-02-16T22:56:27.934 INFO:tasks.workunit.client.0.smithi064.stdout:3/659: mknod d4/d1f/d5b/cde 0 2024-02-16T22:56:27.937 INFO:tasks.workunit.client.0.smithi064.stdout:3/660: mknod d4/d14/d2b/cdf 0 2024-02-16T22:56:27.939 INFO:tasks.workunit.client.1.smithi192.stdout:3/620: dread da/df/f1f [0,4194304] 0 2024-02-16T22:56:27.939 INFO:tasks.workunit.client.1.smithi192.stdout:9/792: write d7/d19/d37/fdc [2059954,34340] 0 2024-02-16T22:56:27.939 INFO:tasks.workunit.client.0.smithi064.stdout:3/661: readlink d4/d1f/ld1 0 2024-02-16T22:56:27.940 INFO:tasks.workunit.client.1.smithi192.stdout:3/621: chown da/d11/d20/f4a 204472 1 2024-02-16T22:56:27.941 INFO:tasks.workunit.client.0.smithi064.stdout:0/517: getdents da/dc/d3b 0 2024-02-16T22:56:27.944 INFO:tasks.workunit.client.0.smithi064.stdout:3/662: creat d4/d13/d1b/d27/d44/fe0 x:0 0 0 2024-02-16T22:56:27.945 INFO:tasks.workunit.client.0.smithi064.stdout:8/553: dwrite df/f3d [0,4194304] 0 2024-02-16T22:56:27.946 INFO:tasks.workunit.client.0.smithi064.stdout:0/518: rmdir da/dc 39 2024-02-16T22:56:27.953 INFO:tasks.workunit.client.0.smithi064.stdout:4/582: write dc/de/f75 [809638,19976] 0 2024-02-16T22:56:27.953 INFO:tasks.workunit.client.0.smithi064.stdout:8/554: dread - df/d31/d2f/d4f/f75 zero size 2024-02-16T22:56:27.955 INFO:tasks.workunit.client.0.smithi064.stdout:8/555: chown df/d31/d61/d79/d81 313 1 2024-02-16T22:56:27.955 INFO:tasks.workunit.client.0.smithi064.stdout:8/556: readlink df/l14 0 2024-02-16T22:56:27.956 INFO:tasks.workunit.client.1.smithi192.stdout:6/696: dwrite da/df/d12/f75 [0,4194304] 0 2024-02-16T22:56:27.956 INFO:tasks.workunit.client.0.smithi064.stdout:8/557: chown df/d31/d2f/d38/d48/l91 3813391 1 2024-02-16T22:56:27.963 INFO:tasks.workunit.client.1.smithi192.stdout:6/697: dwrite da/df/d12/f9b [0,4194304] 0 2024-02-16T22:56:27.963 INFO:tasks.workunit.client.1.smithi192.stdout:6/698: chown da/d40/l6c 1511 1 2024-02-16T22:56:27.964 INFO:tasks.workunit.client.0.smithi064.stdout:7/606: rmdir d5/d33/d45/d73 39 2024-02-16T22:56:27.966 INFO:tasks.workunit.client.1.smithi192.stdout:4/709: creat d1/d26/dbe/ff6 x:0 0 0 2024-02-16T22:56:27.967 INFO:tasks.workunit.client.0.smithi064.stdout:0/519: mknod da/dc/de/cb0 0 2024-02-16T22:56:27.971 INFO:tasks.workunit.client.0.smithi064.stdout:8/558: creat df/d31/d2f/d38/d48/db5/fb8 x:0 0 0 2024-02-16T22:56:27.972 INFO:tasks.workunit.client.0.smithi064.stdout:8/559: dread - df/d31/d2f/d4f/f89 zero size 2024-02-16T22:56:27.975 INFO:tasks.workunit.client.1.smithi192.stdout:5/666: mknod d6/d8/cdb 0 2024-02-16T22:56:27.977 INFO:tasks.workunit.client.0.smithi064.stdout:8/560: symlink df/d31/d61/d79/d81/lb9 0 2024-02-16T22:56:27.978 INFO:tasks.workunit.client.0.smithi064.stdout:8/561: read df/d2a/f34 [3481576,111888] 0 2024-02-16T22:56:27.979 INFO:tasks.workunit.client.1.smithi192.stdout:7/777: getdents d1/d69 0 2024-02-16T22:56:27.989 INFO:tasks.workunit.client.1.smithi192.stdout:6/699: mknod da/df/d12/d57/daf/dcf/ce3 0 2024-02-16T22:56:27.989 INFO:tasks.workunit.client.0.smithi064.stdout:2/657: truncate d7/d14/d1e/f81 1230872 0 2024-02-16T22:56:27.990 INFO:tasks.workunit.client.0.smithi064.stdout:0/520: link da/dc/d3b/ca9 da/dc/d3b/d72/d7c/cb1 0 2024-02-16T22:56:27.994 INFO:tasks.workunit.client.1.smithi192.stdout:4/710: creat d1/d26/d66/ff7 x:0 0 0 2024-02-16T22:56:27.994 INFO:tasks.workunit.client.1.smithi192.stdout:6/700: dwrite da/df/fba [0,4194304] 0 2024-02-16T22:56:27.994 INFO:tasks.workunit.client.0.smithi064.stdout:8/562: creat df/d31/d2f/d38/fba x:0 0 0 2024-02-16T22:56:27.995 INFO:tasks.workunit.client.1.smithi192.stdout:6/701: dread - da/df/d12/d15/d1b/faa zero size 2024-02-16T22:56:27.997 INFO:tasks.workunit.client.1.smithi192.stdout:2/721: rename d22/d24/d39/d42/d85/f86 to d22/d24/d39/fe7 0 2024-02-16T22:56:27.998 INFO:tasks.workunit.client.0.smithi064.stdout:0/521: dread da/dc/de/d12/d1d/f2b [4194304,4194304] 0 2024-02-16T22:56:27.999 INFO:tasks.workunit.client.1.smithi192.stdout:1/627: getdents d3/de/d27/d34/d52/d63/d7a 0 2024-02-16T22:56:27.999 INFO:tasks.workunit.client.0.smithi064.stdout:9/539: write d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/faf [851364,32187] 0 2024-02-16T22:56:28.002 INFO:tasks.workunit.client.1.smithi192.stdout:5/667: chown d6/d12/d4e/l45 31722 1 2024-02-16T22:56:28.004 INFO:tasks.workunit.client.0.smithi064.stdout:2/658: mknod d7/de/d95/d44/dd6/cd8 0 2024-02-16T22:56:28.005 INFO:tasks.workunit.client.0.smithi064.stdout:8/563: readlink df/d31/d2f/d68/lb7 0 2024-02-16T22:56:28.005 INFO:tasks.workunit.client.1.smithi192.stdout:7/778: fdatasync d1/d2/de/f20 0 2024-02-16T22:56:28.006 INFO:tasks.workunit.client.1.smithi192.stdout:7/779: dread - d1/d2/de/d83/d9f/ff0 zero size 2024-02-16T22:56:28.009 INFO:tasks.workunit.client.1.smithi192.stdout:8/654: write d5/db/d47/d43/d67/f8b [4918826,74181] 0 2024-02-16T22:56:28.010 INFO:tasks.workunit.client.1.smithi192.stdout:8/655: write d5/d26/d83/d86/d5f/f9f [1357176,73868] 0 2024-02-16T22:56:28.011 INFO:tasks.workunit.client.1.smithi192.stdout:4/711: mknod d1/d15/d29/dd4/cf8 0 2024-02-16T22:56:28.013 INFO:tasks.workunit.client.1.smithi192.stdout:4/712: read d1/d15/d23/f5c [192688,44934] 0 2024-02-16T22:56:28.014 INFO:tasks.workunit.client.1.smithi192.stdout:6/702: fsync da/df/d12/d57/fc6 0 2024-02-16T22:56:28.016 INFO:tasks.workunit.client.1.smithi192.stdout:9/793: rename d7/d19/d37/fdc to d7/d15/d36/d77/db1/f10f 0 2024-02-16T22:56:28.016 INFO:tasks.workunit.client.1.smithi192.stdout:6/703: read da/df/d12/d15/d1b/fa3 [2391787,20808] 0 2024-02-16T22:56:28.018 INFO:tasks.workunit.client.1.smithi192.stdout:1/628: read d3/de/d27/d34/d36/d47/faf [1693509,70893] 0 2024-02-16T22:56:28.019 INFO:tasks.workunit.client.1.smithi192.stdout:1/629: fdatasync d3/d40/f94 0 2024-02-16T22:56:28.020 INFO:tasks.workunit.client.1.smithi192.stdout:4/713: dwrite d1/d15/d29/d3c/f43 [0,4194304] 0 2024-02-16T22:56:28.021 INFO:tasks.workunit.client.1.smithi192.stdout:4/714: stat d1/d26/d6f/df4 0 2024-02-16T22:56:28.023 INFO:tasks.workunit.client.0.smithi064.stdout:4/583: truncate dc/d1d/d43/fae 100636 0 2024-02-16T22:56:28.023 INFO:tasks.workunit.client.0.smithi064.stdout:3/663: dwrite d4/d14/f2a [0,4194304] 0 2024-02-16T22:56:28.024 INFO:tasks.workunit.client.0.smithi064.stdout:4/584: chown dc/d1d/d43/c91 74018 1 2024-02-16T22:56:28.066 INFO:tasks.workunit.client.1.smithi192.stdout:0/662: dwrite d1/d3/d29/f8d [0,4194304] 0 2024-02-16T22:56:28.067 INFO:tasks.workunit.client.1.smithi192.stdout:0/663: write d1/d3/f8c [1481032,108723] 0 2024-02-16T22:56:28.070 INFO:tasks.workunit.client.1.smithi192.stdout:6/704: stat c7 0 2024-02-16T22:56:28.071 INFO:tasks.workunit.client.1.smithi192.stdout:3/622: dwrite da/d11/d63/d6b/d66/f92 [0,4194304] 0 2024-02-16T22:56:28.074 INFO:tasks.workunit.client.1.smithi192.stdout:5/668: symlink d6/d20/d93/ldc 0 2024-02-16T22:56:28.077 INFO:tasks.workunit.client.1.smithi192.stdout:4/715: dread - d1/d26/d6f/fbc zero size 2024-02-16T22:56:28.080 INFO:tasks.workunit.client.1.smithi192.stdout:7/780: mkdir d1/d2/d2d/d61/d10d 0 2024-02-16T22:56:28.082 INFO:tasks.workunit.client.1.smithi192.stdout:8/656: mkdir d5/db/dcb 0 2024-02-16T22:56:28.086 INFO:tasks.workunit.client.0.smithi064.stdout:2/659: write d7/d14/d4b/fc7 [7860244,102103] 0 2024-02-16T22:56:28.088 INFO:tasks.workunit.client.0.smithi064.stdout:8/564: symlink df/d31/d61/d94/lbb 0 2024-02-16T22:56:28.094 INFO:tasks.workunit.client.1.smithi192.stdout:0/664: creat d1/d3/de/d22/fd5 x:0 0 0 2024-02-16T22:56:28.095 INFO:tasks.workunit.client.0.smithi064.stdout:9/540: symlink d3/d9/d6a/db1/lb7 0 2024-02-16T22:56:28.096 INFO:tasks.workunit.client.1.smithi192.stdout:9/794: creat d7/d15/d36/d38/d3f/d8a/def/df6/f110 x:0 0 0 2024-02-16T22:56:28.097 INFO:tasks.workunit.client.0.smithi064.stdout:0/522: link da/d26/d3f/f6b da/da0/fb2 0 2024-02-16T22:56:28.099 INFO:tasks.workunit.client.0.smithi064.stdout:2/660: fdatasync d7/d14/d1e/d34/f50 0 2024-02-16T22:56:28.100 INFO:tasks.workunit.client.1.smithi192.stdout:9/795: read d7/d15/f6d [1061796,13020] 0 2024-02-16T22:56:28.100 INFO:tasks.workunit.client.1.smithi192.stdout:6/705: truncate da/df/f8d 852045 0 2024-02-16T22:56:28.101 INFO:tasks.workunit.client.1.smithi192.stdout:6/706: write da/df/f1c [5610031,43708] 0 2024-02-16T22:56:28.102 INFO:tasks.workunit.client.1.smithi192.stdout:6/707: chown da/df/d58/d72/fbe 12450 1 2024-02-16T22:56:28.103 INFO:tasks.workunit.client.1.smithi192.stdout:5/669: unlink d6/d12/d4e/d5b/daf/fb2 0 2024-02-16T22:56:28.107 INFO:tasks.workunit.client.1.smithi192.stdout:4/716: dread d1/d15/d48/d8d/f9a [0,4194304] 0 2024-02-16T22:56:28.110 INFO:tasks.workunit.client.1.smithi192.stdout:6/708: dwrite da/df/d12/d15/d1b/d70/fda [0,4194304] 0 2024-02-16T22:56:28.112 INFO:tasks.workunit.client.1.smithi192.stdout:8/657: creat d5/d16/fcc x:0 0 0 2024-02-16T22:56:28.115 INFO:tasks.workunit.client.0.smithi064.stdout:8/565: rename df/d31/d61/d94/l9a to df/d31/d2f/lbc 0 2024-02-16T22:56:28.116 INFO:tasks.workunit.client.1.smithi192.stdout:8/658: write d5/d16/f4f [1914582,89423] 0 2024-02-16T22:56:28.119 INFO:tasks.workunit.client.1.smithi192.stdout:8/659: write d5/d9/fc9 [429498,3743] 0 2024-02-16T22:56:28.119 INFO:tasks.workunit.client.1.smithi192.stdout:4/717: dwrite d1/d15/d48/d8d/d9b/ff1 [0,4194304] 0 2024-02-16T22:56:28.121 INFO:tasks.workunit.client.1.smithi192.stdout:4/718: chown d1/d26/d34/db9 30 1 2024-02-16T22:56:28.121 INFO:tasks.workunit.client.1.smithi192.stdout:6/709: dread da/df/d12/d57/d81/fb7 [4194304,4194304] 0 2024-02-16T22:56:28.121 INFO:tasks.workunit.client.1.smithi192.stdout:4/719: stat d1/d15/d48 0 2024-02-16T22:56:28.123 INFO:tasks.workunit.client.1.smithi192.stdout:9/796: dread f0 [0,4194304] 0 2024-02-16T22:56:28.124 INFO:tasks.workunit.client.0.smithi064.stdout:4/585: link dc/l44 dc/dd/d16/lb6 0 2024-02-16T22:56:28.129 INFO:tasks.workunit.client.0.smithi064.stdout:4/586: creat dc/dd/d5b/fb7 x:0 0 0 2024-02-16T22:56:28.131 INFO:tasks.workunit.client.0.smithi064.stdout:8/566: link df/d2a/d5c/l95 df/d31/d2f/d68/lbd 0 2024-02-16T22:56:28.133 INFO:tasks.workunit.client.0.smithi064.stdout:4/587: fsync dc/de/d6d/f9f 0 2024-02-16T22:56:28.134 INFO:tasks.workunit.client.1.smithi192.stdout:0/665: mkdir d1/dd6 0 2024-02-16T22:56:28.136 INFO:tasks.workunit.client.0.smithi064.stdout:8/567: creat df/d31/d8f/dab/dae/fbe x:0 0 0 2024-02-16T22:56:28.144 INFO:tasks.workunit.client.1.smithi192.stdout:4/720: fdatasync d1/d26/d34/f44 0 2024-02-16T22:56:28.144 INFO:tasks.workunit.client.1.smithi192.stdout:4/721: read d1/d15/d48/d8d/f9a [2682526,29523] 0 2024-02-16T22:56:28.145 INFO:tasks.workunit.client.1.smithi192.stdout:5/670: mknod d6/d12/d4e/d58/d8f/cdd 0 2024-02-16T22:56:28.146 INFO:tasks.workunit.client.0.smithi064.stdout:0/523: fdatasync da/f4d 0 2024-02-16T22:56:28.151 INFO:tasks.workunit.client.0.smithi064.stdout:0/524: chown da/dc/de/d12/d2c/d2e/d62/ca2 110110 1 2024-02-16T22:56:28.153 INFO:tasks.workunit.client.1.smithi192.stdout:9/797: mkdir d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/d104/d111 0 2024-02-16T22:56:28.154 INFO:tasks.workunit.client.1.smithi192.stdout:4/722: symlink d1/d26/d34/lf9 0 2024-02-16T22:56:28.155 INFO:tasks.workunit.client.1.smithi192.stdout:5/671: creat d6/d12/d1d/d56/fde x:0 0 0 2024-02-16T22:56:28.157 INFO:tasks.workunit.client.0.smithi064.stdout:0/525: dwrite da/dc/d3b/d5a/fa1 [0,4194304] 0 2024-02-16T22:56:28.157 INFO:tasks.workunit.client.1.smithi192.stdout:0/666: unlink d1/d3/d29/f9a 0 2024-02-16T22:56:28.158 INFO:tasks.workunit.client.1.smithi192.stdout:0/667: fsync d1/d3/de/d22/d37/d67/fb7 0 2024-02-16T22:56:28.160 INFO:tasks.workunit.client.1.smithi192.stdout:0/668: write d1/d3/de/d22/d37/d67/d75/db0/fb2 [855319,129219] 0 2024-02-16T22:56:28.160 INFO:tasks.workunit.client.1.smithi192.stdout:8/660: creat d5/fcd x:0 0 0 2024-02-16T22:56:28.162 INFO:tasks.workunit.client.0.smithi064.stdout:0/526: mkdir da/dc/d3b/d72/db3 0 2024-02-16T22:56:28.162 INFO:tasks.workunit.client.1.smithi192.stdout:0/669: fdatasync d1/d3/f7c 0 2024-02-16T22:56:28.163 INFO:tasks.workunit.client.1.smithi192.stdout:2/722: sync 2024-02-16T22:56:28.164 INFO:tasks.workunit.client.1.smithi192.stdout:6/710: creat da/df/d18/fe4 x:0 0 0 2024-02-16T22:56:28.165 INFO:tasks.workunit.client.0.smithi064.stdout:3/664: truncate d4/d13/d1b/d27/d99/f87 17577 0 2024-02-16T22:56:28.165 INFO:tasks.workunit.client.1.smithi192.stdout:2/723: write fa [2617745,97048] 0 2024-02-16T22:56:28.166 INFO:tasks.workunit.client.1.smithi192.stdout:4/723: creat d1/d15/d29/dd4/ffa x:0 0 0 2024-02-16T22:56:28.168 INFO:tasks.workunit.client.0.smithi064.stdout:0/527: rename da/dc/de/d12/d2c/d2e/d38/d90 to da/dc/de/d37/daa/db4 0 2024-02-16T22:56:28.170 INFO:tasks.workunit.client.1.smithi192.stdout:3/623: dwrite da/f9c [0,4194304] 0 2024-02-16T22:56:28.172 INFO:tasks.workunit.client.1.smithi192.stdout:4/724: write d1/d26/d66/d89/d11/d58/f90 [1168155,104290] 0 2024-02-16T22:56:28.177 INFO:tasks.workunit.client.1.smithi192.stdout:1/630: dwrite d3/de/d27/d34/d69/f6b [0,4194304] 0 2024-02-16T22:56:28.177 INFO:tasks.workunit.client.0.smithi064.stdout:9/541: dwrite d3/d11/d15/d23/f3d [4194304,4194304] 0 2024-02-16T22:56:28.178 INFO:tasks.workunit.client.0.smithi064.stdout:0/528: dwrite da/dc/de/d37/f4e [0,4194304] 0 2024-02-16T22:56:28.180 INFO:tasks.workunit.client.0.smithi064.stdout:9/542: truncate d3/f18 5249539 0 2024-02-16T22:56:28.180 INFO:tasks.workunit.client.1.smithi192.stdout:8/661: dread d5/d9/f6b [0,4194304] 0 2024-02-16T22:56:28.180 INFO:tasks.workunit.client.1.smithi192.stdout:7/781: dwrite d1/d2/de/d83/fbc [0,4194304] 0 2024-02-16T22:56:28.183 INFO:tasks.workunit.client.1.smithi192.stdout:8/662: write d5/db/f51 [2760820,118375] 0 2024-02-16T22:56:28.183 INFO:tasks.workunit.client.1.smithi192.stdout:3/624: dread da/f9c [0,4194304] 0 2024-02-16T22:56:28.184 INFO:tasks.workunit.client.1.smithi192.stdout:1/631: read d3/f22 [7629147,70191] 0 2024-02-16T22:56:28.185 INFO:tasks.workunit.client.1.smithi192.stdout:6/711: creat da/df/d58/d72/fe5 x:0 0 0 2024-02-16T22:56:28.188 INFO:tasks.workunit.client.0.smithi064.stdout:9/543: creat d3/d9/d1e/d42/fb8 x:0 0 0 2024-02-16T22:56:28.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/724: creat d22/d47/dcb/d3f/dac/fe8 x:0 0 0 2024-02-16T22:56:28.197 INFO:tasks.workunit.client.0.smithi064.stdout:0/529: creat da/dc/de/d12/d2c/d2e/d38/d7e/d89/d93/fb5 x:0 0 0 2024-02-16T22:56:28.197 INFO:tasks.workunit.client.1.smithi192.stdout:6/712: dwrite da/df/fce [0,4194304] 0 2024-02-16T22:56:28.198 INFO:tasks.workunit.client.0.smithi064.stdout:4/588: write dc/dd/d16/f5d [4219623,72609] 0 2024-02-16T22:56:28.201 INFO:tasks.workunit.client.0.smithi064.stdout:0/530: dread - da/d26/f66 zero size 2024-02-16T22:56:28.202 INFO:tasks.workunit.client.1.smithi192.stdout:2/725: dwrite d22/d24/d39/d42/d84/fd4 [0,4194304] 0 2024-02-16T22:56:28.202 INFO:tasks.workunit.client.1.smithi192.stdout:4/725: mknod d1/d15/cfb 0 2024-02-16T22:56:28.203 INFO:tasks.workunit.client.0.smithi064.stdout:3/665: dread d4/d1f/f67 [0,4194304] 0 2024-02-16T22:56:28.205 INFO:tasks.workunit.client.0.smithi064.stdout:9/544: link d3/d9/d6a/db1/d2a/d2e/d31/d3f/faa d3/d9/d6a/db1/fb9 0 2024-02-16T22:56:28.207 INFO:tasks.workunit.client.1.smithi192.stdout:0/670: rmdir d1/d3/d18 39 2024-02-16T22:56:28.212 INFO:tasks.workunit.client.0.smithi064.stdout:0/531: mkdir da/dc/de/d12/d2c/d2e/d62/db6 0 2024-02-16T22:56:28.214 INFO:tasks.workunit.client.0.smithi064.stdout:0/532: stat da/dc/de/d12/d2c/d2e/d38/d87/f9f 0 2024-02-16T22:56:28.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/671: dwrite d1/d3/de/d22/d37/d67/d34/fbc [0,4194304] 0 2024-02-16T22:56:28.219 INFO:tasks.workunit.client.1.smithi192.stdout:9/798: rename d7/d19/d28/fb9 to d7/d19/f112 0 2024-02-16T22:56:28.222 INFO:tasks.workunit.client.1.smithi192.stdout:3/625: fsync da/d11/d63/d6b/d71/f78 0 2024-02-16T22:56:28.224 INFO:tasks.workunit.client.1.smithi192.stdout:5/672: creat d6/d12/d4e/d5b/fdf x:0 0 0 2024-02-16T22:56:28.227 INFO:tasks.workunit.client.1.smithi192.stdout:8/663: unlink d5/d26/d83/d86/d5f/c95 0 2024-02-16T22:56:28.228 INFO:tasks.workunit.client.1.smithi192.stdout:9/799: dwrite d7/d19/d37/f8c [0,4194304] 0 2024-02-16T22:56:28.228 INFO:tasks.workunit.client.0.smithi064.stdout:0/533: mkdir da/dc/d3b/d5a/d8f/da8/db7 0 2024-02-16T22:56:28.231 INFO:tasks.workunit.client.1.smithi192.stdout:1/632: creat d3/de/d27/fd7 x:0 0 0 2024-02-16T22:56:28.236 INFO:tasks.workunit.client.0.smithi064.stdout:0/534: creat da/dc/de/d12/d2c/d2e/d62/db6/fb8 x:0 0 0 2024-02-16T22:56:28.239 INFO:tasks.workunit.client.1.smithi192.stdout:2/726: dread fa [0,4194304] 0 2024-02-16T22:56:28.241 INFO:tasks.workunit.client.0.smithi064.stdout:0/535: creat da/d26/fb9 x:0 0 0 2024-02-16T22:56:28.242 INFO:tasks.workunit.client.1.smithi192.stdout:4/726: creat d1/d26/d5e/ffc x:0 0 0 2024-02-16T22:56:28.244 INFO:tasks.workunit.client.0.smithi064.stdout:0/536: rmdir da/dc/de/d37 39 2024-02-16T22:56:28.245 INFO:tasks.workunit.client.0.smithi064.stdout:0/537: read - da/d71/f8d zero size 2024-02-16T22:56:28.247 INFO:tasks.workunit.client.1.smithi192.stdout:0/672: readlink d1/d3/d29/l2c 0 2024-02-16T22:56:28.248 INFO:tasks.workunit.client.1.smithi192.stdout:4/727: dwrite d1/d6b/fa1 [0,4194304] 0 2024-02-16T22:56:28.253 INFO:tasks.workunit.client.1.smithi192.stdout:4/728: dwrite d1/d26/d66/d89/f3a [0,4194304] 0 2024-02-16T22:56:28.254 INFO:tasks.workunit.client.1.smithi192.stdout:4/729: readlink d1/d26/d66/ld9 0 2024-02-16T22:56:28.257 INFO:tasks.workunit.client.0.smithi064.stdout:4/589: dwrite dc/dd/d32/d46/d5a/f66 [0,4194304] 0 2024-02-16T22:56:28.263 INFO:tasks.workunit.client.1.smithi192.stdout:5/673: symlink d6/d12/d4e/d5b/d67/d78/dae/dbc/le0 0 2024-02-16T22:56:28.264 INFO:tasks.workunit.client.0.smithi064.stdout:3/666: dwrite d4/d13/f62 [0,4194304] 0 2024-02-16T22:56:28.270 INFO:tasks.workunit.client.1.smithi192.stdout:8/664: rename d5/db/d47/d8c to d5/d26/dce 0 2024-02-16T22:56:28.271 INFO:tasks.workunit.client.1.smithi192.stdout:8/665: dread - d5/fc8 zero size 2024-02-16T22:56:28.272 INFO:tasks.workunit.client.1.smithi192.stdout:8/666: chown d5/db/d47/d43/d67/f85 14063098 1 2024-02-16T22:56:28.273 INFO:tasks.workunit.client.1.smithi192.stdout:1/633: creat d3/de/d27/d34/d52/d63/d81/fd8 x:0 0 0 2024-02-16T22:56:28.275 INFO:tasks.workunit.client.0.smithi064.stdout:4/590: mkdir dc/dd/db8 0 2024-02-16T22:56:28.276 INFO:tasks.workunit.client.1.smithi192.stdout:6/713: write da/df/d12/d15/f31 [4175740,13051] 0 2024-02-16T22:56:28.278 INFO:tasks.workunit.client.0.smithi064.stdout:3/667: rename d4/d13/d4d/d5c/d78/d96 to d4/d13/d1b/d27/d61/d63/de1 0 2024-02-16T22:56:28.281 INFO:tasks.workunit.client.1.smithi192.stdout:7/782: creat d1/d2/d2d/d61/f10e x:0 0 0 2024-02-16T22:56:28.282 INFO:tasks.workunit.client.1.smithi192.stdout:6/714: dread da/df/d12/d57/d81/fb2 [0,4194304] 0 2024-02-16T22:56:28.282 INFO:tasks.workunit.client.0.smithi064.stdout:3/668: creat d4/d13/d1b/d27/d99/fe2 x:0 0 0 2024-02-16T22:56:28.283 INFO:tasks.workunit.client.1.smithi192.stdout:0/673: chown d1/d3/de/d22/d37/f87 21951 1 2024-02-16T22:56:28.285 INFO:tasks.workunit.client.1.smithi192.stdout:3/626: mkdir da/d11/d2b/ddc 0 2024-02-16T22:56:28.285 INFO:tasks.workunit.client.0.smithi064.stdout:4/591: dread dc/dd/d32/d46/f55 [0,4194304] 0 2024-02-16T22:56:28.286 INFO:tasks.workunit.client.0.smithi064.stdout:3/669: symlink d4/d13/d1b/le3 0 2024-02-16T22:56:28.286 INFO:tasks.workunit.client.1.smithi192.stdout:6/715: dread da/df/fce [0,4194304] 0 2024-02-16T22:56:28.286 INFO:tasks.workunit.client.1.smithi192.stdout:5/674: write d6/d12/d4e/d58/fb7 [984322,36768] 0 2024-02-16T22:56:28.286 INFO:tasks.workunit.client.1.smithi192.stdout:9/800: mknod d7/d19/c113 0 2024-02-16T22:56:28.289 INFO:tasks.workunit.client.1.smithi192.stdout:2/727: mknod d22/d24/ce9 0 2024-02-16T22:56:28.291 INFO:tasks.workunit.client.0.smithi064.stdout:4/592: creat dc/de/fb9 x:0 0 0 2024-02-16T22:56:28.293 INFO:tasks.workunit.client.0.smithi064.stdout:4/593: stat dc/dd/d32/d46/d82/da1 0 2024-02-16T22:56:28.294 INFO:tasks.workunit.client.0.smithi064.stdout:3/670: dwrite d4/d13/d1b/d27/d99/fe2 [0,4194304] 0 2024-02-16T22:56:28.296 INFO:tasks.workunit.client.1.smithi192.stdout:3/627: rename da/df/f3a to da/d11/d63/d6b/d71/da1/db9/fdd 0 2024-02-16T22:56:28.297 INFO:tasks.workunit.client.1.smithi192.stdout:3/628: readlink da/df/lc0 0 2024-02-16T22:56:28.298 INFO:tasks.workunit.client.1.smithi192.stdout:6/716: unlink da/df/d12/d57/cd7 0 2024-02-16T22:56:28.299 INFO:tasks.workunit.client.1.smithi192.stdout:6/717: dread - da/df/d12/d15/d1b/d70/fd3 zero size 2024-02-16T22:56:28.300 INFO:tasks.workunit.client.0.smithi064.stdout:3/671: creat d4/d1f/d5b/fe4 x:0 0 0 2024-02-16T22:56:28.302 INFO:tasks.workunit.client.1.smithi192.stdout:1/634: unlink d3/d5b/f84 0 2024-02-16T22:56:28.304 INFO:tasks.workunit.client.0.smithi064.stdout:4/594: rename dc/d1d/f95 to dc/fba 0 2024-02-16T22:56:28.305 INFO:tasks.workunit.client.1.smithi192.stdout:8/667: truncate d5/db/de/df/f1b 858161 0 2024-02-16T22:56:28.305 INFO:tasks.workunit.client.0.smithi064.stdout:0/538: dwrite da/dc/de/d37/daa/db4/faf [0,4194304] 0 2024-02-16T22:56:28.307 INFO:tasks.workunit.client.1.smithi192.stdout:1/635: dwrite d3/d5b/f8a [0,4194304] 0 2024-02-16T22:56:28.309 INFO:tasks.workunit.client.0.smithi064.stdout:3/672: truncate d4/d13/d1b/d27/f36 3497867 0 2024-02-16T22:56:28.313 INFO:tasks.workunit.client.1.smithi192.stdout:8/668: dwrite d5/d68/fbb [0,4194304] 0 2024-02-16T22:56:28.315 INFO:tasks.workunit.client.0.smithi064.stdout:0/539: creat da/dc/de/d37/daa/db4/fba x:0 0 0 2024-02-16T22:56:28.316 INFO:tasks.workunit.client.1.smithi192.stdout:7/783: mkdir d1/d2/d2d/d61/d88/d90/d10f 0 2024-02-16T22:56:28.318 INFO:tasks.workunit.client.1.smithi192.stdout:0/674: mknod d1/cd7 0 2024-02-16T22:56:28.320 INFO:tasks.workunit.client.1.smithi192.stdout:0/675: truncate d1/d3/de/d22/d37/d47/fd4 609283 0 2024-02-16T22:56:28.321 INFO:tasks.workunit.client.0.smithi064.stdout:4/595: creat dc/dd/d32/d46/d82/da1/fbb x:0 0 0 2024-02-16T22:56:28.323 INFO:tasks.workunit.client.0.smithi064.stdout:4/596: write dc/dd/d32/d46/d5a/f83 [689438,111271] 0 2024-02-16T22:56:28.326 INFO:tasks.workunit.client.0.smithi064.stdout:3/673: rename d4/d13/d1b/lab to d4/d13/d1b/d27/d61/d63/le5 0 2024-02-16T22:56:28.326 INFO:tasks.workunit.client.0.smithi064.stdout:3/674: dread - d4/d1f/d34/f56 zero size 2024-02-16T22:56:28.327 INFO:tasks.workunit.client.1.smithi192.stdout:4/730: dwrite d1/d26/d66/d89/d11/fb7 [0,4194304] 0 2024-02-16T22:56:28.328 INFO:tasks.workunit.client.0.smithi064.stdout:3/675: readlink d4/d13/d1b/d27/d61/d63/de1/lae 0 2024-02-16T22:56:28.328 INFO:tasks.workunit.client.0.smithi064.stdout:3/676: stat d4/d13/d1b/d27 0 2024-02-16T22:56:28.328 INFO:tasks.workunit.client.1.smithi192.stdout:9/801: symlink d7/d19/d28/d43/dea/l114 0 2024-02-16T22:56:28.329 INFO:tasks.workunit.client.0.smithi064.stdout:3/677: chown d4/d13/d1b/d27/d61/d63/cbb 1518 1 2024-02-16T22:56:28.330 INFO:tasks.workunit.client.1.smithi192.stdout:6/718: mkdir da/df/d18/d52/de6 0 2024-02-16T22:56:28.332 INFO:tasks.workunit.client.1.smithi192.stdout:6/719: write da/df/f6e [739064,49745] 0 2024-02-16T22:56:28.333 INFO:tasks.workunit.client.1.smithi192.stdout:2/728: symlink d22/d24/d2b/lea 0 2024-02-16T22:56:28.334 INFO:tasks.workunit.client.1.smithi192.stdout:6/720: write da/df/d12/d15/f31 [5230435,79858] 0 2024-02-16T22:56:28.334 INFO:tasks.workunit.client.0.smithi064.stdout:3/678: rename d4/f12 to d4/d1f/d34/fe6 0 2024-02-16T22:56:28.335 INFO:tasks.workunit.client.1.smithi192.stdout:6/721: dread - da/df/d18/fe4 zero size 2024-02-16T22:56:28.337 INFO:tasks.workunit.client.1.smithi192.stdout:1/636: mkdir d3/de/d27/d34/d36/d47/dd9 0 2024-02-16T22:56:28.344 INFO:tasks.workunit.client.1.smithi192.stdout:8/669: unlink d5/db/d47/d43/c6a 0 2024-02-16T22:56:28.348 INFO:tasks.workunit.client.0.smithi064.stdout:3/679: creat d4/d13/d1b/d27/fe7 x:0 0 0 2024-02-16T22:56:28.349 INFO:tasks.workunit.client.1.smithi192.stdout:3/629: mknod da/d11/d63/d6b/d66/cde 0 2024-02-16T22:56:28.351 INFO:tasks.workunit.client.1.smithi192.stdout:4/731: creat d1/d26/d34/ffd x:0 0 0 2024-02-16T22:56:28.352 INFO:tasks.workunit.client.0.smithi064.stdout:3/680: write d4/d13/d1b/d27/d44/fc7 [722870,77754] 0 2024-02-16T22:56:28.352 INFO:tasks.workunit.client.1.smithi192.stdout:5/675: write d6/d12/f83 [864347,37717] 0 2024-02-16T22:56:28.353 INFO:tasks.workunit.client.1.smithi192.stdout:2/729: mknod d22/ceb 0 2024-02-16T22:56:28.356 INFO:tasks.workunit.client.0.smithi064.stdout:3/681: mknod d4/d13/d1b/d48/ce8 0 2024-02-16T22:56:28.356 INFO:tasks.workunit.client.0.smithi064.stdout:0/540: write da/f1b [9076076,94470] 0 2024-02-16T22:56:28.357 INFO:tasks.workunit.client.0.smithi064.stdout:3/682: readlink d4/d1f/d34/l98 0 2024-02-16T22:56:28.359 INFO:tasks.workunit.client.0.smithi064.stdout:3/683: chown d4/d13/d1b 9118639 1 2024-02-16T22:56:28.359 INFO:tasks.workunit.client.1.smithi192.stdout:8/670: mkdir d5/d16/dcf 0 2024-02-16T22:56:28.359 INFO:tasks.workunit.client.0.smithi064.stdout:3/684: fdatasync d4/d13/d1b/d27/f58 0 2024-02-16T22:56:28.360 INFO:tasks.workunit.client.1.smithi192.stdout:9/802: dread d7/d15/f6d [0,4194304] 0 2024-02-16T22:56:28.360 INFO:tasks.workunit.client.1.smithi192.stdout:9/803: readlink d7/l64 0 2024-02-16T22:56:28.365 INFO:tasks.workunit.client.0.smithi064.stdout:3/685: creat d4/d14/d2b/fe9 x:0 0 0 2024-02-16T22:56:28.366 INFO:tasks.workunit.client.1.smithi192.stdout:4/732: readlink d1/lf 0 2024-02-16T22:56:28.373 INFO:tasks.workunit.client.1.smithi192.stdout:1/637: dread d3/f21 [4194304,4194304] 0 2024-02-16T22:56:28.378 INFO:tasks.workunit.client.1.smithi192.stdout:1/638: dwrite d3/de/d27/d34/d69/d85/d89/daa/fb3 [0,4194304] 0 2024-02-16T22:56:28.379 INFO:tasks.workunit.client.1.smithi192.stdout:1/639: fdatasync d3/de/d27/d34/dbf/fcf 0 2024-02-16T22:56:28.395 INFO:tasks.workunit.client.0.smithi064.stdout:6/626: sync 2024-02-16T22:56:28.401 INFO:tasks.workunit.client.1.smithi192.stdout:5/676: creat d6/d12/d4e/d58/d8f/fe1 x:0 0 0 2024-02-16T22:56:28.403 INFO:tasks.workunit.client.0.smithi064.stdout:3/686: creat d4/d13/d1b/d27/d81/fea x:0 0 0 2024-02-16T22:56:28.406 INFO:tasks.workunit.client.1.smithi192.stdout:8/671: symlink d5/d26/d83/d86/ld0 0 2024-02-16T22:56:28.407 INFO:tasks.workunit.client.0.smithi064.stdout:6/627: creat d0/d12/fd2 x:0 0 0 2024-02-16T22:56:28.411 INFO:tasks.workunit.client.1.smithi192.stdout:7/784: truncate d1/d2/de/d2c/d49/da4/f85 1393824 0 2024-02-16T22:56:28.412 INFO:tasks.workunit.client.0.smithi064.stdout:0/541: write da/d26/d3f/f75 [5176291,90195] 0 2024-02-16T22:56:28.414 INFO:tasks.workunit.client.0.smithi064.stdout:3/687: link d4/fd9 d4/d14/d2b/feb 0 2024-02-16T22:56:28.415 INFO:tasks.workunit.client.0.smithi064.stdout:3/688: dread d4/d13/d4d/d5c/f6c [0,4194304] 0 2024-02-16T22:56:28.416 INFO:tasks.workunit.client.0.smithi064.stdout:6/628: truncate d0/d12/f74 2215235 0 2024-02-16T22:56:28.417 INFO:tasks.workunit.client.1.smithi192.stdout:4/733: mknod d1/d26/d6f/cfe 0 2024-02-16T22:56:28.418 INFO:tasks.workunit.client.0.smithi064.stdout:0/542: truncate da/f4d 146918 0 2024-02-16T22:56:28.420 INFO:tasks.workunit.client.0.smithi064.stdout:3/689: rmdir d4/d13/d41 39 2024-02-16T22:56:28.422 INFO:tasks.workunit.client.1.smithi192.stdout:6/722: dwrite da/df/f8d [0,4194304] 0 2024-02-16T22:56:28.424 INFO:tasks.workunit.client.1.smithi192.stdout:2/730: mkdir d22/d47/dcb/dda/dec 0 2024-02-16T22:56:28.426 INFO:tasks.workunit.client.1.smithi192.stdout:2/731: write d22/d47/f7d [4693075,9131] 0 2024-02-16T22:56:28.428 INFO:tasks.workunit.client.0.smithi064.stdout:3/690: dwrite d4/d13/d1b/fcb [0,4194304] 0 2024-02-16T22:56:28.430 INFO:tasks.workunit.client.1.smithi192.stdout:9/804: dwrite d7/f9d [0,4194304] 0 2024-02-16T22:56:28.430 INFO:tasks.workunit.client.1.smithi192.stdout:2/732: write d22/d24/d39/d42/d85/d98/db0/fc1 [56218,87972] 0 2024-02-16T22:56:28.434 INFO:tasks.workunit.client.1.smithi192.stdout:9/805: read d7/d15/d36/d38/f47 [1063380,34137] 0 2024-02-16T22:56:28.436 INFO:tasks.workunit.client.1.smithi192.stdout:8/672: creat d5/d26/dce/fd1 x:0 0 0 2024-02-16T22:56:28.438 INFO:tasks.workunit.client.0.smithi064.stdout:3/691: dread d4/d13/fa7 [0,4194304] 0 2024-02-16T22:56:28.443 INFO:tasks.workunit.client.1.smithi192.stdout:0/676: link d1/d3/de/d22/d37/d47/c49 d1/d3/de/d22/d37/d67/d34/cd8 0 2024-02-16T22:56:28.444 INFO:tasks.workunit.client.1.smithi192.stdout:7/785: rmdir d1/d2/d2d/d61/d88/d90/db2 39 2024-02-16T22:56:28.445 INFO:tasks.workunit.client.0.smithi064.stdout:0/543: link da/dc/de/f86 da/dc/d3b/d5a/d8f/fbb 0 2024-02-16T22:56:28.446 INFO:tasks.workunit.client.0.smithi064.stdout:0/544: write da/dc/de/d12/d2c/f92 [6575309,14723] 0 2024-02-16T22:56:28.446 INFO:tasks.workunit.client.1.smithi192.stdout:3/630: creat da/d11/fdf x:0 0 0 2024-02-16T22:56:28.451 INFO:tasks.workunit.client.0.smithi064.stdout:6/629: dread d0/d5/f27 [0,4194304] 0 2024-02-16T22:56:28.452 INFO:tasks.workunit.client.1.smithi192.stdout:1/640: rename d3/de/d27/d34/d52/f56 to d3/de/da4/fda 0 2024-02-16T22:56:28.453 INFO:tasks.workunit.client.1.smithi192.stdout:3/631: dwrite da/d6a/f81 [0,4194304] 0 2024-02-16T22:56:28.454 INFO:tasks.workunit.client.1.smithi192.stdout:1/641: chown d3/de/d27/c37 4036 1 2024-02-16T22:56:28.454 INFO:tasks.workunit.client.1.smithi192.stdout:6/723: dread da/df/d12/f47 [0,4194304] 0 2024-02-16T22:56:28.457 INFO:tasks.workunit.client.1.smithi192.stdout:5/677: mknod d6/d12/d4e/d5b/ce2 0 2024-02-16T22:56:28.459 INFO:tasks.workunit.client.1.smithi192.stdout:2/733: mkdir d22/d47/dcb/d78/ded 0 2024-02-16T22:56:28.461 INFO:tasks.workunit.client.1.smithi192.stdout:9/806: symlink d7/d15/d36/d38/d3f/d8a/d98/d6e/l115 0 2024-02-16T22:56:28.462 INFO:tasks.workunit.client.1.smithi192.stdout:9/807: fdatasync d7/d15/d36/d38/d3f/d8a/def/df6/f110 0 2024-02-16T22:56:28.463 INFO:tasks.workunit.client.0.smithi064.stdout:6/630: creat d0/d5/df/d17/da1/fd3 x:0 0 0 2024-02-16T22:56:28.463 INFO:tasks.workunit.client.1.smithi192.stdout:4/734: write d1/d26/d6f/fbc [530904,67123] 0 2024-02-16T22:56:28.464 INFO:tasks.workunit.client.0.smithi064.stdout:3/692: dwrite d4/f84 [0,4194304] 0 2024-02-16T22:56:28.464 INFO:tasks.workunit.client.1.smithi192.stdout:0/677: read d1/d3/de/d22/d37/d67/f66 [60199,83301] 0 2024-02-16T22:56:28.466 INFO:tasks.workunit.client.1.smithi192.stdout:2/734: dread d22/d24/d39/d42/d65/d66/fd6 [0,4194304] 0 2024-02-16T22:56:28.472 INFO:tasks.workunit.client.0.smithi064.stdout:6/631: rename d0/d1d/d21/f7f to d0/d6/d10/d33/da4/fd4 0 2024-02-16T22:56:28.472 INFO:tasks.workunit.client.0.smithi064.stdout:3/693: dwrite d4/d1f/d5b/fd5 [0,4194304] 0 2024-02-16T22:56:28.474 INFO:tasks.workunit.client.0.smithi064.stdout:0/545: dread f9 [0,4194304] 0 2024-02-16T22:56:28.475 INFO:tasks.workunit.client.1.smithi192.stdout:8/673: rename d5/d68/fbb to d5/d16/fd2 0 2024-02-16T22:56:28.478 INFO:tasks.workunit.client.1.smithi192.stdout:1/642: creat d3/de/d27/d34/d52/d63/d81/fdb x:0 0 0 2024-02-16T22:56:28.482 INFO:tasks.workunit.client.0.smithi064.stdout:6/632: symlink d0/d1d/ld5 0 2024-02-16T22:56:28.483 INFO:tasks.workunit.client.1.smithi192.stdout:5/678: dread - d6/d12/d4e/d46/d4f/f62 zero size 2024-02-16T22:56:28.483 INFO:tasks.workunit.client.0.smithi064.stdout:6/633: readlink d0/d5/df/d17/d18/l4e 0 2024-02-16T22:56:28.484 INFO:tasks.workunit.client.1.smithi192.stdout:5/679: fsync d6/d20/fc5 0 2024-02-16T22:56:28.485 INFO:tasks.workunit.client.0.smithi064.stdout:3/694: truncate d4/d13/d1b/fb2 924862 0 2024-02-16T22:56:28.485 INFO:tasks.workunit.client.0.smithi064.stdout:3/695: chown d4/d1f/ld4 0 1 2024-02-16T22:56:28.490 INFO:tasks.workunit.client.1.smithi192.stdout:9/808: creat d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f116 x:0 0 0 2024-02-16T22:56:28.491 INFO:tasks.workunit.client.0.smithi064.stdout:3/696: creat d4/d13/d1b/d27/d44/fec x:0 0 0 2024-02-16T22:56:28.492 INFO:tasks.workunit.client.0.smithi064.stdout:0/546: write da/dc/de/d12/f49 [206119,55639] 0 2024-02-16T22:56:28.492 INFO:tasks.workunit.client.1.smithi192.stdout:2/735: fsync d22/d24/fa0 0 2024-02-16T22:56:28.493 INFO:tasks.workunit.client.1.smithi192.stdout:2/736: fdatasync d22/d24/d39/d42/d65/d69/fa1 0 2024-02-16T22:56:28.494 INFO:tasks.workunit.client.0.smithi064.stdout:3/697: truncate d4/d1f/d34/fba 408353 0 2024-02-16T22:56:28.495 INFO:tasks.workunit.client.0.smithi064.stdout:6/634: fsync d0/d5/fce 0 2024-02-16T22:56:28.495 INFO:tasks.workunit.client.1.smithi192.stdout:2/737: chown d22/d24/d39/d42/d85/d98/db0 54969 1 2024-02-16T22:56:28.496 INFO:tasks.workunit.client.0.smithi064.stdout:3/698: chown d4/d13/d1b/d27/d99/d7d 8269869 1 2024-02-16T22:56:28.496 INFO:tasks.workunit.client.1.smithi192.stdout:7/786: rename d1/d69/d10a to d1/d2/de/d83/d9f/d110 0 2024-02-16T22:56:28.498 INFO:tasks.workunit.client.0.smithi064.stdout:6/635: write d0/d5/df/fc2 [47179,72388] 0 2024-02-16T22:56:28.498 INFO:tasks.workunit.client.0.smithi064.stdout:3/699: dread - d4/d1f/d5b/fe4 zero size 2024-02-16T22:56:28.500 INFO:tasks.workunit.client.1.smithi192.stdout:6/724: creat da/d76/dd5/de2/fe7 x:0 0 0 2024-02-16T22:56:28.500 INFO:tasks.workunit.client.0.smithi064.stdout:6/636: write d0/d1b/fcb [18542,31338] 0 2024-02-16T22:56:28.500 INFO:tasks.workunit.client.0.smithi064.stdout:0/547: dwrite da/dc/d3b/d5a/f73 [0,4194304] 0 2024-02-16T22:56:28.501 INFO:tasks.workunit.client.1.smithi192.stdout:1/643: dread d3/d40/f59 [0,4194304] 0 2024-02-16T22:56:28.503 INFO:tasks.workunit.client.1.smithi192.stdout:2/738: dwrite d22/d24/d39/d42/d65/d69/f7a [4194304,4194304] 0 2024-02-16T22:56:28.504 INFO:tasks.workunit.client.1.smithi192.stdout:3/632: link da/d11/d20/d39/d53/f93 da/d11/d63/d6b/d4e/fe0 0 2024-02-16T22:56:28.504 INFO:tasks.workunit.client.1.smithi192.stdout:1/644: write d3/de/d27/d34/d52/d5a/fa2 [85902,42622] 0 2024-02-16T22:56:28.510 INFO:tasks.workunit.client.0.smithi064.stdout:0/548: mknod da/dc/de/d37/daa/cbc 0 2024-02-16T22:56:28.513 INFO:tasks.workunit.client.1.smithi192.stdout:9/809: chown d7/d15/d36/d77/c7f 0 1 2024-02-16T22:56:28.514 INFO:tasks.workunit.client.1.smithi192.stdout:3/633: truncate da/d11/d63/d6b/d71/da1/fb0 644278 0 2024-02-16T22:56:28.514 INFO:tasks.workunit.client.1.smithi192.stdout:9/810: stat d7/d19/d37/ddf/cfb 0 2024-02-16T22:56:28.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/674: mknod d5/db/de/cd3 0 2024-02-16T22:56:28.517 INFO:tasks.workunit.client.1.smithi192.stdout:7/787: rename d1/d2/de/d2c/d49/f6f to d1/d2/de/d2c/d49/d54/d9d/f111 0 2024-02-16T22:56:28.518 INFO:tasks.workunit.client.0.smithi064.stdout:6/637: getdents d0/d5 0 2024-02-16T22:56:28.519 INFO:tasks.workunit.client.1.smithi192.stdout:1/645: dwrite d3/de/d27/d34/d69/d85/d89/fbe [0,4194304] 0 2024-02-16T22:56:28.521 INFO:tasks.workunit.client.0.smithi064.stdout:6/638: symlink d0/d6/d10/d33/da4/ld6 0 2024-02-16T22:56:28.526 INFO:tasks.workunit.client.1.smithi192.stdout:1/646: dwrite d3/de/d87/fa9 [0,4194304] 0 2024-02-16T22:56:28.528 INFO:tasks.workunit.client.0.smithi064.stdout:3/700: write d4/d13/d1b/d27/d44/f5f [389375,59336] 0 2024-02-16T22:56:28.530 INFO:tasks.workunit.client.0.smithi064.stdout:6/639: symlink d0/d6/da/d56/db9/ld7 0 2024-02-16T22:56:28.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/680: dwrite d6/d12/d4e/d5b/f5e [0,4194304] 0 2024-02-16T22:56:28.533 INFO:tasks.workunit.client.0.smithi064.stdout:3/701: rename d4/d14/fce to d4/d13/d4d/d5c/d78/fed 0 2024-02-16T22:56:28.533 INFO:tasks.workunit.client.1.smithi192.stdout:5/681: write d6/d12/d2d/d5c/f99 [2342003,15050] 0 2024-02-16T22:56:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:0/549: dread da/d26/d3f/f6b [0,4194304] 0 2024-02-16T22:56:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:6/640: rename d0/d6/d10/d33/da4/ld6 to d0/d6/d10/db2/ld8 0 2024-02-16T22:56:28.544 INFO:tasks.workunit.client.0.smithi064.stdout:0/550: write da/d26/d3f/f6b [4484267,109004] 0 2024-02-16T22:56:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:6/641: dwrite d0/d6/d10/f40 [4194304,4194304] 0 2024-02-16T22:56:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:0/551: symlink da/dc/d3b/d5a/d8f/lbd 0 2024-02-16T22:56:28.549 INFO:tasks.workunit.client.1.smithi192.stdout:9/811: dread d7/d15/f5d [0,4194304] 0 2024-02-16T22:56:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:0/552: truncate da/dc/de/d12/d2c/d2e/d38/f7a 88318 0 2024-02-16T22:56:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:6/642: dwrite d0/d1b/fc8 [0,4194304] 0 2024-02-16T22:56:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:0/553: dread da/dc/de/d12/d2c/f31 [0,4194304] 0 2024-02-16T22:56:28.562 INFO:tasks.workunit.client.0.smithi064.stdout:6/643: read d0/d5/df/d17/f62 [102518,38337] 0 2024-02-16T22:56:28.564 INFO:tasks.workunit.client.0.smithi064.stdout:0/554: readlink da/dc/de/d12/d2c/l50 0 2024-02-16T22:56:28.568 INFO:tasks.workunit.client.0.smithi064.stdout:0/555: mkdir da/dc/de/d12/d2c/d2e/d38/dbe 0 2024-02-16T22:56:28.568 INFO:tasks.workunit.client.0.smithi064.stdout:0/556: chown da/dc/f4b 10 1 2024-02-16T22:56:28.574 INFO:tasks.workunit.client.0.smithi064.stdout:0/557: dwrite da/da0/fb2 [4194304,4194304] 0 2024-02-16T22:56:28.578 INFO:tasks.workunit.client.1.smithi192.stdout:3/634: chown da/d11/d63/d6b/d4e/ca2 0 1 2024-02-16T22:56:28.579 INFO:tasks.workunit.client.1.smithi192.stdout:3/635: chown da/d11/d63/d6b/d71 11929882 1 2024-02-16T22:56:28.588 INFO:tasks.workunit.client.1.smithi192.stdout:9/812: mkdir d7/d15/d36/d38/d3f/d8a/d117 0 2024-02-16T22:56:28.589 INFO:tasks.workunit.client.1.smithi192.stdout:4/735: getdents d1/d15/d29/d3c 0 2024-02-16T22:56:28.590 INFO:tasks.workunit.client.1.smithi192.stdout:3/636: dread - da/d11/d20/d39/d53/f5b zero size 2024-02-16T22:56:28.594 INFO:tasks.workunit.client.1.smithi192.stdout:9/813: truncate d7/d19/f94 482214 0 2024-02-16T22:56:28.595 INFO:tasks.workunit.client.1.smithi192.stdout:9/814: chown d7/d15/d36/d38/d3f/d8a/def/d100/d10e 35749343 1 2024-02-16T22:56:28.596 INFO:tasks.workunit.client.1.smithi192.stdout:4/736: read d1/d15/d48/d8d/d9b/fb0 [321163,29107] 0 2024-02-16T22:56:28.596 INFO:tasks.workunit.client.1.smithi192.stdout:2/739: rename d22/d24/d39/d3d/ca5 to d22/d47/dcb/cee 0 2024-02-16T22:56:28.597 INFO:tasks.workunit.client.1.smithi192.stdout:6/725: getdents da/df/d12/d57/daf/dcf 0 2024-02-16T22:56:28.599 INFO:tasks.workunit.client.1.smithi192.stdout:4/737: truncate d1/d15/d29/dd4/ff5 1001509 0 2024-02-16T22:56:28.602 INFO:tasks.workunit.client.1.smithi192.stdout:1/647: dread d3/de/d27/f44 [0,4194304] 0 2024-02-16T22:56:28.602 INFO:tasks.workunit.client.1.smithi192.stdout:4/738: chown d1/d26/d6f/fe3 1 1 2024-02-16T22:56:28.603 INFO:tasks.workunit.client.1.smithi192.stdout:8/675: rename d5/db/f9b to d5/db/d47/fd4 0 2024-02-16T22:56:28.604 INFO:tasks.workunit.client.1.smithi192.stdout:2/740: mknod d22/d24/d2b/d99/d9b/cef 0 2024-02-16T22:56:28.605 INFO:tasks.workunit.client.1.smithi192.stdout:0/678: sync 2024-02-16T22:56:28.606 INFO:tasks.workunit.client.1.smithi192.stdout:8/676: chown d5/d16/f27 0 1 2024-02-16T22:56:28.606 INFO:tasks.workunit.client.1.smithi192.stdout:6/726: dwrite da/d76/dd5/de2/fe7 [0,4194304] 0 2024-02-16T22:56:28.606 INFO:tasks.workunit.client.1.smithi192.stdout:1/648: dread - d3/de/d27/d34/d52/d63/d81/fdb zero size 2024-02-16T22:56:28.607 INFO:tasks.workunit.client.1.smithi192.stdout:4/739: creat d1/d26/d66/d89/da8/fff x:0 0 0 2024-02-16T22:56:28.611 INFO:tasks.workunit.client.1.smithi192.stdout:5/682: write d6/f6f [639526,56898] 0 2024-02-16T22:56:28.613 INFO:tasks.workunit.client.1.smithi192.stdout:9/815: dread f1 [0,4194304] 0 2024-02-16T22:56:28.614 INFO:tasks.workunit.client.0.smithi064.stdout:5/495: sync 2024-02-16T22:56:28.616 INFO:tasks.workunit.client.1.smithi192.stdout:7/788: dwrite d1/d2/d2d/d61/f74 [0,4194304] 0 2024-02-16T22:56:28.616 INFO:tasks.workunit.client.1.smithi192.stdout:8/677: rename d5/d26/dce to d5/d16/dd5 0 2024-02-16T22:56:28.618 INFO:tasks.workunit.client.1.smithi192.stdout:3/637: write da/d11/f50 [44209,89018] 0 2024-02-16T22:56:28.619 INFO:tasks.workunit.client.1.smithi192.stdout:3/638: dread - da/d11/d20/fab zero size 2024-02-16T22:56:28.620 INFO:tasks.workunit.client.0.smithi064.stdout:5/496: unlink d6/c96 0 2024-02-16T22:56:28.623 INFO:tasks.workunit.client.1.smithi192.stdout:1/649: creat d3/de/d27/d34/d69/d85/fdc x:0 0 0 2024-02-16T22:56:28.625 INFO:tasks.workunit.client.1.smithi192.stdout:4/740: symlink d1/d15/d48/l100 0 2024-02-16T22:56:28.627 INFO:tasks.workunit.client.1.smithi192.stdout:0/679: write d1/d3/de/d22/d37/d67/d34/f88 [144345,58262] 0 2024-02-16T22:56:28.627 INFO:tasks.workunit.client.0.smithi064.stdout:5/497: truncate d6/f34 1105015 0 2024-02-16T22:56:28.629 INFO:tasks.workunit.client.1.smithi192.stdout:5/683: rename d6/d12/l2a to d6/d8/d75/d8c/le3 0 2024-02-16T22:56:28.630 INFO:tasks.workunit.client.0.smithi064.stdout:5/498: rmdir d6/d18/d95 39 2024-02-16T22:56:28.631 INFO:tasks.workunit.client.1.smithi192.stdout:8/678: creat d5/db/d47/d43/d67/fd6 x:0 0 0 2024-02-16T22:56:28.631 INFO:tasks.workunit.client.1.smithi192.stdout:6/727: creat da/df/d71/dde/fe8 x:0 0 0 2024-02-16T22:56:28.636 INFO:tasks.workunit.client.1.smithi192.stdout:2/741: creat d22/d24/d39/d42/d85/d98/db0/db6/de2/ff0 x:0 0 0 2024-02-16T22:56:28.638 INFO:tasks.workunit.client.1.smithi192.stdout:1/650: creat d3/de/d27/d34/d69/d85/d89/daa/fdd x:0 0 0 2024-02-16T22:56:28.640 INFO:tasks.workunit.client.1.smithi192.stdout:4/741: mknod d1/d15/d48/d8d/d9b/c101 0 2024-02-16T22:56:28.644 INFO:tasks.workunit.client.1.smithi192.stdout:7/789: symlink d1/d2/de/d2c/d49/d54/l112 0 2024-02-16T22:56:28.644 INFO:tasks.workunit.client.1.smithi192.stdout:7/790: chown d1/d2/de/d83/d9f/ff0 0 1 2024-02-16T22:56:28.645 INFO:tasks.workunit.client.1.smithi192.stdout:5/684: rename d6/d8/la8 to d6/d12/d1d/d56/le4 0 2024-02-16T22:56:28.646 INFO:tasks.workunit.client.1.smithi192.stdout:8/679: creat d5/db/d47/d43/d67/fd7 x:0 0 0 2024-02-16T22:56:28.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/728: truncate da/df/d12/d15/d1b/f37 3591313 0 2024-02-16T22:56:28.647 INFO:tasks.workunit.client.1.smithi192.stdout:3/639: truncate da/fa8 178275 0 2024-02-16T22:56:28.648 INFO:tasks.workunit.client.1.smithi192.stdout:2/742: rmdir d22/d47/dcb/d78 39 2024-02-16T22:56:28.649 INFO:tasks.workunit.client.0.smithi064.stdout:5/499: write d6/f13 [44687,844] 0 2024-02-16T22:56:28.650 INFO:tasks.workunit.client.1.smithi192.stdout:0/680: getdents d1/d3/d29/dcd/dcf/dd1 0 2024-02-16T22:56:28.654 INFO:tasks.workunit.client.1.smithi192.stdout:7/791: mkdir d1/d2/d2d/ddf/d113 0 2024-02-16T22:56:28.655 INFO:tasks.workunit.client.0.smithi064.stdout:5/500: symlink d6/d18/d95/lac 0 2024-02-16T22:56:28.662 INFO:tasks.workunit.client.1.smithi192.stdout:6/729: dread da/df/f1c [4194304,4194304] 0 2024-02-16T22:56:28.662 INFO:tasks.workunit.client.0.smithi064.stdout:5/501: creat d6/d18/d19/d1a/d85/fad x:0 0 0 2024-02-16T22:56:28.665 INFO:tasks.workunit.client.1.smithi192.stdout:1/651: creat d3/d5b/db1/fde x:0 0 0 2024-02-16T22:56:28.667 INFO:tasks.workunit.client.0.smithi064.stdout:5/502: rename d6/d1c/d43/c73 to d6/d25/d48/d67/d91/cae 0 2024-02-16T22:56:28.669 INFO:tasks.workunit.client.1.smithi192.stdout:9/816: getdents d7/d15/d36/d38/d3f/d8a/def 0 2024-02-16T22:56:28.669 INFO:tasks.workunit.client.0.smithi064.stdout:5/503: rmdir d6/d18/d19 39 2024-02-16T22:56:28.671 INFO:tasks.workunit.client.1.smithi192.stdout:5/685: symlink d6/d12/d4e/le5 0 2024-02-16T22:56:28.674 INFO:tasks.workunit.client.1.smithi192.stdout:8/680: fsync d5/d16/dd5/f9c 0 2024-02-16T22:56:28.676 INFO:tasks.workunit.client.0.smithi064.stdout:5/504: mknod d6/d1c/caf 0 2024-02-16T22:56:28.679 INFO:tasks.workunit.client.1.smithi192.stdout:3/640: creat da/d11/dd4/fe1 x:0 0 0 2024-02-16T22:56:28.679 INFO:tasks.workunit.client.1.smithi192.stdout:0/681: write d1/dc7/f8b [238811,76246] 0 2024-02-16T22:56:28.680 INFO:tasks.workunit.client.1.smithi192.stdout:2/743: creat d22/d24/d39/d42/d85/d98/dc4/ff1 x:0 0 0 2024-02-16T22:56:28.681 INFO:tasks.workunit.client.1.smithi192.stdout:2/744: chown d22/d47/dcb/dc9 31 1 2024-02-16T22:56:28.682 INFO:tasks.workunit.client.1.smithi192.stdout:6/730: write da/df/d6b/d7d/fdc [768396,2568] 0 2024-02-16T22:56:28.683 INFO:tasks.workunit.client.1.smithi192.stdout:9/817: symlink d7/l118 0 2024-02-16T22:56:28.684 INFO:tasks.workunit.client.1.smithi192.stdout:3/641: dwrite da/df/f79 [0,4194304] 0 2024-02-16T22:56:28.685 INFO:tasks.workunit.client.0.smithi064.stdout:5/505: creat d6/d18/d19/fb0 x:0 0 0 2024-02-16T22:56:28.685 INFO:tasks.workunit.client.1.smithi192.stdout:3/642: write da/d11/fd9 [861410,51653] 0 2024-02-16T22:56:28.689 INFO:tasks.workunit.client.1.smithi192.stdout:8/681: mknod d5/d68/cd8 0 2024-02-16T22:56:28.690 INFO:tasks.workunit.client.1.smithi192.stdout:8/682: fsync d5/d16/dd5/fd1 0 2024-02-16T22:56:28.691 INFO:tasks.workunit.client.1.smithi192.stdout:0/682: fsync d1/d3/de/d22/d37/d67/f66 0 2024-02-16T22:56:28.694 INFO:tasks.workunit.client.1.smithi192.stdout:2/745: rename d22/d24/d2b to d22/d24/d39/d3d/df2 0 2024-02-16T22:56:28.695 INFO:tasks.workunit.client.1.smithi192.stdout:6/731: creat da/df/d12/d15/d1b/d70/fe9 x:0 0 0 2024-02-16T22:56:28.696 INFO:tasks.workunit.client.1.smithi192.stdout:6/732: stat da/d40/fbc 0 2024-02-16T22:56:28.697 INFO:tasks.workunit.client.1.smithi192.stdout:7/792: truncate d1/d2/d2d/f40 1057644 0 2024-02-16T22:56:28.699 INFO:tasks.workunit.client.1.smithi192.stdout:8/683: creat d5/d16/dd5/d9d/fd9 x:0 0 0 2024-02-16T22:56:28.700 INFO:tasks.workunit.client.1.smithi192.stdout:8/684: dread - d5/fae zero size 2024-02-16T22:56:28.700 INFO:tasks.workunit.client.1.smithi192.stdout:8/685: chown d5/db/d47/f92 441347516 1 2024-02-16T22:56:28.701 INFO:tasks.workunit.client.1.smithi192.stdout:0/683: creat d1/dc7/fd9 x:0 0 0 2024-02-16T22:56:28.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/652: rmdir d3/de/d27/d34/dbf/dc4 0 2024-02-16T22:56:28.704 INFO:tasks.workunit.client.0.smithi064.stdout:5/506: dwrite d6/d18/d19/f2e [0,4194304] 0 2024-02-16T22:56:28.707 INFO:tasks.workunit.client.1.smithi192.stdout:6/733: creat da/df/d58/d72/fea x:0 0 0 2024-02-16T22:56:28.708 INFO:tasks.workunit.client.1.smithi192.stdout:8/686: fsync d5/db/de/f37 0 2024-02-16T22:56:28.709 INFO:tasks.workunit.client.1.smithi192.stdout:1/653: symlink d3/de/da4/ldf 0 2024-02-16T22:56:28.711 INFO:tasks.workunit.client.0.smithi064.stdout:5/507: rename d6/d25/d48/f55 to d6/fb1 0 2024-02-16T22:56:28.713 INFO:tasks.workunit.client.1.smithi192.stdout:3/643: rename da/d11/d63/d6b/d66/f86 to da/df/fe2 0 2024-02-16T22:56:28.714 INFO:tasks.workunit.client.1.smithi192.stdout:2/746: symlink d22/d24/d39/d3d/df2/lf3 0 2024-02-16T22:56:28.714 INFO:tasks.workunit.client.1.smithi192.stdout:7/793: dwrite d1/d2/de/d2c/d49/d54/d9d/f111 [0,4194304] 0 2024-02-16T22:56:28.715 INFO:tasks.workunit.client.0.smithi064.stdout:5/508: creat d6/d25/d48/d67/fb2 x:0 0 0 2024-02-16T22:56:28.716 INFO:tasks.workunit.client.0.smithi064.stdout:5/509: fdatasync d6/d1c/d43/d7d/f9d 0 2024-02-16T22:56:28.718 INFO:tasks.workunit.client.1.smithi192.stdout:5/686: getdents d6/d12 0 2024-02-16T22:56:28.723 INFO:tasks.workunit.client.1.smithi192.stdout:2/747: dwrite d22/d24/d39/d42/d85/d98/dc4/ff1 [0,4194304] 0 2024-02-16T22:56:28.726 INFO:tasks.workunit.client.1.smithi192.stdout:8/687: mknod d5/d26/cda 0 2024-02-16T22:56:28.726 INFO:tasks.workunit.client.1.smithi192.stdout:2/748: chown d22/d24/d39/d3d/df2/d99/d9b/fe5 1514 1 2024-02-16T22:56:28.726 INFO:tasks.workunit.client.1.smithi192.stdout:5/687: dwrite d6/d20/fc5 [0,4194304] 0 2024-02-16T22:56:28.728 INFO:tasks.workunit.client.1.smithi192.stdout:1/654: truncate d3/de/d27/d34/d52/f70 23158 0 2024-02-16T22:56:28.729 INFO:tasks.workunit.client.0.smithi064.stdout:5/510: rename d6/d18/f8a to d6/d18/d19/fb3 0 2024-02-16T22:56:28.731 INFO:tasks.workunit.client.1.smithi192.stdout:5/688: readlink d6/d12/d1d/lab 0 2024-02-16T22:56:28.731 INFO:tasks.workunit.client.1.smithi192.stdout:2/749: write fe [7916336,119591] 0 2024-02-16T22:56:28.731 INFO:tasks.workunit.client.1.smithi192.stdout:8/688: write d5/d26/d83/db4/f4b [4341765,55191] 0 2024-02-16T22:56:28.731 INFO:tasks.workunit.client.1.smithi192.stdout:5/689: write d6/d12/d1d/f3a [2598646,63608] 0 2024-02-16T22:56:28.739 INFO:tasks.workunit.client.0.smithi064.stdout:5/511: link d6/d1c/f93 d6/d1c/d43/d7d/fb4 0 2024-02-16T22:56:28.742 INFO:tasks.workunit.client.0.smithi064.stdout:5/512: creat d6/d1c/d4b/fb5 x:0 0 0 2024-02-16T22:56:28.744 INFO:tasks.workunit.client.1.smithi192.stdout:8/689: rmdir d5/d16 39 2024-02-16T22:56:28.745 INFO:tasks.workunit.client.1.smithi192.stdout:7/794: link d1/d2/fb4 d1/d2/de/d2c/d49/da4/d104/f114 0 2024-02-16T22:56:28.746 INFO:tasks.workunit.client.1.smithi192.stdout:7/795: write d1/d2/d2d/d61/f10e [643931,118519] 0 2024-02-16T22:56:28.747 INFO:tasks.workunit.client.1.smithi192.stdout:6/734: creat da/df/d12/d15/feb x:0 0 0 2024-02-16T22:56:28.747 INFO:tasks.workunit.client.1.smithi192.stdout:2/750: rmdir d22/d24/d39/d42/d85/d98/db0/db6 39 2024-02-16T22:56:28.748 INFO:tasks.workunit.client.0.smithi064.stdout:5/513: mknod d6/d18/d19/cb6 0 2024-02-16T22:56:28.749 INFO:tasks.workunit.client.0.smithi064.stdout:5/514: truncate d6/d18/d19/d1a/d85/f8b 147786 0 2024-02-16T22:56:28.750 INFO:tasks.workunit.client.0.smithi064.stdout:5/515: write d6/d18/d19/f2e [1888727,68821] 0 2024-02-16T22:56:28.751 INFO:tasks.workunit.client.1.smithi192.stdout:3/644: creat da/d11/d63/d6b/d66/fe3 x:0 0 0 2024-02-16T22:56:28.754 INFO:tasks.workunit.client.1.smithi192.stdout:8/690: write d5/db/de/df/fb3 [749016,28508] 0 2024-02-16T22:56:28.756 INFO:tasks.workunit.client.1.smithi192.stdout:7/796: symlink d1/d2/l115 0 2024-02-16T22:56:28.758 INFO:tasks.workunit.client.1.smithi192.stdout:6/735: rename da/c45 to da/df/d6b/d74/cec 0 2024-02-16T22:56:28.759 INFO:tasks.workunit.client.1.smithi192.stdout:6/736: chown da/df/d71/dde/fe8 10104 1 2024-02-16T22:56:28.760 INFO:tasks.workunit.client.1.smithi192.stdout:6/737: truncate da/df/d18/fc0 4707601 0 2024-02-16T22:56:28.761 INFO:tasks.workunit.client.0.smithi064.stdout:5/516: fsync d6/d1c/f93 0 2024-02-16T22:56:28.761 INFO:tasks.workunit.client.1.smithi192.stdout:3/645: read da/f3d [895777,2839] 0 2024-02-16T22:56:28.762 INFO:tasks.workunit.client.1.smithi192.stdout:1/655: creat d3/de/d27/d34/d52/fe0 x:0 0 0 2024-02-16T22:56:28.764 INFO:tasks.workunit.client.1.smithi192.stdout:8/691: creat d5/d26/d83/fdb x:0 0 0 2024-02-16T22:56:28.764 INFO:tasks.workunit.client.1.smithi192.stdout:1/656: read d3/de/d27/d34/d69/d85/d89/daa/fb3 [2030800,117032] 0 2024-02-16T22:56:28.765 INFO:tasks.workunit.client.0.smithi064.stdout:5/517: dread d6/d25/d48/d67/f7f [0,4194304] 0 2024-02-16T22:56:28.765 INFO:tasks.workunit.client.0.smithi064.stdout:5/518: fsync d6/d1c/f54 0 2024-02-16T22:56:28.767 INFO:tasks.workunit.client.0.smithi064.stdout:5/519: write f4 [4415501,69227] 0 2024-02-16T22:56:28.772 INFO:tasks.workunit.client.1.smithi192.stdout:5/690: write d6/d12/f1e [7398883,108] 0 2024-02-16T22:56:28.773 INFO:tasks.workunit.client.0.smithi064.stdout:5/520: dwrite d6/d18/d19/d1a/f60 [0,4194304] 0 2024-02-16T22:56:28.775 INFO:tasks.workunit.client.1.smithi192.stdout:2/751: dread d22/d47/dcb/f43 [0,4194304] 0 2024-02-16T22:56:28.776 INFO:tasks.workunit.client.1.smithi192.stdout:8/692: rmdir d5/db 39 2024-02-16T22:56:28.776 INFO:tasks.workunit.client.1.smithi192.stdout:2/752: readlink d22/d24/d39/d42/d52/l59 0 2024-02-16T22:56:28.782 INFO:tasks.workunit.client.1.smithi192.stdout:6/738: rmdir da 39 2024-02-16T22:56:28.784 INFO:tasks.workunit.client.1.smithi192.stdout:5/691: fdatasync d6/f9d 0 2024-02-16T22:56:28.785 INFO:tasks.workunit.client.0.smithi064.stdout:5/521: link d6/d1c/l6b d6/d18/d19/d1a/d8e/lb7 0 2024-02-16T22:56:28.785 INFO:tasks.workunit.client.0.smithi064.stdout:5/522: chown d6/f31 95 1 2024-02-16T22:56:28.786 INFO:tasks.workunit.client.1.smithi192.stdout:4/742: sync 2024-02-16T22:56:28.787 INFO:tasks.workunit.client.1.smithi192.stdout:8/693: chown d5/db/de/l8a 53389 1 2024-02-16T22:56:28.787 INFO:tasks.workunit.client.1.smithi192.stdout:8/694: chown d5/d68/cb1 1737 1 2024-02-16T22:56:28.790 INFO:tasks.workunit.client.1.smithi192.stdout:3/646: write da/f94 [607021,75423] 0 2024-02-16T22:56:28.791 INFO:tasks.workunit.client.0.smithi064.stdout:5/523: dwrite d6/d18/d19/d1a/d85/fa3 [0,4194304] 0 2024-02-16T22:56:28.791 INFO:tasks.workunit.client.0.smithi064.stdout:5/524: dread - d6/d18/d19/d1a/d85/f9f zero size 2024-02-16T22:56:28.792 INFO:tasks.workunit.client.1.smithi192.stdout:6/739: write da/df/d71/fd0 [719728,75724] 0 2024-02-16T22:56:28.795 INFO:tasks.workunit.client.1.smithi192.stdout:7/797: dwrite d1/d2/f25 [4194304,4194304] 0 2024-02-16T22:56:28.829 INFO:tasks.workunit.client.1.smithi192.stdout:9/818: sync 2024-02-16T22:56:28.847 INFO:tasks.workunit.client.0.smithi064.stdout:1/529: sync 2024-02-16T22:56:28.850 INFO:tasks.workunit.client.1.smithi192.stdout:4/743: rmdir d1/d15 39 2024-02-16T22:56:28.852 INFO:tasks.workunit.client.0.smithi064.stdout:1/530: symlink d2/df/d14/d2c/db5/lbb 0 2024-02-16T22:56:28.856 INFO:tasks.workunit.client.1.smithi192.stdout:3/647: creat da/d6a/fe4 x:0 0 0 2024-02-16T22:56:28.856 INFO:tasks.workunit.client.1.smithi192.stdout:3/648: write da/d11/d63/d6b/d71/f77 [852223,80574] 0 2024-02-16T22:56:28.856 INFO:tasks.workunit.client.1.smithi192.stdout:6/740: mknod da/df/d6b/d7d/ced 0 2024-02-16T22:56:28.856 INFO:tasks.workunit.client.0.smithi064.stdout:1/531: truncate d2/df/f1e 1919162 0 2024-02-16T22:56:28.856 INFO:tasks.workunit.client.0.smithi064.stdout:5/525: read d6/fa [1580056,117868] 0 2024-02-16T22:56:28.857 INFO:tasks.workunit.client.1.smithi192.stdout:7/798: readlink d1/d2/de/l3f 0 2024-02-16T22:56:28.859 INFO:tasks.workunit.client.1.smithi192.stdout:5/692: mkdir d6/d12/d4e/de6 0 2024-02-16T22:56:28.860 INFO:tasks.workunit.client.0.smithi064.stdout:5/526: creat d6/d25/d48/d67/d80/fb8 x:0 0 0 2024-02-16T22:56:28.861 INFO:tasks.workunit.client.1.smithi192.stdout:4/744: creat d1/d6b/f102 x:0 0 0 2024-02-16T22:56:28.862 INFO:tasks.workunit.client.1.smithi192.stdout:2/753: link d22/d24/d39/d42/d85/d98/fc8 d22/d24/d39/d42/ff4 0 2024-02-16T22:56:28.864 INFO:tasks.workunit.client.0.smithi064.stdout:5/527: rename d6/d18/d39 to d6/d1c/d43/db9 0 2024-02-16T22:56:28.865 INFO:tasks.workunit.client.0.smithi064.stdout:5/528: write d6/d1c/d4b/fb5 [374118,115047] 0 2024-02-16T22:56:28.867 INFO:tasks.workunit.client.1.smithi192.stdout:3/649: creat da/df/fe5 x:0 0 0 2024-02-16T22:56:28.868 INFO:tasks.workunit.client.0.smithi064.stdout:1/532: link d2/df/d14/d18/l4b d2/df/d14/lbc 0 2024-02-16T22:56:28.868 INFO:tasks.workunit.client.1.smithi192.stdout:3/650: chown da/d11/d35/c4f 60816 1 2024-02-16T22:56:28.870 INFO:tasks.workunit.client.1.smithi192.stdout:5/693: symlink d6/d20/d93/le7 0 2024-02-16T22:56:28.871 INFO:tasks.workunit.client.1.smithi192.stdout:8/695: link d5/d16/f4f d5/d16/dcf/fdc 0 2024-02-16T22:56:28.873 INFO:tasks.workunit.client.1.smithi192.stdout:1/657: getdents d3/de/d27/d34/d52/d72 0 2024-02-16T22:56:28.873 INFO:tasks.workunit.client.1.smithi192.stdout:5/694: truncate d6/d12/d4e/d58/fb7 1298283 0 2024-02-16T22:56:28.874 INFO:tasks.workunit.client.1.smithi192.stdout:7/799: creat d1/d2/d2d/d61/d8a/f116 x:0 0 0 2024-02-16T22:56:28.875 INFO:tasks.workunit.client.1.smithi192.stdout:4/745: dread d1/d26/d66/d89/da8/fb8 [0,4194304] 0 2024-02-16T22:56:28.875 INFO:tasks.workunit.client.1.smithi192.stdout:3/651: dwrite da/fcc [0,4194304] 0 2024-02-16T22:56:28.876 INFO:tasks.workunit.client.0.smithi064.stdout:1/533: rename d2/df/d14/d2c/d62 to d2/df/d11/d1a/d30/dbd 0 2024-02-16T22:56:28.880 INFO:tasks.workunit.client.0.smithi064.stdout:1/534: symlink d2/df/d14/d18/d48/d6b/lbe 0 2024-02-16T22:56:28.886 INFO:tasks.workunit.client.0.smithi064.stdout:1/535: rename d2/df/d14/d2c/f5e to d2/df/d14/d2a/fbf 0 2024-02-16T22:56:28.892 INFO:tasks.workunit.client.1.smithi192.stdout:5/695: creat d6/d8/d75/d8c/fe8 x:0 0 0 2024-02-16T22:56:28.895 INFO:tasks.workunit.client.1.smithi192.stdout:3/652: truncate da/d11/d20/d39/fad 36055 0 2024-02-16T22:56:28.896 INFO:tasks.workunit.client.1.smithi192.stdout:9/819: getdents d7/d15/d36/d77/db1 0 2024-02-16T22:56:28.896 INFO:tasks.workunit.client.0.smithi064.stdout:1/536: link d2/df/d14/d18/d48/d6b/lbe d2/df/d11/d1a/d30/dbd/lc0 0 2024-02-16T22:56:28.898 INFO:tasks.workunit.client.1.smithi192.stdout:9/820: write d7/d19/d5e/d81/fcc [4591154,10356] 0 2024-02-16T22:56:28.898 INFO:tasks.workunit.client.1.smithi192.stdout:2/754: creat d22/d24/d39/ff5 x:0 0 0 2024-02-16T22:56:28.900 INFO:tasks.workunit.client.1.smithi192.stdout:8/696: unlink d5/db/de/df/f62 0 2024-02-16T22:56:28.901 INFO:tasks.workunit.client.1.smithi192.stdout:7/800: dread d1/d2/f1e [0,4194304] 0 2024-02-16T22:56:28.901 INFO:tasks.workunit.client.1.smithi192.stdout:2/755: truncate d22/d24/d39/d42/d85/d98/fc8 173433 0 2024-02-16T22:56:28.902 INFO:tasks.workunit.client.0.smithi064.stdout:1/537: read d2/df/d14/d18/d40/f68 [3591544,32395] 0 2024-02-16T22:56:28.903 INFO:tasks.workunit.client.0.smithi064.stdout:1/538: readlink d2/df/d14/d18/d48/la1 0 2024-02-16T22:56:28.904 INFO:tasks.workunit.client.1.smithi192.stdout:5/696: unlink d6/d12/d4e/d46/d4f/c71 0 2024-02-16T22:56:28.904 INFO:tasks.workunit.client.1.smithi192.stdout:5/697: stat d6/d8/fa1 0 2024-02-16T22:56:28.906 INFO:tasks.workunit.client.1.smithi192.stdout:8/697: dwrite d5/d26/d83/db4/fac [0,4194304] 0 2024-02-16T22:56:28.908 INFO:tasks.workunit.client.1.smithi192.stdout:1/658: write d3/de/f1b [1891002,90835] 0 2024-02-16T22:56:28.912 INFO:tasks.workunit.client.0.smithi064.stdout:1/539: truncate d2/df/f54 2092970 0 2024-02-16T22:56:28.912 INFO:tasks.workunit.client.1.smithi192.stdout:9/821: rename d7/d19/d37/f55 to d7/d15/d36/d38/d3f/d8a/def/d100/d10e/f119 0 2024-02-16T22:56:28.916 INFO:tasks.workunit.client.1.smithi192.stdout:5/698: mknod d6/d12/ce9 0 2024-02-16T22:56:28.917 INFO:tasks.workunit.client.1.smithi192.stdout:4/746: dwrite d1/d26/fd7 [0,4194304] 0 2024-02-16T22:56:28.917 INFO:tasks.workunit.client.1.smithi192.stdout:8/698: dwrite d5/d9/f73 [0,4194304] 0 2024-02-16T22:56:28.918 INFO:tasks.workunit.client.1.smithi192.stdout:8/699: stat d5/db/d47/f92 0 2024-02-16T22:56:28.919 INFO:tasks.workunit.client.1.smithi192.stdout:3/653: unlink da/d11/d2b/l7c 0 2024-02-16T22:56:28.921 INFO:tasks.workunit.client.0.smithi064.stdout:1/540: creat d2/df/d11/fc1 x:0 0 0 2024-02-16T22:56:28.921 INFO:tasks.workunit.client.1.smithi192.stdout:3/654: fsync da/d11/d63/d6b/faa 0 2024-02-16T22:56:28.922 INFO:tasks.workunit.client.0.smithi064.stdout:1/541: write d2/df/d14/d2c/f49 [3862019,128899] 0 2024-02-16T22:56:28.923 INFO:tasks.workunit.client.1.smithi192.stdout:1/659: dread d3/de/d27/d34/d52/d5a/f90 [0,4194304] 0 2024-02-16T22:56:28.929 INFO:tasks.workunit.client.1.smithi192.stdout:7/801: rename d1/d2/d2d/d92 to d1/d2/de/d2c/d49/d54/dd8/d117 0 2024-02-16T22:56:28.931 INFO:tasks.workunit.client.0.smithi064.stdout:1/542: write d2/df/d14/d18/f26 [1002417,77274] 0 2024-02-16T22:56:28.931 INFO:tasks.workunit.client.0.smithi064.stdout:1/543: write d2/df/d11/fc1 [506413,80037] 0 2024-02-16T22:56:28.932 INFO:tasks.workunit.client.1.smithi192.stdout:2/756: creat d22/d47/dcb/d78/ff6 x:0 0 0 2024-02-16T22:56:28.932 INFO:tasks.workunit.client.0.smithi064.stdout:1/544: fsync d2/df/d11/f8f 0 2024-02-16T22:56:28.939 INFO:tasks.workunit.client.1.smithi192.stdout:0/684: sync 2024-02-16T22:56:28.944 INFO:tasks.workunit.client.0.smithi064.stdout:1/545: truncate d2/df/d11/d1a/d30/f63 1817776 0 2024-02-16T22:56:28.946 INFO:tasks.workunit.client.0.smithi064.stdout:1/546: write d2/df/d14/f16 [1355223,98702] 0 2024-02-16T22:56:28.947 INFO:tasks.workunit.client.0.smithi064.stdout:1/547: write d2/df/d14/d18/d48/fb8 [235036,72017] 0 2024-02-16T22:56:28.949 INFO:tasks.workunit.client.1.smithi192.stdout:5/699: symlink d6/d12/d4e/d5b/d67/d78/dae/dbc/lea 0 2024-02-16T22:56:28.949 INFO:tasks.workunit.client.1.smithi192.stdout:8/700: creat d5/d26/d83/d86/fdd x:0 0 0 2024-02-16T22:56:28.950 INFO:tasks.workunit.client.1.smithi192.stdout:3/655: symlink da/d11/d20/d39/le6 0 2024-02-16T22:56:28.950 INFO:tasks.workunit.client.1.smithi192.stdout:1/660: rmdir d3/de/d27/d34/d52/d72 39 2024-02-16T22:56:28.951 INFO:tasks.workunit.client.1.smithi192.stdout:9/822: write d7/fe [4191628,44303] 0 2024-02-16T22:56:28.951 INFO:tasks.workunit.client.1.smithi192.stdout:7/802: rmdir d1/d2/d2d/d61/d88/d90/d4c/d60 39 2024-02-16T22:56:28.953 INFO:tasks.workunit.client.1.smithi192.stdout:1/661: write d3/de/d27/f42 [1746052,15114] 0 2024-02-16T22:56:28.954 INFO:tasks.workunit.client.1.smithi192.stdout:2/757: rename d22/d47/dcb/d78/c94 to d22/d47/dcb/d3f/dac/cf7 0 2024-02-16T22:56:28.955 INFO:tasks.workunit.client.1.smithi192.stdout:5/700: creat d6/d12/d4e/d5b/d67/d78/feb x:0 0 0 2024-02-16T22:56:28.956 INFO:tasks.workunit.client.1.smithi192.stdout:1/662: truncate d3/d40/fa1 4624919 0 2024-02-16T22:56:28.957 INFO:tasks.workunit.client.1.smithi192.stdout:0/685: creat d1/d3/de/d22/d37/d47/fda x:0 0 0 2024-02-16T22:56:28.961 INFO:tasks.workunit.client.1.smithi192.stdout:7/803: dwrite d1/d2/de/d2c/d49/f71 [0,4194304] 0 2024-02-16T22:56:28.961 INFO:tasks.workunit.client.1.smithi192.stdout:9/823: fdatasync d7/d15/d36/d38/d3f/fc3 0 2024-02-16T22:56:28.962 INFO:tasks.workunit.client.1.smithi192.stdout:7/804: readlink d1/d2/de/d2c/d49/d54/dd8/d117/la0 0 2024-02-16T22:56:28.963 INFO:tasks.workunit.client.1.smithi192.stdout:3/656: rename da/d11/d63/d6b/d80/f83 to da/d6a/fe7 0 2024-02-16T22:56:28.964 INFO:tasks.workunit.client.0.smithi064.stdout:1/548: dwrite d2/df/d11/d1a/f34 [0,4194304] 0 2024-02-16T22:56:28.965 INFO:tasks.workunit.client.1.smithi192.stdout:5/701: readlink d6/d8/la4 0 2024-02-16T22:56:28.968 INFO:tasks.workunit.client.1.smithi192.stdout:5/702: write d6/d8/d77/f85 [383787,65213] 0 2024-02-16T22:56:28.975 INFO:tasks.workunit.client.1.smithi192.stdout:9/824: write d7/d15/f20 [3531425,95689] 0 2024-02-16T22:56:28.976 INFO:tasks.workunit.client.0.smithi064.stdout:7/607: sync 2024-02-16T22:56:28.977 INFO:tasks.workunit.client.1.smithi192.stdout:3/657: chown da/d11/d63/d6b/d71/da1/db9/fdd 221557 1 2024-02-16T22:56:28.977 INFO:tasks.workunit.client.1.smithi192.stdout:9/825: fsync f2 0 2024-02-16T22:56:28.977 INFO:tasks.workunit.client.1.smithi192.stdout:8/701: dread d5/f61 [0,4194304] 0 2024-02-16T22:56:28.977 INFO:tasks.workunit.client.1.smithi192.stdout:5/703: dwrite d6/d20/fc5 [0,4194304] 0 2024-02-16T22:56:28.978 INFO:tasks.workunit.client.1.smithi192.stdout:1/663: creat d3/de/d27/d34/d52/d63/d7a/dc9/fe1 x:0 0 0 2024-02-16T22:56:28.978 INFO:tasks.workunit.client.0.smithi064.stdout:7/608: write d5/d33/d45/d64/d56/d5d/fb6 [638210,97276] 0 2024-02-16T22:56:28.979 INFO:tasks.workunit.client.0.smithi064.stdout:7/609: chown d5/d33/d53/f99 213 1 2024-02-16T22:56:28.986 INFO:tasks.workunit.client.1.smithi192.stdout:1/664: symlink d3/de/d27/d34/d36/le2 0 2024-02-16T22:56:28.987 INFO:tasks.workunit.client.0.smithi064.stdout:7/610: dread d5/d33/d42/d81/f94 [0,4194304] 0 2024-02-16T22:56:28.988 INFO:tasks.workunit.client.1.smithi192.stdout:7/805: rmdir d1/d2/d2d/d61/d10d 0 2024-02-16T22:56:28.989 INFO:tasks.workunit.client.1.smithi192.stdout:1/665: stat d3/de/ff 0 2024-02-16T22:56:28.990 INFO:tasks.workunit.client.0.smithi064.stdout:7/611: stat d5/d33/d45/d73/dab 0 2024-02-16T22:56:28.990 INFO:tasks.workunit.client.1.smithi192.stdout:2/758: dread d22/d24/f96 [0,4194304] 0 2024-02-16T22:56:28.991 INFO:tasks.workunit.client.1.smithi192.stdout:5/704: dwrite d6/d12/d4e/d58/d8f/fe1 [0,4194304] 0 2024-02-16T22:56:28.991 INFO:tasks.workunit.client.0.smithi064.stdout:7/612: write d5/d33/d45/d64/d56/d5d/f80 [2168894,107961] 0 2024-02-16T22:56:28.991 INFO:tasks.workunit.client.1.smithi192.stdout:1/666: stat d3/de/f82 0 2024-02-16T22:56:28.992 INFO:tasks.workunit.client.0.smithi064.stdout:7/613: chown d5/d6/d54/d41/f7e 0 1 2024-02-16T22:56:28.996 INFO:tasks.workunit.client.0.smithi064.stdout:7/614: creat d5/d33/d45/d73/fc0 x:0 0 0 2024-02-16T22:56:28.997 INFO:tasks.workunit.client.0.smithi064.stdout:7/615: dread - d5/d33/f60 zero size 2024-02-16T22:56:29.000 INFO:tasks.workunit.client.1.smithi192.stdout:7/806: dwrite d1/d2/de/d2c/f96 [0,4194304] 0 2024-02-16T22:56:29.003 INFO:tasks.workunit.client.0.smithi064.stdout:1/549: dwrite d2/df/d11/d1a/d1d/f58 [0,4194304] 0 2024-02-16T22:56:29.003 INFO:tasks.workunit.client.0.smithi064.stdout:7/616: creat d5/d6/d54/d7b/fc1 x:0 0 0 2024-02-16T22:56:29.008 INFO:tasks.workunit.client.1.smithi192.stdout:3/658: dwrite da/f3d [0,4194304] 0 2024-02-16T22:56:29.010 INFO:tasks.workunit.client.1.smithi192.stdout:9/826: dwrite d7/d15/d36/d38/d3f/fc3 [0,4194304] 0 2024-02-16T22:56:29.012 INFO:tasks.workunit.client.1.smithi192.stdout:9/827: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/lfa 735631 1 2024-02-16T22:56:29.012 INFO:tasks.workunit.client.1.smithi192.stdout:8/702: write d5/d16/d81/fca [277669,105459] 0 2024-02-16T22:56:29.017 INFO:tasks.workunit.client.1.smithi192.stdout:2/759: mknod d22/d24/d39/d42/d52/cf8 0 2024-02-16T22:56:29.018 INFO:tasks.workunit.client.1.smithi192.stdout:2/760: write d22/d24/d39/d42/d65/d69/fa1 [362989,32709] 0 2024-02-16T22:56:29.020 INFO:tasks.workunit.client.1.smithi192.stdout:2/761: readlink d22/d24/d39/d42/d65/d69/l79 0 2024-02-16T22:56:29.020 INFO:tasks.workunit.client.1.smithi192.stdout:5/705: link d6/d20/fc5 d6/d12/d4e/d46/d4f/fec 0 2024-02-16T22:56:29.026 INFO:tasks.workunit.client.1.smithi192.stdout:7/807: mknod d1/d2/de/d2c/d49/da4/d104/c118 0 2024-02-16T22:56:29.029 INFO:tasks.workunit.client.1.smithi192.stdout:8/703: creat d5/d16/dd5/d9d/fde x:0 0 0 2024-02-16T22:56:29.032 INFO:tasks.workunit.client.1.smithi192.stdout:3/659: truncate da/d11/d20/f69 1157421 0 2024-02-16T22:56:29.033 INFO:tasks.workunit.client.1.smithi192.stdout:3/660: chown da/d11/d35/ld6 44586719 1 2024-02-16T22:56:29.034 INFO:tasks.workunit.client.1.smithi192.stdout:1/667: link d3/de/d27/d34/d52/d72/l98 d3/de/d27/d34/d52/d63/d7a/le3 0 2024-02-16T22:56:29.036 INFO:tasks.workunit.client.1.smithi192.stdout:7/808: write d1/d2/d2d/d61/d88/d90/d4c/f6b [2850600,59875] 0 2024-02-16T22:56:29.037 INFO:tasks.workunit.client.1.smithi192.stdout:8/704: symlink d5/d26/d83/d86/d5f/db2/ldf 0 2024-02-16T22:56:29.044 INFO:tasks.workunit.client.1.smithi192.stdout:8/705: dwrite d5/d26/fa0 [0,4194304] 0 2024-02-16T22:56:29.044 INFO:tasks.workunit.client.1.smithi192.stdout:5/706: creat d6/d12/d4e/d5b/fed x:0 0 0 2024-02-16T22:56:29.045 INFO:tasks.workunit.client.1.smithi192.stdout:5/707: stat d6/d12/d2d/d5c/f99 0 2024-02-16T22:56:29.046 INFO:tasks.workunit.client.1.smithi192.stdout:1/668: symlink d3/de/d27/d34/le4 0 2024-02-16T22:56:29.047 INFO:tasks.workunit.client.1.smithi192.stdout:1/669: stat d3/f50 0 2024-02-16T22:56:29.048 INFO:tasks.workunit.client.1.smithi192.stdout:7/809: dread d1/d2/de/f24 [0,4194304] 0 2024-02-16T22:56:29.049 INFO:tasks.workunit.client.1.smithi192.stdout:1/670: fdatasync d3/de/d27/d34/d52/d63/d81/fd8 0 2024-02-16T22:56:29.054 INFO:tasks.workunit.client.1.smithi192.stdout:1/671: rmdir d3/de/d27/d34/d52/d72 39 2024-02-16T22:56:29.054 INFO:tasks.workunit.client.1.smithi192.stdout:1/672: chown d3/cac 962367298 1 2024-02-16T22:56:29.058 INFO:tasks.workunit.client.1.smithi192.stdout:7/810: truncate d1/d2/fb4 3117960 0 2024-02-16T22:56:29.059 INFO:tasks.workunit.client.1.smithi192.stdout:5/708: getdents d6/d12/d4e/d5b/d67/d78 0 2024-02-16T22:56:29.060 INFO:tasks.workunit.client.1.smithi192.stdout:1/673: creat d3/de/d27/d34/d36/d47/d5d/fe5 x:0 0 0 2024-02-16T22:56:29.060 INFO:tasks.workunit.client.1.smithi192.stdout:2/762: dwrite d22/fa9 [0,4194304] 0 2024-02-16T22:56:29.060 INFO:tasks.workunit.client.1.smithi192.stdout:5/709: fdatasync d6/d12/d1d/f3a 0 2024-02-16T22:56:29.064 INFO:tasks.workunit.client.0.smithi064.stdout:1/550: unlink d2/df/fa2 0 2024-02-16T22:56:29.065 INFO:tasks.workunit.client.0.smithi064.stdout:7/617: creat d5/d6/d65/da7/dad/fc2 x:0 0 0 2024-02-16T22:56:29.066 INFO:tasks.workunit.client.1.smithi192.stdout:1/674: dread d3/d5b/f8a [0,4194304] 0 2024-02-16T22:56:29.067 INFO:tasks.workunit.client.1.smithi192.stdout:3/661: dread da/f22 [0,4194304] 0 2024-02-16T22:56:29.067 INFO:tasks.workunit.client.1.smithi192.stdout:2/763: rmdir d22/d24/d39/d42/d85/d98 39 2024-02-16T22:56:29.068 INFO:tasks.workunit.client.0.smithi064.stdout:1/551: mkdir d2/df/d14/d2a/dc2 0 2024-02-16T22:56:29.069 INFO:tasks.workunit.client.1.smithi192.stdout:5/710: dread d6/d8/f9e [0,4194304] 0 2024-02-16T22:56:29.070 INFO:tasks.workunit.client.1.smithi192.stdout:5/711: chown d6/d8/d75 1961119 1 2024-02-16T22:56:29.071 INFO:tasks.workunit.client.1.smithi192.stdout:7/811: rename d1/d2/de/d2c/d49/d54/f105 to d1/d2/de/d2c/d49/d54/de3/f119 0 2024-02-16T22:56:29.091 INFO:tasks.workunit.client.1.smithi192.stdout:6/741: sync 2024-02-16T22:56:29.095 INFO:tasks.workunit.client.1.smithi192.stdout:6/742: dwrite da/d76/dd5/de2/fe7 [0,4194304] 0 2024-02-16T22:56:29.096 INFO:tasks.workunit.client.1.smithi192.stdout:6/743: write da/df/d12/d15/f31 [4300685,89400] 0 2024-02-16T22:56:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:28 smithi064 ceph-mon[29226]: pgmap v19: 65 pgs: 65 active+clean; 2.6 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 106 MiB/s rd, 315 MiB/s wr, 589 op/s 2024-02-16T22:56:29.155 INFO:tasks.workunit.client.1.smithi192.stdout:4/747: sync 2024-02-16T22:56:29.166 INFO:tasks.workunit.client.0.smithi064.stdout:2/661: sync 2024-02-16T22:56:29.167 INFO:tasks.workunit.client.0.smithi064.stdout:1/552: symlink d2/df/d11/d1a/d30/d3d/lc3 0 2024-02-16T22:56:29.168 INFO:tasks.workunit.client.1.smithi192.stdout:1/675: read - d3/de/d27/d34/d69/f9d zero size 2024-02-16T22:56:29.169 INFO:tasks.workunit.client.0.smithi064.stdout:7/618: unlink d5/d6/c2c 0 2024-02-16T22:56:29.172 INFO:tasks.workunit.client.0.smithi064.stdout:7/619: write d5/d33/d45/d64/f9e [535303,102024] 0 2024-02-16T22:56:29.174 INFO:tasks.workunit.client.0.smithi064.stdout:1/553: dwrite d2/df/d14/d18/d48/d84/fa4 [0,4194304] 0 2024-02-16T22:56:29.175 INFO:tasks.workunit.client.0.smithi064.stdout:1/554: fsync d2/df/d11/f8f 0 2024-02-16T22:56:29.213 INFO:tasks.workunit.client.1.smithi192.stdout:0/686: sync 2024-02-16T22:56:29.213 INFO:tasks.workunit.client.1.smithi192.stdout:8/706: sync 2024-02-16T22:56:29.213 INFO:tasks.workunit.client.1.smithi192.stdout:9/828: sync 2024-02-16T22:56:29.213 INFO:tasks.workunit.client.1.smithi192.stdout:9/829: stat d7/d19/d37/dbf/le4 0 2024-02-16T22:56:29.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/687: read - d1/d3/de/d22/d37/d47/fda zero size 2024-02-16T22:56:29.214 INFO:tasks.workunit.client.1.smithi192.stdout:9/830: readlink d7/d19/d28/d43/l50 0 2024-02-16T22:56:29.234 INFO:tasks.workunit.client.0.smithi064.stdout:2/662: rename d7/de to d7/d14/d1e/d34/d3f/d9a/dd9 0 2024-02-16T22:56:29.235 INFO:tasks.workunit.client.0.smithi064.stdout:2/663: chown d7/d14/d1e/d34/d65/c9e 2185 1 2024-02-16T22:56:29.242 INFO:tasks.workunit.client.1.smithi192.stdout:5/712: rename d6/d12/d4e/d5b/d67/d78/fb4 to d6/d12/fee 0 2024-02-16T22:56:29.243 INFO:tasks.workunit.client.0.smithi064.stdout:2/664: creat d7/d14/d25/d46/d55/fda x:0 0 0 2024-02-16T22:56:29.244 INFO:tasks.workunit.client.0.smithi064.stdout:2/665: read - d7/d14/d1e/d34/d3f/d9a/dd9/d95/fac zero size 2024-02-16T22:56:29.244 INFO:tasks.workunit.client.1.smithi192.stdout:7/812: write d1/d2/de/d2c/d49/d54/de3/f119 [575152,4573] 0 2024-02-16T22:56:29.245 INFO:tasks.workunit.client.0.smithi064.stdout:2/666: fdatasync d7/d14/d3e/fd2 0 2024-02-16T22:56:29.246 INFO:tasks.workunit.client.0.smithi064.stdout:2/667: chown d7/d14/l26 120926451 1 2024-02-16T22:56:29.250 INFO:tasks.workunit.client.1.smithi192.stdout:6/744: symlink da/df/d6b/d7d/lee 0 2024-02-16T22:56:29.250 INFO:tasks.workunit.client.1.smithi192.stdout:7/813: dwrite d1/d2/de/d2c/d49/dba/ff4 [0,4194304] 0 2024-02-16T22:56:29.251 INFO:tasks.workunit.client.1.smithi192.stdout:7/814: chown d1/d2/de/l16 543 1 2024-02-16T22:56:29.254 INFO:tasks.workunit.client.1.smithi192.stdout:7/815: truncate d1/d2/de/d83/d9f/fa2 1016525 0 2024-02-16T22:56:29.254 INFO:tasks.workunit.client.0.smithi064.stdout:2/668: rename d7/d14/d1e/d34/l82 to d7/d14/d25/d46/d55/d57/ldb 0 2024-02-16T22:56:29.255 INFO:tasks.workunit.client.1.smithi192.stdout:1/676: mkdir d3/de/d27/d34/d69/d85/d89/daa/de6 0 2024-02-16T22:56:29.256 INFO:tasks.workunit.client.1.smithi192.stdout:1/677: readlink d3/de/d27/l8f 0 2024-02-16T22:56:29.257 INFO:tasks.workunit.client.0.smithi064.stdout:2/669: write d7/d14/d1e/d34/d3f/d9a/dd9/f15 [2162663,16336] 0 2024-02-16T22:56:29.258 INFO:tasks.workunit.client.1.smithi192.stdout:1/678: read d3/de/f65 [2194248,43165] 0 2024-02-16T22:56:29.259 INFO:tasks.workunit.client.0.smithi064.stdout:2/670: fdatasync d7/d14/d1e/d34/d3f/d9a/dd9/d95/f47 0 2024-02-16T22:56:29.259 INFO:tasks.workunit.client.1.smithi192.stdout:0/688: mknod d1/d3/dac/cdb 0 2024-02-16T22:56:29.263 INFO:tasks.workunit.client.1.smithi192.stdout:9/831: truncate d7/d15/f6d 1615332 0 2024-02-16T22:56:29.263 INFO:tasks.workunit.client.0.smithi064.stdout:9/545: sync 2024-02-16T22:56:29.264 INFO:tasks.workunit.client.0.smithi064.stdout:8/568: sync 2024-02-16T22:56:29.264 INFO:tasks.workunit.client.1.smithi192.stdout:2/764: symlink d22/d47/dcb/dda/dec/lf9 0 2024-02-16T22:56:29.264 INFO:tasks.workunit.client.1.smithi192.stdout:8/707: dwrite d5/d26/d83/db4/fba [0,4194304] 0 2024-02-16T22:56:29.264 INFO:tasks.workunit.client.0.smithi064.stdout:8/569: chown df/d2a/fa3 0 1 2024-02-16T22:56:29.265 INFO:tasks.workunit.client.0.smithi064.stdout:2/671: rename d7/d14/d3e/cb2 to d7/d14/d25/daf/cdc 0 2024-02-16T22:56:29.266 INFO:tasks.workunit.client.1.smithi192.stdout:8/708: dread d5/d16/fd2 [0,4194304] 0 2024-02-16T22:56:29.267 INFO:tasks.workunit.client.1.smithi192.stdout:5/713: creat d6/d12/d4e/d5b/d67/d78/fef x:0 0 0 2024-02-16T22:56:29.272 INFO:tasks.workunit.client.0.smithi064.stdout:8/570: mknod df/d31/d2f/d3e/d5f/d93/cbf 0 2024-02-16T22:56:29.275 INFO:tasks.workunit.client.0.smithi064.stdout:8/571: dread df/d31/d2f/d3e/f9c [0,4194304] 0 2024-02-16T22:56:29.280 INFO:tasks.workunit.client.0.smithi064.stdout:8/572: rename df/d31/d2f/d3e/d5f/d93/l98 to df/d31/d2f/d3e/lc0 0 2024-02-16T22:56:29.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:28 smithi192 ceph-mon[36254]: pgmap v19: 65 pgs: 65 active+clean; 2.6 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 106 MiB/s rd, 315 MiB/s wr, 589 op/s 2024-02-16T22:56:29.285 INFO:tasks.workunit.client.1.smithi192.stdout:2/765: mkdir d22/d47/dca/dfa 0 2024-02-16T22:56:29.286 INFO:tasks.workunit.client.0.smithi064.stdout:8/573: mkdir df/d31/dc1 0 2024-02-16T22:56:29.286 INFO:tasks.workunit.client.0.smithi064.stdout:1/555: write d2/df/d14/f2d [152958,55749] 0 2024-02-16T22:56:29.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/709: symlink d5/d68/le0 0 2024-02-16T22:56:29.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/710: readlink d5/db/lb8 0 2024-02-16T22:56:29.288 INFO:tasks.workunit.client.0.smithi064.stdout:7/620: dwrite d5/d33/d45/f46 [0,4194304] 0 2024-02-16T22:56:29.294 INFO:tasks.workunit.client.0.smithi064.stdout:1/556: unlink d2/df/d14/d18/d40/c66 0 2024-02-16T22:56:29.295 INFO:tasks.workunit.client.1.smithi192.stdout:4/748: dwrite d1/d26/fa5 [0,4194304] 0 2024-02-16T22:56:29.298 INFO:tasks.workunit.client.0.smithi064.stdout:7/621: unlink d5/d6/d65/fbd 0 2024-02-16T22:56:29.300 INFO:tasks.workunit.client.1.smithi192.stdout:7/816: unlink d1/d2/de/d2c/d49/d54/c84 0 2024-02-16T22:56:29.302 INFO:tasks.workunit.client.0.smithi064.stdout:1/557: fsync d2/df/d14/d2c/f3c 0 2024-02-16T22:56:29.305 INFO:tasks.workunit.client.0.smithi064.stdout:1/558: dread d2/df/d14/d2c/f3c [0,4194304] 0 2024-02-16T22:56:29.306 INFO:tasks.workunit.client.1.smithi192.stdout:3/662: getdents da/d11/d2b 0 2024-02-16T22:56:29.307 INFO:tasks.workunit.client.0.smithi064.stdout:1/559: write d2/df/d14/d18/d48/fb8 [1310636,59542] 0 2024-02-16T22:56:29.309 INFO:tasks.workunit.client.1.smithi192.stdout:1/679: mkdir d3/de/d27/d34/de7 0 2024-02-16T22:56:29.311 INFO:tasks.workunit.client.0.smithi064.stdout:9/546: write d3/d9/d6a/db1/d2a/d2e/f39 [5263732,114330] 0 2024-02-16T22:56:29.312 INFO:tasks.workunit.client.0.smithi064.stdout:4/597: sync 2024-02-16T22:56:29.312 INFO:tasks.workunit.client.0.smithi064.stdout:3/702: sync 2024-02-16T22:56:29.313 INFO:tasks.workunit.client.0.smithi064.stdout:8/574: dread df/d31/d2f/d38/f5d [0,4194304] 0 2024-02-16T22:56:29.314 INFO:tasks.workunit.client.0.smithi064.stdout:2/672: write d7/d14/d25/d46/d55/f5a [1775241,88647] 0 2024-02-16T22:56:29.316 INFO:tasks.workunit.client.0.smithi064.stdout:9/547: write d3/f18 [6072068,117520] 0 2024-02-16T22:56:29.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/689: write d1/d3/d39/f76 [2579685,18407] 0 2024-02-16T22:56:29.317 INFO:tasks.workunit.client.0.smithi064.stdout:4/598: chown dc/dd/f59 65196 1 2024-02-16T22:56:29.318 INFO:tasks.workunit.client.0.smithi064.stdout:8/575: dread - df/d31/d61/d94/fa6 zero size 2024-02-16T22:56:29.319 INFO:tasks.workunit.client.0.smithi064.stdout:1/560: dwrite d2/df/d11/d25/fb9 [0,4194304] 0 2024-02-16T22:56:29.325 INFO:tasks.workunit.client.0.smithi064.stdout:3/703: chown d4/d13/d1b/d48/c7a 4 1 2024-02-16T22:56:29.326 INFO:tasks.workunit.client.0.smithi064.stdout:7/622: dread d5/d33/d45/d64/f85 [0,4194304] 0 2024-02-16T22:56:29.326 INFO:tasks.workunit.client.0.smithi064.stdout:2/673: mknod d7/d14/d25/cdd 0 2024-02-16T22:56:29.327 INFO:tasks.workunit.client.0.smithi064.stdout:7/623: stat c2 0 2024-02-16T22:56:29.329 INFO:tasks.workunit.client.0.smithi064.stdout:3/704: chown d4/d13/d1b/d27/d44/fc7 78888896 1 2024-02-16T22:56:29.330 INFO:tasks.workunit.client.0.smithi064.stdout:4/599: dwrite dc/de/f99 [0,4194304] 0 2024-02-16T22:56:29.332 INFO:tasks.workunit.client.0.smithi064.stdout:4/600: fsync dc/dd/d16/d9c/fad 0 2024-02-16T22:56:29.334 INFO:tasks.workunit.client.0.smithi064.stdout:1/561: dread d2/f5c [0,4194304] 0 2024-02-16T22:56:29.335 INFO:tasks.workunit.client.1.smithi192.stdout:3/663: mknod da/d11/dd4/ce8 0 2024-02-16T22:56:29.338 INFO:tasks.workunit.client.0.smithi064.stdout:7/624: dwrite d5/d33/d53/f9f [0,4194304] 0 2024-02-16T22:56:29.339 INFO:tasks.workunit.client.0.smithi064.stdout:7/625: stat d5/d6/c48 0 2024-02-16T22:56:29.339 INFO:tasks.workunit.client.0.smithi064.stdout:9/548: symlink d3/d9/d91/lba 0 2024-02-16T22:56:29.341 INFO:tasks.workunit.client.0.smithi064.stdout:7/626: write d5/f8b [986915,31007] 0 2024-02-16T22:56:29.341 INFO:tasks.workunit.client.1.smithi192.stdout:9/832: creat d7/d15/d36/d38/f11a x:0 0 0 2024-02-16T22:56:29.342 INFO:tasks.workunit.client.1.smithi192.stdout:9/833: chown d7/d19/d5b/c66 0 1 2024-02-16T22:56:29.343 INFO:tasks.workunit.client.1.smithi192.stdout:1/680: dwrite d3/d5b/f8a [0,4194304] 0 2024-02-16T22:56:29.344 INFO:tasks.workunit.client.0.smithi064.stdout:2/674: dread - d7/d14/d1e/fa2 zero size 2024-02-16T22:56:29.345 INFO:tasks.workunit.client.1.smithi192.stdout:5/714: link d6/d12/d2d/d5c/f99 d6/d12/d4e/ff0 0 2024-02-16T22:56:29.345 INFO:tasks.workunit.client.0.smithi064.stdout:2/675: chown d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44 1474685 1 2024-02-16T22:56:29.346 INFO:tasks.workunit.client.0.smithi064.stdout:2/676: dread - d7/d14/d25/d46/d55/fda zero size 2024-02-16T22:56:29.347 INFO:tasks.workunit.client.1.smithi192.stdout:5/715: stat d6/d12/d4e/d5b/d67/d78/fef 0 2024-02-16T22:56:29.347 INFO:tasks.workunit.client.0.smithi064.stdout:6/644: sync 2024-02-16T22:56:29.347 INFO:tasks.workunit.client.0.smithi064.stdout:5/529: sync 2024-02-16T22:56:29.347 INFO:tasks.workunit.client.0.smithi064.stdout:0/558: sync 2024-02-16T22:56:29.347 INFO:tasks.workunit.client.1.smithi192.stdout:0/690: dread - d1/fa6 zero size 2024-02-16T22:56:29.348 INFO:tasks.workunit.client.1.smithi192.stdout:0/691: chown d1/dc7/d9c 1695 1 2024-02-16T22:56:29.350 INFO:tasks.workunit.client.0.smithi064.stdout:4/601: write dc/dd/d32/d58/f8e [319439,93670] 0 2024-02-16T22:56:29.351 INFO:tasks.workunit.client.0.smithi064.stdout:8/576: unlink df/d31/d8f/dab/dae/fb3 0 2024-02-16T22:56:29.352 INFO:tasks.workunit.client.1.smithi192.stdout:4/749: mknod d1/d15/c103 0 2024-02-16T22:56:29.355 INFO:tasks.workunit.client.0.smithi064.stdout:9/549: mknod d3/d9/d6a/cbb 0 2024-02-16T22:56:29.356 INFO:tasks.workunit.client.1.smithi192.stdout:8/711: sync 2024-02-16T22:56:29.356 INFO:tasks.workunit.client.0.smithi064.stdout:1/562: dread d2/f43 [0,4194304] 0 2024-02-16T22:56:29.359 INFO:tasks.workunit.client.0.smithi064.stdout:5/530: mknod d6/d25/d48/d67/cba 0 2024-02-16T22:56:29.359 INFO:tasks.workunit.client.0.smithi064.stdout:5/531: chown d6/d1c/caf 53 1 2024-02-16T22:56:29.360 INFO:tasks.workunit.client.1.smithi192.stdout:9/834: creat d7/d15/f11b x:0 0 0 2024-02-16T22:56:29.361 INFO:tasks.workunit.client.0.smithi064.stdout:0/559: symlink da/dc/de/d37/d88/lbf 0 2024-02-16T22:56:29.363 INFO:tasks.workunit.client.1.smithi192.stdout:1/681: rename d3/d40/lae to d3/de/d27/d34/d36/le8 0 2024-02-16T22:56:29.363 INFO:tasks.workunit.client.1.smithi192.stdout:3/664: dwrite da/d6a/fe7 [0,4194304] 0 2024-02-16T22:56:29.363 INFO:tasks.workunit.client.0.smithi064.stdout:6/645: creat d0/d3c/da9/d76/fd9 x:0 0 0 2024-02-16T22:56:29.364 INFO:tasks.workunit.client.1.smithi192.stdout:3/665: read da/f22 [384607,99547] 0 2024-02-16T22:56:29.364 INFO:tasks.workunit.client.0.smithi064.stdout:6/646: chown d0/d6/da/lc3 50 1 2024-02-16T22:56:29.366 INFO:tasks.workunit.client.0.smithi064.stdout:4/602: rmdir dc/de/d28/d57/d77 39 2024-02-16T22:56:29.367 INFO:tasks.workunit.client.1.smithi192.stdout:0/692: chown d1/d3/d39/cb1 1345219 1 2024-02-16T22:56:29.370 INFO:tasks.workunit.client.0.smithi064.stdout:8/577: mknod df/d31/d61/cc2 0 2024-02-16T22:56:29.370 INFO:tasks.workunit.client.0.smithi064.stdout:3/705: write d4/d13/fad [967843,62004] 0 2024-02-16T22:56:29.370 INFO:tasks.workunit.client.1.smithi192.stdout:4/750: dread d1/d15/d23/f4a [0,4194304] 0 2024-02-16T22:56:29.371 INFO:tasks.workunit.client.1.smithi192.stdout:6/745: link da/df/d12/f9c da/df/fef 0 2024-02-16T22:56:29.373 INFO:tasks.workunit.client.0.smithi064.stdout:9/550: chown d3/f10 41 1 2024-02-16T22:56:29.375 INFO:tasks.workunit.client.1.smithi192.stdout:7/817: link d1/d2/de/d2c/d49/d54/l112 d1/de1/dd0/l11a 0 2024-02-16T22:56:29.375 INFO:tasks.workunit.client.0.smithi064.stdout:7/627: symlink d5/d33/d42/d81/d82/lc3 0 2024-02-16T22:56:29.377 INFO:tasks.workunit.client.0.smithi064.stdout:9/551: dread d3/d9/d6a/db1/d2a/d2e/f39 [0,4194304] 0 2024-02-16T22:56:29.378 INFO:tasks.workunit.client.1.smithi192.stdout:7/818: chown d1/d2/d2d/d61/d88/d90/d4c/f6b 96492 1 2024-02-16T22:56:29.379 INFO:tasks.workunit.client.1.smithi192.stdout:6/746: dwrite da/df/d12/d15/d1b/f8a [0,4194304] 0 2024-02-16T22:56:29.380 INFO:tasks.workunit.client.0.smithi064.stdout:1/563: creat d2/df/d14/d18/d48/d84/fc4 x:0 0 0 2024-02-16T22:56:29.380 INFO:tasks.workunit.client.0.smithi064.stdout:2/677: dread d7/f1a [0,4194304] 0 2024-02-16T22:56:29.380 INFO:tasks.workunit.client.0.smithi064.stdout:9/552: dread - d3/d9/d1e/fad zero size 2024-02-16T22:56:29.381 INFO:tasks.workunit.client.1.smithi192.stdout:2/766: getdents d22/d24 0 2024-02-16T22:56:29.381 INFO:tasks.workunit.client.0.smithi064.stdout:5/532: readlink d6/d1c/l6b 0 2024-02-16T22:56:29.383 INFO:tasks.workunit.client.0.smithi064.stdout:0/560: truncate da/dc/de/d12/f61 705836 0 2024-02-16T22:56:29.386 INFO:tasks.workunit.client.0.smithi064.stdout:6/647: symlink d0/d5/lda 0 2024-02-16T22:56:29.388 INFO:tasks.workunit.client.1.smithi192.stdout:1/682: chown d3/de/c18 455149088 1 2024-02-16T22:56:29.390 INFO:tasks.workunit.client.0.smithi064.stdout:3/706: symlink d4/d1f/lee 0 2024-02-16T22:56:29.391 INFO:tasks.workunit.client.0.smithi064.stdout:3/707: read d4/d13/d1b/fcb [1888139,64088] 0 2024-02-16T22:56:29.391 INFO:tasks.workunit.client.1.smithi192.stdout:5/716: dwrite d6/d12/f44 [0,4194304] 0 2024-02-16T22:56:29.396 INFO:tasks.workunit.client.0.smithi064.stdout:2/678: fdatasync d7/d14/d25/f4a 0 2024-02-16T22:56:29.397 INFO:tasks.workunit.client.0.smithi064.stdout:9/553: mknod d3/d11/d15/dab/cbc 0 2024-02-16T22:56:29.398 INFO:tasks.workunit.client.1.smithi192.stdout:7/819: mknod d1/d2/de/d2c/c11b 0 2024-02-16T22:56:29.405 INFO:tasks.workunit.client.1.smithi192.stdout:2/767: unlink l1e 0 2024-02-16T22:56:29.406 INFO:tasks.workunit.client.1.smithi192.stdout:2/768: write fe [1128672,11118] 0 2024-02-16T22:56:29.407 INFO:tasks.workunit.client.0.smithi064.stdout:6/648: dread d0/d1b/f23 [4194304,4194304] 0 2024-02-16T22:56:29.408 INFO:tasks.workunit.client.1.smithi192.stdout:8/712: truncate d5/d26/f88 330227 0 2024-02-16T22:56:29.409 INFO:tasks.workunit.client.0.smithi064.stdout:4/603: symlink dc/de/d28/d57/d77/lbc 0 2024-02-16T22:56:29.409 INFO:tasks.workunit.client.1.smithi192.stdout:2/769: chown d22/d24/d39/d42/d65/d69/f68 692574 1 2024-02-16T22:56:29.410 INFO:tasks.workunit.client.1.smithi192.stdout:4/751: dwrite d1/d26/d6f/f86 [0,4194304] 0 2024-02-16T22:56:29.411 INFO:tasks.workunit.client.1.smithi192.stdout:1/683: readlink d3/d40/d9f/la8 0 2024-02-16T22:56:29.412 INFO:tasks.workunit.client.1.smithi192.stdout:1/684: chown d3/l3d 36 1 2024-02-16T22:56:29.413 INFO:tasks.workunit.client.1.smithi192.stdout:1/685: dread - d3/de/d27/d34/d52/d63/d81/fd8 zero size 2024-02-16T22:56:29.413 INFO:tasks.workunit.client.1.smithi192.stdout:3/666: rename da/d11/d63/d6b/d66/cc9 to da/d11/d20/d39/ce9 0 2024-02-16T22:56:29.414 INFO:tasks.workunit.client.0.smithi064.stdout:5/533: write d6/d18/d19/f47 [1993797,96692] 0 2024-02-16T22:56:29.415 INFO:tasks.workunit.client.0.smithi064.stdout:3/708: dread - d4/d1f/d5b/d51/f6a zero size 2024-02-16T22:56:29.416 INFO:tasks.workunit.client.1.smithi192.stdout:1/686: dread - d3/de/d27/d34/d69/d85/fbd zero size 2024-02-16T22:56:29.417 INFO:tasks.workunit.client.1.smithi192.stdout:6/747: write da/df/d18/d46/fa4 [180094,55629] 0 2024-02-16T22:56:29.417 INFO:tasks.workunit.client.1.smithi192.stdout:5/717: creat d6/d12/d1d/d56/ff1 x:0 0 0 2024-02-16T22:56:29.423 INFO:tasks.workunit.client.0.smithi064.stdout:7/628: truncate d5/f27 295813 0 2024-02-16T22:56:29.425 INFO:tasks.workunit.client.1.smithi192.stdout:9/835: link d7/d19/d28/faa d7/d15/d36/d77/f11c 0 2024-02-16T22:56:29.426 INFO:tasks.workunit.client.0.smithi064.stdout:2/679: mkdir d7/d14/d25/d46/d55/d7f/dde 0 2024-02-16T22:56:29.429 INFO:tasks.workunit.client.0.smithi064.stdout:9/554: fsync d3/f10 0 2024-02-16T22:56:29.432 INFO:tasks.workunit.client.0.smithi064.stdout:3/709: dread d4/d13/d4d/fb8 [0,4194304] 0 2024-02-16T22:56:29.432 INFO:tasks.workunit.client.1.smithi192.stdout:2/770: dread d22/f3c [0,4194304] 0 2024-02-16T22:56:29.433 INFO:tasks.workunit.client.1.smithi192.stdout:4/752: unlink d1/d15/d29/d3c/ce9 0 2024-02-16T22:56:29.434 INFO:tasks.workunit.client.1.smithi192.stdout:4/753: chown d1/d26/d66/f9e 3093598 1 2024-02-16T22:56:29.437 INFO:tasks.workunit.client.1.smithi192.stdout:0/693: creat d1/d3/de/d22/d37/fdc x:0 0 0 2024-02-16T22:56:29.440 INFO:tasks.workunit.client.1.smithi192.stdout:3/667: mknod da/d11/d35/cea 0 2024-02-16T22:56:29.441 INFO:tasks.workunit.client.0.smithi064.stdout:4/604: mkdir dc/d1d/dbd 0 2024-02-16T22:56:29.442 INFO:tasks.workunit.client.1.smithi192.stdout:1/687: mknod d3/d5b/d77/ce9 0 2024-02-16T22:56:29.444 INFO:tasks.workunit.client.1.smithi192.stdout:9/836: dread d7/d15/d36/d77/faf [0,4194304] 0 2024-02-16T22:56:29.444 INFO:tasks.workunit.client.1.smithi192.stdout:5/718: write d6/fd9 [122484,78476] 0 2024-02-16T22:56:29.445 INFO:tasks.workunit.client.1.smithi192.stdout:1/688: fsync d3/de/d27/d34/d36/d47/f79 0 2024-02-16T22:56:29.446 INFO:tasks.workunit.client.0.smithi064.stdout:5/534: symlink d6/d18/d19/d1a/lbb 0 2024-02-16T22:56:29.448 INFO:tasks.workunit.client.1.smithi192.stdout:3/668: dwrite da/d11/fdf [0,4194304] 0 2024-02-16T22:56:29.448 INFO:tasks.workunit.client.0.smithi064.stdout:7/629: mkdir d5/d33/d42/d81/dc4 0 2024-02-16T22:56:29.448 INFO:tasks.workunit.client.0.smithi064.stdout:7/630: readlink d5/d6/l69 0 2024-02-16T22:56:29.448 INFO:tasks.workunit.client.1.smithi192.stdout:5/719: dread - d6/d8/d75/d8c/fe8 zero size 2024-02-16T22:56:29.449 INFO:tasks.workunit.client.1.smithi192.stdout:5/720: chown d6/d20/d93 5 1 2024-02-16T22:56:29.450 INFO:tasks.workunit.client.0.smithi064.stdout:1/564: creat d2/df/d11/d25/fc5 x:0 0 0 2024-02-16T22:56:29.451 INFO:tasks.workunit.client.1.smithi192.stdout:4/754: mkdir d1/d15/d48/db5/dd5/d104 0 2024-02-16T22:56:29.454 INFO:tasks.workunit.client.0.smithi064.stdout:2/680: creat d7/d14/d25/d46/d55/d7f/fdf x:0 0 0 2024-02-16T22:56:29.455 INFO:tasks.workunit.client.1.smithi192.stdout:0/694: fsync d1/d3/de/d22/d37/d67/f42 0 2024-02-16T22:56:29.458 INFO:tasks.workunit.client.1.smithi192.stdout:1/689: dwrite d3/de/d27/d34/d52/d63/d7a/dc9/fe1 [0,4194304] 0 2024-02-16T22:56:29.458 INFO:tasks.workunit.client.1.smithi192.stdout:0/695: fsync d1/d3/de/d22/fd2 0 2024-02-16T22:56:29.460 INFO:tasks.workunit.client.0.smithi064.stdout:6/649: stat d0/d5/fcd 0 2024-02-16T22:56:29.462 INFO:tasks.workunit.client.0.smithi064.stdout:4/605: unlink dc/dd/d32/d58/l30 0 2024-02-16T22:56:29.464 INFO:tasks.workunit.client.1.smithi192.stdout:7/820: getdents d1/d2/de/d2c/d49 0 2024-02-16T22:56:29.464 INFO:tasks.workunit.client.0.smithi064.stdout:8/578: getdents df/d31/d2f/d3e/d5f/d93 0 2024-02-16T22:56:29.469 INFO:tasks.workunit.client.1.smithi192.stdout:6/748: dread da/df/d12/d15/d1b/f37 [0,4194304] 0 2024-02-16T22:56:29.470 INFO:tasks.workunit.client.0.smithi064.stdout:5/535: creat d6/d18/d90/fbc x:0 0 0 2024-02-16T22:56:29.470 INFO:tasks.workunit.client.1.smithi192.stdout:9/837: dread d7/d15/d36/dbe/ffe [4194304,4194304] 0 2024-02-16T22:56:29.470 INFO:tasks.workunit.client.0.smithi064.stdout:5/536: chown d6/d18/d19/l57 292501 1 2024-02-16T22:56:29.471 INFO:tasks.workunit.client.1.smithi192.stdout:9/838: chown d7/d19/d5e/d6f/la3 5 1 2024-02-16T22:56:29.471 INFO:tasks.workunit.client.0.smithi064.stdout:7/631: read - d5/d6/d54/d41/f68 zero size 2024-02-16T22:56:29.471 INFO:tasks.workunit.client.0.smithi064.stdout:5/537: read d6/d1c/d43/f58 [507662,12472] 0 2024-02-16T22:56:29.472 INFO:tasks.workunit.client.1.smithi192.stdout:1/690: write d3/de/fc6 [4888986,129341] 0 2024-02-16T22:56:29.473 INFO:tasks.workunit.client.0.smithi064.stdout:1/565: mkdir d2/df/d14/d2c/db5/dc6 0 2024-02-16T22:56:29.473 INFO:tasks.workunit.client.1.smithi192.stdout:1/691: dread - d3/de/d27/d34/d52/d63/d81/fdb zero size 2024-02-16T22:56:29.474 INFO:tasks.workunit.client.0.smithi064.stdout:1/566: chown d2/df/d11/d1a/d30/dbd/d91/d56/c89 38228 1 2024-02-16T22:56:29.475 INFO:tasks.workunit.client.1.smithi192.stdout:0/696: symlink d1/d3/de/d22/d37/d47/ldd 0 2024-02-16T22:56:29.475 INFO:tasks.workunit.client.1.smithi192.stdout:0/697: fsync d1/fcb 0 2024-02-16T22:56:29.477 INFO:tasks.workunit.client.1.smithi192.stdout:2/771: getdents d22/d47 0 2024-02-16T22:56:29.479 INFO:tasks.workunit.client.1.smithi192.stdout:1/692: dwrite d3/de/d27/f4a [0,4194304] 0 2024-02-16T22:56:29.480 INFO:tasks.workunit.client.0.smithi064.stdout:1/567: dwrite d2/df/d14/fa7 [0,4194304] 0 2024-02-16T22:56:29.481 INFO:tasks.workunit.client.1.smithi192.stdout:9/839: creat d7/f11d x:0 0 0 2024-02-16T22:56:29.482 INFO:tasks.workunit.client.1.smithi192.stdout:9/840: chown d7/d15/d36/d77/db1/ff8 7872 1 2024-02-16T22:56:29.484 INFO:tasks.workunit.client.1.smithi192.stdout:6/749: rename da/df/d12/d15/d1b/d70/fda to da/d40/db6/db8/ff0 0 2024-02-16T22:56:29.485 INFO:tasks.workunit.client.1.smithi192.stdout:6/750: stat da/df/d71/dab 0 2024-02-16T22:56:29.486 INFO:tasks.workunit.client.1.smithi192.stdout:6/751: write da/df/d71/dab/fcd [231886,61071] 0 2024-02-16T22:56:29.490 INFO:tasks.workunit.client.1.smithi192.stdout:6/752: dread da/df/f1c [4194304,4194304] 0 2024-02-16T22:56:29.490 INFO:tasks.workunit.client.1.smithi192.stdout:2/772: chown d22/d24/d39/d42/d85/d98 27167839 1 2024-02-16T22:56:29.491 INFO:tasks.workunit.client.1.smithi192.stdout:4/755: link d1/d26/d66/d89/f64 d1/d26/d6f/df4/f105 0 2024-02-16T22:56:29.494 INFO:tasks.workunit.client.0.smithi064.stdout:7/632: rmdir d5/d33 39 2024-02-16T22:56:29.497 INFO:tasks.workunit.client.1.smithi192.stdout:5/721: getdents d6/d12/d4e/d46/d4f 0 2024-02-16T22:56:29.498 INFO:tasks.workunit.client.0.smithi064.stdout:0/561: truncate da/dc/de/d12/f61 40247 0 2024-02-16T22:56:29.501 INFO:tasks.workunit.client.0.smithi064.stdout:2/681: mknod d7/d14/d3e/db8/ce0 0 2024-02-16T22:56:29.501 INFO:tasks.workunit.client.1.smithi192.stdout:0/698: write d1/d3/de/d22/f60 [4214694,15440] 0 2024-02-16T22:56:29.502 INFO:tasks.workunit.client.0.smithi064.stdout:2/682: dread - d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f80 zero size 2024-02-16T22:56:29.503 INFO:tasks.workunit.client.1.smithi192.stdout:9/841: creat d7/d15/d36/d77/dbd/de0/f11e x:0 0 0 2024-02-16T22:56:29.503 INFO:tasks.workunit.client.0.smithi064.stdout:2/683: write d7/d14/d25/f90 [3317684,31864] 0 2024-02-16T22:56:29.507 INFO:tasks.workunit.client.0.smithi064.stdout:3/710: rename d4/d1f/d5b/d51/d7c/l94 to d4/d13/d1b/lef 0 2024-02-16T22:56:29.510 INFO:tasks.workunit.client.1.smithi192.stdout:6/753: unlink da/df/d12/f9b 0 2024-02-16T22:56:29.512 INFO:tasks.workunit.client.0.smithi064.stdout:4/606: truncate dc/d1d/f2b 332821 0 2024-02-16T22:56:29.516 INFO:tasks.workunit.client.0.smithi064.stdout:7/633: read d5/d6/d54/d41/f4b [4024461,56112] 0 2024-02-16T22:56:29.517 INFO:tasks.workunit.client.0.smithi064.stdout:5/538: creat d6/d25/d48/d67/d91/da5/fbd x:0 0 0 2024-02-16T22:56:29.518 INFO:tasks.workunit.client.1.smithi192.stdout:1/693: symlink d3/lea 0 2024-02-16T22:56:29.518 INFO:tasks.workunit.client.1.smithi192.stdout:4/756: creat d1/d15/d48/db5/f106 x:0 0 0 2024-02-16T22:56:29.518 INFO:tasks.workunit.client.1.smithi192.stdout:5/722: rename d6/d12/d4e/l59 to d6/d12/d4e/d5b/d67/lf2 0 2024-02-16T22:56:29.518 INFO:tasks.workunit.client.1.smithi192.stdout:0/699: symlink d1/dc7/lde 0 2024-02-16T22:56:29.519 INFO:tasks.workunit.client.1.smithi192.stdout:6/754: creat da/df/d18/d46/ff1 x:0 0 0 2024-02-16T22:56:29.520 INFO:tasks.workunit.client.1.smithi192.stdout:2/773: symlink d22/d47/dca/dfa/lfb 0 2024-02-16T22:56:29.521 INFO:tasks.workunit.client.1.smithi192.stdout:2/774: chown d22/d47/dcb/d3f/dac/fde 1987 1 2024-02-16T22:56:29.521 INFO:tasks.workunit.client.0.smithi064.stdout:3/711: fdatasync d4/d13/d4d/d5c/d78/fed 0 2024-02-16T22:56:29.522 INFO:tasks.workunit.client.0.smithi064.stdout:3/712: fsync d4/d13/d1b/d27/fd6 0 2024-02-16T22:56:29.522 INFO:tasks.workunit.client.1.smithi192.stdout:8/713: sync 2024-02-16T22:56:29.522 INFO:tasks.workunit.client.0.smithi064.stdout:1/568: fdatasync d2/df/d14/d2a/fbf 0 2024-02-16T22:56:29.525 INFO:tasks.workunit.client.1.smithi192.stdout:2/775: dwrite d22/d24/d39/fb7 [0,4194304] 0 2024-02-16T22:56:29.527 INFO:tasks.workunit.client.1.smithi192.stdout:2/776: write d22/d47/dcb/dc9/fcf [253863,31269] 0 2024-02-16T22:56:29.529 INFO:tasks.workunit.client.0.smithi064.stdout:8/579: link df/f24 df/d31/d2f/fc3 0 2024-02-16T22:56:29.536 INFO:tasks.workunit.client.0.smithi064.stdout:0/562: mkdir da/dc/de/d12/d2c/d2e/d38/d7e/d89/d9e/dc0 0 2024-02-16T22:56:29.538 INFO:tasks.workunit.client.0.smithi064.stdout:7/634: rmdir d5/d6/d54/d41 39 2024-02-16T22:56:29.541 INFO:tasks.workunit.client.0.smithi064.stdout:2/684: write d7/d14/d25/f8a [300141,81203] 0 2024-02-16T22:56:29.542 INFO:tasks.workunit.client.0.smithi064.stdout:3/713: mknod d4/d1f/d34/d7b/cf0 0 2024-02-16T22:56:29.543 INFO:tasks.workunit.client.1.smithi192.stdout:8/714: dread d5/d26/f3f [0,4194304] 0 2024-02-16T22:56:29.544 INFO:tasks.workunit.client.1.smithi192.stdout:8/715: write d5/f84 [4733985,58793] 0 2024-02-16T22:56:29.544 INFO:tasks.workunit.client.0.smithi064.stdout:6/650: getdents d0/d1d/d21 0 2024-02-16T22:56:29.549 INFO:tasks.workunit.client.0.smithi064.stdout:4/607: write dc/dd/d32/d22/f56 [422097,123071] 0 2024-02-16T22:56:29.550 INFO:tasks.workunit.client.0.smithi064.stdout:8/580: mkdir df/d31/d61/d79/dc4 0 2024-02-16T22:56:29.550 INFO:tasks.workunit.client.0.smithi064.stdout:4/608: fdatasync dc/dd/d4f/fa7 0 2024-02-16T22:56:29.552 INFO:tasks.workunit.client.0.smithi064.stdout:8/581: write df/d31/d2f/d38/fba [64140,48979] 0 2024-02-16T22:56:29.552 INFO:tasks.workunit.client.0.smithi064.stdout:5/539: fdatasync d6/f17 0 2024-02-16T22:56:29.556 INFO:tasks.workunit.client.0.smithi064.stdout:2/685: dread d7/f54 [0,4194304] 0 2024-02-16T22:56:29.557 INFO:tasks.workunit.client.0.smithi064.stdout:2/686: chown d7/d14/d1e/d34/f87 147 1 2024-02-16T22:56:29.559 INFO:tasks.workunit.client.0.smithi064.stdout:3/714: creat d4/d1f/d34/ff1 x:0 0 0 2024-02-16T22:56:29.561 INFO:tasks.workunit.client.0.smithi064.stdout:6/651: chown d0/d5/df/d17/f7d 2040283741 1 2024-02-16T22:56:29.562 INFO:tasks.workunit.client.0.smithi064.stdout:6/652: chown d0/d5/df/d17/d18/l52 4015711 1 2024-02-16T22:56:29.568 INFO:tasks.workunit.client.0.smithi064.stdout:4/609: mknod dc/dd/d32/d46/d82/cbe 0 2024-02-16T22:56:29.571 INFO:tasks.workunit.client.0.smithi064.stdout:5/540: rmdir d6/d25/da4 39 2024-02-16T22:56:29.572 INFO:tasks.workunit.client.0.smithi064.stdout:8/582: mknod df/d31/d2f/d68/cc5 0 2024-02-16T22:56:29.574 INFO:tasks.workunit.client.0.smithi064.stdout:0/563: mknod da/cc1 0 2024-02-16T22:56:29.576 INFO:tasks.workunit.client.0.smithi064.stdout:0/564: read - da/d71/f8d zero size 2024-02-16T22:56:29.580 INFO:tasks.workunit.client.0.smithi064.stdout:0/565: chown da/dc/f4b 43181 1 2024-02-16T22:56:29.580 INFO:tasks.workunit.client.0.smithi064.stdout:3/715: fdatasync d4/d1f/d5b/d51/f6a 0 2024-02-16T22:56:29.581 INFO:tasks.workunit.client.0.smithi064.stdout:1/569: dwrite d2/df/d11/d1a/d30/f63 [0,4194304] 0 2024-02-16T22:56:29.582 INFO:tasks.workunit.client.1.smithi192.stdout:3/669: sync 2024-02-16T22:56:29.582 INFO:tasks.workunit.client.1.smithi192.stdout:7/821: sync 2024-02-16T22:56:29.583 INFO:tasks.workunit.client.1.smithi192.stdout:3/670: fdatasync da/d11/d20/fd3 0 2024-02-16T22:56:29.583 INFO:tasks.workunit.client.1.smithi192.stdout:7/822: chown d1/d2/de/d2c/d49/d54/d6a 111613458 1 2024-02-16T22:56:29.583 INFO:tasks.workunit.client.0.smithi064.stdout:8/583: dwrite df/d31/d8f/dab/dae/fb2 [0,4194304] 0 2024-02-16T22:56:29.584 INFO:tasks.workunit.client.1.smithi192.stdout:3/671: stat da/d11/d63/d6b/faa 0 2024-02-16T22:56:29.585 INFO:tasks.workunit.client.0.smithi064.stdout:3/716: write d4/d13/d1b/d27/d44/f5f [812359,47852] 0 2024-02-16T22:56:29.585 INFO:tasks.workunit.client.0.smithi064.stdout:1/570: chown d2/df/d11/d1a/d30/dbd/d91/c6c 11578311 1 2024-02-16T22:56:29.585 INFO:tasks.workunit.client.0.smithi064.stdout:6/653: write d0/d5/fab [490202,106085] 0 2024-02-16T22:56:29.591 INFO:tasks.workunit.client.1.smithi192.stdout:4/757: chown d1/d15/d48/d8d/cde 31020405 1 2024-02-16T22:56:29.596 INFO:tasks.workunit.client.1.smithi192.stdout:1/694: creat d3/de/d27/d34/d36/feb x:0 0 0 2024-02-16T22:56:29.597 INFO:tasks.workunit.client.1.smithi192.stdout:1/695: chown d3/de/d27/d34/d69/d85/d89/fbe 1051723 1 2024-02-16T22:56:29.598 INFO:tasks.workunit.client.0.smithi064.stdout:5/541: mkdir d6/d18/d95/dbe 0 2024-02-16T22:56:29.598 INFO:tasks.workunit.client.0.smithi064.stdout:5/542: readlink d6/d25/la8 0 2024-02-16T22:56:29.599 INFO:tasks.workunit.client.1.smithi192.stdout:0/700: creat d1/d3/de/d22/da7/fdf x:0 0 0 2024-02-16T22:56:29.600 INFO:tasks.workunit.client.0.smithi064.stdout:2/687: write d7/d14/d3e/f75 [231366,109832] 0 2024-02-16T22:56:29.604 INFO:tasks.workunit.client.0.smithi064.stdout:7/635: dwrite d5/d33/d53/f6e [0,4194304] 0 2024-02-16T22:56:29.607 INFO:tasks.workunit.client.0.smithi064.stdout:5/543: dwrite d6/d1c/f53 [0,4194304] 0 2024-02-16T22:56:29.608 INFO:tasks.workunit.client.1.smithi192.stdout:9/842: truncate d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f9a 3790689 0 2024-02-16T22:56:29.613 INFO:tasks.workunit.client.1.smithi192.stdout:8/716: unlink d5/db/de/l1d 0 2024-02-16T22:56:29.614 INFO:tasks.workunit.client.1.smithi192.stdout:8/717: fsync d5/d26/d83/d86/f93 0 2024-02-16T22:56:29.614 INFO:tasks.workunit.client.1.smithi192.stdout:8/718: readlink d5/db/l82 0 2024-02-16T22:56:29.614 INFO:tasks.workunit.client.1.smithi192.stdout:8/719: fdatasync d5/d9/f5a 0 2024-02-16T22:56:29.615 INFO:tasks.workunit.client.0.smithi064.stdout:0/566: mkdir da/dc/d3b/dc2 0 2024-02-16T22:56:29.616 INFO:tasks.workunit.client.1.smithi192.stdout:7/823: dread - d1/d2/de/d2c/d49/d54/dd8/d117/dcd/fe0 zero size 2024-02-16T22:56:29.616 INFO:tasks.workunit.client.0.smithi064.stdout:4/610: dwrite dc/d1d/d43/f5f [0,4194304] 0 2024-02-16T22:56:29.619 INFO:tasks.workunit.client.1.smithi192.stdout:3/672: dread da/d11/d63/d6b/faa [0,4194304] 0 2024-02-16T22:56:29.621 INFO:tasks.workunit.client.1.smithi192.stdout:5/723: creat d6/d12/d4e/de6/ff3 x:0 0 0 2024-02-16T22:56:29.627 INFO:tasks.workunit.client.0.smithi064.stdout:3/717: chown d4/d13/d1b/d4f/f6f 93693 1 2024-02-16T22:56:29.628 INFO:tasks.workunit.client.1.smithi192.stdout:0/701: chown d1/l8f 34484 1 2024-02-16T22:56:29.630 INFO:tasks.workunit.client.0.smithi064.stdout:6/654: creat d0/d1d/da8/fdb x:0 0 0 2024-02-16T22:56:29.631 INFO:tasks.workunit.client.1.smithi192.stdout:6/755: symlink da/df/lf2 0 2024-02-16T22:56:29.634 INFO:tasks.workunit.client.1.smithi192.stdout:7/824: dread d1/d2/de/d2c/f8b [0,4194304] 0 2024-02-16T22:56:29.636 INFO:tasks.workunit.client.1.smithi192.stdout:7/825: stat d1/d2/de/d2c/d49/da4/d104 0 2024-02-16T22:56:29.637 INFO:tasks.workunit.client.1.smithi192.stdout:4/758: dwrite d1/d26/d66/d89/d11/fa6 [0,4194304] 0 2024-02-16T22:56:29.637 INFO:tasks.workunit.client.0.smithi064.stdout:2/688: mknod d7/d14/d1e/d34/d65/ce1 0 2024-02-16T22:56:29.639 INFO:tasks.workunit.client.1.smithi192.stdout:8/720: mkdir d5/d16/de1 0 2024-02-16T22:56:29.639 INFO:tasks.workunit.client.1.smithi192.stdout:6/756: dread da/d40/fcb [0,4194304] 0 2024-02-16T22:56:29.639 INFO:tasks.workunit.client.0.smithi064.stdout:7/636: rename d5/d33/d45/d64/d56/d66/fa1 to d5/d33/d45/d73/d88/fc5 0 2024-02-16T22:56:29.640 INFO:tasks.workunit.client.0.smithi064.stdout:7/637: dread - d5/d33/d53/f99 zero size 2024-02-16T22:56:29.642 INFO:tasks.workunit.client.0.smithi064.stdout:5/544: symlink d6/d1c/d4b/lbf 0 2024-02-16T22:56:29.642 INFO:tasks.workunit.client.1.smithi192.stdout:8/721: write d5/fcd [320340,105725] 0 2024-02-16T22:56:29.643 INFO:tasks.workunit.client.0.smithi064.stdout:8/584: write f8 [1168668,59761] 0 2024-02-16T22:56:29.646 INFO:tasks.workunit.client.1.smithi192.stdout:5/724: read - d6/d12/d1d/fb5 zero size 2024-02-16T22:56:29.648 INFO:tasks.workunit.client.0.smithi064.stdout:0/567: creat da/dc/de/d12/d2c/d2e/d38/d7e/d89/d93/fc3 x:0 0 0 2024-02-16T22:56:29.649 INFO:tasks.workunit.client.0.smithi064.stdout:0/568: fdatasync da/dc/de/d12/d2c/d2e/f36 0 2024-02-16T22:56:29.649 INFO:tasks.workunit.client.1.smithi192.stdout:1/696: creat d3/de/d27/d34/d69/d85/d89/daa/de6/fec x:0 0 0 2024-02-16T22:56:29.651 INFO:tasks.workunit.client.1.smithi192.stdout:5/725: dread d6/d12/d4e/d46/d4f/fec [0,4194304] 0 2024-02-16T22:56:29.653 INFO:tasks.workunit.client.1.smithi192.stdout:2/777: dwrite f1 [0,4194304] 0 2024-02-16T22:56:29.654 INFO:tasks.workunit.client.1.smithi192.stdout:5/726: write d6/d12/d1d/fb8 [991513,6692] 0 2024-02-16T22:56:29.654 INFO:tasks.workunit.client.0.smithi064.stdout:3/718: mknod d4/d13/d1b/d27/d44/d97/cf2 0 2024-02-16T22:56:29.659 INFO:tasks.workunit.client.0.smithi064.stdout:9/555: sync 2024-02-16T22:56:29.662 INFO:tasks.workunit.client.0.smithi064.stdout:5/545: dread d6/d18/f56 [0,4194304] 0 2024-02-16T22:56:29.665 INFO:tasks.workunit.client.1.smithi192.stdout:4/759: mkdir d1/d26/dbe/d107 0 2024-02-16T22:56:29.666 INFO:tasks.workunit.client.0.smithi064.stdout:2/689: read - d7/d14/d1e/d34/d65/f67 zero size 2024-02-16T22:56:29.667 INFO:tasks.workunit.client.1.smithi192.stdout:4/760: write d1/d26/d6f/fbc [133166,9793] 0 2024-02-16T22:56:29.667 INFO:tasks.workunit.client.0.smithi064.stdout:2/690: dread - d7/d14/d4b/d63/fc3 zero size 2024-02-16T22:56:29.668 INFO:tasks.workunit.client.0.smithi064.stdout:2/691: dread - d7/fb1 zero size 2024-02-16T22:56:29.669 INFO:tasks.workunit.client.1.smithi192.stdout:6/757: mknod da/df/d6b/d83/cf3 0 2024-02-16T22:56:29.669 INFO:tasks.workunit.client.1.smithi192.stdout:3/673: dwrite da/d11/d63/d6b/d66/f60 [0,4194304] 0 2024-02-16T22:56:29.670 INFO:tasks.workunit.client.0.smithi064.stdout:7/638: mkdir d5/d6/d54/d7b/dc6 0 2024-02-16T22:56:29.671 INFO:tasks.workunit.client.1.smithi192.stdout:2/778: dread d22/d24/d39/d3d/df2/f8a [0,4194304] 0 2024-02-16T22:56:29.672 INFO:tasks.workunit.client.1.smithi192.stdout:2/779: readlink d22/d24/d39/d42/d52/l59 0 2024-02-16T22:56:29.672 INFO:tasks.workunit.client.0.smithi064.stdout:4/611: dwrite dc/dd/d32/f3d [0,4194304] 0 2024-02-16T22:56:29.674 INFO:tasks.workunit.client.0.smithi064.stdout:6/655: write d0/d12/d7b/fa7 [693069,9273] 0 2024-02-16T22:56:29.674 INFO:tasks.workunit.client.0.smithi064.stdout:8/585: chown df/d31/d61/d79/d81/fb1 4617 1 2024-02-16T22:56:29.679 INFO:tasks.workunit.client.0.smithi064.stdout:6/656: fsync d0/d89/fb6 0 2024-02-16T22:56:29.679 INFO:tasks.workunit.client.0.smithi064.stdout:1/571: truncate d2/df/d14/d2c/f49 1010318 0 2024-02-16T22:56:29.681 INFO:tasks.workunit.client.1.smithi192.stdout:2/780: dwrite d22/d24/d39/d42/d65/d69/f7a [0,4194304] 0 2024-02-16T22:56:29.683 INFO:tasks.workunit.client.0.smithi064.stdout:9/556: mknod d3/d9/d1e/d42/cbd 0 2024-02-16T22:56:29.684 INFO:tasks.workunit.client.1.smithi192.stdout:8/722: dread d5/d16/dd5/f9c [0,4194304] 0 2024-02-16T22:56:29.684 INFO:tasks.workunit.client.1.smithi192.stdout:0/702: fsync d1/f3e 0 2024-02-16T22:56:29.686 INFO:tasks.workunit.client.0.smithi064.stdout:5/546: creat d6/d18/d19/d1a/fc0 x:0 0 0 2024-02-16T22:56:29.686 INFO:tasks.workunit.client.0.smithi064.stdout:0/569: dwrite da/d71/f7d [0,4194304] 0 2024-02-16T22:56:29.687 INFO:tasks.workunit.client.1.smithi192.stdout:0/703: fsync d1/d3/de/d22/fd2 0 2024-02-16T22:56:29.690 INFO:tasks.workunit.client.0.smithi064.stdout:7/639: symlink d5/d7d/lc7 0 2024-02-16T22:56:29.695 INFO:tasks.workunit.client.0.smithi064.stdout:2/692: read d7/d14/d1e/f81 [181715,4424] 0 2024-02-16T22:56:29.700 INFO:tasks.workunit.client.0.smithi064.stdout:1/572: mkdir d2/df/d14/d18/d48/d6b/dc7 0 2024-02-16T22:56:29.702 INFO:tasks.workunit.client.0.smithi064.stdout:4/612: dread dc/dd/d32/d58/f38 [0,4194304] 0 2024-02-16T22:56:29.702 INFO:tasks.workunit.client.0.smithi064.stdout:1/573: write d2/df/d11/d1a/d30/dbd/fb7 [247831,59556] 0 2024-02-16T22:56:29.707 INFO:tasks.workunit.client.0.smithi064.stdout:9/557: symlink d3/d11/lbe 0 2024-02-16T22:56:29.712 INFO:tasks.workunit.client.1.smithi192.stdout:3/674: rename da/d11/d63/d6b/d71 to da/d11/dd4/deb 0 2024-02-16T22:56:29.713 INFO:tasks.workunit.client.0.smithi064.stdout:1/574: dwrite d2/df/d14/d18/d48/d84/fa4 [0,4194304] 0 2024-02-16T22:56:29.716 INFO:tasks.workunit.client.0.smithi064.stdout:1/575: chown d2/df/d14/d18/d48/d6b/d75/f8c 457202 1 2024-02-16T22:56:29.716 INFO:tasks.workunit.client.0.smithi064.stdout:4/613: dwrite dc/dd/d16/fa6 [0,4194304] 0 2024-02-16T22:56:29.718 INFO:tasks.workunit.client.0.smithi064.stdout:3/719: write d4/d13/d1b/d27/d99/d7d/f9c [816264,99665] 0 2024-02-16T22:56:29.723 INFO:tasks.workunit.client.1.smithi192.stdout:1/697: mknod d3/de/d27/d34/ced 0 2024-02-16T22:56:29.726 INFO:tasks.workunit.client.0.smithi064.stdout:8/586: symlink df/lc6 0 2024-02-16T22:56:29.728 INFO:tasks.workunit.client.0.smithi064.stdout:2/693: mknod d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/ce2 0 2024-02-16T22:56:29.728 INFO:tasks.workunit.client.0.smithi064.stdout:8/587: write df/d31/d2f/d3e/f83 [137918,117463] 0 2024-02-16T22:56:29.729 INFO:tasks.workunit.client.0.smithi064.stdout:2/694: fdatasync d7/d14/d25/d46/d55/fda 0 2024-02-16T22:56:29.729 INFO:tasks.workunit.client.1.smithi192.stdout:0/704: chown d1/d3/de/d22/d37/fa0 4351599 1 2024-02-16T22:56:29.731 INFO:tasks.workunit.client.1.smithi192.stdout:5/727: mknod d6/d12/d4e/d5b/cf4 0 2024-02-16T22:56:29.735 INFO:tasks.workunit.client.1.smithi192.stdout:7/826: truncate d1/d2/de/d2c/d49/d54/d9d/f111 3938702 0 2024-02-16T22:56:29.737 INFO:tasks.workunit.client.0.smithi064.stdout:6/657: dwrite d0/d1d/d21/f2d [0,4194304] 0 2024-02-16T22:56:29.737 INFO:tasks.workunit.client.0.smithi064.stdout:9/558: stat d3/d9/cd 0 2024-02-16T22:56:29.738 INFO:tasks.workunit.client.1.smithi192.stdout:9/843: getdents d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/da2 0 2024-02-16T22:56:29.741 INFO:tasks.workunit.client.1.smithi192.stdout:4/761: link d1/d26/d6f/f8f d1/d15/d48/d8d/d9b/f108 0 2024-02-16T22:56:29.743 INFO:tasks.workunit.client.0.smithi064.stdout:5/547: write d6/d18/d19/d1a/f69 [1249303,81360] 0 2024-02-16T22:56:29.743 INFO:tasks.workunit.client.0.smithi064.stdout:0/570: dread da/dc/de/d12/d1d/f34 [0,4194304] 0 2024-02-16T22:56:29.744 INFO:tasks.workunit.client.1.smithi192.stdout:6/758: creat da/d76/dc8/ff4 x:0 0 0 2024-02-16T22:56:29.746 INFO:tasks.workunit.client.0.smithi064.stdout:4/614: creat dc/de/d28/fbf x:0 0 0 2024-02-16T22:56:29.747 INFO:tasks.workunit.client.1.smithi192.stdout:3/675: mknod da/d11/d63/d6b/d80/cec 0 2024-02-16T22:56:29.749 INFO:tasks.workunit.client.1.smithi192.stdout:4/762: dwrite d1/d15/d29/dd4/ff5 [0,4194304] 0 2024-02-16T22:56:29.750 INFO:tasks.workunit.client.0.smithi064.stdout:1/576: rename d2/df/d14/l20 to d2/df/d11/dba/lc8 0 2024-02-16T22:56:29.750 INFO:tasks.workunit.client.1.smithi192.stdout:6/759: dread da/df/d6b/d7d/fdc [0,4194304] 0 2024-02-16T22:56:29.751 INFO:tasks.workunit.client.1.smithi192.stdout:2/781: creat d22/d47/dcb/d78/ded/ffc x:0 0 0 2024-02-16T22:56:29.751 INFO:tasks.workunit.client.0.smithi064.stdout:1/577: chown d2/df/d11/d25/fb9 215012 1 2024-02-16T22:56:29.751 INFO:tasks.workunit.client.0.smithi064.stdout:4/615: dread dc/de/d6d/f87 [0,4194304] 0 2024-02-16T22:56:29.754 INFO:tasks.workunit.client.1.smithi192.stdout:1/698: mknod d3/de/d27/d34/d52/d63/d81/cee 0 2024-02-16T22:56:29.760 INFO:tasks.workunit.client.1.smithi192.stdout:8/723: creat d5/d16/dcf/fe2 x:0 0 0 2024-02-16T22:56:29.761 INFO:tasks.workunit.client.0.smithi064.stdout:2/695: creat d7/d14/d1e/fe3 x:0 0 0 2024-02-16T22:56:29.762 INFO:tasks.workunit.client.1.smithi192.stdout:0/705: mknod d1/d3/d29/dcd/ce0 0 2024-02-16T22:56:29.767 INFO:tasks.workunit.client.0.smithi064.stdout:5/548: dread d6/d1c/f59 [0,4194304] 0 2024-02-16T22:56:29.769 INFO:tasks.workunit.client.0.smithi064.stdout:6/658: readlink d0/d6/d10/db2/ld8 0 2024-02-16T22:56:29.769 INFO:tasks.workunit.client.0.smithi064.stdout:2/696: dwrite d7/d14/d25/d46/d55/d7f/f99 [0,4194304] 0 2024-02-16T22:56:29.769 INFO:tasks.workunit.client.0.smithi064.stdout:5/549: chown d6/d1c/d43/f58 2167 1 2024-02-16T22:56:29.770 INFO:tasks.workunit.client.1.smithi192.stdout:9/844: write d7/d19/f112 [957043,77665] 0 2024-02-16T22:56:29.772 INFO:tasks.workunit.client.0.smithi064.stdout:5/550: write d6/d1c/d4b/f94 [4812254,2257] 0 2024-02-16T22:56:29.778 INFO:tasks.workunit.client.1.smithi192.stdout:9/845: dwrite d7/d19/d37/d39/f44 [0,4194304] 0 2024-02-16T22:56:29.780 INFO:tasks.workunit.client.1.smithi192.stdout:9/846: write d7/d19/f112 [1564228,65495] 0 2024-02-16T22:56:29.780 INFO:tasks.workunit.client.1.smithi192.stdout:9/847: chown d7/d15/c30 0 1 2024-02-16T22:56:29.782 INFO:tasks.workunit.client.1.smithi192.stdout:3/676: mkdir da/d11/dd4/deb/da1/db9/ded 0 2024-02-16T22:56:29.784 INFO:tasks.workunit.client.0.smithi064.stdout:4/616: mknod dc/dd/d16/d6e/cc0 0 2024-02-16T22:56:29.785 INFO:tasks.workunit.client.1.smithi192.stdout:8/724: dread d5/d16/dd5/d9d/fa8 [0,4194304] 0 2024-02-16T22:56:29.788 INFO:tasks.workunit.client.0.smithi064.stdout:9/559: write f1 [1276421,107709] 0 2024-02-16T22:56:29.789 INFO:tasks.workunit.client.1.smithi192.stdout:8/725: dread d5/f84 [0,4194304] 0 2024-02-16T22:56:29.791 INFO:tasks.workunit.client.1.smithi192.stdout:7/827: write d1/d2/de/d2c/d49/d54/dd8/fe8 [249681,80722] 0 2024-02-16T22:56:29.793 INFO:tasks.workunit.client.0.smithi064.stdout:7/640: creat d5/d33/d45/fc8 x:0 0 0 2024-02-16T22:56:29.793 INFO:tasks.workunit.client.0.smithi064.stdout:4/617: dwrite dc/dd/d16/f5d [0,4194304] 0 2024-02-16T22:56:29.793 INFO:tasks.workunit.client.1.smithi192.stdout:2/782: creat d22/d24/d39/d3d/ffd x:0 0 0 2024-02-16T22:56:29.797 INFO:tasks.workunit.client.1.smithi192.stdout:1/699: creat d3/de/d27/d34/d69/fef x:0 0 0 2024-02-16T22:56:29.798 INFO:tasks.workunit.client.1.smithi192.stdout:1/700: fsync d3/de/d27/d34/d36/d47/f79 0 2024-02-16T22:56:29.799 INFO:tasks.workunit.client.0.smithi064.stdout:9/560: read f1 [158893,81901] 0 2024-02-16T22:56:29.800 INFO:tasks.workunit.client.0.smithi064.stdout:9/561: fsync d3/d9/ff 0 2024-02-16T22:56:29.800 INFO:tasks.workunit.client.0.smithi064.stdout:9/562: readlink d3/d9/d6a/db1/d38/la7 0 2024-02-16T22:56:29.804 INFO:tasks.workunit.client.0.smithi064.stdout:6/659: mknod d0/d5/df/cdc 0 2024-02-16T22:56:29.806 INFO:tasks.workunit.client.0.smithi064.stdout:2/697: unlink d7/d14/d25/d46/c96 0 2024-02-16T22:56:29.810 INFO:tasks.workunit.client.0.smithi064.stdout:5/551: unlink d6/d1c/l6b 0 2024-02-16T22:56:29.814 INFO:tasks.workunit.client.0.smithi064.stdout:8/588: write df/d2a/f57 [331664,105545] 0 2024-02-16T22:56:29.815 INFO:tasks.workunit.client.0.smithi064.stdout:8/589: read df/d31/d2f/d38/f5d [2396055,71504] 0 2024-02-16T22:56:29.816 INFO:tasks.workunit.client.1.smithi192.stdout:9/848: rename d7/d19/d37/dbf/le4 to d7/d15/d36/d77/l11f 0 2024-02-16T22:56:29.817 INFO:tasks.workunit.client.0.smithi064.stdout:8/590: dread df/f15 [0,4194304] 0 2024-02-16T22:56:29.817 INFO:tasks.workunit.client.1.smithi192.stdout:4/763: symlink d1/d15/d48/d8d/def/l109 0 2024-02-16T22:56:29.819 INFO:tasks.workunit.client.0.smithi064.stdout:3/720: creat d4/d13/d1b/ff3 x:0 0 0 2024-02-16T22:56:29.824 INFO:tasks.workunit.client.1.smithi192.stdout:6/760: symlink da/df/d6b/lf5 0 2024-02-16T22:56:29.826 INFO:tasks.workunit.client.0.smithi064.stdout:3/721: dwrite d4/d13/d1b/d27/d44/d97/fdc [0,4194304] 0 2024-02-16T22:56:29.827 INFO:tasks.workunit.client.1.smithi192.stdout:7/828: creat d1/d2/de/d2c/d49/d54/d9d/f11c x:0 0 0 2024-02-16T22:56:29.829 INFO:tasks.workunit.client.1.smithi192.stdout:4/764: dread d1/d26/d66/d89/f9 [0,4194304] 0 2024-02-16T22:56:29.829 INFO:tasks.workunit.client.1.smithi192.stdout:1/701: mkdir d3/d40/d9f/df0 0 2024-02-16T22:56:29.833 INFO:tasks.workunit.client.0.smithi064.stdout:6/660: mkdir d0/d6/d10/d33/da4/ddd 0 2024-02-16T22:56:29.834 INFO:tasks.workunit.client.0.smithi064.stdout:0/571: dwrite da/dc/de/d12/d1d/f2b [0,4194304] 0 2024-02-16T22:56:29.834 INFO:tasks.workunit.client.1.smithi192.stdout:9/849: mkdir d7/d15/d36/d77/db1/d120 0 2024-02-16T22:56:29.836 INFO:tasks.workunit.client.0.smithi064.stdout:2/698: mknod d7/d14/d25/ce4 0 2024-02-16T22:56:29.845 INFO:tasks.workunit.client.0.smithi064.stdout:0/572: dwrite da/dc/d3b/f7b [0,4194304] 0 2024-02-16T22:56:29.845 INFO:tasks.workunit.client.0.smithi064.stdout:6/661: dwrite d0/d6/d10/f9f [0,4194304] 0 2024-02-16T22:56:29.846 INFO:tasks.workunit.client.0.smithi064.stdout:0/573: chown da/d26/f28 72 1 2024-02-16T22:56:29.847 INFO:tasks.workunit.client.1.smithi192.stdout:6/761: dread da/df/d12/f47 [0,4194304] 0 2024-02-16T22:56:29.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/762: chown da/df/d12/d57/f50 3241 1 2024-02-16T22:56:29.849 INFO:tasks.workunit.client.1.smithi192.stdout:5/728: sync 2024-02-16T22:56:29.854 INFO:tasks.workunit.client.1.smithi192.stdout:5/729: read d6/d12/f1e [5823756,118579] 0 2024-02-16T22:56:29.858 INFO:tasks.workunit.client.1.smithi192.stdout:9/850: creat d7/d15/d36/d77/dbd/de0/f121 x:0 0 0 2024-02-16T22:56:29.859 INFO:tasks.workunit.client.0.smithi064.stdout:3/722: creat d4/d13/d4d/d5c/ff4 x:0 0 0 2024-02-16T22:56:29.859 INFO:tasks.workunit.client.0.smithi064.stdout:3/723: fdatasync d4/d13/f62 0 2024-02-16T22:56:29.864 INFO:tasks.workunit.client.0.smithi064.stdout:5/552: write d6/d18/f72 [1172701,61767] 0 2024-02-16T22:56:29.866 INFO:tasks.workunit.client.0.smithi064.stdout:8/591: write df/d31/d2f/d4f/f75 [215578,1985] 0 2024-02-16T22:56:29.869 INFO:tasks.workunit.client.0.smithi064.stdout:7/641: truncate d5/d33/d42/d81/fae 1113101 0 2024-02-16T22:56:29.871 INFO:tasks.workunit.client.0.smithi064.stdout:2/699: symlink d7/d14/d1e/d34/d3f/d9a/dcf/le5 0 2024-02-16T22:56:29.873 INFO:tasks.workunit.client.1.smithi192.stdout:0/706: getdents d1/d3/d29/dcd 0 2024-02-16T22:56:29.875 INFO:tasks.workunit.client.1.smithi192.stdout:8/726: rename d5/d26/d83/db4/l7b to d5/le3 0 2024-02-16T22:56:29.877 INFO:tasks.workunit.client.0.smithi064.stdout:7/642: read d5/d33/d42/f4f [3686054,126000] 0 2024-02-16T22:56:29.883 INFO:tasks.workunit.client.0.smithi064.stdout:6/662: fdatasync d0/d3c/da9/f67 0 2024-02-16T22:56:29.884 INFO:tasks.workunit.client.1.smithi192.stdout:9/851: dread f2 [0,4194304] 0 2024-02-16T22:56:29.884 INFO:tasks.workunit.client.1.smithi192.stdout:1/702: write d3/d5b/d77/f88 [574780,108616] 0 2024-02-16T22:56:29.884 INFO:tasks.workunit.client.0.smithi064.stdout:9/563: truncate d3/d9/d6a/db1/f29 429787 0 2024-02-16T22:56:29.885 INFO:tasks.workunit.client.0.smithi064.stdout:4/618: link dc/d4a/f4d dc/dd/d5b/fc1 0 2024-02-16T22:56:29.886 INFO:tasks.workunit.client.0.smithi064.stdout:9/564: dread - d3/d9/d1e/d42/f9f zero size 2024-02-16T22:56:29.886 INFO:tasks.workunit.client.1.smithi192.stdout:1/703: truncate d3/de/d27/d34/d36/d47/d5d/fe5 236450 0 2024-02-16T22:56:29.889 INFO:tasks.workunit.client.1.smithi192.stdout:4/765: truncate d1/d26/d66/d89/da8/da9/ff2 620491 0 2024-02-16T22:56:29.890 INFO:tasks.workunit.client.1.smithi192.stdout:6/763: dwrite da/df/d12/d57/fc6 [0,4194304] 0 2024-02-16T22:56:29.891 INFO:tasks.workunit.client.1.smithi192.stdout:3/677: sync 2024-02-16T22:56:29.892 INFO:tasks.workunit.client.1.smithi192.stdout:3/678: chown da/d11/d63/d6b/d66/fb4 95546529 1 2024-02-16T22:56:29.895 INFO:tasks.workunit.client.0.smithi064.stdout:3/724: unlink d4/d13/d1b/d27/d99/f89 0 2024-02-16T22:56:29.897 INFO:tasks.workunit.client.1.smithi192.stdout:2/783: rename d22/f3c to d22/d24/d39/d42/d85/d98/ffe 0 2024-02-16T22:56:29.901 INFO:tasks.workunit.client.0.smithi064.stdout:5/553: rename d6/d18/d19/d1a to d6/d25/da4/dc1 0 2024-02-16T22:56:29.903 INFO:tasks.workunit.client.0.smithi064.stdout:5/554: truncate d6/d25/d48/d67/f87 892543 0 2024-02-16T22:56:29.904 INFO:tasks.workunit.client.0.smithi064.stdout:8/592: fsync df/d31/d47/fad 0 2024-02-16T22:56:29.905 INFO:tasks.workunit.client.0.smithi064.stdout:9/565: dread d3/d9/f21 [0,4194304] 0 2024-02-16T22:56:29.909 INFO:tasks.workunit.client.1.smithi192.stdout:7/829: getdents d1/d2/de/d2c/d49/dba 0 2024-02-16T22:56:29.909 INFO:tasks.workunit.client.0.smithi064.stdout:9/566: write d3/d9/d1e/f8d [49190,59786] 0 2024-02-16T22:56:29.910 INFO:tasks.workunit.client.1.smithi192.stdout:7/830: fdatasync d1/d2/de/d2c/d49/d54/d9d/f11c 0 2024-02-16T22:56:29.912 INFO:tasks.workunit.client.1.smithi192.stdout:9/852: mknod d7/d19/d28/d43/c122 0 2024-02-16T22:56:29.914 INFO:tasks.workunit.client.0.smithi064.stdout:5/555: dwrite d6/d1c/f62 [0,4194304] 0 2024-02-16T22:56:29.914 INFO:tasks.workunit.client.0.smithi064.stdout:7/643: mknod d5/d33/d45/d73/d88/cc9 0 2024-02-16T22:56:29.916 INFO:tasks.workunit.client.1.smithi192.stdout:1/704: chown d3/de/c14 24 1 2024-02-16T22:56:29.916 INFO:tasks.workunit.client.0.smithi064.stdout:9/567: write d3/d9/d1e/d42/f9f [833491,92343] 0 2024-02-16T22:56:29.917 INFO:tasks.workunit.client.0.smithi064.stdout:0/574: dwrite da/dc/f1e [0,4194304] 0 2024-02-16T22:56:29.919 INFO:tasks.workunit.client.0.smithi064.stdout:7/644: read - d5/d33/d45/d73/fa4 zero size 2024-02-16T22:56:29.920 INFO:tasks.workunit.client.0.smithi064.stdout:0/575: chown da/da0/fb2 1066802 1 2024-02-16T22:56:29.921 INFO:tasks.workunit.client.1.smithi192.stdout:0/707: dwrite d1/d3/de/d22/f31 [0,4194304] 0 2024-02-16T22:56:29.921 INFO:tasks.workunit.client.0.smithi064.stdout:6/663: fsync d0/d6/f15 0 2024-02-16T22:56:29.923 INFO:tasks.workunit.client.1.smithi192.stdout:0/708: write d1/d3/de/d22/d37/d67/d34/f92 [4852404,5012] 0 2024-02-16T22:56:29.925 INFO:tasks.workunit.client.0.smithi064.stdout:8/593: dwrite df/d31/d2f/d3e/f83 [4194304,4194304] 0 2024-02-16T22:56:29.925 INFO:tasks.workunit.client.1.smithi192.stdout:4/766: mkdir d1/d15/d48/d8d/d9b/d10a 0 2024-02-16T22:56:29.925 INFO:tasks.workunit.client.1.smithi192.stdout:0/709: write d1/d3/de/d22/d37/d67/fb7 [401470,58336] 0 2024-02-16T22:56:29.928 INFO:tasks.workunit.client.1.smithi192.stdout:6/764: fsync da/df/d58/f9a 0 2024-02-16T22:56:29.930 INFO:tasks.workunit.client.1.smithi192.stdout:3/679: write da/d6a/f84 [120601,8473] 0 2024-02-16T22:56:29.935 INFO:tasks.workunit.client.1.smithi192.stdout:8/727: dwrite d5/db/d47/d43/d67/f85 [0,4194304] 0 2024-02-16T22:56:29.936 INFO:tasks.workunit.client.1.smithi192.stdout:6/765: chown da/df/d12/d57/f2f 1 1 2024-02-16T22:56:29.937 INFO:tasks.workunit.client.0.smithi064.stdout:2/700: dwrite d7/d14/d1e/f81 [0,4194304] 0 2024-02-16T22:56:29.941 INFO:tasks.workunit.client.1.smithi192.stdout:5/730: creat d6/d12/d1d/ff5 x:0 0 0 2024-02-16T22:56:29.944 INFO:tasks.workunit.client.1.smithi192.stdout:9/853: symlink d7/d19/d37/d5a/l123 0 2024-02-16T22:56:29.949 INFO:tasks.workunit.client.1.smithi192.stdout:4/767: creat d1/d26/d66/d89/da8/f10b x:0 0 0 2024-02-16T22:56:29.952 INFO:tasks.workunit.client.0.smithi064.stdout:7/645: creat d5/d33/d45/d64/d56/d5d/fca x:0 0 0 2024-02-16T22:56:29.953 INFO:tasks.workunit.client.0.smithi064.stdout:7/646: dread - d5/d6/d54/d7b/fc1 zero size 2024-02-16T22:56:29.956 INFO:tasks.workunit.client.1.smithi192.stdout:4/768: dwrite d1/d15/d48/db5/f106 [0,4194304] 0 2024-02-16T22:56:29.956 INFO:tasks.workunit.client.0.smithi064.stdout:6/664: rename d0/d6/d10/f1c to d0/d1d/dbc/fde 0 2024-02-16T22:56:29.957 INFO:tasks.workunit.client.0.smithi064.stdout:4/619: creat dc/d1d/dbd/fc2 x:0 0 0 2024-02-16T22:56:29.959 INFO:tasks.workunit.client.1.smithi192.stdout:6/766: mknod da/d40/d85/cf6 0 2024-02-16T22:56:29.965 INFO:tasks.workunit.client.1.smithi192.stdout:5/731: rename d6/d12/d4e/d5b/d67/d78/dae to d6/d12/d2d/d5c/df6 0 2024-02-16T22:56:29.966 INFO:tasks.workunit.client.1.smithi192.stdout:5/732: chown d6/d12/d1d/lab 87678 1 2024-02-16T22:56:29.966 INFO:tasks.workunit.client.0.smithi064.stdout:2/701: creat d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/dd6/fe6 x:0 0 0 2024-02-16T22:56:29.967 INFO:tasks.workunit.client.1.smithi192.stdout:5/733: chown d6/d12/d4e/d58 3 1 2024-02-16T22:56:29.967 INFO:tasks.workunit.client.1.smithi192.stdout:9/854: creat d7/d15/d36/d77/f124 x:0 0 0 2024-02-16T22:56:29.967 INFO:tasks.workunit.client.0.smithi064.stdout:2/702: read - d7/d14/d1e/d34/d73/db5/fbd zero size 2024-02-16T22:56:29.968 INFO:tasks.workunit.client.0.smithi064.stdout:9/568: truncate d3/d9/f48 964444 0 2024-02-16T22:56:29.968 INFO:tasks.workunit.client.0.smithi064.stdout:9/569: chown d3/d11 317744492 1 2024-02-16T22:56:29.969 INFO:tasks.workunit.client.0.smithi064.stdout:9/570: write d3/d9/d1e/d3e/d76/f79 [694172,28281] 0 2024-02-16T22:56:29.970 INFO:tasks.workunit.client.0.smithi064.stdout:0/576: truncate f3 1050729 0 2024-02-16T22:56:29.974 INFO:tasks.workunit.client.1.smithi192.stdout:7/831: creat d1/d2/d2d/d61/f11d x:0 0 0 2024-02-16T22:56:29.976 INFO:tasks.workunit.client.1.smithi192.stdout:5/734: readlink d6/d12/d4e/d58/l6a 0 2024-02-16T22:56:29.982 INFO:tasks.workunit.client.1.smithi192.stdout:6/767: creat da/db9/ff7 x:0 0 0 2024-02-16T22:56:29.984 INFO:tasks.workunit.client.1.smithi192.stdout:7/832: truncate d1/d2/de/d2c/d49/d89/f97 468900 0 2024-02-16T22:56:29.987 INFO:tasks.workunit.client.1.smithi192.stdout:0/710: getdents d1/d3/de/d22 0 2024-02-16T22:56:29.988 INFO:tasks.workunit.client.1.smithi192.stdout:6/768: truncate da/df/d12/d15/f21 1404784 0 2024-02-16T22:56:29.990 INFO:tasks.workunit.client.1.smithi192.stdout:7/833: write d1/d2/d37/fe7 [154919,108390] 0 2024-02-16T22:56:29.992 INFO:tasks.workunit.client.1.smithi192.stdout:4/769: rename d1/d26/c32 to d1/d26/d66/d89/c10c 0 2024-02-16T22:56:29.996 INFO:tasks.workunit.client.1.smithi192.stdout:2/784: write d22/d24/d39/d3d/fbb [114002,113110] 0 2024-02-16T22:56:29.997 INFO:tasks.workunit.client.1.smithi192.stdout:0/711: mkdir d1/d3/de/d22/d37/d47/dd0/de1 0 2024-02-16T22:56:29.998 INFO:tasks.workunit.client.1.smithi192.stdout:0/712: readlink d1/d3/de/l84 0 2024-02-16T22:56:29.999 INFO:tasks.workunit.client.1.smithi192.stdout:4/770: dwrite d1/d26/d66/d89/d11/fb7 [0,4194304] 0 2024-02-16T22:56:29.999 INFO:tasks.workunit.client.1.smithi192.stdout:7/834: fsync d1/d2/de/d2c/d49/d54/f67 0 2024-02-16T22:56:29.999 INFO:tasks.workunit.client.1.smithi192.stdout:0/713: chown d1/fcb 129635797 1 2024-02-16T22:56:30.000 INFO:tasks.workunit.client.1.smithi192.stdout:4/771: readlink d1/d26/d66/d89/la2 0 2024-02-16T22:56:30.000 INFO:tasks.workunit.client.1.smithi192.stdout:2/785: symlink d22/lff 0 2024-02-16T22:56:30.001 INFO:tasks.workunit.client.1.smithi192.stdout:1/705: write d3/de/d27/f44 [1830243,57086] 0 2024-02-16T22:56:30.005 INFO:tasks.workunit.client.1.smithi192.stdout:4/772: fsync d1/d26/d6f/fd8 0 2024-02-16T22:56:30.007 INFO:tasks.workunit.client.1.smithi192.stdout:1/706: symlink d3/de/d27/d34/d69/d85/d89/daa/lf1 0 2024-02-16T22:56:30.008 INFO:tasks.workunit.client.1.smithi192.stdout:2/786: dwrite d22/d47/dcb/f2d [0,4194304] 0 2024-02-16T22:56:30.009 INFO:tasks.workunit.client.1.smithi192.stdout:1/707: chown d3/de/d27/d34/d52/d63/l6f 2122 1 2024-02-16T22:56:30.011 INFO:tasks.workunit.client.1.smithi192.stdout:3/680: write da/d11/d20/f24 [3745896,70057] 0 2024-02-16T22:56:30.014 INFO:tasks.workunit.client.1.smithi192.stdout:0/714: getdents d1/d3/d29/dcd 0 2024-02-16T22:56:30.020 INFO:tasks.workunit.client.0.smithi064.stdout:4/620: creat dc/dd/d32/d46/d82/da1/fc3 x:0 0 0 2024-02-16T22:56:30.020 INFO:tasks.workunit.client.0.smithi064.stdout:6/665: write d0/d6/f15 [3010920,83694] 0 2024-02-16T22:56:30.021 INFO:tasks.workunit.client.1.smithi192.stdout:3/681: chown da/l2c 952 1 2024-02-16T22:56:30.025 INFO:tasks.workunit.client.1.smithi192.stdout:8/728: sync 2024-02-16T22:56:30.029 INFO:tasks.workunit.client.1.smithi192.stdout:2/787: mknod d22/d24/d39/d42/d52/c100 0 2024-02-16T22:56:30.031 INFO:tasks.workunit.client.1.smithi192.stdout:7/835: link d1/d2/fb4 d1/d2/d2d/d61/d8a/de6/f11e 0 2024-02-16T22:56:30.032 INFO:tasks.workunit.client.1.smithi192.stdout:9/855: write d7/d19/d28/d43/d70/df0/f96 [31710,124476] 0 2024-02-16T22:56:30.033 INFO:tasks.workunit.client.1.smithi192.stdout:3/682: creat da/d11/dd4/deb/da1/db9/fee x:0 0 0 2024-02-16T22:56:30.034 INFO:tasks.workunit.client.1.smithi192.stdout:4/773: dread d1/d26/d34/f60 [0,4194304] 0 2024-02-16T22:56:30.037 INFO:tasks.workunit.client.1.smithi192.stdout:5/735: write d6/d12/f18 [1527038,29322] 0 2024-02-16T22:56:30.037 INFO:tasks.workunit.client.1.smithi192.stdout:8/729: write d5/db/de/df/fb7 [638390,129260] 0 2024-02-16T22:56:30.039 INFO:tasks.workunit.client.1.smithi192.stdout:0/715: fsync d1/d3/de/d22/d37/d67/d75/f80 0 2024-02-16T22:56:30.039 INFO:tasks.workunit.client.1.smithi192.stdout:8/730: chown d5/db/d47/d43/c77 1805964677 1 2024-02-16T22:56:30.041 INFO:tasks.workunit.client.1.smithi192.stdout:2/788: creat d22/d24/d39/d42/d84/f101 x:0 0 0 2024-02-16T22:56:30.044 INFO:tasks.workunit.client.1.smithi192.stdout:7/836: chown d1/de1/c76 2707774 1 2024-02-16T22:56:30.046 INFO:tasks.workunit.client.1.smithi192.stdout:9/856: rename d7/d19/d28/d43/l50 to d7/d19/d5e/d81/l125 0 2024-02-16T22:56:30.047 INFO:tasks.workunit.client.1.smithi192.stdout:7/837: stat d1/d2/de/d2c/d49/d54/de3/ld6 0 2024-02-16T22:56:30.047 INFO:tasks.workunit.client.1.smithi192.stdout:3/683: rmdir da/d11/d20/d39/d53 39 2024-02-16T22:56:30.048 INFO:tasks.workunit.client.1.smithi192.stdout:0/716: dwrite d1/d3/de/d22/d37/d67/d34/fc1 [0,4194304] 0 2024-02-16T22:56:30.049 INFO:tasks.workunit.client.1.smithi192.stdout:8/731: dread d5/db/de/f80 [0,4194304] 0 2024-02-16T22:56:30.051 INFO:tasks.workunit.client.1.smithi192.stdout:6/769: dwrite da/d40/f4e [0,4194304] 0 2024-02-16T22:56:30.051 INFO:tasks.workunit.client.1.smithi192.stdout:5/736: rmdir d6/d12/d4e 39 2024-02-16T22:56:30.053 INFO:tasks.workunit.client.1.smithi192.stdout:5/737: chown d6/d8 225931081 1 2024-02-16T22:56:30.053 INFO:tasks.workunit.client.1.smithi192.stdout:7/838: write d1/d2/de/d2c/d49/d54/de3/f119 [282648,124218] 0 2024-02-16T22:56:30.053 INFO:tasks.workunit.client.1.smithi192.stdout:4/774: dread d1/d26/fd7 [0,4194304] 0 2024-02-16T22:56:30.055 INFO:tasks.workunit.client.1.smithi192.stdout:4/775: read - d1/d26/d34/ffd zero size 2024-02-16T22:56:30.059 INFO:tasks.workunit.client.1.smithi192.stdout:6/770: dwrite da/df/d12/d15/d1b/faa [0,4194304] 0 2024-02-16T22:56:30.062 INFO:tasks.workunit.client.1.smithi192.stdout:2/789: truncate d22/d47/f88 501408 0 2024-02-16T22:56:30.066 INFO:tasks.workunit.client.1.smithi192.stdout:3/684: rmdir da 39 2024-02-16T22:56:30.069 INFO:tasks.workunit.client.1.smithi192.stdout:2/790: dwrite d22/d24/d39/fb7 [0,4194304] 0 2024-02-16T22:56:30.070 INFO:tasks.workunit.client.1.smithi192.stdout:1/708: write d3/de/d27/f30 [2564803,116519] 0 2024-02-16T22:56:30.071 INFO:tasks.workunit.client.1.smithi192.stdout:8/732: creat d5/d16/fe4 x:0 0 0 2024-02-16T22:56:30.075 INFO:tasks.workunit.client.1.smithi192.stdout:0/717: write d1/d3/de/d22/d37/d67/d34/da1/fab [603471,100326] 0 2024-02-16T22:56:30.076 INFO:tasks.workunit.client.1.smithi192.stdout:5/738: readlink d6/d20/d93/lbd 0 2024-02-16T22:56:30.079 INFO:tasks.workunit.client.1.smithi192.stdout:7/839: mknod d1/d2/de/d83/d9f/c11f 0 2024-02-16T22:56:30.083 INFO:tasks.workunit.client.1.smithi192.stdout:7/840: dread d1/d2/de/d2c/d49/dba/ff4 [0,4194304] 0 2024-02-16T22:56:30.092 INFO:tasks.workunit.client.1.smithi192.stdout:3/685: write da/df/f2e [4042219,68986] 0 2024-02-16T22:56:30.092 INFO:tasks.workunit.client.1.smithi192.stdout:2/791: mkdir d22/d24/d39/d42/d65/d66/d102 0 2024-02-16T22:56:30.093 INFO:tasks.workunit.client.1.smithi192.stdout:3/686: read da/d11/f4b [743004,16962] 0 2024-02-16T22:56:30.098 INFO:tasks.workunit.client.1.smithi192.stdout:1/709: dread d3/f21 [0,4194304] 0 2024-02-16T22:56:30.098 INFO:tasks.workunit.client.1.smithi192.stdout:0/718: mkdir d1/d3/de2 0 2024-02-16T22:56:30.101 INFO:tasks.workunit.client.1.smithi192.stdout:7/841: dread d1/d2/de/d83/f8d [0,4194304] 0 2024-02-16T22:56:30.101 INFO:tasks.workunit.client.1.smithi192.stdout:5/739: read d6/d12/d4e/d58/fbb [2818362,100563] 0 2024-02-16T22:56:30.103 INFO:tasks.workunit.client.1.smithi192.stdout:5/740: read d6/d20/d93/fc3 [491816,41618] 0 2024-02-16T22:56:30.104 INFO:tasks.workunit.client.1.smithi192.stdout:3/687: rename da/l2c to da/d6a/lef 0 2024-02-16T22:56:30.104 INFO:tasks.workunit.client.1.smithi192.stdout:3/688: fsync da/d11/f50 0 2024-02-16T22:56:30.105 INFO:tasks.workunit.client.1.smithi192.stdout:9/857: getdents d7/d19/d28/d43/d70/d8e 0 2024-02-16T22:56:30.109 INFO:tasks.workunit.client.1.smithi192.stdout:0/719: rmdir d1/d3/de/d22/d37/d67/d34/da1 39 2024-02-16T22:56:30.110 INFO:tasks.workunit.client.1.smithi192.stdout:9/858: dwrite d7/d19/d5e/fcd [0,4194304] 0 2024-02-16T22:56:30.111 INFO:tasks.workunit.client.1.smithi192.stdout:4/776: getdents d1/d15/d48/dc3 0 2024-02-16T22:56:30.113 INFO:tasks.workunit.client.1.smithi192.stdout:7/842: mknod d1/d2/de/d2c/d49/d54/d9d/c120 0 2024-02-16T22:56:30.116 INFO:tasks.workunit.client.1.smithi192.stdout:6/771: getdents da/df 0 2024-02-16T22:56:30.117 INFO:tasks.workunit.client.1.smithi192.stdout:6/772: read da/df/d71/fd0 [427779,110003] 0 2024-02-16T22:56:30.118 INFO:tasks.workunit.client.1.smithi192.stdout:6/773: readlink da/df/d12/d57/d81/lca 0 2024-02-16T22:56:30.119 INFO:tasks.workunit.client.1.smithi192.stdout:3/689: truncate da/d6a/fc6 98458 0 2024-02-16T22:56:30.119 INFO:tasks.workunit.client.1.smithi192.stdout:6/774: chown da/df/d12 5 1 2024-02-16T22:56:30.120 INFO:tasks.workunit.client.1.smithi192.stdout:3/690: chown da/d11/d20/d39/ccb 4055623 1 2024-02-16T22:56:30.120 INFO:tasks.workunit.client.1.smithi192.stdout:8/733: getdents d5/d26/d83/d86/d5f/dbf 0 2024-02-16T22:56:30.121 INFO:tasks.workunit.client.1.smithi192.stdout:8/734: write d5/d26/d83/fdb [383981,30987] 0 2024-02-16T22:56:30.122 INFO:tasks.workunit.client.1.smithi192.stdout:1/710: creat d3/de/d27/d34/d52/db7/ff2 x:0 0 0 2024-02-16T22:56:30.125 INFO:tasks.workunit.client.1.smithi192.stdout:9/859: mknod d7/d19/d5b/c126 0 2024-02-16T22:56:30.127 INFO:tasks.workunit.client.1.smithi192.stdout:7/843: truncate d1/d2/de/d2c/d49/d54/d6a/f95 2598183 0 2024-02-16T22:56:30.129 INFO:tasks.workunit.client.1.smithi192.stdout:6/775: unlink da/df/d6b/l99 0 2024-02-16T22:56:30.131 INFO:tasks.workunit.client.1.smithi192.stdout:8/735: rename d5/d9/f6b to d5/d26/d83/fe5 0 2024-02-16T22:56:30.132 INFO:tasks.workunit.client.1.smithi192.stdout:2/792: getdents d22/d47/dcb/d3f/dac 0 2024-02-16T22:56:30.133 INFO:tasks.workunit.client.1.smithi192.stdout:8/736: write d5/d9/f73 [2278620,122889] 0 2024-02-16T22:56:30.133 INFO:tasks.workunit.client.1.smithi192.stdout:7/844: dread d1/fc1 [0,4194304] 0 2024-02-16T22:56:30.134 INFO:tasks.workunit.client.1.smithi192.stdout:0/720: truncate d1/f1f 1454272 0 2024-02-16T22:56:30.134 INFO:tasks.workunit.client.0.smithi064.stdout:3/725: write d4/d1f/f67 [1836245,121474] 0 2024-02-16T22:56:30.136 INFO:tasks.workunit.client.0.smithi064.stdout:4/621: dread - dc/dd/d5b/fb2 zero size 2024-02-16T22:56:30.136 INFO:tasks.workunit.client.1.smithi192.stdout:8/737: write d5/db/de/df/fb7 [1517072,88767] 0 2024-02-16T22:56:30.137 INFO:tasks.workunit.client.1.smithi192.stdout:9/860: dread d7/d19/d5e/d81/dda/f87 [0,4194304] 0 2024-02-16T22:56:30.138 INFO:tasks.workunit.client.0.smithi064.stdout:0/577: dread da/dc/de/d12/d1d/f1f [4194304,4194304] 0 2024-02-16T22:56:30.139 INFO:tasks.workunit.client.0.smithi064.stdout:2/703: write d7/d14/d1e/d34/d3f/f79 [75328,34404] 0 2024-02-16T22:56:30.139 INFO:tasks.workunit.client.1.smithi192.stdout:9/861: truncate d7/d15/d36/d77/f124 671931 0 2024-02-16T22:56:30.141 INFO:tasks.workunit.client.0.smithi064.stdout:0/578: chown da/dc/d3b/d72/d7c/fad 123403 1 2024-02-16T22:56:30.142 INFO:tasks.workunit.client.0.smithi064.stdout:2/704: dread - d7/d14/d1e/d34/d65/f67 zero size 2024-02-16T22:56:30.143 INFO:tasks.workunit.client.1.smithi192.stdout:6/776: creat da/df/d6b/d74/ff8 x:0 0 0 2024-02-16T22:56:30.143 INFO:tasks.workunit.client.1.smithi192.stdout:4/777: write d1/d26/d6f/df4/f105 [818506,105830] 0 2024-02-16T22:56:30.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/777: dread - da/df/d12/d15/d1b/fbf zero size 2024-02-16T22:56:30.145 INFO:tasks.workunit.client.1.smithi192.stdout:6/778: write da/df/d18/d46/ff1 [211068,3796] 0 2024-02-16T22:56:30.145 INFO:tasks.workunit.client.0.smithi064.stdout:6/666: rename d0/d1b/f7a to d0/d3c/fdf 0 2024-02-16T22:56:30.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/691: rename da/d11/d20/d39/d53/dc7/f88 to da/d11/dd4/deb/ff0 0 2024-02-16T22:56:30.150 INFO:tasks.workunit.client.0.smithi064.stdout:0/579: dwrite da/dc/de/d12/f49 [0,4194304] 0 2024-02-16T22:56:30.154 INFO:tasks.workunit.client.0.smithi064.stdout:7/647: dread d5/d33/f57 [0,4194304] 0 2024-02-16T22:56:30.156 INFO:tasks.workunit.client.0.smithi064.stdout:9/571: dread d3/f63 [0,4194304] 0 2024-02-16T22:56:30.158 INFO:tasks.workunit.client.1.smithi192.stdout:1/711: truncate d3/f22 8251315 0 2024-02-16T22:56:30.159 INFO:tasks.workunit.client.0.smithi064.stdout:3/726: mkdir d4/d13/d1b/d27/d61/dd8/df5 0 2024-02-16T22:56:30.161 INFO:tasks.workunit.client.1.smithi192.stdout:2/793: mkdir d22/d47/dcb/dc9/d103 0 2024-02-16T22:56:30.164 INFO:tasks.workunit.client.1.smithi192.stdout:5/741: getdents d6/d12/d1d 0 2024-02-16T22:56:30.167 INFO:tasks.workunit.client.0.smithi064.stdout:2/705: creat d7/d14/d25/d46/d55/d7f/fe7 x:0 0 0 2024-02-16T22:56:30.168 INFO:tasks.workunit.client.1.smithi192.stdout:7/845: mknod d1/d2/de/d2c/d49/da1/c121 0 2024-02-16T22:56:30.169 INFO:tasks.workunit.client.0.smithi064.stdout:6/667: symlink d0/d1d/le0 0 2024-02-16T22:56:30.171 INFO:tasks.workunit.client.0.smithi064.stdout:0/580: creat da/dc/de/d12/d2c/d2e/d38/d7e/fc4 x:0 0 0 2024-02-16T22:56:30.173 INFO:tasks.workunit.client.1.smithi192.stdout:8/738: dread d5/d26/f88 [0,4194304] 0 2024-02-16T22:56:30.174 INFO:tasks.workunit.client.1.smithi192.stdout:0/721: write d1/d3/de/d22/f45 [1982175,99326] 0 2024-02-16T22:56:30.175 INFO:tasks.workunit.client.1.smithi192.stdout:0/722: read d1/d3/de/d22/d37/d67/f66 [206503,114526] 0 2024-02-16T22:56:30.175 INFO:tasks.workunit.client.0.smithi064.stdout:7/648: read d5/d33/d42/d81/fae [712374,68029] 0 2024-02-16T22:56:30.177 INFO:tasks.workunit.client.1.smithi192.stdout:6/779: creat da/db9/ff9 x:0 0 0 2024-02-16T22:56:30.179 INFO:tasks.workunit.client.1.smithi192.stdout:4/778: rename d1/d15/d29/f8b to d1/d26/d34/db9/f10d 0 2024-02-16T22:56:30.179 INFO:tasks.workunit.client.0.smithi064.stdout:2/706: rmdir d7/d14/d25/d46/d55/d57 39 2024-02-16T22:56:30.180 INFO:tasks.workunit.client.0.smithi064.stdout:6/668: readlink d0/d5/l45 0 2024-02-16T22:56:30.181 INFO:tasks.workunit.client.1.smithi192.stdout:0/723: dwrite d1/d3/de/d22/d37/d67/d75/db0/fb2 [0,4194304] 0 2024-02-16T22:56:30.182 INFO:tasks.workunit.client.0.smithi064.stdout:3/727: dwrite d4/d13/d1b/d27/f7f [0,4194304] 0 2024-02-16T22:56:30.183 INFO:tasks.workunit.client.0.smithi064.stdout:0/581: creat da/dc/de/d12/d2c/fc5 x:0 0 0 2024-02-16T22:56:30.186 INFO:tasks.workunit.client.0.smithi064.stdout:0/582: stat da/dc/de/d12/d2c/d2e/d38/d7e/c97 0 2024-02-16T22:56:30.188 INFO:tasks.workunit.client.0.smithi064.stdout:9/572: unlink d3/ce 0 2024-02-16T22:56:30.188 INFO:tasks.workunit.client.0.smithi064.stdout:2/707: dwrite d7/d14/d25/f90 [0,4194304] 0 2024-02-16T22:56:30.237 INFO:tasks.workunit.client.0.smithi064.stdout:4/622: getdents dc/de/d28 0 2024-02-16T22:56:30.240 INFO:tasks.workunit.client.0.smithi064.stdout:2/708: rmdir d7 39 2024-02-16T22:56:30.241 INFO:tasks.workunit.client.1.smithi192.stdout:9/862: mknod d7/d19/d28/d43/d70/c127 0 2024-02-16T22:56:30.244 INFO:tasks.workunit.client.0.smithi064.stdout:7/649: write d5/f25 [2332430,19152] 0 2024-02-16T22:56:30.247 INFO:tasks.workunit.client.1.smithi192.stdout:6/780: mkdir da/d40/db6/db8/dfa 0 2024-02-16T22:56:30.248 INFO:tasks.workunit.client.1.smithi192.stdout:6/781: chown da/df/d12/d15/f33 165 1 2024-02-16T22:56:30.248 INFO:tasks.workunit.client.0.smithi064.stdout:6/669: truncate d0/d1d/f9e 2013269 0 2024-02-16T22:56:30.249 INFO:tasks.workunit.client.1.smithi192.stdout:7/846: dwrite d1/d2/d2d/d61/d88/d90/f64 [0,4194304] 0 2024-02-16T22:56:30.249 INFO:tasks.workunit.client.0.smithi064.stdout:6/670: fdatasync d0/d89/fb6 0 2024-02-16T22:56:30.250 INFO:tasks.workunit.client.1.smithi192.stdout:3/692: rename da/d11/d20/cb2 to da/d11/d20/d39/d53/cf1 0 2024-02-16T22:56:30.251 INFO:tasks.workunit.client.0.smithi064.stdout:7/650: dwrite d5/d6/d65/d7f/fba [0,4194304] 0 2024-02-16T22:56:30.252 INFO:tasks.workunit.client.1.smithi192.stdout:4/779: symlink d1/d26/d66/d89/d11/l10e 0 2024-02-16T22:56:30.256 INFO:tasks.workunit.client.0.smithi064.stdout:0/583: mkdir da/dc/de/d12/d2c/d2e/d38/dbe/dc6 0 2024-02-16T22:56:30.260 INFO:tasks.workunit.client.1.smithi192.stdout:2/794: symlink d22/d24/d39/d42/d52/l104 0 2024-02-16T22:56:30.260 INFO:tasks.workunit.client.0.smithi064.stdout:2/709: chown d7/d14/d25/la9 0 1 2024-02-16T22:56:30.264 INFO:tasks.workunit.client.0.smithi064.stdout:9/573: dread d3/d11/d15/d23/f73 [0,4194304] 0 2024-02-16T22:56:30.266 INFO:tasks.workunit.client.1.smithi192.stdout:0/724: write d1/f24 [1127762,37464] 0 2024-02-16T22:56:30.266 INFO:tasks.workunit.client.1.smithi192.stdout:5/742: sync 2024-02-16T22:56:30.268 INFO:tasks.workunit.client.0.smithi064.stdout:6/671: symlink d0/d12/le1 0 2024-02-16T22:56:30.268 INFO:tasks.workunit.client.1.smithi192.stdout:5/743: truncate d6/d12/d4e/d58/d8f/fa9 1429932 0 2024-02-16T22:56:30.271 INFO:tasks.workunit.client.0.smithi064.stdout:7/651: unlink d5/d33/d45/fc8 0 2024-02-16T22:56:30.271 INFO:tasks.workunit.client.0.smithi064.stdout:1/578: sync 2024-02-16T22:56:30.272 INFO:tasks.workunit.client.0.smithi064.stdout:5/556: sync 2024-02-16T22:56:30.274 INFO:tasks.workunit.client.1.smithi192.stdout:6/782: unlink da/df/d18/d52/f7c 0 2024-02-16T22:56:30.274 INFO:tasks.workunit.client.1.smithi192.stdout:6/783: stat da/df/d6b/d74/f8b 0 2024-02-16T22:56:30.275 INFO:tasks.workunit.client.1.smithi192.stdout:6/784: chown da/df/d12/d15/f33 0 1 2024-02-16T22:56:30.281 INFO:tasks.workunit.client.1.smithi192.stdout:8/739: truncate d5/d16/f4f 1901259 0 2024-02-16T22:56:30.284 INFO:tasks.workunit.client.0.smithi064.stdout:4/623: link f3 dc/d1d/fc4 0 2024-02-16T22:56:30.285 INFO:tasks.workunit.client.0.smithi064.stdout:2/710: dread d7/d14/d25/daf/fd3 [0,4194304] 0 2024-02-16T22:56:30.285 INFO:tasks.workunit.client.0.smithi064.stdout:4/624: chown dc/dd/d32/d46/l62 54 1 2024-02-16T22:56:30.287 INFO:tasks.workunit.client.0.smithi064.stdout:1/579: dread d2/df/d14/d2a/f87 [0,4194304] 0 2024-02-16T22:56:30.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/863: dwrite d7/d15/d36/ff3 [0,4194304] 0 2024-02-16T22:56:30.290 INFO:tasks.workunit.client.1.smithi192.stdout:2/795: symlink d22/d24/d39/d3d/df2/d99/l105 0 2024-02-16T22:56:30.295 INFO:tasks.workunit.client.0.smithi064.stdout:7/652: mknod d5/d33/d45/d64/d56/d5d/ccb 0 2024-02-16T22:56:30.295 INFO:tasks.workunit.client.0.smithi064.stdout:6/672: write d0/f41 [300201,16679] 0 2024-02-16T22:56:30.295 INFO:tasks.workunit.client.0.smithi064.stdout:6/673: read - d0/d1d/dbc/fc9 zero size 2024-02-16T22:56:30.296 INFO:tasks.workunit.client.1.smithi192.stdout:2/796: readlink d22/d24/d39/d42/d52/laa 0 2024-02-16T22:56:30.296 INFO:tasks.workunit.client.1.smithi192.stdout:3/693: dwrite da/d11/d2b/f5e [0,4194304] 0 2024-02-16T22:56:30.296 INFO:tasks.workunit.client.1.smithi192.stdout:0/725: rename d1/dc7/f7e to d1/d3/de/d22/d37/d67/d75/db0/fe3 0 2024-02-16T22:56:30.299 INFO:tasks.workunit.client.0.smithi064.stdout:5/557: fdatasync d6/d1c/d43/d7d/fb4 0 2024-02-16T22:56:30.300 INFO:tasks.workunit.client.0.smithi064.stdout:3/728: getdents d4/d14/d2b 0 2024-02-16T22:56:30.302 INFO:tasks.workunit.client.0.smithi064.stdout:3/729: write d4/d1f/d34/f45 [928643,78227] 0 2024-02-16T22:56:30.306 INFO:tasks.workunit.client.1.smithi192.stdout:7/847: symlink d1/d2/d2d/d61/d88/d90/d4c/d60/l122 0 2024-02-16T22:56:30.307 INFO:tasks.workunit.client.1.smithi192.stdout:7/848: write d1/d2/de/d2c/d49/fd7 [912603,91590] 0 2024-02-16T22:56:30.308 INFO:tasks.workunit.client.0.smithi064.stdout:9/574: truncate d3/d9/fa2 100203 0 2024-02-16T22:56:30.309 INFO:tasks.workunit.client.1.smithi192.stdout:6/785: write da/df/d58/f91 [774643,30404] 0 2024-02-16T22:56:30.309 INFO:tasks.workunit.client.1.smithi192.stdout:7/849: truncate d1/d2/d2d/ddf/fe5 975305 0 2024-02-16T22:56:30.310 INFO:tasks.workunit.client.0.smithi064.stdout:4/625: mknod dc/dd/d32/d46/d5a/d7e/cc5 0 2024-02-16T22:56:30.311 INFO:tasks.workunit.client.1.smithi192.stdout:4/780: mknod d1/d15/d48/db5/dd5/d104/c10f 0 2024-02-16T22:56:30.313 INFO:tasks.workunit.client.1.smithi192.stdout:1/712: getdents d3/de/d27/d34/d52/d63/d7a 0 2024-02-16T22:56:30.313 INFO:tasks.workunit.client.1.smithi192.stdout:7/850: truncate d1/d2/de/d2c/d49/d89/ffa 620403 0 2024-02-16T22:56:30.316 INFO:tasks.workunit.client.0.smithi064.stdout:6/674: creat d0/d5/df/d17/d4a/fe2 x:0 0 0 2024-02-16T22:56:30.320 INFO:tasks.workunit.client.1.smithi192.stdout:2/797: rename d22/d24/d39/d3d/cb4 to d22/d24/d39/d42/d65/c106 0 2024-02-16T22:56:30.328 INFO:tasks.workunit.client.0.smithi064.stdout:5/558: dread d6/d1c/d43/f4d [0,4194304] 0 2024-02-16T22:56:30.330 INFO:tasks.workunit.client.0.smithi064.stdout:2/711: mknod d7/d14/ce8 0 2024-02-16T22:56:30.332 INFO:tasks.workunit.client.1.smithi192.stdout:8/740: mknod d5/ce6 0 2024-02-16T22:56:30.332 INFO:tasks.workunit.client.1.smithi192.stdout:0/726: write d1/d3/de/d22/d37/d47/f81 [209373,52073] 0 2024-02-16T22:56:30.333 INFO:tasks.workunit.client.0.smithi064.stdout:7/653: dwrite d5/d33/d53/da3/fb7 [0,4194304] 0 2024-02-16T22:56:30.333 INFO:tasks.workunit.client.1.smithi192.stdout:8/741: stat d5/d26/d83/db4/f4b 0 2024-02-16T22:56:30.334 INFO:tasks.workunit.client.0.smithi064.stdout:4/626: mkdir dc/de/d28/d57/d77/dc6 0 2024-02-16T22:56:30.334 INFO:tasks.workunit.client.1.smithi192.stdout:8/742: dread - d5/d16/dd5/d9d/fd9 zero size 2024-02-16T22:56:30.334 INFO:tasks.workunit.client.1.smithi192.stdout:7/851: dread d1/de1/f63 [0,4194304] 0 2024-02-16T22:56:30.335 INFO:tasks.workunit.client.1.smithi192.stdout:6/786: mkdir da/df/dfb 0 2024-02-16T22:56:30.336 INFO:tasks.workunit.client.0.smithi064.stdout:7/654: chown d5/d33/d53/f99 650 1 2024-02-16T22:56:30.337 INFO:tasks.workunit.client.0.smithi064.stdout:2/712: write d7/d14/d1e/d34/d3f/d9a/dd9/f15 [2871809,52222] 0 2024-02-16T22:56:30.339 INFO:tasks.workunit.client.1.smithi192.stdout:4/781: mkdir d1/d26/d34/db9/d110 0 2024-02-16T22:56:30.339 INFO:tasks.workunit.client.0.smithi064.stdout:3/730: write d4/fd9 [777293,66137] 0 2024-02-16T22:56:30.339 INFO:tasks.workunit.client.0.smithi064.stdout:5/559: dwrite d6/d25/d48/d67/fb2 [0,4194304] 0 2024-02-16T22:56:30.340 INFO:tasks.workunit.client.1.smithi192.stdout:4/782: dread - d1/d26/d66/d89/da8/f10b zero size 2024-02-16T22:56:30.340 INFO:tasks.workunit.client.1.smithi192.stdout:9/864: creat d7/d19/d37/dbf/d10b/f128 x:0 0 0 2024-02-16T22:56:30.343 INFO:tasks.workunit.client.0.smithi064.stdout:6/675: fsync d0/d12/f8b 0 2024-02-16T22:56:30.344 INFO:tasks.workunit.client.1.smithi192.stdout:1/713: mknod d3/de/d27/d34/d52/d63/d7a/db0/cf3 0 2024-02-16T22:56:30.346 INFO:tasks.workunit.client.1.smithi192.stdout:3/694: creat da/d11/d2b/ddc/ff2 x:0 0 0 2024-02-16T22:56:30.350 INFO:tasks.workunit.client.0.smithi064.stdout:9/575: fdatasync d3/d9/f48 0 2024-02-16T22:56:30.351 INFO:tasks.workunit.client.0.smithi064.stdout:9/576: write d3/d11/f28 [1235578,126346] 0 2024-02-16T22:56:30.353 INFO:tasks.workunit.client.1.smithi192.stdout:0/727: creat d1/dc7/d9c/fe4 x:0 0 0 2024-02-16T22:56:30.355 INFO:tasks.workunit.client.0.smithi064.stdout:4/627: fdatasync dc/dd/d5b/fc1 0 2024-02-16T22:56:30.356 INFO:tasks.workunit.client.0.smithi064.stdout:1/580: link d2/c9f d2/df/d11/d1a/d30/dbd/d91/cc9 0 2024-02-16T22:56:30.356 INFO:tasks.workunit.client.0.smithi064.stdout:4/628: write dc/dd/d32/d22/f56 [1132272,76655] 0 2024-02-16T22:56:30.358 INFO:tasks.workunit.client.1.smithi192.stdout:7/852: symlink d1/d2/d2d/d61/d88/l123 0 2024-02-16T22:56:30.358 INFO:tasks.workunit.client.1.smithi192.stdout:7/853: chown d1/d2/d2d/d61/d8a/de6 2 1 2024-02-16T22:56:30.359 INFO:tasks.workunit.client.1.smithi192.stdout:4/783: rmdir d1/d15/d48/dc3 39 2024-02-16T22:56:30.360 INFO:tasks.workunit.client.1.smithi192.stdout:4/784: fsync d1/d26/d66/d89/d11/d58/f90 0 2024-02-16T22:56:30.361 INFO:tasks.workunit.client.1.smithi192.stdout:4/785: write d1/d26/d66/d89/da8/f10b [471799,25006] 0 2024-02-16T22:56:30.366 INFO:tasks.workunit.client.1.smithi192.stdout:2/798: getdents d22/d47/dcb/d3f/dac 0 2024-02-16T22:56:30.366 INFO:tasks.workunit.client.1.smithi192.stdout:2/799: readlink d22/d24/d39/d42/l77 0 2024-02-16T22:56:30.367 INFO:tasks.workunit.client.1.smithi192.stdout:5/744: getdents d6/d8/d75/d8c/dc4 0 2024-02-16T22:56:30.369 INFO:tasks.workunit.client.1.smithi192.stdout:0/728: creat d1/d3/de/d22/fe5 x:0 0 0 2024-02-16T22:56:30.369 INFO:tasks.workunit.client.1.smithi192.stdout:0/729: read - d1/d3/de/d22/da7/fdf zero size 2024-02-16T22:56:30.370 INFO:tasks.workunit.client.0.smithi064.stdout:3/731: dwrite d4/d1f/d5b/d51/f6a [0,4194304] 0 2024-02-16T22:56:30.372 INFO:tasks.workunit.client.0.smithi064.stdout:1/581: creat d2/df/d11/d1a/d1d/fca x:0 0 0 2024-02-16T22:56:30.372 INFO:tasks.workunit.client.1.smithi192.stdout:7/854: mkdir d1/d2/de/d2c/d49/dba/d124 0 2024-02-16T22:56:30.375 INFO:tasks.workunit.client.0.smithi064.stdout:4/629: rename dc/dd/d32/d22/f52 to dc/dd/d32/d46/d79/fc7 0 2024-02-16T22:56:30.375 INFO:tasks.workunit.client.0.smithi064.stdout:4/630: fsync dc/d1d/f8b 0 2024-02-16T22:56:30.376 INFO:tasks.workunit.client.1.smithi192.stdout:6/787: dwrite da/df/d12/d15/f2d [0,4194304] 0 2024-02-16T22:56:30.377 INFO:tasks.workunit.client.1.smithi192.stdout:4/786: mkdir d1/d26/d66/d89/da8/dc5/d111 0 2024-02-16T22:56:30.378 INFO:tasks.workunit.client.0.smithi064.stdout:5/560: link d6/d25/d48/d67/d91/da5/fbd d6/d18/d90/fc2 0 2024-02-16T22:56:30.382 INFO:tasks.workunit.client.0.smithi064.stdout:4/631: dwrite dc/dd/d16/fa6 [0,4194304] 0 2024-02-16T22:56:30.386 INFO:tasks.workunit.client.1.smithi192.stdout:2/800: rename d22/d24/d39/d42/d52/c76 to d22/d24/d39/d42/d65/d66/c107 0 2024-02-16T22:56:30.389 INFO:tasks.workunit.client.0.smithi064.stdout:9/577: dread - d3/d9/d6a/db1/d38/fa6 zero size 2024-02-16T22:56:30.390 INFO:tasks.workunit.client.1.smithi192.stdout:0/730: creat d1/dc7/d9c/fe6 x:0 0 0 2024-02-16T22:56:30.391 INFO:tasks.workunit.client.1.smithi192.stdout:8/743: link d5/d26/f88 d5/d26/d83/d86/d5f/db2/fe7 0 2024-02-16T22:56:30.394 INFO:tasks.workunit.client.1.smithi192.stdout:9/865: link d7/d19/d37/d5a/l123 d7/d15/d36/d77/l129 0 2024-02-16T22:56:30.396 INFO:tasks.workunit.client.1.smithi192.stdout:6/788: mkdir da/df/d12/d57/d81/dfc 0 2024-02-16T22:56:30.397 INFO:tasks.workunit.client.1.smithi192.stdout:4/787: creat d1/d15/d29/d3c/f112 x:0 0 0 2024-02-16T22:56:30.398 INFO:tasks.workunit.client.1.smithi192.stdout:4/788: stat d1/d15/d48/d8d/f7b 0 2024-02-16T22:56:30.399 INFO:tasks.workunit.client.1.smithi192.stdout:4/789: fdatasync d1/d26/d66/d89/da8/fff 0 2024-02-16T22:56:30.400 INFO:tasks.workunit.client.1.smithi192.stdout:4/790: write d1/d15/d29/d3c/f112 [913236,13549] 0 2024-02-16T22:56:30.401 INFO:tasks.workunit.client.1.smithi192.stdout:0/731: mknod d1/d3/dac/ce7 0 2024-02-16T22:56:30.401 INFO:tasks.workunit.client.0.smithi064.stdout:4/632: rmdir dc/d1d/d43/d7a 39 2024-02-16T22:56:30.402 INFO:tasks.workunit.client.1.smithi192.stdout:8/744: truncate d5/d16/f7d 1930984 0 2024-02-16T22:56:30.404 INFO:tasks.workunit.client.0.smithi064.stdout:3/732: link d4/d14/fc2 d4/d1f/d34/d7b/ff6 0 2024-02-16T22:56:30.405 INFO:tasks.workunit.client.1.smithi192.stdout:9/866: rmdir d7/d15/d36/d77/dbd/de0 39 2024-02-16T22:56:30.406 INFO:tasks.workunit.client.0.smithi064.stdout:2/713: getdents d7/d14/d3e 0 2024-02-16T22:56:30.409 INFO:tasks.workunit.client.0.smithi064.stdout:1/582: rename d2/df/d14/d18/d48/d6b/d75/l9b to d2/df/lcb 0 2024-02-16T22:56:30.410 INFO:tasks.workunit.client.0.smithi064.stdout:1/583: stat d2/df/d11/d25/fb9 0 2024-02-16T22:56:30.411 INFO:tasks.workunit.client.1.smithi192.stdout:1/714: getdents d3/de/d27/d34/d52/d5a 0 2024-02-16T22:56:30.412 INFO:tasks.workunit.client.0.smithi064.stdout:2/714: dread d7/d14/d1e/d34/d3f/d9a/dd9/f11 [4194304,4194304] 0 2024-02-16T22:56:30.412 INFO:tasks.workunit.client.0.smithi064.stdout:6/676: getdents d0/d89 0 2024-02-16T22:56:30.412 INFO:tasks.workunit.client.0.smithi064.stdout:1/584: chown d2/df/d14/d18/d40/fa0 145 1 2024-02-16T22:56:30.414 INFO:tasks.workunit.client.1.smithi192.stdout:4/791: dread d1/d15/d23/fea [0,4194304] 0 2024-02-16T22:56:30.416 INFO:tasks.workunit.client.1.smithi192.stdout:0/732: unlink d1/d3/dac/ce7 0 2024-02-16T22:56:30.416 INFO:tasks.workunit.client.1.smithi192.stdout:5/745: dread d6/f2b [0,4194304] 0 2024-02-16T22:56:30.417 INFO:tasks.workunit.client.1.smithi192.stdout:8/745: symlink d5/d16/d81/le8 0 2024-02-16T22:56:30.418 INFO:tasks.workunit.client.1.smithi192.stdout:0/733: dread d1/d3/de/d22/d37/d67/f66 [0,4194304] 0 2024-02-16T22:56:30.419 INFO:tasks.workunit.client.1.smithi192.stdout:0/734: rename d1/d3/de/d22/d37/d47 to d1/d3/de/d22/d37/d47/de8 22 2024-02-16T22:56:30.419 INFO:tasks.workunit.client.0.smithi064.stdout:9/578: creat d3/d9/d6a/db1/d2a/fbf x:0 0 0 2024-02-16T22:56:30.423 INFO:tasks.workunit.client.0.smithi064.stdout:3/733: rename d4/d13/d1b/d27/d99/ca0 to d4/d13/d1b/d27/d44/d97/cf7 0 2024-02-16T22:56:30.425 INFO:tasks.workunit.client.0.smithi064.stdout:6/677: symlink d0/d3c/le3 0 2024-02-16T22:56:30.426 INFO:tasks.workunit.client.0.smithi064.stdout:6/678: fsync d0/d6/d10/f5e 0 2024-02-16T22:56:30.427 INFO:tasks.workunit.client.0.smithi064.stdout:2/715: mkdir d7/d14/d25/daf/de9 0 2024-02-16T22:56:30.429 INFO:tasks.workunit.client.1.smithi192.stdout:3/695: dwrite da/df/fe2 [0,4194304] 0 2024-02-16T22:56:30.430 INFO:tasks.workunit.client.0.smithi064.stdout:1/585: truncate d2/df/f13 2458507 0 2024-02-16T22:56:30.431 INFO:tasks.workunit.client.1.smithi192.stdout:4/792: fdatasync d1/d26/d66/d89/fd6 0 2024-02-16T22:56:30.437 INFO:tasks.workunit.client.0.smithi064.stdout:6/679: dwrite d0/d1d/da8/fdb [0,4194304] 0 2024-02-16T22:56:30.437 INFO:tasks.workunit.client.1.smithi192.stdout:4/793: dwrite d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:30.441 INFO:tasks.workunit.client.1.smithi192.stdout:7/855: dwrite d1/d69/f93 [0,4194304] 0 2024-02-16T22:56:30.441 INFO:tasks.workunit.client.0.smithi064.stdout:2/716: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f80 [0,4194304] 0 2024-02-16T22:56:30.442 INFO:tasks.workunit.client.0.smithi064.stdout:1/586: dread d2/df/d11/d1a/d1d/f4d [0,4194304] 0 2024-02-16T22:56:30.443 INFO:tasks.workunit.client.0.smithi064.stdout:2/717: write d7/d14/d3e/fd2 [396455,121498] 0 2024-02-16T22:56:30.444 INFO:tasks.workunit.client.0.smithi064.stdout:2/718: stat d7/d14/d3e/fd2 0 2024-02-16T22:56:30.449 INFO:tasks.workunit.client.0.smithi064.stdout:2/719: readlink d7/d14/d1e/d34/d3f/d9a/dd9/l5f 0 2024-02-16T22:56:30.450 INFO:tasks.workunit.client.0.smithi064.stdout:4/633: creat dc/dd/fc8 x:0 0 0 2024-02-16T22:56:30.454 INFO:tasks.workunit.client.0.smithi064.stdout:2/720: chown d7/d14/l26 812 1 2024-02-16T22:56:30.456 INFO:tasks.workunit.client.0.smithi064.stdout:1/587: dwrite d2/df/d11/d25/f85 [0,4194304] 0 2024-02-16T22:56:30.459 INFO:tasks.workunit.client.0.smithi064.stdout:5/561: dwrite d6/d1c/d43/d7d/fb4 [0,4194304] 0 2024-02-16T22:56:30.459 INFO:tasks.workunit.client.0.smithi064.stdout:4/634: write dc/dd/d4f/fa7 [495617,124889] 0 2024-02-16T22:56:30.462 INFO:tasks.workunit.client.0.smithi064.stdout:4/635: stat dc/de/d28/d57/d77/dc6 0 2024-02-16T22:56:30.463 INFO:tasks.workunit.client.0.smithi064.stdout:1/588: chown d2/df/d11/d1a/d1d/f4d 2 1 2024-02-16T22:56:30.464 INFO:tasks.workunit.client.1.smithi192.stdout:9/867: write d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/feb [735691,95146] 0 2024-02-16T22:56:30.465 INFO:tasks.workunit.client.0.smithi064.stdout:6/680: creat d0/d1d/dbc/fe4 x:0 0 0 2024-02-16T22:56:30.465 INFO:tasks.workunit.client.1.smithi192.stdout:9/868: chown d7/d15/d36/d77/db1/ff8 1855 1 2024-02-16T22:56:30.468 INFO:tasks.workunit.client.0.smithi064.stdout:4/636: truncate dc/dd/d32/d46/d82/da1/fc3 699213 0 2024-02-16T22:56:30.468 INFO:tasks.workunit.client.1.smithi192.stdout:6/789: write da/df/d12/d15/f21 [2396682,8641] 0 2024-02-16T22:56:30.469 INFO:tasks.workunit.client.1.smithi192.stdout:5/746: truncate d6/d12/d1d/f3a 1982488 0 2024-02-16T22:56:30.470 INFO:tasks.workunit.client.1.smithi192.stdout:6/790: write da/df/d12/d15/d1b/f97 [3522310,6156] 0 2024-02-16T22:56:30.471 INFO:tasks.workunit.client.1.smithi192.stdout:5/747: write d6/d20/d93/fba [2045317,127188] 0 2024-02-16T22:56:30.473 INFO:tasks.workunit.client.0.smithi064.stdout:9/579: creat d3/d9/d6a/db1/d2a/d2e/fc0 x:0 0 0 2024-02-16T22:56:30.476 INFO:tasks.workunit.client.0.smithi064.stdout:2/721: write d7/d14/d1e/d34/d3f/d9a/dd9/d95/fac [885991,46167] 0 2024-02-16T22:56:30.479 INFO:tasks.workunit.client.1.smithi192.stdout:4/794: rmdir d1/d26/d66 39 2024-02-16T22:56:30.480 INFO:tasks.workunit.client.0.smithi064.stdout:3/734: write d4/d13/f16 [5110466,121516] 0 2024-02-16T22:56:30.481 INFO:tasks.workunit.client.0.smithi064.stdout:3/735: chown d4/d13/fa7 88091 1 2024-02-16T22:56:30.481 INFO:tasks.workunit.client.0.smithi064.stdout:5/562: creat d6/d1c/d43/db9/fc3 x:0 0 0 2024-02-16T22:56:30.482 INFO:tasks.workunit.client.0.smithi064.stdout:5/563: stat d6/d25/d48/f5e 0 2024-02-16T22:56:30.483 INFO:tasks.workunit.client.0.smithi064.stdout:9/580: dread - d3/f69 zero size 2024-02-16T22:56:30.483 INFO:tasks.workunit.client.0.smithi064.stdout:5/564: chown d6/d25/c86 11144466 1 2024-02-16T22:56:30.486 INFO:tasks.workunit.client.0.smithi064.stdout:3/736: fdatasync d4/d1f/f20 0 2024-02-16T22:56:30.486 INFO:tasks.workunit.client.0.smithi064.stdout:9/581: read d3/f8f [694621,16188] 0 2024-02-16T22:56:30.487 INFO:tasks.workunit.client.0.smithi064.stdout:8/594: sync 2024-02-16T22:56:30.487 INFO:tasks.workunit.client.1.smithi192.stdout:9/869: readlink d7/d19/d5e/d81/l125 0 2024-02-16T22:56:30.487 INFO:tasks.workunit.client.0.smithi064.stdout:6/681: dread d0/d3c/da9/f67 [0,4194304] 0 2024-02-16T22:56:30.492 INFO:tasks.workunit.client.0.smithi064.stdout:5/565: rename d6/d18/f72 to d6/d25/d48/d67/d91/fc4 0 2024-02-16T22:56:30.493 INFO:tasks.workunit.client.0.smithi064.stdout:5/566: chown d6/d18/d19/fb0 588841934 1 2024-02-16T22:56:30.495 INFO:tasks.workunit.client.1.smithi192.stdout:8/746: creat d5/db/de/fe9 x:0 0 0 2024-02-16T22:56:30.499 INFO:tasks.workunit.client.0.smithi064.stdout:3/737: mknod d4/d13/d4d/d5c/d78/cf8 0 2024-02-16T22:56:30.500 INFO:tasks.workunit.client.1.smithi192.stdout:5/748: unlink d6/d20/d93/fba 0 2024-02-16T22:56:30.500 INFO:tasks.workunit.client.0.smithi064.stdout:4/637: dread dc/d1d/f8b [0,4194304] 0 2024-02-16T22:56:30.501 INFO:tasks.workunit.client.0.smithi064.stdout:8/595: creat df/d31/d2f/d3e/d5f/fc7 x:0 0 0 2024-02-16T22:56:30.502 INFO:tasks.workunit.client.1.smithi192.stdout:0/735: rmdir d1/d3/de/d22/d37/d47/dd0/de1 0 2024-02-16T22:56:30.503 INFO:tasks.workunit.client.0.smithi064.stdout:6/682: read d0/d5/f90 [31511,26970] 0 2024-02-16T22:56:30.504 INFO:tasks.workunit.client.1.smithi192.stdout:1/715: getdents d3/d5b/db1 0 2024-02-16T22:56:30.508 INFO:tasks.workunit.client.0.smithi064.stdout:9/582: dread d3/d9/f3c [0,4194304] 0 2024-02-16T22:56:30.510 INFO:tasks.workunit.client.0.smithi064.stdout:3/738: rename d4/d13/d1b/fb2 to d4/d1f/d34/ff9 0 2024-02-16T22:56:30.511 INFO:tasks.workunit.client.0.smithi064.stdout:4/638: mknod dc/dd/d32/d46/d82/da1/cc9 0 2024-02-16T22:56:30.513 INFO:tasks.workunit.client.1.smithi192.stdout:8/747: read d5/db/d47/f92 [2422870,32050] 0 2024-02-16T22:56:30.516 INFO:tasks.workunit.client.1.smithi192.stdout:8/748: dread d5/d26/d83/fdb [0,4194304] 0 2024-02-16T22:56:30.517 INFO:tasks.workunit.client.0.smithi064.stdout:2/722: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/d95/f66 [0,4194304] 0 2024-02-16T22:56:30.517 INFO:tasks.workunit.client.1.smithi192.stdout:5/749: mknod d6/d12/d4e/d46/d4f/cf7 0 2024-02-16T22:56:30.523 INFO:tasks.workunit.client.0.smithi064.stdout:9/583: chown d3/l8 18 1 2024-02-16T22:56:30.524 INFO:tasks.workunit.client.1.smithi192.stdout:0/736: symlink d1/d3/de/d22/d37/d47/le9 0 2024-02-16T22:56:30.526 INFO:tasks.workunit.client.1.smithi192.stdout:0/737: fsync d1/d3/de/d22/d37/d47/fda 0 2024-02-16T22:56:30.526 INFO:tasks.workunit.client.1.smithi192.stdout:2/801: sync 2024-02-16T22:56:30.527 INFO:tasks.workunit.client.1.smithi192.stdout:0/738: chown d1/d3/de/d22/d37/d67/d75/db0/fb2 194 1 2024-02-16T22:56:30.527 INFO:tasks.workunit.client.0.smithi064.stdout:3/739: creat d4/d13/d1b/d27/d61/d63/de1/ffa x:0 0 0 2024-02-16T22:56:30.531 INFO:tasks.workunit.client.0.smithi064.stdout:8/596: creat df/d31/d5e/d73/fc8 x:0 0 0 2024-02-16T22:56:30.532 INFO:tasks.workunit.client.1.smithi192.stdout:9/870: write d7/d19/d37/d5a/fa7 [477654,84471] 0 2024-02-16T22:56:30.533 INFO:tasks.workunit.client.1.smithi192.stdout:9/871: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/d92 51100 1 2024-02-16T22:56:30.534 INFO:tasks.workunit.client.0.smithi064.stdout:6/683: mkdir d0/d1b/de5 0 2024-02-16T22:56:30.536 INFO:tasks.workunit.client.1.smithi192.stdout:8/749: dread d5/d26/d83/d86/d5f/db2/fe7 [0,4194304] 0 2024-02-16T22:56:30.539 INFO:tasks.workunit.client.0.smithi064.stdout:5/567: write d6/f37 [708464,105161] 0 2024-02-16T22:56:30.540 INFO:tasks.workunit.client.0.smithi064.stdout:9/584: mknod d3/d11/d9e/da9/cc1 0 2024-02-16T22:56:30.541 INFO:tasks.workunit.client.1.smithi192.stdout:8/750: dread d5/d9/f5a [0,4194304] 0 2024-02-16T22:56:30.543 INFO:tasks.workunit.client.0.smithi064.stdout:3/740: creat d4/d13/d1b/d27/d61/d63/ffb x:0 0 0 2024-02-16T22:56:30.544 INFO:tasks.workunit.client.0.smithi064.stdout:3/741: write d4/d13/f16 [5432466,2804] 0 2024-02-16T22:56:30.548 INFO:tasks.workunit.client.0.smithi064.stdout:8/597: rename df/d31/d5e to df/d31/d8f/dc9 0 2024-02-16T22:56:30.556 INFO:tasks.workunit.client.0.smithi064.stdout:2/723: dwrite d7/d14/d1e/fa2 [0,4194304] 0 2024-02-16T22:56:30.558 INFO:tasks.workunit.client.0.smithi064.stdout:5/568: dread d6/d25/f89 [0,4194304] 0 2024-02-16T22:56:30.558 INFO:tasks.workunit.client.0.smithi064.stdout:6/684: write d0/d5/df/d17/d18/f3f [2692859,130314] 0 2024-02-16T22:56:30.558 INFO:tasks.workunit.client.0.smithi064.stdout:3/742: symlink d4/d13/d1b/d27/d44/d97/lfc 0 2024-02-16T22:56:30.560 INFO:tasks.workunit.client.0.smithi064.stdout:5/569: chown d6/d25/d48/d67/f87 9857 1 2024-02-16T22:56:30.566 INFO:tasks.workunit.client.0.smithi064.stdout:2/724: rename d7/d14/d1e/d34/d65 to d7/d14/d1e/d34/d73/db5/dea 0 2024-02-16T22:56:30.569 INFO:tasks.workunit.client.0.smithi064.stdout:3/743: fdatasync d4/d1f/d34/fe6 0 2024-02-16T22:56:30.571 INFO:tasks.workunit.client.0.smithi064.stdout:5/570: stat d6/d1c/c42 0 2024-02-16T22:56:30.573 INFO:tasks.workunit.client.0.smithi064.stdout:2/725: dwrite d7/d14/d25/d46/d55/d7f/f99 [4194304,4194304] 0 2024-02-16T22:56:30.581 INFO:tasks.workunit.client.0.smithi064.stdout:3/744: rename d4/d14/c2f to d4/d13/d1b/d27/d99/cfd 0 2024-02-16T22:56:30.583 INFO:tasks.workunit.client.0.smithi064.stdout:2/726: dwrite d7/d14/d25/f90 [0,4194304] 0 2024-02-16T22:56:30.586 INFO:tasks.workunit.client.0.smithi064.stdout:3/745: dread d4/d13/d1b/d27/d44/d97/fdc [0,4194304] 0 2024-02-16T22:56:30.587 INFO:tasks.workunit.client.0.smithi064.stdout:9/585: getdents d3/d11/d9e/da9 0 2024-02-16T22:56:30.588 INFO:tasks.workunit.client.0.smithi064.stdout:8/598: dread df/d2a/f2d [0,4194304] 0 2024-02-16T22:56:30.589 INFO:tasks.workunit.client.0.smithi064.stdout:9/586: write d3/d11/d15/d23/f3d [5705864,11739] 0 2024-02-16T22:56:30.590 INFO:tasks.workunit.client.0.smithi064.stdout:9/587: chown d3/d9/d1e/l67 610 1 2024-02-16T22:56:30.596 INFO:tasks.workunit.client.0.smithi064.stdout:5/571: mkdir d6/d18/dc5 0 2024-02-16T22:56:30.598 INFO:tasks.workunit.client.0.smithi064.stdout:2/727: rename d7/d14/d4b/d63/l78 to d7/d14/d25/d46/leb 0 2024-02-16T22:56:30.599 INFO:tasks.workunit.client.1.smithi192.stdout:3/696: sync 2024-02-16T22:56:30.600 INFO:tasks.workunit.client.1.smithi192.stdout:3/697: chown da/d11/d63/d6b/d66 398408 1 2024-02-16T22:56:30.641 INFO:tasks.workunit.client.1.smithi192.stdout:7/856: sync 2024-02-16T22:56:30.642 INFO:tasks.workunit.client.1.smithi192.stdout:7/857: chown d1/d2/d2d/d61/d88/d90/d4c/cb3 0 1 2024-02-16T22:56:30.642 INFO:tasks.workunit.client.1.smithi192.stdout:7/858: chown d1/d2/de/f20 4 1 2024-02-16T22:56:30.646 INFO:tasks.workunit.client.0.smithi064.stdout:3/746: dread d4/d13/f55 [0,4194304] 0 2024-02-16T22:56:30.682 INFO:tasks.workunit.client.0.smithi064.stdout:8/599: unlink df/d31/d2f/d38/f53 0 2024-02-16T22:56:30.685 INFO:tasks.workunit.client.0.smithi064.stdout:6/685: getdents d0/d6 0 2024-02-16T22:56:30.686 INFO:tasks.workunit.client.0.smithi064.stdout:6/686: dread - d0/d1d/dbc/fc9 zero size 2024-02-16T22:56:30.688 INFO:tasks.workunit.client.0.smithi064.stdout:5/572: mknod d6/d25/d48/d67/d91/cc6 0 2024-02-16T22:56:30.690 INFO:tasks.workunit.client.0.smithi064.stdout:2/728: mknod d7/d14/d25/cec 0 2024-02-16T22:56:30.691 INFO:tasks.workunit.client.0.smithi064.stdout:2/729: write d7/d14/d25/d46/d55/f5a [5092272,21687] 0 2024-02-16T22:56:30.692 INFO:tasks.workunit.client.0.smithi064.stdout:3/747: creat d4/d13/d1b/d27/d61/d63/ffe x:0 0 0 2024-02-16T22:56:30.693 INFO:tasks.workunit.client.1.smithi192.stdout:4/795: symlink d1/d26/d66/d89/l113 0 2024-02-16T22:56:30.693 INFO:tasks.workunit.client.1.smithi192.stdout:4/796: chown d1/d26/d66/fcf 9593264 1 2024-02-16T22:56:30.693 INFO:tasks.workunit.client.0.smithi064.stdout:8/600: mknod df/d2a/d5c/cca 0 2024-02-16T22:56:30.695 INFO:tasks.workunit.client.1.smithi192.stdout:1/716: symlink d3/de/d27/d34/lf4 0 2024-02-16T22:56:30.696 INFO:tasks.workunit.client.1.smithi192.stdout:2/802: creat d22/d24/d39/d3d/f108 x:0 0 0 2024-02-16T22:56:30.697 INFO:tasks.workunit.client.1.smithi192.stdout:0/739: truncate d1/dc7/f68 646033 0 2024-02-16T22:56:30.700 INFO:tasks.workunit.client.1.smithi192.stdout:6/791: creat da/df/d6b/ffd x:0 0 0 2024-02-16T22:56:30.703 INFO:tasks.workunit.client.0.smithi064.stdout:3/748: mkdir d4/d13/d4d/d5c/d78/dff 0 2024-02-16T22:56:30.704 INFO:tasks.workunit.client.0.smithi064.stdout:3/749: chown d4/d14/d2b/feb 15 1 2024-02-16T22:56:30.707 INFO:tasks.workunit.client.0.smithi064.stdout:6/687: symlink d0/d5/le6 0 2024-02-16T22:56:30.709 INFO:tasks.workunit.client.0.smithi064.stdout:9/588: write d3/d9/d6a/db1/f26 [3138692,64878] 0 2024-02-16T22:56:30.709 INFO:tasks.workunit.client.0.smithi064.stdout:5/573: creat d6/d18/dc5/fc7 x:0 0 0 2024-02-16T22:56:30.710 INFO:tasks.workunit.client.1.smithi192.stdout:8/751: read d5/d26/d83/fdb [382608,33130] 0 2024-02-16T22:56:30.711 INFO:tasks.workunit.client.0.smithi064.stdout:5/574: chown d6/d18/d95/dbe 59294 1 2024-02-16T22:56:30.711 INFO:tasks.workunit.client.0.smithi064.stdout:2/730: symlink d7/d14/d1e/d34/d3f/d9a/dd9/led 0 2024-02-16T22:56:30.714 INFO:tasks.workunit.client.0.smithi064.stdout:8/601: symlink df/d31/d2f/d7f/lcb 0 2024-02-16T22:56:30.715 INFO:tasks.workunit.client.1.smithi192.stdout:7/859: symlink d1/d69/l125 0 2024-02-16T22:56:30.719 INFO:tasks.workunit.client.1.smithi192.stdout:1/717: symlink d3/d5b/db1/lf5 0 2024-02-16T22:56:30.720 INFO:tasks.workunit.client.1.smithi192.stdout:2/803: fdatasync d22/d24/fb9 0 2024-02-16T22:56:30.721 INFO:tasks.workunit.client.0.smithi064.stdout:2/731: truncate d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f5c 2140597 0 2024-02-16T22:56:30.722 INFO:tasks.workunit.client.1.smithi192.stdout:2/804: fdatasync d22/d24/d39/d42/d85/d98/db0/dcd/fd8 0 2024-02-16T22:56:30.724 INFO:tasks.workunit.client.1.smithi192.stdout:0/740: rename d1/d3/de/d22/d37/d67/d75/fca to d1/d3/de/d22/d37/d47/dd0/fea 0 2024-02-16T22:56:30.724 INFO:tasks.workunit.client.1.smithi192.stdout:5/750: dwrite d6/d12/d1d/d56/fc8 [0,4194304] 0 2024-02-16T22:56:30.725 INFO:tasks.workunit.client.1.smithi192.stdout:6/792: fsync da/df/f6e 0 2024-02-16T22:56:30.727 INFO:tasks.workunit.client.0.smithi064.stdout:2/732: fdatasync d7/d14/d1e/d34/d73/f77 0 2024-02-16T22:56:30.728 INFO:tasks.workunit.client.1.smithi192.stdout:2/805: truncate d22/d24/d39/d3d/df2/d67/fd3 1000068 0 2024-02-16T22:56:30.729 INFO:tasks.workunit.client.0.smithi064.stdout:3/750: link d4/d13/d1b/d27/d44/l57 d4/d1f/d5b/l100 0 2024-02-16T22:56:30.732 INFO:tasks.workunit.client.0.smithi064.stdout:9/589: rename d3/d11/d15/l46 to d3/d9/d6a/db1/lc2 0 2024-02-16T22:56:30.733 INFO:tasks.workunit.client.1.smithi192.stdout:4/797: dwrite d1/d26/d66/d89/f99 [0,4194304] 0 2024-02-16T22:56:30.736 INFO:tasks.workunit.client.0.smithi064.stdout:5/575: creat d6/d18/d19/fc8 x:0 0 0 2024-02-16T22:56:30.737 INFO:tasks.workunit.client.0.smithi064.stdout:5/576: write d6/d1c/d43/db9/f77 [1477305,100847] 0 2024-02-16T22:56:30.740 INFO:tasks.workunit.client.0.smithi064.stdout:2/733: truncate d7/d14/d25/d46/d55/d7f/faa 168640 0 2024-02-16T22:56:30.740 INFO:tasks.workunit.client.1.smithi192.stdout:8/752: write d5/d26/d83/d86/d5f/db2/fe7 [1287572,26606] 0 2024-02-16T22:56:30.741 INFO:tasks.workunit.client.1.smithi192.stdout:8/753: read - d5/fae zero size 2024-02-16T22:56:30.742 INFO:tasks.workunit.client.0.smithi064.stdout:3/751: mknod d4/d1f/d5b/d51/dc1/c101 0 2024-02-16T22:56:30.743 INFO:tasks.workunit.client.1.smithi192.stdout:1/718: fdatasync d3/d5b/f71 0 2024-02-16T22:56:30.744 INFO:tasks.workunit.client.0.smithi064.stdout:3/752: chown d4/d13/f16 1289283 1 2024-02-16T22:56:30.747 INFO:tasks.workunit.client.0.smithi064.stdout:2/734: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/d95/fac [0,4194304] 0 2024-02-16T22:56:30.748 INFO:tasks.workunit.client.0.smithi064.stdout:5/577: creat d6/d25/d48/d67/d91/fc9 x:0 0 0 2024-02-16T22:56:30.748 INFO:tasks.workunit.client.1.smithi192.stdout:3/698: dwrite da/d11/d20/d39/fad [0,4194304] 0 2024-02-16T22:56:30.752 INFO:tasks.workunit.client.1.smithi192.stdout:0/741: creat d1/d3/d29/dcd/dcf/feb x:0 0 0 2024-02-16T22:56:30.754 INFO:tasks.workunit.client.0.smithi064.stdout:3/753: truncate d4/d1f/d5b/d51/fc5 882841 0 2024-02-16T22:56:30.757 INFO:tasks.workunit.client.0.smithi064.stdout:2/735: mkdir d7/d14/d25/dee 0 2024-02-16T22:56:30.758 INFO:tasks.workunit.client.0.smithi064.stdout:2/736: readlink d7/d14/d1e/d34/d3f/d9a/dd9/l29 0 2024-02-16T22:56:30.759 INFO:tasks.workunit.client.0.smithi064.stdout:2/737: readlink d7/d14/d1e/d34/d3f/d9a/dcf/le5 0 2024-02-16T22:56:30.761 INFO:tasks.workunit.client.0.smithi064.stdout:5/578: stat d6/d25/da4/dc1/d8e 0 2024-02-16T22:56:30.763 INFO:tasks.workunit.client.0.smithi064.stdout:3/754: fdatasync d4/d13/d41/f8d 0 2024-02-16T22:56:30.763 INFO:tasks.workunit.client.0.smithi064.stdout:2/738: fsync d7/d14/d1e/d34/f62 0 2024-02-16T22:56:30.764 INFO:tasks.workunit.client.0.smithi064.stdout:3/755: chown d4/d1f/f20 394403 1 2024-02-16T22:56:30.765 INFO:tasks.workunit.client.1.smithi192.stdout:2/806: rename d22/d24/d39/d3d/df2/l44 to d22/d47/dcb/dc9/l109 0 2024-02-16T22:56:30.768 INFO:tasks.workunit.client.0.smithi064.stdout:5/579: dwrite d6/d1c/d4b/fb5 [0,4194304] 0 2024-02-16T22:56:30.769 INFO:tasks.workunit.client.0.smithi064.stdout:2/739: readlink d7/d14/d25/d46/d55/lc2 0 2024-02-16T22:56:30.770 INFO:tasks.workunit.client.0.smithi064.stdout:5/580: stat d6/d1c/c21 0 2024-02-16T22:56:30.772 INFO:tasks.workunit.client.0.smithi064.stdout:3/756: mkdir d4/d13/d1b/d27/d81/d102 0 2024-02-16T22:56:30.772 INFO:tasks.workunit.client.0.smithi064.stdout:2/740: write d7/d14/d25/d46/d55/d7f/fe7 [112094,1961] 0 2024-02-16T22:56:30.773 INFO:tasks.workunit.client.0.smithi064.stdout:3/757: dread - d4/d1f/d34/ff1 zero size 2024-02-16T22:56:30.777 INFO:tasks.workunit.client.1.smithi192.stdout:1/719: rmdir d3/de/d27/d34/d69/d85/d89/daa 39 2024-02-16T22:56:30.778 INFO:tasks.workunit.client.1.smithi192.stdout:1/720: write d3/de/d27/d34/d36/feb [515823,55187] 0 2024-02-16T22:56:30.779 INFO:tasks.workunit.client.0.smithi064.stdout:5/581: symlink d6/d25/lca 0 2024-02-16T22:56:30.781 INFO:tasks.workunit.client.0.smithi064.stdout:3/758: dread - d4/d13/d1b/d27/d99/d7d/fbf zero size 2024-02-16T22:56:30.787 INFO:tasks.workunit.client.1.smithi192.stdout:9/872: getdents d7/d19/d37/dbf/d10b 0 2024-02-16T22:56:30.788 INFO:tasks.workunit.client.1.smithi192.stdout:8/754: write d5/d26/d8f/f72 [308511,38971] 0 2024-02-16T22:56:30.788 INFO:tasks.workunit.client.1.smithi192.stdout:8/755: dread - d5/d16/d81/fc6 zero size 2024-02-16T22:56:30.789 INFO:tasks.workunit.client.1.smithi192.stdout:8/756: readlink d5/db/d47/l59 0 2024-02-16T22:56:30.790 INFO:tasks.workunit.client.1.smithi192.stdout:8/757: truncate d5/d16/fd2 4469617 0 2024-02-16T22:56:30.790 INFO:tasks.workunit.client.1.smithi192.stdout:4/798: fsync d1/f19 0 2024-02-16T22:56:30.791 INFO:tasks.workunit.client.1.smithi192.stdout:4/799: stat d1/d26/d66/d89/da8/da9 0 2024-02-16T22:56:30.792 INFO:tasks.workunit.client.0.smithi064.stdout:2/741: dread d7/f1a [0,4194304] 0 2024-02-16T22:56:30.793 INFO:tasks.workunit.client.1.smithi192.stdout:7/860: creat d1/d2/d2d/d61/d88/d90/d4c/f126 x:0 0 0 2024-02-16T22:56:30.794 INFO:tasks.workunit.client.0.smithi064.stdout:5/582: write d6/d25/da4/dc1/f45 [400345,22936] 0 2024-02-16T22:56:30.794 INFO:tasks.workunit.client.0.smithi064.stdout:2/742: chown d7/d14/d1e/d34/d3f/d9a/dd9/l74 0 1 2024-02-16T22:56:30.796 INFO:tasks.workunit.client.0.smithi064.stdout:5/583: rmdir d6/d25 39 2024-02-16T22:56:30.797 INFO:tasks.workunit.client.1.smithi192.stdout:0/742: truncate d1/d3/de/d22/d37/f71 1180789 0 2024-02-16T22:56:30.798 INFO:tasks.workunit.client.1.smithi192.stdout:0/743: write d1/d3/de/d22/fe5 [266690,364] 0 2024-02-16T22:56:30.798 INFO:tasks.workunit.client.0.smithi064.stdout:3/759: dread d4/d13/d1b/d4f/f6f [0,4194304] 0 2024-02-16T22:56:30.799 INFO:tasks.workunit.client.1.smithi192.stdout:0/744: chown d1/d3/f6d 52297076 1 2024-02-16T22:56:30.799 INFO:tasks.workunit.client.1.smithi192.stdout:5/751: creat d6/d12/d4e/d58/ff8 x:0 0 0 2024-02-16T22:56:30.800 INFO:tasks.workunit.client.0.smithi064.stdout:5/584: readlink d6/d1c/l2d 0 2024-02-16T22:56:30.800 INFO:tasks.workunit.client.1.smithi192.stdout:5/752: chown d6/d8/fa1 7 1 2024-02-16T22:56:30.800 INFO:tasks.workunit.client.1.smithi192.stdout:2/807: dread d22/d24/d39/d42/d65/d66/f83 [0,4194304] 0 2024-02-16T22:56:30.801 INFO:tasks.workunit.client.1.smithi192.stdout:9/873: creat d7/d19/d28/d43/f12a x:0 0 0 2024-02-16T22:56:30.801 INFO:tasks.workunit.client.1.smithi192.stdout:9/874: stat d7/d19/d28/d43/fc9 0 2024-02-16T22:56:30.802 INFO:tasks.workunit.client.1.smithi192.stdout:6/793: symlink da/df/d18/lfe 0 2024-02-16T22:56:30.802 INFO:tasks.workunit.client.1.smithi192.stdout:9/875: write d7/d19/f112 [1829636,21121] 0 2024-02-16T22:56:30.803 INFO:tasks.workunit.client.0.smithi064.stdout:3/760: rmdir d4/d13/d1b/d27/d61/dd8 39 2024-02-16T22:56:30.803 INFO:tasks.workunit.client.0.smithi064.stdout:3/761: chown d4/d14/d2b/feb 0 1 2024-02-16T22:56:30.804 INFO:tasks.workunit.client.1.smithi192.stdout:8/758: readlink d5/l10 0 2024-02-16T22:56:30.807 INFO:tasks.workunit.client.0.smithi064.stdout:5/585: rmdir d6 39 2024-02-16T22:56:30.812 INFO:tasks.workunit.client.1.smithi192.stdout:4/800: mkdir d1/d26/d66/d89/da8/da9/d114 0 2024-02-16T22:56:30.812 INFO:tasks.workunit.client.1.smithi192.stdout:1/721: creat d3/de/d27/d34/d69/d85/d89/daa/ff6 x:0 0 0 2024-02-16T22:56:30.813 INFO:tasks.workunit.client.0.smithi064.stdout:3/762: stat d4/d13/d1b/d27/l3a 0 2024-02-16T22:56:30.813 INFO:tasks.workunit.client.0.smithi064.stdout:3/763: fsync d4/d13/d4d/d5c/ff4 0 2024-02-16T22:56:30.813 INFO:tasks.workunit.client.1.smithi192.stdout:1/722: dread d3/de/d27/d34/d36/d47/d5d/fe5 [0,4194304] 0 2024-02-16T22:56:30.815 INFO:tasks.workunit.client.0.smithi064.stdout:5/586: write d6/f1b [598699,51870] 0 2024-02-16T22:56:30.816 INFO:tasks.workunit.client.0.smithi064.stdout:2/743: dwrite d7/d14/d25/d46/d55/d7f/faa [0,4194304] 0 2024-02-16T22:56:30.817 INFO:tasks.workunit.client.0.smithi064.stdout:5/587: dread - d6/d1c/d43/d7d/f88 zero size 2024-02-16T22:56:30.819 INFO:tasks.workunit.client.0.smithi064.stdout:2/744: write d7/d14/d25/d46/d55/d7f/fdf [459736,87850] 0 2024-02-16T22:56:30.819 INFO:tasks.workunit.client.0.smithi064.stdout:3/764: creat d4/d13/d1b/d4f/f103 x:0 0 0 2024-02-16T22:56:30.820 INFO:tasks.workunit.client.0.smithi064.stdout:3/765: chown d4/d14/d2b 0 1 2024-02-16T22:56:30.821 INFO:tasks.workunit.client.1.smithi192.stdout:2/808: symlink d22/d24/d39/d3d/df2/d99/d9b/l10a 0 2024-02-16T22:56:30.824 INFO:tasks.workunit.client.0.smithi064.stdout:3/766: read d4/d13/d4d/d5c/f6c [3021,61588] 0 2024-02-16T22:56:30.826 INFO:tasks.workunit.client.1.smithi192.stdout:8/759: dread d5/d26/d83/db4/f4b [0,4194304] 0 2024-02-16T22:56:30.829 INFO:tasks.workunit.client.0.smithi064.stdout:3/767: dwrite d4/d13/d1b/d27/d44/f5f [0,4194304] 0 2024-02-16T22:56:30.833 INFO:tasks.workunit.client.0.smithi064.stdout:5/588: link d6/f13 d6/d25/d48/d67/d91/da5/fcb 0 2024-02-16T22:56:30.835 INFO:tasks.workunit.client.0.smithi064.stdout:5/589: truncate d6/d1c/d43/d7d/faa 1037707 0 2024-02-16T22:56:30.836 INFO:tasks.workunit.client.0.smithi064.stdout:3/768: mkdir d4/d13/d4d/d5c/d78/d104 0 2024-02-16T22:56:30.840 INFO:tasks.workunit.client.0.smithi064.stdout:5/590: mknod d6/d25/d48/d67/d91/ccc 0 2024-02-16T22:56:30.840 INFO:tasks.workunit.client.0.smithi064.stdout:2/745: write d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f49 [597950,109731] 0 2024-02-16T22:56:30.843 INFO:tasks.workunit.client.1.smithi192.stdout:1/723: dread - d3/d5b/fc0 zero size 2024-02-16T22:56:30.844 INFO:tasks.workunit.client.0.smithi064.stdout:2/746: read d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/fbc [731486,47369] 0 2024-02-16T22:56:30.844 INFO:tasks.workunit.client.0.smithi064.stdout:2/747: chown d7/d14/d25/c3c 267671581 1 2024-02-16T22:56:30.845 INFO:tasks.workunit.client.1.smithi192.stdout:3/699: getdents da/d11/d63/d6b 0 2024-02-16T22:56:30.846 INFO:tasks.workunit.client.1.smithi192.stdout:2/809: dread d22/d24/d39/d3d/df2/d67/fd9 [0,4194304] 0 2024-02-16T22:56:30.846 INFO:tasks.workunit.client.1.smithi192.stdout:5/753: symlink d6/d8/lf9 0 2024-02-16T22:56:30.847 INFO:tasks.workunit.client.1.smithi192.stdout:0/745: dread d1/d3/de/f10 [4194304,4194304] 0 2024-02-16T22:56:30.848 INFO:tasks.workunit.client.1.smithi192.stdout:2/810: write d22/d47/dcb/d78/fe4 [50728,110846] 0 2024-02-16T22:56:30.849 INFO:tasks.workunit.client.1.smithi192.stdout:8/760: rename d5/d68/cc4 to d5/d26/d83/d86/d5f/cea 0 2024-02-16T22:56:30.850 INFO:tasks.workunit.client.1.smithi192.stdout:8/761: write d5/db/de/df/fb3 [1471506,67242] 0 2024-02-16T22:56:30.852 INFO:tasks.workunit.client.0.smithi064.stdout:2/748: symlink d7/d14/d25/d46/d55/d7f/lef 0 2024-02-16T22:56:30.855 INFO:tasks.workunit.client.1.smithi192.stdout:4/801: mkdir d1/d15/d48/dc3/d115 0 2024-02-16T22:56:30.857 INFO:tasks.workunit.client.1.smithi192.stdout:7/861: creat d1/f127 x:0 0 0 2024-02-16T22:56:30.857 INFO:tasks.workunit.client.1.smithi192.stdout:7/862: write d1/d2/de/d83/d9f/ff0 [85725,120983] 0 2024-02-16T22:56:30.858 INFO:tasks.workunit.client.0.smithi064.stdout:3/769: getdents d4/d13/d1b/d27/d61 0 2024-02-16T22:56:30.859 INFO:tasks.workunit.client.0.smithi064.stdout:3/770: truncate d4/fd9 857878 0 2024-02-16T22:56:30.860 INFO:tasks.workunit.client.1.smithi192.stdout:1/724: mkdir d3/de/d27/d34/d69/d85/df7 0 2024-02-16T22:56:30.862 INFO:tasks.workunit.client.0.smithi064.stdout:5/591: truncate d6/d1c/f20 3623209 0 2024-02-16T22:56:30.862 INFO:tasks.workunit.client.0.smithi064.stdout:5/592: chown d6/d18/d19/l22 1367955 1 2024-02-16T22:56:30.863 INFO:tasks.workunit.client.1.smithi192.stdout:3/700: truncate da/d11/d20/fab 796887 0 2024-02-16T22:56:30.863 INFO:tasks.workunit.client.0.smithi064.stdout:5/593: truncate d6/d18/d19/fc8 1015553 0 2024-02-16T22:56:30.865 INFO:tasks.workunit.client.0.smithi064.stdout:2/749: link d7/d14/d1e/d34/d3f/f79 d7/d14/d1e/d34/d3f/d9a/dd9/ff0 0 2024-02-16T22:56:30.865 INFO:tasks.workunit.client.1.smithi192.stdout:5/754: creat d6/d12/d4e/d58/d8f/ffa x:0 0 0 2024-02-16T22:56:30.866 INFO:tasks.workunit.client.0.smithi064.stdout:2/750: chown d7/d14/d25/f8a 59 1 2024-02-16T22:56:30.866 INFO:tasks.workunit.client.0.smithi064.stdout:3/771: mknod d4/d13/d1b/d27/d61/dd8/df5/c105 0 2024-02-16T22:56:30.869 INFO:tasks.workunit.client.1.smithi192.stdout:2/811: creat d22/d24/d39/d3d/df2/d99/f10b x:0 0 0 2024-02-16T22:56:30.869 INFO:tasks.workunit.client.0.smithi064.stdout:5/594: dwrite d6/d1c/d43/f4c [0,4194304] 0 2024-02-16T22:56:30.873 INFO:tasks.workunit.client.0.smithi064.stdout:2/751: mkdir d7/d14/d1e/d34/d73/db5/df1 0 2024-02-16T22:56:30.874 INFO:tasks.workunit.client.0.smithi064.stdout:3/772: dwrite d4/d1f/d5b/d51/fa4 [0,4194304] 0 2024-02-16T22:56:30.879 INFO:tasks.workunit.client.0.smithi064.stdout:2/752: mknod d7/d14/d3e/cf2 0 2024-02-16T22:56:30.890 INFO:tasks.workunit.client.1.smithi192.stdout:9/876: link d7/d15/d36/d77/c7f d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/d104/c12b 0 2024-02-16T22:56:30.890 INFO:tasks.workunit.client.1.smithi192.stdout:4/802: creat d1/d26/d34/db9/f116 x:0 0 0 2024-02-16T22:56:30.891 INFO:tasks.workunit.client.0.smithi064.stdout:2/753: stat d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f5c 0 2024-02-16T22:56:30.891 INFO:tasks.workunit.client.1.smithi192.stdout:5/755: fsync d6/d20/fc5 0 2024-02-16T22:56:30.893 INFO:tasks.workunit.client.1.smithi192.stdout:6/794: getdents da/df 0 2024-02-16T22:56:30.893 INFO:tasks.workunit.client.1.smithi192.stdout:2/812: fsync d22/d24/d39/d42/d84/fc5 0 2024-02-16T22:56:30.897 INFO:tasks.workunit.client.1.smithi192.stdout:3/701: dread da/d11/d63/fae [0,4194304] 0 2024-02-16T22:56:30.899 INFO:tasks.workunit.client.1.smithi192.stdout:9/877: mkdir d7/d19/d37/dbf/d12c 0 2024-02-16T22:56:30.900 INFO:tasks.workunit.client.1.smithi192.stdout:4/803: symlink d1/d26/d66/d89/d11/l117 0 2024-02-16T22:56:30.900 INFO:tasks.workunit.client.1.smithi192.stdout:7/863: write d1/d2/d2d/d61/d8a/fde [4054486,1027] 0 2024-02-16T22:56:30.903 INFO:tasks.workunit.client.1.smithi192.stdout:6/795: mkdir da/df/d18/d46/dff 0 2024-02-16T22:56:30.904 INFO:tasks.workunit.client.1.smithi192.stdout:2/813: unlink d22/d24/d39/d42/d52/fb5 0 2024-02-16T22:56:30.907 INFO:tasks.workunit.client.1.smithi192.stdout:2/814: chown d22/d24/c2f 144343321 1 2024-02-16T22:56:30.907 INFO:tasks.workunit.client.0.smithi064.stdout:5/595: truncate d6/d18/d19/f2e 1858432 0 2024-02-16T22:56:30.907 INFO:tasks.workunit.client.0.smithi064.stdout:2/754: write d7/d14/f2e [294010,52320] 0 2024-02-16T22:56:30.908 INFO:tasks.workunit.client.0.smithi064.stdout:0/584: sync 2024-02-16T22:56:30.908 INFO:tasks.workunit.client.0.smithi064.stdout:7/655: sync 2024-02-16T22:56:30.908 INFO:tasks.workunit.client.0.smithi064.stdout:1/589: sync 2024-02-16T22:56:30.910 INFO:tasks.workunit.client.0.smithi064.stdout:5/596: creat d6/d25/d48/d67/d91/fcd x:0 0 0 2024-02-16T22:56:30.911 INFO:tasks.workunit.client.1.smithi192.stdout:9/878: rename d7/d19/d5e/d81/dda/ca9 to d7/d19/d37/dbf/d10b/c12d 0 2024-02-16T22:56:30.911 INFO:tasks.workunit.client.0.smithi064.stdout:3/773: dwrite d4/d14/f46 [0,4194304] 0 2024-02-16T22:56:30.912 INFO:tasks.workunit.client.0.smithi064.stdout:1/590: readlink d2/df/d11/d1a/d30/l55 0 2024-02-16T22:56:30.912 INFO:tasks.workunit.client.1.smithi192.stdout:5/756: mknod d6/d8/d80/cfb 0 2024-02-16T22:56:30.912 INFO:tasks.workunit.client.1.smithi192.stdout:9/879: write d7/d19/d37/d5a/fa7 [1848221,122039] 0 2024-02-16T22:56:30.913 INFO:tasks.workunit.client.1.smithi192.stdout:7/864: stat d1/d2/f7e 0 2024-02-16T22:56:30.913 INFO:tasks.workunit.client.0.smithi064.stdout:7/656: write d5/d33/d45/d64/d56/faa [834683,52804] 0 2024-02-16T22:56:30.915 INFO:tasks.workunit.client.1.smithi192.stdout:7/865: chown d1/d2/de/d2c/d49/d54/d9d/ddd/lf2 0 1 2024-02-16T22:56:30.917 INFO:tasks.workunit.client.1.smithi192.stdout:0/746: link d1/d3/d29/f27 d1/d3/fec 0 2024-02-16T22:56:30.917 INFO:tasks.workunit.client.0.smithi064.stdout:2/755: mknod d7/d14/d1e/d34/d73/db5/cf3 0 2024-02-16T22:56:30.918 INFO:tasks.workunit.client.1.smithi192.stdout:0/747: dread - d1/dc7/d9c/fe6 zero size 2024-02-16T22:56:30.918 INFO:tasks.workunit.client.1.smithi192.stdout:9/880: dwrite d7/d15/d36/d38/d3f/d8a/def/df6/f110 [0,4194304] 0 2024-02-16T22:56:30.919 INFO:tasks.workunit.client.1.smithi192.stdout:9/881: chown d7/d15/d36/f7b 0 1 2024-02-16T22:56:30.923 INFO:tasks.workunit.client.0.smithi064.stdout:0/585: dwrite da/d26/d3f/f6b [4194304,4194304] 0 2024-02-16T22:56:30.924 INFO:tasks.workunit.client.0.smithi064.stdout:3/774: mknod d4/d13/d4d/d5c/c106 0 2024-02-16T22:56:30.924 INFO:tasks.workunit.client.1.smithi192.stdout:5/757: rmdir d6/d20 39 2024-02-16T22:56:30.926 INFO:tasks.workunit.client.0.smithi064.stdout:0/586: chown da/dc/de/d37/d88/f9c 0 1 2024-02-16T22:56:30.927 INFO:tasks.workunit.client.0.smithi064.stdout:5/597: rename d6/d25/d48/f5e to d6/d25/d48/d67/fce 0 2024-02-16T22:56:30.929 INFO:tasks.workunit.client.0.smithi064.stdout:1/591: dwrite d2/df/d14/d2a/f87 [4194304,4194304] 0 2024-02-16T22:56:30.933 INFO:tasks.workunit.client.0.smithi064.stdout:5/598: truncate d6/d25/da4/dc1/f76 541467 0 2024-02-16T22:56:30.936 INFO:tasks.workunit.client.0.smithi064.stdout:2/756: write d7/fb1 [28179,25661] 0 2024-02-16T22:56:30.937 INFO:tasks.workunit.client.1.smithi192.stdout:7/866: creat d1/d2/d37/d86/f128 x:0 0 0 2024-02-16T22:56:30.937 INFO:tasks.workunit.client.0.smithi064.stdout:0/587: dwrite da/d26/f82 [0,4194304] 0 2024-02-16T22:56:30.939 INFO:tasks.workunit.client.0.smithi064.stdout:0/588: fsync da/dc/f4b 0 2024-02-16T22:56:30.944 INFO:tasks.workunit.client.1.smithi192.stdout:0/748: unlink d1/d3/de/d22/f60 0 2024-02-16T22:56:30.946 INFO:tasks.workunit.client.0.smithi064.stdout:7/657: mkdir d5/d33/d42/d81/dc4/dcc 0 2024-02-16T22:56:30.947 INFO:tasks.workunit.client.0.smithi064.stdout:7/658: stat d5/d33/d45/d64/d56/c5a 0 2024-02-16T22:56:30.948 INFO:tasks.workunit.client.1.smithi192.stdout:9/882: unlink d7/d15/d36/c73 0 2024-02-16T22:56:30.950 INFO:tasks.workunit.client.1.smithi192.stdout:5/758: chown d6/d12/d4e/c95 1151117122 1 2024-02-16T22:56:30.952 INFO:tasks.workunit.client.1.smithi192.stdout:4/804: link d1/d26/d6f/fab d1/d15/d48/dc3/deb/f118 0 2024-02-16T22:56:30.953 INFO:tasks.workunit.client.0.smithi064.stdout:5/599: symlink d6/d25/d48/d67/lcf 0 2024-02-16T22:56:30.954 INFO:tasks.workunit.client.0.smithi064.stdout:5/600: read - d6/d25/d48/d67/d91/da5/fbd zero size 2024-02-16T22:56:30.955 INFO:tasks.workunit.client.1.smithi192.stdout:7/867: rmdir d1/d2/de/d2c/d49/d54/d9d/ddd 39 2024-02-16T22:56:30.957 INFO:tasks.workunit.client.1.smithi192.stdout:3/702: dwrite da/d11/d63/d6b/d66/fd2 [0,4194304] 0 2024-02-16T22:56:30.959 INFO:tasks.workunit.client.1.smithi192.stdout:3/703: chown da/d6a/f84 29 1 2024-02-16T22:56:30.960 INFO:tasks.workunit.client.1.smithi192.stdout:4/805: dwrite d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:30.962 INFO:tasks.workunit.client.1.smithi192.stdout:4/806: chown d1/d26/d66/ld9 702 1 2024-02-16T22:56:30.962 INFO:tasks.workunit.client.0.smithi064.stdout:7/659: truncate d5/d6/f5b 766630 0 2024-02-16T22:56:30.962 INFO:tasks.workunit.client.0.smithi064.stdout:7/660: chown d5/cc 43155981 1 2024-02-16T22:56:30.963 INFO:tasks.workunit.client.1.smithi192.stdout:4/807: stat d1/d26/d6f/df4 0 2024-02-16T22:56:30.963 INFO:tasks.workunit.client.0.smithi064.stdout:7/661: fdatasync d5/d6/d54/f5e 0 2024-02-16T22:56:30.965 INFO:tasks.workunit.client.1.smithi192.stdout:5/759: unlink d6/d12/d4e/d46/f5d 0 2024-02-16T22:56:30.965 INFO:tasks.workunit.client.0.smithi064.stdout:3/775: write d4/d1f/f20 [3474613,32212] 0 2024-02-16T22:56:30.966 INFO:tasks.workunit.client.1.smithi192.stdout:3/704: dwrite da/f3d [0,4194304] 0 2024-02-16T22:56:30.968 INFO:tasks.workunit.client.1.smithi192.stdout:3/705: write da/d11/fd9 [1778527,58143] 0 2024-02-16T22:56:30.978 INFO:tasks.workunit.client.1.smithi192.stdout:0/749: mknod d1/d3/de/d22/d37/d67/d34/ced 0 2024-02-16T22:56:30.982 INFO:tasks.workunit.client.1.smithi192.stdout:4/808: fsync d1/d15/d48/d8d/fba 0 2024-02-16T22:56:30.983 INFO:tasks.workunit.client.1.smithi192.stdout:4/809: chown d1/d15/d48/fbd 84289 1 2024-02-16T22:56:30.983 INFO:tasks.workunit.client.0.smithi064.stdout:7/662: mkdir d5/d6/d65/d7f/dcd 0 2024-02-16T22:56:30.984 INFO:tasks.workunit.client.1.smithi192.stdout:4/810: fsync d1/d26/d66/d89/f68 0 2024-02-16T22:56:30.984 INFO:tasks.workunit.client.0.smithi064.stdout:2/757: dwrite d7/d14/d25/f4a [0,4194304] 0 2024-02-16T22:56:30.986 INFO:tasks.workunit.client.0.smithi064.stdout:0/589: dwrite da/d26/f66 [0,4194304] 0 2024-02-16T22:56:30.986 INFO:tasks.workunit.client.0.smithi064.stdout:1/592: rename d2/df/d14/d18/d48/c98 to d2/df/d11/d1a/d30/dbd/d91/d56/ccc 0 2024-02-16T22:56:30.992 INFO:tasks.workunit.client.0.smithi064.stdout:3/776: symlink d4/d13/d4d/d5c/d78/l107 0 2024-02-16T22:56:30.992 INFO:tasks.workunit.client.0.smithi064.stdout:5/601: write d6/fa [2818082,70685] 0 2024-02-16T22:56:30.997 INFO:tasks.workunit.client.1.smithi192.stdout:9/883: creat d7/d19/d37/f12e x:0 0 0 2024-02-16T22:56:30.999 INFO:tasks.workunit.client.1.smithi192.stdout:7/868: dwrite d1/d2/de/d2c/f8b [0,4194304] 0 2024-02-16T22:56:31.000 INFO:tasks.workunit.client.0.smithi064.stdout:7/663: creat d5/d6/d65/da7/fce x:0 0 0 2024-02-16T22:56:31.001 INFO:tasks.workunit.client.0.smithi064.stdout:7/664: readlink d5/d7d/lc7 0 2024-02-16T22:56:31.002 INFO:tasks.workunit.client.0.smithi064.stdout:2/758: rmdir d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44 39 2024-02-16T22:56:31.003 INFO:tasks.workunit.client.0.smithi064.stdout:6/688: sync 2024-02-16T22:56:31.003 INFO:tasks.workunit.client.0.smithi064.stdout:4/639: sync 2024-02-16T22:56:31.004 INFO:tasks.workunit.client.0.smithi064.stdout:1/593: stat d2/df/d14/c22 0 2024-02-16T22:56:31.005 INFO:tasks.workunit.client.1.smithi192.stdout:4/811: creat d1/d15/d48/dc3/deb/f119 x:0 0 0 2024-02-16T22:56:31.006 INFO:tasks.workunit.client.1.smithi192.stdout:4/812: write d1/d15/f4f [5095005,100641] 0 2024-02-16T22:56:31.010 INFO:tasks.workunit.client.1.smithi192.stdout:3/706: unlink da/f51 0 2024-02-16T22:56:31.010 INFO:tasks.workunit.client.1.smithi192.stdout:3/707: chown da/d11/d35/cbb 4027 1 2024-02-16T22:56:31.012 INFO:tasks.workunit.client.0.smithi064.stdout:5/602: mknod d6/d1c/d4b/cd0 0 2024-02-16T22:56:31.014 INFO:tasks.workunit.client.0.smithi064.stdout:2/759: fdatasync d7/d14/d1e/d34/d3f/d9a/dd9/ff0 0 2024-02-16T22:56:31.016 INFO:tasks.workunit.client.0.smithi064.stdout:2/760: write d7/d14/d1e/f92 [1546333,92273] 0 2024-02-16T22:56:31.016 INFO:tasks.workunit.client.0.smithi064.stdout:6/689: chown d0/d6/d10/l34 87299430 1 2024-02-16T22:56:31.018 INFO:tasks.workunit.client.1.smithi192.stdout:7/869: creat d1/d2/de/d2c/d49/d54/dd8/d117/f129 x:0 0 0 2024-02-16T22:56:31.018 INFO:tasks.workunit.client.0.smithi064.stdout:3/777: dread d4/d1f/f32 [0,4194304] 0 2024-02-16T22:56:31.021 INFO:tasks.workunit.client.1.smithi192.stdout:5/760: creat d6/d8/ffc x:0 0 0 2024-02-16T22:56:31.025 INFO:tasks.workunit.client.0.smithi064.stdout:1/594: mknod d2/df/d11/d1a/d1d/ccd 0 2024-02-16T22:56:31.025 INFO:tasks.workunit.client.1.smithi192.stdout:4/813: readlink d1/l27 0 2024-02-16T22:56:31.026 INFO:tasks.workunit.client.0.smithi064.stdout:4/640: fsync dc/dd/d32/d46/d82/f9e 0 2024-02-16T22:56:31.030 INFO:tasks.workunit.client.0.smithi064.stdout:6/690: dwrite d0/d12/d7b/fa7 [0,4194304] 0 2024-02-16T22:56:31.032 INFO:tasks.workunit.client.1.smithi192.stdout:9/884: mknod d7/d19/c12f 0 2024-02-16T22:56:31.034 INFO:tasks.workunit.client.0.smithi064.stdout:2/761: mknod d7/d14/d1e/d34/d3f/d9a/dcf/cf4 0 2024-02-16T22:56:31.034 INFO:tasks.workunit.client.1.smithi192.stdout:7/870: mknod d1/d2/de/d2c/d49/c12a 0 2024-02-16T22:56:31.036 INFO:tasks.workunit.client.1.smithi192.stdout:5/761: creat d6/d12/d1d/d56/ffd x:0 0 0 2024-02-16T22:56:31.036 INFO:tasks.workunit.client.1.smithi192.stdout:3/708: dread da/d11/d20/d39/d53/fce [0,4194304] 0 2024-02-16T22:56:31.037 INFO:tasks.workunit.client.1.smithi192.stdout:3/709: dread - da/d11/d20/d39/d53/f5b zero size 2024-02-16T22:56:31.040 INFO:tasks.workunit.client.1.smithi192.stdout:0/750: getdents d1/d3/de/d22/d37/d47/dd0 0 2024-02-16T22:56:31.042 INFO:tasks.workunit.client.1.smithi192.stdout:8/762: sync 2024-02-16T22:56:31.045 INFO:tasks.workunit.client.1.smithi192.stdout:7/871: chown d1/d2/de/d2c/d49/da4/d104/f114 4878 1 2024-02-16T22:56:31.045 INFO:tasks.workunit.client.0.smithi064.stdout:7/665: dwrite d5/d6/d54/f3f [0,4194304] 0 2024-02-16T22:56:31.046 INFO:tasks.workunit.client.0.smithi064.stdout:7/666: rename d5 to d5/d33/d53/dcf 22 2024-02-16T22:56:31.051 INFO:tasks.workunit.client.1.smithi192.stdout:4/814: creat d1/d26/d6f/df4/f11a x:0 0 0 2024-02-16T22:56:31.052 INFO:tasks.workunit.client.1.smithi192.stdout:5/762: dwrite d6/f29 [0,4194304] 0 2024-02-16T22:56:31.053 INFO:tasks.workunit.client.1.smithi192.stdout:3/710: creat da/d11/d2b/ff3 x:0 0 0 2024-02-16T22:56:31.066 INFO:tasks.workunit.client.1.smithi192.stdout:4/815: mkdir d1/d15/d48/d8d/d11b 0 2024-02-16T22:56:31.069 INFO:tasks.workunit.client.1.smithi192.stdout:3/711: mknod da/d11/d20/d39/d53/dc7/cf4 0 2024-02-16T22:56:31.073 INFO:tasks.workunit.client.1.smithi192.stdout:6/796: sync 2024-02-16T22:56:31.073 INFO:tasks.workunit.client.1.smithi192.stdout:1/725: sync 2024-02-16T22:56:31.074 INFO:tasks.workunit.client.1.smithi192.stdout:0/751: rename d1/d3/de/d22/d37/d67/d34/da1/fa9 to d1/dc7/d9c/fee 0 2024-02-16T22:56:31.077 INFO:tasks.workunit.client.1.smithi192.stdout:8/763: mknod d5/db/ceb 0 2024-02-16T22:56:31.085 INFO:tasks.workunit.client.1.smithi192.stdout:4/816: rmdir d1/d15 39 2024-02-16T22:56:31.088 INFO:tasks.workunit.client.1.smithi192.stdout:5/763: fsync d6/d8/d80/f8a 0 2024-02-16T22:56:31.089 INFO:tasks.workunit.client.1.smithi192.stdout:3/712: dread da/f22 [0,4194304] 0 2024-02-16T22:56:31.089 INFO:tasks.workunit.client.1.smithi192.stdout:4/817: dread d1/d26/d6f/df4/f105 [0,4194304] 0 2024-02-16T22:56:31.090 INFO:tasks.workunit.client.1.smithi192.stdout:5/764: truncate d6/d8/d77/f85 1243010 0 2024-02-16T22:56:31.093 INFO:tasks.workunit.client.1.smithi192.stdout:6/797: rename da/df/d6b/d7d/fdc to da/d76/dd5/de2/f100 0 2024-02-16T22:56:31.096 INFO:tasks.workunit.client.1.smithi192.stdout:7/872: truncate d1/d2/d2d/d61/d88/d90/f64 789843 0 2024-02-16T22:56:31.098 INFO:tasks.workunit.client.1.smithi192.stdout:6/798: dwrite da/df/d12/d15/f21 [0,4194304] 0 2024-02-16T22:56:31.098 INFO:tasks.workunit.client.1.smithi192.stdout:9/885: link d7/d19/d37/d5a/l123 d7/d19/d5b/l130 0 2024-02-16T22:56:31.100 INFO:tasks.workunit.client.1.smithi192.stdout:9/886: write d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/fa1 [3976580,59468] 0 2024-02-16T22:56:31.101 INFO:tasks.workunit.client.1.smithi192.stdout:9/887: chown d7/d19/d28/ded 8657 1 2024-02-16T22:56:31.107 INFO:tasks.workunit.client.1.smithi192.stdout:8/764: write d5/db/d47/f4a [1714192,41002] 0 2024-02-16T22:56:31.108 INFO:tasks.workunit.client.1.smithi192.stdout:3/713: fsync da/d11/d35/fd8 0 2024-02-16T22:56:31.110 INFO:tasks.workunit.client.1.smithi192.stdout:4/818: mkdir d1/d26/d34/d11c 0 2024-02-16T22:56:31.119 INFO:tasks.workunit.client.1.smithi192.stdout:4/819: dread d1/d26/d6f/f8f [0,4194304] 0 2024-02-16T22:56:31.120 INFO:tasks.workunit.client.1.smithi192.stdout:4/820: fdatasync d1/d26/d66/d89/da8/fff 0 2024-02-16T22:56:31.123 INFO:tasks.workunit.client.1.smithi192.stdout:9/888: mkdir d7/d15/d36/d38/d3f/d8a/d131 0 2024-02-16T22:56:31.125 INFO:tasks.workunit.client.1.smithi192.stdout:2/815: sync 2024-02-16T22:56:31.129 INFO:tasks.workunit.client.1.smithi192.stdout:8/765: dwrite d5/d16/fd2 [0,4194304] 0 2024-02-16T22:56:31.132 INFO:tasks.workunit.client.1.smithi192.stdout:1/726: link d3/de/d27/d34/d69/fef d3/de/d27/d34/d52/ff8 0 2024-02-16T22:56:31.133 INFO:tasks.workunit.client.1.smithi192.stdout:7/873: creat d1/d2/de/d83/d9f/dfb/f12b x:0 0 0 2024-02-16T22:56:31.134 INFO:tasks.workunit.client.1.smithi192.stdout:1/727: write d3/d5b/db1/fde [265902,78649] 0 2024-02-16T22:56:31.135 INFO:tasks.workunit.client.1.smithi192.stdout:7/874: dread - d1/d2/de/d2c/d49/d54/d9d/f11c zero size 2024-02-16T22:56:31.136 INFO:tasks.workunit.client.1.smithi192.stdout:6/799: mkdir da/df/d12/d57/d81/dfc/d101 0 2024-02-16T22:56:31.140 INFO:tasks.workunit.client.0.smithi064.stdout:0/590: link da/dc/f1e da/dc/de/d12/fc7 0 2024-02-16T22:56:31.141 INFO:tasks.workunit.client.0.smithi064.stdout:0/591: chown da/dc/de/d12/d2c/d2e/f36 36 1 2024-02-16T22:56:31.142 INFO:tasks.workunit.client.0.smithi064.stdout:0/592: write da/d26/f66 [3608273,67683] 0 2024-02-16T22:56:31.145 INFO:tasks.workunit.client.0.smithi064.stdout:3/778: symlink d4/d13/d1b/d27/d44/d97/l108 0 2024-02-16T22:56:31.145 INFO:tasks.workunit.client.1.smithi192.stdout:9/889: dread - d7/d15/d36/d77/dbd/de0/f11e zero size 2024-02-16T22:56:31.145 INFO:tasks.workunit.client.0.smithi064.stdout:3/779: chown d4/d14/d2b/cdf 948753118 1 2024-02-16T22:56:31.146 INFO:tasks.workunit.client.1.smithi192.stdout:7/875: dread d1/d2/f7e [0,4194304] 0 2024-02-16T22:56:31.146 INFO:tasks.workunit.client.0.smithi064.stdout:1/595: fdatasync d2/df/d14/d18/d48/d84/f9c 0 2024-02-16T22:56:31.149 INFO:tasks.workunit.client.0.smithi064.stdout:6/691: truncate d0/d3c/fb7 98942 0 2024-02-16T22:56:31.149 INFO:tasks.workunit.client.0.smithi064.stdout:2/762: truncate d7/d14/f36 2500624 0 2024-02-16T22:56:31.150 INFO:tasks.workunit.client.1.smithi192.stdout:2/816: dread f2 [0,4194304] 0 2024-02-16T22:56:31.151 INFO:tasks.workunit.client.1.smithi192.stdout:5/765: creat d6/d12/d4e/d58/ffe x:0 0 0 2024-02-16T22:56:31.151 INFO:tasks.workunit.client.0.smithi064.stdout:2/763: chown d7/d14/d1e/cae 138 1 2024-02-16T22:56:31.151 INFO:tasks.workunit.client.0.smithi064.stdout:8/602: sync 2024-02-16T22:56:31.151 INFO:tasks.workunit.client.0.smithi064.stdout:9/590: sync 2024-02-16T22:56:31.151 INFO:tasks.workunit.client.1.smithi192.stdout:5/766: fdatasync d6/d12/d2d/fc0 0 2024-02-16T22:56:31.152 INFO:tasks.workunit.client.1.smithi192.stdout:6/800: dread da/df/d12/d15/d1b/f60 [0,4194304] 0 2024-02-16T22:56:31.153 INFO:tasks.workunit.client.1.smithi192.stdout:0/752: rename d1/d3/de/d22/d37/d67/d34/cb4 to d1/d3/de/cef 0 2024-02-16T22:56:31.161 INFO:tasks.workunit.client.1.smithi192.stdout:7/876: dread d1/d2/d2d/d61/d88/d90/d4c/f70 [0,4194304] 0 2024-02-16T22:56:31.184 INFO:tasks.workunit.client.1.smithi192.stdout:9/890: fsync d7/f2e 0 2024-02-16T22:56:31.185 INFO:tasks.workunit.client.0.smithi064.stdout:1/596: creat d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce x:0 0 0 2024-02-16T22:56:31.186 INFO:tasks.workunit.client.1.smithi192.stdout:9/891: write d7/d15/d36/d38/f11a [401345,69195] 0 2024-02-16T22:56:31.186 INFO:tasks.workunit.client.1.smithi192.stdout:9/892: chown d7/d15/d36/l7c 385116 1 2024-02-16T22:56:31.191 INFO:tasks.workunit.client.0.smithi064.stdout:2/764: truncate d7/d14/d25/f52 2975592 0 2024-02-16T22:56:31.191 INFO:tasks.workunit.client.1.smithi192.stdout:3/714: symlink da/d6a/lf5 0 2024-02-16T22:56:31.193 INFO:tasks.workunit.client.0.smithi064.stdout:9/591: truncate d3/d9/d6a/db1/d2a/d2e/f5f 229659 0 2024-02-16T22:56:31.194 INFO:tasks.workunit.client.1.smithi192.stdout:2/817: creat d22/d47/dcb/d78/f10c x:0 0 0 2024-02-16T22:56:31.195 INFO:tasks.workunit.client.0.smithi064.stdout:4/641: rmdir dc/d4a/d92 0 2024-02-16T22:56:31.195 INFO:tasks.workunit.client.1.smithi192.stdout:2/818: dread - d22/d24/d39/d42/d85/d98/fdb zero size 2024-02-16T22:56:31.195 INFO:tasks.workunit.client.1.smithi192.stdout:2/819: dread - d22/d47/dcb/d78/f10c zero size 2024-02-16T22:56:31.197 INFO:tasks.workunit.client.0.smithi064.stdout:3/780: mknod d4/d14/c109 0 2024-02-16T22:56:31.198 INFO:tasks.workunit.client.0.smithi064.stdout:3/781: readlink d4/d13/d1b/d27/d61/d63/ldd 0 2024-02-16T22:56:31.199 INFO:tasks.workunit.client.0.smithi064.stdout:1/597: creat d2/df/d14/d18/d48/d6b/d75/fcf x:0 0 0 2024-02-16T22:56:31.200 INFO:tasks.workunit.client.0.smithi064.stdout:6/692: dread d0/d5/df/f8d [0,4194304] 0 2024-02-16T22:56:31.200 INFO:tasks.workunit.client.0.smithi064.stdout:1/598: stat d2/df/d14/l1b 0 2024-02-16T22:56:31.201 INFO:tasks.workunit.client.1.smithi192.stdout:0/753: mknod d1/d3/de/d22/cf0 0 2024-02-16T22:56:31.201 INFO:tasks.workunit.client.1.smithi192.stdout:0/754: stat d1/d3/de/d22/c25 0 2024-02-16T22:56:31.201 INFO:tasks.workunit.client.1.smithi192.stdout:0/755: fdatasync d1/d3/de/d22/d37/d47/fda 0 2024-02-16T22:56:31.202 INFO:tasks.workunit.client.1.smithi192.stdout:0/756: chown d1/d3/de/d22/d37/d47/c49 1701 1 2024-02-16T22:56:31.203 INFO:tasks.workunit.client.0.smithi064.stdout:8/603: symlink df/d31/dc1/lcc 0 2024-02-16T22:56:31.203 INFO:tasks.workunit.client.1.smithi192.stdout:0/757: fdatasync d1/d3/d39/f79 0 2024-02-16T22:56:31.209 INFO:tasks.workunit.client.0.smithi064.stdout:9/592: unlink d3/f18 0 2024-02-16T22:56:31.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/767: write d6/d12/d4e/d46/f6b [1101795,110330] 0 2024-02-16T22:56:31.209 INFO:tasks.workunit.client.1.smithi192.stdout:4/821: mkdir d1/d15/d48/dc3/d115/d11d 0 2024-02-16T22:56:31.210 INFO:tasks.workunit.client.1.smithi192.stdout:4/822: fdatasync d1/d15/d29/d3c/f112 0 2024-02-16T22:56:31.211 INFO:tasks.workunit.client.0.smithi064.stdout:4/642: rmdir dc 39 2024-02-16T22:56:31.212 INFO:tasks.workunit.client.0.smithi064.stdout:3/782: mknod d4/d13/d1b/d27/d44/c10a 0 2024-02-16T22:56:31.217 INFO:tasks.workunit.client.1.smithi192.stdout:2/820: stat d22/ca7 0 2024-02-16T22:56:31.220 INFO:tasks.workunit.client.0.smithi064.stdout:8/604: creat df/d31/d8f/dab/fcd x:0 0 0 2024-02-16T22:56:31.221 INFO:tasks.workunit.client.0.smithi064.stdout:2/765: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/f1f [0,4194304] 0 2024-02-16T22:56:31.223 INFO:tasks.workunit.client.1.smithi192.stdout:8/766: creat d5/db/de/fec x:0 0 0 2024-02-16T22:56:31.224 INFO:tasks.workunit.client.1.smithi192.stdout:7/877: creat d1/d2/de/d2c/d49/d54/d9d/ddd/f12c x:0 0 0 2024-02-16T22:56:31.227 INFO:tasks.workunit.client.0.smithi064.stdout:9/593: creat d3/d9/d6a/db1/d2a/d2e/d31/fc3 x:0 0 0 2024-02-16T22:56:31.231 INFO:tasks.workunit.client.0.smithi064.stdout:6/693: dwrite d0/d12/d7b/f99 [0,4194304] 0 2024-02-16T22:56:31.231 INFO:tasks.workunit.client.0.smithi064.stdout:1/599: truncate d2/df/d14/f52 754092 0 2024-02-16T22:56:31.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/878: dwrite d1/d2/de/d83/d9f/ff0 [0,4194304] 0 2024-02-16T22:56:31.233 INFO:tasks.workunit.client.0.smithi064.stdout:1/600: fdatasync d2/df/d11/d1a/d1d/fca 0 2024-02-16T22:56:31.233 INFO:tasks.workunit.client.1.smithi192.stdout:9/893: creat d7/d15/d36/d77/dbd/d101/f132 x:0 0 0 2024-02-16T22:56:31.234 INFO:tasks.workunit.client.0.smithi064.stdout:8/605: symlink df/d31/dc1/lce 0 2024-02-16T22:56:31.234 INFO:tasks.workunit.client.1.smithi192.stdout:7/879: dread - d1/d2/de/d2c/d49/d54/d9d/ddd/f12c zero size 2024-02-16T22:56:31.234 INFO:tasks.workunit.client.1.smithi192.stdout:9/894: write d7/d19/d37/f12e [595369,18066] 0 2024-02-16T22:56:31.235 INFO:tasks.workunit.client.1.smithi192.stdout:7/880: readlink d1/d2/d2d/d61/d88/lbf 0 2024-02-16T22:56:31.236 INFO:tasks.workunit.client.0.smithi064.stdout:8/606: dread df/d31/d2f/d4f/f99 [0,4194304] 0 2024-02-16T22:56:31.237 INFO:tasks.workunit.client.1.smithi192.stdout:6/801: creat da/df/d12/d15/f102 x:0 0 0 2024-02-16T22:56:31.240 INFO:tasks.workunit.client.0.smithi064.stdout:9/594: rename d3/d9/d1e/d3e/c6b to d3/d9/d6a/db1/d2a/cc4 0 2024-02-16T22:56:31.245 INFO:tasks.workunit.client.0.smithi064.stdout:6/694: fsync d0/d1d/f84 0 2024-02-16T22:56:31.246 INFO:tasks.workunit.client.0.smithi064.stdout:6/695: chown d0/d12/d7b/la3 2304618 1 2024-02-16T22:56:31.246 INFO:tasks.workunit.client.0.smithi064.stdout:6/696: stat d0/d12/d7b/la3 0 2024-02-16T22:56:31.248 INFO:tasks.workunit.client.0.smithi064.stdout:1/601: creat d2/df/d14/d18/d48/d6b/fd0 x:0 0 0 2024-02-16T22:56:31.248 INFO:tasks.workunit.client.1.smithi192.stdout:1/728: link d3/d5b/d77/ccb d3/de/d27/d34/d52/d72/cf9 0 2024-02-16T22:56:31.254 INFO:tasks.workunit.client.0.smithi064.stdout:8/607: symlink df/d31/d2f/d68/lcf 0 2024-02-16T22:56:31.257 INFO:tasks.workunit.client.0.smithi064.stdout:9/595: creat d3/d88/fc5 x:0 0 0 2024-02-16T22:56:31.257 INFO:tasks.workunit.client.0.smithi064.stdout:2/766: write d7/d14/d1e/d34/f61 [3333395,102317] 0 2024-02-16T22:56:31.258 INFO:tasks.workunit.client.1.smithi192.stdout:7/881: dread - d1/de1/dd0/fef zero size 2024-02-16T22:56:31.262 INFO:tasks.workunit.client.1.smithi192.stdout:0/758: truncate d1/d3/de/d22/d37/d67/d75/db0/fb2 3451312 0 2024-02-16T22:56:31.263 INFO:tasks.workunit.client.0.smithi064.stdout:6/697: rename d0/d6/cd to d0/d6/da/d56/db9/ce7 0 2024-02-16T22:56:31.265 INFO:tasks.workunit.client.1.smithi192.stdout:1/729: unlink d3/de/d27/d34/d36/d47/f4b 0 2024-02-16T22:56:31.266 INFO:tasks.workunit.client.0.smithi064.stdout:1/602: read - d2/df/d11/d1a/d30/dbd/d91/f65 zero size 2024-02-16T22:56:31.266 INFO:tasks.workunit.client.1.smithi192.stdout:1/730: stat d3/de/d27/d34/d69/d85/d89/l95 0 2024-02-16T22:56:31.266 INFO:tasks.workunit.client.1.smithi192.stdout:5/768: rename d6/d8/d80/l66 to d6/d12/d4e/d58/lff 0 2024-02-16T22:56:31.269 INFO:tasks.workunit.client.0.smithi064.stdout:8/608: creat df/d31/d2f/d7f/fd0 x:0 0 0 2024-02-16T22:56:31.270 INFO:tasks.workunit.client.1.smithi192.stdout:9/895: write d7/d15/d36/d77/ff1 [125515,48121] 0 2024-02-16T22:56:31.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/769: dwrite d6/d12/d4e/d58/ff8 [0,4194304] 0 2024-02-16T22:56:31.273 INFO:tasks.workunit.client.1.smithi192.stdout:9/896: truncate d7/d15/d36/d77/dbd/ffc 807062 0 2024-02-16T22:56:31.273 INFO:tasks.workunit.client.0.smithi064.stdout:4/643: link dc/dd/d32/d58/f70 dc/d1d/dbd/fca 0 2024-02-16T22:56:31.275 INFO:tasks.workunit.client.1.smithi192.stdout:6/802: creat da/d40/db6/db8/dfa/f103 x:0 0 0 2024-02-16T22:56:31.277 INFO:tasks.workunit.client.0.smithi064.stdout:4/644: dread dc/dd/d32/d58/f8e [0,4194304] 0 2024-02-16T22:56:31.280 INFO:tasks.workunit.client.1.smithi192.stdout:1/731: readlink d3/de/d27/d34/d36/d47/l91 0 2024-02-16T22:56:31.281 INFO:tasks.workunit.client.1.smithi192.stdout:1/732: readlink d3/de/d27/l8f 0 2024-02-16T22:56:31.282 INFO:tasks.workunit.client.1.smithi192.stdout:5/770: dwrite d6/d12/d4e/d5b/daf/fcd [0,4194304] 0 2024-02-16T22:56:31.284 INFO:tasks.workunit.client.0.smithi064.stdout:1/603: dread d2/df/d14/f96 [0,4194304] 0 2024-02-16T22:56:31.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:30 smithi192 ceph-mon[36254]: pgmap v20: 65 pgs: 65 active+clean; 2.6 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 81 MiB/s rd, 197 MiB/s wr, 402 op/s 2024-02-16T22:56:31.286 INFO:tasks.workunit.client.1.smithi192.stdout:4/823: rename d1/d26/d66/d89/d11/fb7 to d1/d15/d48/dc3/d115/d11d/f11e 0 2024-02-16T22:56:31.288 INFO:tasks.workunit.client.1.smithi192.stdout:4/824: truncate d1/d26/fa5 4510050 0 2024-02-16T22:56:31.288 INFO:tasks.workunit.client.1.smithi192.stdout:2/821: link d22/d47/f88 d22/d24/d39/d3d/f10d 0 2024-02-16T22:56:31.289 INFO:tasks.workunit.client.0.smithi064.stdout:8/609: creat df/d31/d8f/dab/fd1 x:0 0 0 2024-02-16T22:56:31.292 INFO:tasks.workunit.client.0.smithi064.stdout:2/767: fsync d7/d14/d25/d46/d55/d57/f7b 0 2024-02-16T22:56:31.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/767: getdents d5/d26/d8f/dc0 0 2024-02-16T22:56:31.295 INFO:tasks.workunit.client.1.smithi192.stdout:7/882: dwrite d1/d69/f7f [0,4194304] 0 2024-02-16T22:56:31.295 INFO:tasks.workunit.client.1.smithi192.stdout:0/759: dread d1/d3/f12 [0,4194304] 0 2024-02-16T22:56:31.296 INFO:tasks.workunit.client.1.smithi192.stdout:6/803: mknod da/db9/c104 0 2024-02-16T22:56:31.296 INFO:tasks.workunit.client.1.smithi192.stdout:3/715: sync 2024-02-16T22:56:31.297 INFO:tasks.workunit.client.0.smithi064.stdout:4/645: dread dc/dd/d16/f5d [0,4194304] 0 2024-02-16T22:56:31.297 INFO:tasks.workunit.client.1.smithi192.stdout:6/804: read - da/df/d58/f9a zero size 2024-02-16T22:56:31.298 INFO:tasks.workunit.client.0.smithi064.stdout:9/596: dwrite d3/d9/d6a/db1/d2a/d2e/d31/f4e [0,4194304] 0 2024-02-16T22:56:31.300 INFO:tasks.workunit.client.0.smithi064.stdout:4/646: read - dc/dd/d5b/fb2 zero size 2024-02-16T22:56:31.301 INFO:tasks.workunit.client.0.smithi064.stdout:8/610: mkdir df/d31/d2f/d3e/d5f/d93/dd2 0 2024-02-16T22:56:31.301 INFO:tasks.workunit.client.0.smithi064.stdout:9/597: dread - d3/d9/d1e/d42/d7b/fa8 zero size 2024-02-16T22:56:31.301 INFO:tasks.workunit.client.0.smithi064.stdout:4/647: chown dc/fba 0 1 2024-02-16T22:56:31.302 INFO:tasks.workunit.client.0.smithi064.stdout:8/611: readlink df/d31/d47/l7b 0 2024-02-16T22:56:31.305 INFO:tasks.workunit.client.0.smithi064.stdout:2/768: chown d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44 1015537587 1 2024-02-16T22:56:31.307 INFO:tasks.workunit.client.0.smithi064.stdout:2/769: stat d7/d14/d25/d46/d55/d57/dbe/cc0 0 2024-02-16T22:56:31.310 INFO:tasks.workunit.client.1.smithi192.stdout:2/822: truncate d22/fc7 537102 0 2024-02-16T22:56:31.313 INFO:tasks.workunit.client.0.smithi064.stdout:6/698: mkdir d0/d6/d10/d33/d72/de8 0 2024-02-16T22:56:31.314 INFO:tasks.workunit.client.1.smithi192.stdout:0/760: stat d1/d3/d18/l57 0 2024-02-16T22:56:31.315 INFO:tasks.workunit.client.0.smithi064.stdout:4/648: mkdir dc/dd/d32/d46/d5a/dcb 0 2024-02-16T22:56:31.316 INFO:tasks.workunit.client.1.smithi192.stdout:8/768: creat d5/db/d47/d43/d67/fed x:0 0 0 2024-02-16T22:56:31.320 INFO:tasks.workunit.client.0.smithi064.stdout:8/612: mknod df/d31/d47/cd3 0 2024-02-16T22:56:31.320 INFO:tasks.workunit.client.1.smithi192.stdout:3/716: symlink da/df/lf6 0 2024-02-16T22:56:31.320 INFO:tasks.workunit.client.1.smithi192.stdout:6/805: creat da/d40/db6/db8/dfa/f105 x:0 0 0 2024-02-16T22:56:31.320 INFO:tasks.workunit.client.1.smithi192.stdout:5/771: unlink d6/d12/d4e/d5b/d67/lf2 0 2024-02-16T22:56:31.324 INFO:tasks.workunit.client.0.smithi064.stdout:8/613: dread df/d31/d2f/f59 [0,4194304] 0 2024-02-16T22:56:31.325 INFO:tasks.workunit.client.0.smithi064.stdout:4/649: mknod dc/dd/ccc 0 2024-02-16T22:56:31.326 INFO:tasks.workunit.client.0.smithi064.stdout:8/614: fsync df/d31/d47/fad 0 2024-02-16T22:56:31.327 INFO:tasks.workunit.client.0.smithi064.stdout:2/770: dread d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f5c [0,4194304] 0 2024-02-16T22:56:31.328 INFO:tasks.workunit.client.1.smithi192.stdout:0/761: chown d1/d3/d29/l33 11534 1 2024-02-16T22:56:31.330 INFO:tasks.workunit.client.1.smithi192.stdout:8/769: mkdir d5/d68/dee 0 2024-02-16T22:56:31.331 INFO:tasks.workunit.client.0.smithi064.stdout:9/598: dread d3/d9/d6a/db1/d2a/f4b [0,4194304] 0 2024-02-16T22:56:31.331 INFO:tasks.workunit.client.0.smithi064.stdout:9/599: chown d3/d11 153 1 2024-02-16T22:56:31.332 INFO:tasks.workunit.client.1.smithi192.stdout:8/770: chown d5/d26/d83/db4/fba 150204 1 2024-02-16T22:56:31.332 INFO:tasks.workunit.client.1.smithi192.stdout:8/771: chown d5/d68 265 1 2024-02-16T22:56:31.335 INFO:tasks.workunit.client.1.smithi192.stdout:0/762: dwrite d1/d3/d29/f8d [4194304,4194304] 0 2024-02-16T22:56:31.335 INFO:tasks.workunit.client.0.smithi064.stdout:4/650: dread dc/dd/d32/d46/d5a/f84 [0,4194304] 0 2024-02-16T22:56:31.335 INFO:tasks.workunit.client.0.smithi064.stdout:4/651: stat dc/fba 0 2024-02-16T22:56:31.335 INFO:tasks.workunit.client.0.smithi064.stdout:8/615: creat df/d31/d8f/dc9/fd4 x:0 0 0 2024-02-16T22:56:31.337 INFO:tasks.workunit.client.1.smithi192.stdout:5/772: symlink d6/d8/d86/l100 0 2024-02-16T22:56:31.338 INFO:tasks.workunit.client.1.smithi192.stdout:4/825: write d1/d26/d66/d89/da8/da9/ff2 [250541,105153] 0 2024-02-16T22:56:31.340 INFO:tasks.workunit.client.1.smithi192.stdout:9/897: rename d7/d19/d5e/fcf to d7/d19/d28/f133 0 2024-02-16T22:56:31.340 INFO:tasks.workunit.client.0.smithi064.stdout:6/699: getdents d0/d6/da/d56/db9 0 2024-02-16T22:56:31.341 INFO:tasks.workunit.client.1.smithi192.stdout:4/826: write d1/d15/d29/dd4/ffa [802975,96658] 0 2024-02-16T22:56:31.342 INFO:tasks.workunit.client.1.smithi192.stdout:9/898: dread - d7/d19/d28/f9f zero size 2024-02-16T22:56:31.343 INFO:tasks.workunit.client.1.smithi192.stdout:2/823: mknod d22/d24/d39/d42/d85/d98/db0/c10e 0 2024-02-16T22:56:31.345 INFO:tasks.workunit.client.1.smithi192.stdout:5/773: dwrite d6/d12/d4e/d46/fca [0,4194304] 0 2024-02-16T22:56:31.346 INFO:tasks.workunit.client.0.smithi064.stdout:2/771: write d7/d14/d1e/d34/d3f/d9a/dd9/f24 [4779731,89638] 0 2024-02-16T22:56:31.349 INFO:tasks.workunit.client.1.smithi192.stdout:8/772: rmdir d5/db/d47 39 2024-02-16T22:56:31.350 INFO:tasks.workunit.client.0.smithi064.stdout:9/600: mkdir d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6 0 2024-02-16T22:56:31.351 INFO:tasks.workunit.client.0.smithi064.stdout:9/601: chown d3/d9/ff 55851 1 2024-02-16T22:56:31.351 INFO:tasks.workunit.client.1.smithi192.stdout:5/774: dwrite d6/d12/d4e/d5b/daf/fcd [0,4194304] 0 2024-02-16T22:56:31.357 INFO:tasks.workunit.client.1.smithi192.stdout:5/775: dread d6/d12/d1d/f3a [0,4194304] 0 2024-02-16T22:56:31.357 INFO:tasks.workunit.client.1.smithi192.stdout:3/717: write da/d11/d63/d6b/d66/d75/f76 [1652045,55078] 0 2024-02-16T22:56:31.358 INFO:tasks.workunit.client.0.smithi064.stdout:4/652: dwrite dc/f6c [0,4194304] 0 2024-02-16T22:56:31.359 INFO:tasks.workunit.client.1.smithi192.stdout:1/733: rename d3/de/d27/d34/d36/d47/f79 to d3/de/d27/d34/d52/db7/ffa 0 2024-02-16T22:56:31.360 INFO:tasks.workunit.client.1.smithi192.stdout:1/734: chown d3/d5b/f71 3 1 2024-02-16T22:56:31.363 INFO:tasks.workunit.client.0.smithi064.stdout:2/772: symlink d7/d14/d1e/d34/d73/db5/dea/lf5 0 2024-02-16T22:56:31.363 INFO:tasks.workunit.client.0.smithi064.stdout:2/773: chown d7/d14/d1e/fa4 954673 1 2024-02-16T22:56:31.365 INFO:tasks.workunit.client.0.smithi064.stdout:2/774: chown d7/d14/d1e/d34/d73/db5/ld7 993 1 2024-02-16T22:56:31.367 INFO:tasks.workunit.client.0.smithi064.stdout:4/653: truncate dc/dd/d32/d46/d5a/f65 2630994 0 2024-02-16T22:56:31.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:30 smithi064 ceph-mon[29226]: pgmap v20: 65 pgs: 65 active+clean; 2.6 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 81 MiB/s rd, 197 MiB/s wr, 402 op/s 2024-02-16T22:56:31.370 INFO:tasks.workunit.client.0.smithi064.stdout:8/616: getdents df/d31/d2f/d4f 0 2024-02-16T22:56:31.373 INFO:tasks.workunit.client.0.smithi064.stdout:2/775: rename d7/d14/d1e/d34/d3f/d9a/dcf/le5 to d7/d14/d1e/d34/d3f/d9a/dd9/d60/lf6 0 2024-02-16T22:56:31.378 INFO:tasks.workunit.client.1.smithi192.stdout:1/735: mkdir d3/de/d27/d34/d69/d85/d89/daa/dfb 0 2024-02-16T22:56:31.378 INFO:tasks.workunit.client.0.smithi064.stdout:8/617: link f9 df/d31/d2f/d7f/fd5 0 2024-02-16T22:56:31.380 INFO:tasks.workunit.client.1.smithi192.stdout:4/827: mkdir d1/d26/d66/d89/da8/dc5/d111/d11f 0 2024-02-16T22:56:31.382 INFO:tasks.workunit.client.0.smithi064.stdout:2/776: mkdir d7/d14/d1e/d34/d3f/d9a/dd9/df7 0 2024-02-16T22:56:31.383 INFO:tasks.workunit.client.0.smithi064.stdout:9/602: dread d3/d11/d15/f94 [4194304,4194304] 0 2024-02-16T22:56:31.384 INFO:tasks.workunit.client.1.smithi192.stdout:8/773: mkdir d5/d68/dee/def 0 2024-02-16T22:56:31.385 INFO:tasks.workunit.client.1.smithi192.stdout:6/806: getdents da/df/d6b/d83 0 2024-02-16T22:56:31.386 INFO:tasks.workunit.client.1.smithi192.stdout:0/763: write d1/d3/de/d22/d37/d67/f66 [644182,112759] 0 2024-02-16T22:56:31.386 INFO:tasks.workunit.client.0.smithi064.stdout:8/618: symlink df/d31/d61/d79/dc4/ld6 0 2024-02-16T22:56:31.387 INFO:tasks.workunit.client.1.smithi192.stdout:5/776: dread d6/d8/d80/f40 [0,4194304] 0 2024-02-16T22:56:31.387 INFO:tasks.workunit.client.1.smithi192.stdout:9/899: dread d7/d19/d28/f52 [0,4194304] 0 2024-02-16T22:56:31.387 INFO:tasks.workunit.client.0.smithi064.stdout:9/603: dread d3/d11/d15/f94 [4194304,4194304] 0 2024-02-16T22:56:31.388 INFO:tasks.workunit.client.1.smithi192.stdout:5/777: dread - d6/d12/d4e/d58/d8f/ffa zero size 2024-02-16T22:56:31.389 INFO:tasks.workunit.client.0.smithi064.stdout:9/604: readlink d3/d9/d1e/d42/l99 0 2024-02-16T22:56:31.389 INFO:tasks.workunit.client.0.smithi064.stdout:9/605: stat d3/d89/d9a/f9b 0 2024-02-16T22:56:31.390 INFO:tasks.workunit.client.1.smithi192.stdout:5/778: dread d6/d12/d4e/faa [0,4194304] 0 2024-02-16T22:56:31.390 INFO:tasks.workunit.client.1.smithi192.stdout:3/718: truncate da/f26 3722468 0 2024-02-16T22:56:31.391 INFO:tasks.workunit.client.0.smithi064.stdout:8/619: dread - df/d31/d2f/d38/d48/f76 zero size 2024-02-16T22:56:31.394 INFO:tasks.workunit.client.1.smithi192.stdout:7/883: rename d1/d2/de/d2c/d49/d54/d6a/f95 to d1/d2/de/d2c/d49/da4/d104/f12d 0 2024-02-16T22:56:31.395 INFO:tasks.workunit.client.0.smithi064.stdout:2/777: symlink d7/d14/d25/d46/d55/d7f/dde/lf8 0 2024-02-16T22:56:31.396 INFO:tasks.workunit.client.1.smithi192.stdout:7/884: readlink d1/d2/de/d2c/d49/d54/dd8/d117/la0 0 2024-02-16T22:56:31.397 INFO:tasks.workunit.client.0.smithi064.stdout:6/700: dwrite d0/d12/f78 [0,4194304] 0 2024-02-16T22:56:31.400 INFO:tasks.workunit.client.1.smithi192.stdout:1/736: creat d3/de/d87/ffc x:0 0 0 2024-02-16T22:56:31.400 INFO:tasks.workunit.client.0.smithi064.stdout:4/654: dread dc/d1d/f4c [0,4194304] 0 2024-02-16T22:56:31.401 INFO:tasks.workunit.client.1.smithi192.stdout:4/828: write d1/d26/d34/f60 [1454162,27618] 0 2024-02-16T22:56:31.407 INFO:tasks.workunit.client.1.smithi192.stdout:6/807: dread da/df/d18/fc0 [0,4194304] 0 2024-02-16T22:56:31.407 INFO:tasks.workunit.client.0.smithi064.stdout:8/620: dread df/d31/d2f/d4f/fa7 [0,4194304] 0 2024-02-16T22:56:31.410 INFO:tasks.workunit.client.0.smithi064.stdout:9/606: creat d3/d9/d6a/db1/d2a/d2e/da3/fc7 x:0 0 0 2024-02-16T22:56:31.412 INFO:tasks.workunit.client.1.smithi192.stdout:9/900: creat d7/d19/d37/d5a/f134 x:0 0 0 2024-02-16T22:56:31.412 INFO:tasks.workunit.client.0.smithi064.stdout:9/607: truncate d3/d9/d6a/db1/d2a/d2e/f9d 489572 0 2024-02-16T22:56:31.419 INFO:tasks.workunit.client.1.smithi192.stdout:7/885: mkdir d1/d2/d2d/d61/d88/d12e 0 2024-02-16T22:56:31.422 INFO:tasks.workunit.client.1.smithi192.stdout:5/779: dread d6/d12/d4e/f2e [0,4194304] 0 2024-02-16T22:56:31.422 INFO:tasks.workunit.client.1.smithi192.stdout:4/829: creat d1/d26/d66/d89/da8/da9/f120 x:0 0 0 2024-02-16T22:56:31.422 INFO:tasks.workunit.client.1.smithi192.stdout:1/737: dread d3/d40/f59 [0,4194304] 0 2024-02-16T22:56:31.423 INFO:tasks.workunit.client.1.smithi192.stdout:2/824: getdents d22/d24/d39/d42/d52 0 2024-02-16T22:56:31.423 INFO:tasks.workunit.client.1.smithi192.stdout:1/738: write d3/de/f32 [3548728,107406] 0 2024-02-16T22:56:31.424 INFO:tasks.workunit.client.0.smithi064.stdout:4/655: link dc/dd/d32/d4b/fa4 dc/dd/d5b/fcd 0 2024-02-16T22:56:31.425 INFO:tasks.workunit.client.1.smithi192.stdout:2/825: chown d22/d24/d39/d42/d65/d69/c61 4567063 1 2024-02-16T22:56:31.425 INFO:tasks.workunit.client.0.smithi064.stdout:6/701: write d0/d12/f9b [758158,72697] 0 2024-02-16T22:56:31.426 INFO:tasks.workunit.client.0.smithi064.stdout:4/656: chown dc/dd/d32/d46/d5a/d7e/d9a/da2 182226 1 2024-02-16T22:56:31.436 INFO:tasks.workunit.client.0.smithi064.stdout:6/702: rmdir d0/d6/d10/d33/da4 39 2024-02-16T22:56:31.437 INFO:tasks.workunit.client.1.smithi192.stdout:3/719: dread - da/d6a/fe4 zero size 2024-02-16T22:56:31.438 INFO:tasks.workunit.client.0.smithi064.stdout:2/778: link d7/d14/d1e/lc5 d7/d14/d1e/d34/d3f/d9a/dd9/lf9 0 2024-02-16T22:56:31.439 INFO:tasks.workunit.client.0.smithi064.stdout:9/608: truncate d3/d9/d6a/db1/d2a/d2e/f59 19403 0 2024-02-16T22:56:31.440 INFO:tasks.workunit.client.1.smithi192.stdout:6/808: dwrite da/d40/f4c [0,4194304] 0 2024-02-16T22:56:31.441 INFO:tasks.workunit.client.1.smithi192.stdout:6/809: write da/df/d12/d15/f33 [481115,57788] 0 2024-02-16T22:56:31.441 INFO:tasks.workunit.client.1.smithi192.stdout:0/764: truncate d1/d3/de/d22/f31 552750 0 2024-02-16T22:56:31.442 INFO:tasks.workunit.client.0.smithi064.stdout:4/657: chown dc/de/c2f 63922 1 2024-02-16T22:56:31.443 INFO:tasks.workunit.client.0.smithi064.stdout:8/621: link df/d31/d61/d79/dc4/ld6 df/d31/d8f/dab/dae/ld7 0 2024-02-16T22:56:31.445 INFO:tasks.workunit.client.1.smithi192.stdout:2/826: dread fe [4194304,4194304] 0 2024-02-16T22:56:31.445 INFO:tasks.workunit.client.1.smithi192.stdout:4/830: read d1/d26/d34/f50 [1306676,11305] 0 2024-02-16T22:56:31.447 INFO:tasks.workunit.client.1.smithi192.stdout:4/831: truncate d1/d15/d48/dc3/deb/f119 976496 0 2024-02-16T22:56:31.447 INFO:tasks.workunit.client.0.smithi064.stdout:6/703: rename d0/d1d/d21/cb8 to d0/d5/df/d17/ce9 0 2024-02-16T22:56:31.449 INFO:tasks.workunit.client.0.smithi064.stdout:2/779: symlink d7/d14/d4b/d63/lfa 0 2024-02-16T22:56:31.451 INFO:tasks.workunit.client.1.smithi192.stdout:5/780: rename d6/d12/d4e/d5b/daf/ld1 to d6/d8/d77/l101 0 2024-02-16T22:56:31.452 INFO:tasks.workunit.client.1.smithi192.stdout:1/739: dread - d3/de/da4/fa7 zero size 2024-02-16T22:56:31.453 INFO:tasks.workunit.client.0.smithi064.stdout:4/658: creat dc/dd/d32/d4b/fce x:0 0 0 2024-02-16T22:56:31.453 INFO:tasks.workunit.client.1.smithi192.stdout:1/740: write d3/de/d27/d34/d52/d63/d81/fdb [102504,119208] 0 2024-02-16T22:56:31.454 INFO:tasks.workunit.client.0.smithi064.stdout:6/704: creat d0/d3c/fea x:0 0 0 2024-02-16T22:56:31.454 INFO:tasks.workunit.client.1.smithi192.stdout:1/741: stat d3/de/d27/d34/d52/d5a/c93 0 2024-02-16T22:56:31.455 INFO:tasks.workunit.client.0.smithi064.stdout:6/705: readlink d0/d5/df/d17/d18/l39 0 2024-02-16T22:56:31.456 INFO:tasks.workunit.client.0.smithi064.stdout:9/609: fsync d3/d9/fa2 0 2024-02-16T22:56:31.458 INFO:tasks.workunit.client.1.smithi192.stdout:8/774: link d5/d26/d83/d86/d5f/db2/fe7 d5/db/de/ff0 0 2024-02-16T22:56:31.461 INFO:tasks.workunit.client.1.smithi192.stdout:1/742: dwrite d3/de/d27/d34/d52/fe0 [0,4194304] 0 2024-02-16T22:56:31.462 INFO:tasks.workunit.client.1.smithi192.stdout:1/743: write d3/de/fd4 [81377,125047] 0 2024-02-16T22:56:31.470 INFO:tasks.workunit.client.1.smithi192.stdout:7/886: creat d1/d2/d2d/d61/d88/d90/d10f/f12f x:0 0 0 2024-02-16T22:56:31.470 INFO:tasks.workunit.client.1.smithi192.stdout:7/887: dread - d1/d2/de/fc7 zero size 2024-02-16T22:56:31.473 INFO:tasks.workunit.client.0.smithi064.stdout:4/659: rename dc/dd/d32/d58/f70 to dc/dd/d32/d46/d82/fcf 0 2024-02-16T22:56:31.473 INFO:tasks.workunit.client.0.smithi064.stdout:2/780: write d7/d14/d25/daf/fd3 [4939274,46698] 0 2024-02-16T22:56:31.481 INFO:tasks.workunit.client.0.smithi064.stdout:6/706: dread d0/d6/f15 [0,4194304] 0 2024-02-16T22:56:31.511 INFO:tasks.workunit.client.0.smithi064.stdout:5/603: sync 2024-02-16T22:56:31.511 INFO:tasks.workunit.client.0.smithi064.stdout:5/604: dread - d6/d18/d90/fc2 zero size 2024-02-16T22:56:31.512 INFO:tasks.workunit.client.1.smithi192.stdout:6/810: rmdir da/df/d71/dab 39 2024-02-16T22:56:31.512 INFO:tasks.workunit.client.1.smithi192.stdout:6/811: chown da/df/d12/c5d 79182227 1 2024-02-16T22:56:31.513 INFO:tasks.workunit.client.1.smithi192.stdout:0/765: creat d1/d3/de/d22/da7/ff1 x:0 0 0 2024-02-16T22:56:31.514 INFO:tasks.workunit.client.0.smithi064.stdout:9/610: fsync d3/d9/d6a/db1/f29 0 2024-02-16T22:56:31.519 INFO:tasks.workunit.client.1.smithi192.stdout:9/901: truncate d7/d19/d37/ff2 7382060 0 2024-02-16T22:56:31.520 INFO:tasks.workunit.client.1.smithi192.stdout:5/781: mkdir d6/d12/d2d/d5c/df6/dbc/d102 0 2024-02-16T22:56:31.520 INFO:tasks.workunit.client.1.smithi192.stdout:3/720: write da/d11/d63/d6b/d4e/f56 [3492559,41838] 0 2024-02-16T22:56:31.521 INFO:tasks.workunit.client.1.smithi192.stdout:5/782: write d6/d12/d4e/d58/ff8 [630834,71194] 0 2024-02-16T22:56:31.528 INFO:tasks.workunit.client.1.smithi192.stdout:8/775: creat d5/d16/dd5/ff1 x:0 0 0 2024-02-16T22:56:31.528 INFO:tasks.workunit.client.1.smithi192.stdout:3/721: dread da/df/f2e [0,4194304] 0 2024-02-16T22:56:31.529 INFO:tasks.workunit.client.0.smithi064.stdout:9/611: fdatasync d3/f10 0 2024-02-16T22:56:31.529 INFO:tasks.workunit.client.1.smithi192.stdout:1/744: write d3/d40/f59 [2405375,86388] 0 2024-02-16T22:56:31.531 INFO:tasks.workunit.client.1.smithi192.stdout:8/776: write d5/d26/d83/d86/d5f/dbf/fc2 [2055492,89360] 0 2024-02-16T22:56:31.533 INFO:tasks.workunit.client.0.smithi064.stdout:4/660: truncate dc/dd/d32/d46/d5a/f65 3512109 0 2024-02-16T22:56:31.534 INFO:tasks.workunit.client.0.smithi064.stdout:4/661: chown dc/dd/d32/d46/d5a/d7e 7997139 1 2024-02-16T22:56:31.535 INFO:tasks.workunit.client.1.smithi192.stdout:2/827: dwrite d22/d24/d39/d42/d65/d69/f68 [0,4194304] 0 2024-02-16T22:56:31.535 INFO:tasks.workunit.client.1.smithi192.stdout:6/812: rename da/df/d12/d57/d81/fb7 to da/d40/db6/f106 0 2024-02-16T22:56:31.536 INFO:tasks.workunit.client.1.smithi192.stdout:2/828: stat d22/d24/c28 0 2024-02-16T22:56:31.538 INFO:tasks.workunit.client.1.smithi192.stdout:6/813: write da/df/d12/d15/d1b/f8a [2850286,70345] 0 2024-02-16T22:56:31.538 INFO:tasks.workunit.client.1.smithi192.stdout:6/814: stat da/df/d12/c5d 0 2024-02-16T22:56:31.541 INFO:tasks.workunit.client.0.smithi064.stdout:5/605: mknod d6/d25/da4/dc1/d8e/cd1 0 2024-02-16T22:56:31.541 INFO:tasks.workunit.client.1.smithi192.stdout:4/832: getdents d1/d26/d34/db9/ddb 0 2024-02-16T22:56:31.546 INFO:tasks.workunit.client.0.smithi064.stdout:9/612: rmdir d3/d9/d1e/d42 39 2024-02-16T22:56:31.547 INFO:tasks.workunit.client.0.smithi064.stdout:9/613: dread - d3/d9/d6a/db1/d38/fa6 zero size 2024-02-16T22:56:31.547 INFO:tasks.workunit.client.1.smithi192.stdout:9/902: symlink d7/d19/d37/d5a/l135 0 2024-02-16T22:56:31.548 INFO:tasks.workunit.client.0.smithi064.stdout:9/614: dread - d3/d88/fc5 zero size 2024-02-16T22:56:31.549 INFO:tasks.workunit.client.0.smithi064.stdout:2/781: link d7/d14/d1e/d34/d73/c7a d7/d14/d1e/d34/d3f/d9a/dd9/df7/cfb 0 2024-02-16T22:56:31.550 INFO:tasks.workunit.client.0.smithi064.stdout:6/707: link d0/d5/df/d17/f7d d0/d1d/da8/d60/feb 0 2024-02-16T22:56:31.551 INFO:tasks.workunit.client.0.smithi064.stdout:6/708: write d0/d5/df/d17/d4a/fe2 [483519,71794] 0 2024-02-16T22:56:31.552 INFO:tasks.workunit.client.0.smithi064.stdout:5/606: mknod d6/d1c/d43/d7d/cd2 0 2024-02-16T22:56:31.552 INFO:tasks.workunit.client.0.smithi064.stdout:6/709: fsync d0/d6/d10/f9f 0 2024-02-16T22:56:31.553 INFO:tasks.workunit.client.0.smithi064.stdout:5/607: fdatasync d6/d25/d48/d67/f7f 0 2024-02-16T22:56:31.553 INFO:tasks.workunit.client.0.smithi064.stdout:6/710: chown d0/d1d/dbc/fe4 123476107 1 2024-02-16T22:56:31.555 INFO:tasks.workunit.client.0.smithi064.stdout:4/662: unlink dc/l44 0 2024-02-16T22:56:31.558 INFO:tasks.workunit.client.0.smithi064.stdout:2/782: unlink d7/fbf 0 2024-02-16T22:56:31.561 INFO:tasks.workunit.client.0.smithi064.stdout:4/663: dwrite dc/dd/d16/fa6 [4194304,4194304] 0 2024-02-16T22:56:31.564 INFO:tasks.workunit.client.0.smithi064.stdout:6/711: mkdir d0/d1d/dbc/dec 0 2024-02-16T22:56:31.570 INFO:tasks.workunit.client.0.smithi064.stdout:9/615: creat d3/d11/d15/dab/db6/fc8 x:0 0 0 2024-02-16T22:56:31.573 INFO:tasks.workunit.client.0.smithi064.stdout:2/783: creat d7/d14/d4b/d63/ffc x:0 0 0 2024-02-16T22:56:31.574 INFO:tasks.workunit.client.0.smithi064.stdout:2/784: stat d7/d14/d1e/d34/d73/db5/fbd 0 2024-02-16T22:56:31.575 INFO:tasks.workunit.client.0.smithi064.stdout:4/664: symlink dc/dd/d32/d22/ld0 0 2024-02-16T22:56:31.579 INFO:tasks.workunit.client.0.smithi064.stdout:6/712: unlink d0/d1d/c46 0 2024-02-16T22:56:31.585 INFO:tasks.workunit.client.0.smithi064.stdout:2/785: mkdir d7/d14/d1e/dfd 0 2024-02-16T22:56:31.587 INFO:tasks.workunit.client.0.smithi064.stdout:4/665: symlink dc/de/d6d/ld1 0 2024-02-16T22:56:31.588 INFO:tasks.workunit.client.0.smithi064.stdout:4/666: chown dc/dd/d32/d46/d5a/d7e/d9a/da2 85 1 2024-02-16T22:56:31.594 INFO:tasks.workunit.client.0.smithi064.stdout:5/608: link d6/d25/d48/d67/c71 d6/d1c/d43/d7d/cd3 0 2024-02-16T22:56:31.598 INFO:tasks.workunit.client.0.smithi064.stdout:2/786: creat d7/d14/d1e/d34/d3f/d9a/ffe x:0 0 0 2024-02-16T22:56:31.600 INFO:tasks.workunit.client.0.smithi064.stdout:6/713: dread d0/d1d/f9e [0,4194304] 0 2024-02-16T22:56:31.604 INFO:tasks.workunit.client.0.smithi064.stdout:2/787: unlink d7/d14/d3e/f75 0 2024-02-16T22:56:31.606 INFO:tasks.workunit.client.0.smithi064.stdout:4/667: creat dc/dd/db8/fd2 x:0 0 0 2024-02-16T22:56:31.606 INFO:tasks.workunit.client.0.smithi064.stdout:9/616: dwrite d3/f69 [0,4194304] 0 2024-02-16T22:56:31.608 INFO:tasks.workunit.client.0.smithi064.stdout:6/714: mkdir d0/d5/df/d17/ded 0 2024-02-16T22:56:31.618 INFO:tasks.workunit.client.1.smithi192.stdout:3/722: creat da/d11/d63/d6b/ff7 x:0 0 0 2024-02-16T22:56:31.620 INFO:tasks.workunit.client.0.smithi064.stdout:2/788: rename d7/d14/d1e/d34/d3f/l89 to d7/d14/d1e/dfd/lff 0 2024-02-16T22:56:31.621 INFO:tasks.workunit.client.1.smithi192.stdout:7/888: fdatasync d1/d2/de/d2c/d49/da4/f85 0 2024-02-16T22:56:31.622 INFO:tasks.workunit.client.0.smithi064.stdout:2/789: truncate d7/d14/d4b/d63/fc3 305476 0 2024-02-16T22:56:31.622 INFO:tasks.workunit.client.1.smithi192.stdout:7/889: write d1/f11 [7934145,21247] 0 2024-02-16T22:56:31.628 INFO:tasks.workunit.client.1.smithi192.stdout:5/783: rename c4 to d6/d12/d2d/d5c/c103 0 2024-02-16T22:56:31.630 INFO:tasks.workunit.client.0.smithi064.stdout:5/609: dwrite d6/f24 [0,4194304] 0 2024-02-16T22:56:31.631 INFO:tasks.workunit.client.0.smithi064.stdout:5/610: stat d6/d25/da4/dc1/d85/f8b 0 2024-02-16T22:56:31.631 INFO:tasks.workunit.client.0.smithi064.stdout:6/715: rename d0/d1b/de5 to d0/d6/d10/d33/d72/de8/dee 0 2024-02-16T22:56:31.632 INFO:tasks.workunit.client.0.smithi064.stdout:6/716: stat d0/d12/f78 0 2024-02-16T22:56:31.634 INFO:tasks.workunit.client.1.smithi192.stdout:2/829: unlink d22/d24/d39/d42/d85/d98/ffe 0 2024-02-16T22:56:31.637 INFO:tasks.workunit.client.0.smithi064.stdout:2/790: mknod d7/d14/d1e/d34/d73/c100 0 2024-02-16T22:56:31.643 INFO:tasks.workunit.client.0.smithi064.stdout:5/611: readlink d6/d1c/l40 0 2024-02-16T22:56:31.643 INFO:tasks.workunit.client.0.smithi064.stdout:9/617: dread d3/d9/d6a/db1/d2a/d2e/d31/f84 [0,4194304] 0 2024-02-16T22:56:31.646 INFO:tasks.workunit.client.0.smithi064.stdout:6/717: creat d0/d3c/fef x:0 0 0 2024-02-16T22:56:31.648 INFO:tasks.workunit.client.1.smithi192.stdout:0/766: write d1/d3/de/d22/d37/d67/d75/db0/fe3 [241299,115151] 0 2024-02-16T22:56:31.650 INFO:tasks.workunit.client.0.smithi064.stdout:5/612: mkdir d6/d25/da4/dc1/dd4 0 2024-02-16T22:56:31.652 INFO:tasks.workunit.client.0.smithi064.stdout:9/618: creat d3/d9/d6a/db1/d2a/fc9 x:0 0 0 2024-02-16T22:56:31.653 INFO:tasks.workunit.client.1.smithi192.stdout:1/745: creat d3/de/d27/d34/d36/d47/dcd/ffd x:0 0 0 2024-02-16T22:56:31.653 INFO:tasks.workunit.client.0.smithi064.stdout:9/619: write d3/d88/fc5 [67061,109713] 0 2024-02-16T22:56:31.655 INFO:tasks.workunit.client.1.smithi192.stdout:4/833: dwrite d1/d26/d66/d89/d11/f98 [0,4194304] 0 2024-02-16T22:56:31.655 INFO:tasks.workunit.client.0.smithi064.stdout:9/620: stat d3/d9/d6a/db1/c2f 0 2024-02-16T22:56:31.656 INFO:tasks.workunit.client.1.smithi192.stdout:4/834: fsync d1/d26/d66/d89/d11/d58/f90 0 2024-02-16T22:56:31.657 INFO:tasks.workunit.client.1.smithi192.stdout:4/835: dread - d1/d26/d34/ffd zero size 2024-02-16T22:56:31.658 INFO:tasks.workunit.client.1.smithi192.stdout:2/830: fdatasync d22/d47/dcb/d3f/dac/fde 0 2024-02-16T22:56:31.658 INFO:tasks.workunit.client.0.smithi064.stdout:5/613: mknod d6/d25/da4/dc1/d8e/cd5 0 2024-02-16T22:56:31.660 INFO:tasks.workunit.client.0.smithi064.stdout:9/621: read d3/d9/d6a/db1/d2a/d2e/f5f [82789,28568] 0 2024-02-16T22:56:31.661 INFO:tasks.workunit.client.1.smithi192.stdout:6/815: symlink da/l107 0 2024-02-16T22:56:31.663 INFO:tasks.workunit.client.1.smithi192.stdout:9/903: symlink d7/d19/d37/dbf/d12c/l136 0 2024-02-16T22:56:31.663 INFO:tasks.workunit.client.1.smithi192.stdout:6/816: fsync da/df/d12/d15/f2d 0 2024-02-16T22:56:31.663 INFO:tasks.workunit.client.0.smithi064.stdout:4/668: dread dc/dd/d16/f3a [0,4194304] 0 2024-02-16T22:56:31.664 INFO:tasks.workunit.client.1.smithi192.stdout:6/817: chown da/df/d18/fc0 371602451 1 2024-02-16T22:56:31.664 INFO:tasks.workunit.client.0.smithi064.stdout:5/614: mknod d6/d1c/cd6 0 2024-02-16T22:56:31.668 INFO:tasks.workunit.client.0.smithi064.stdout:9/622: rmdir d3/d9/d6a/db1/d2a/d2e/d31 39 2024-02-16T22:56:31.671 INFO:tasks.workunit.client.0.smithi064.stdout:6/718: dread d0/d6/d10/f5e [0,4194304] 0 2024-02-16T22:56:31.671 INFO:tasks.workunit.client.0.smithi064.stdout:2/791: dread d7/d14/d4b/fc7 [0,4194304] 0 2024-02-16T22:56:31.672 INFO:tasks.workunit.client.1.smithi192.stdout:0/767: symlink d1/d3/de/d22/d37/d67/d34/da1/lf2 0 2024-02-16T22:56:31.673 INFO:tasks.workunit.client.0.smithi064.stdout:4/669: dwrite dc/dd/db8/fd2 [0,4194304] 0 2024-02-16T22:56:31.678 INFO:tasks.workunit.client.0.smithi064.stdout:9/623: creat d3/d9/d6a/db1/d38/fca x:0 0 0 2024-02-16T22:56:31.680 INFO:tasks.workunit.client.0.smithi064.stdout:9/624: chown d3/d9/d1e/d3e/d76/c78 905121 1 2024-02-16T22:56:31.683 INFO:tasks.workunit.client.1.smithi192.stdout:8/777: rmdir d5/d68/dee/def 0 2024-02-16T22:56:31.684 INFO:tasks.workunit.client.0.smithi064.stdout:2/792: mkdir d7/d14/d25/d46/d55/d7f/d101 0 2024-02-16T22:56:31.686 INFO:tasks.workunit.client.1.smithi192.stdout:5/784: link d6/d12/d4e/de6/ff3 d6/d12/d2d/d5c/df6/dbc/d102/f104 0 2024-02-16T22:56:31.686 INFO:tasks.workunit.client.1.smithi192.stdout:7/890: truncate d1/d2/de/d83/fbc 3307349 0 2024-02-16T22:56:31.687 INFO:tasks.workunit.client.0.smithi064.stdout:5/615: truncate d6/d18/d19/f2e 2716427 0 2024-02-16T22:56:31.687 INFO:tasks.workunit.client.1.smithi192.stdout:8/778: dwrite d5/d16/f18 [0,4194304] 0 2024-02-16T22:56:31.689 INFO:tasks.workunit.client.0.smithi064.stdout:4/670: write dc/fba [593543,118868] 0 2024-02-16T22:56:31.691 INFO:tasks.workunit.client.0.smithi064.stdout:4/671: chown dc/dd/d32/d46/d5a/d7e/d9a/da2 0 1 2024-02-16T22:56:31.695 INFO:tasks.workunit.client.1.smithi192.stdout:9/904: rmdir d7/d15/d36 39 2024-02-16T22:56:31.695 INFO:tasks.workunit.client.1.smithi192.stdout:9/905: chown d7/d19/d37/d39/lad 48940771 1 2024-02-16T22:56:31.702 INFO:tasks.workunit.client.1.smithi192.stdout:7/891: dread d1/d2/de/d2c/d49/f71 [0,4194304] 0 2024-02-16T22:56:31.703 INFO:tasks.workunit.client.1.smithi192.stdout:3/723: rmdir da/d11/dd4/deb/da1/db9/dd7 0 2024-02-16T22:56:31.705 INFO:tasks.workunit.client.1.smithi192.stdout:1/746: symlink d3/de/lfe 0 2024-02-16T22:56:31.709 INFO:tasks.workunit.client.1.smithi192.stdout:4/836: link d1/d15/d48/dc3/d115/d11d/f11e d1/d26/d5e/f121 0 2024-02-16T22:56:31.710 INFO:tasks.workunit.client.1.smithi192.stdout:4/837: readlink d1/d15/d23/le1 0 2024-02-16T22:56:31.710 INFO:tasks.workunit.client.1.smithi192.stdout:6/818: symlink da/df/d12/d15/d1b/l108 0 2024-02-16T22:56:31.711 INFO:tasks.workunit.client.1.smithi192.stdout:9/906: fdatasync d7/d15/d36/d38/d3f/d8a/def/df6/f110 0 2024-02-16T22:56:31.712 INFO:tasks.workunit.client.1.smithi192.stdout:3/724: fdatasync da/df/f23 0 2024-02-16T22:56:31.714 INFO:tasks.workunit.client.1.smithi192.stdout:5/785: mknod d6/d20/d93/c105 0 2024-02-16T22:56:31.716 INFO:tasks.workunit.client.1.smithi192.stdout:5/786: chown d6/d12/d2d/d5c/df6 46955 1 2024-02-16T22:56:31.716 INFO:tasks.workunit.client.1.smithi192.stdout:9/907: truncate d7/d15/d36/d77/dbd/de0/f11e 640947 0 2024-02-16T22:56:31.716 INFO:tasks.workunit.client.1.smithi192.stdout:8/779: rmdir d5/db/d47/d43/d67 39 2024-02-16T22:56:31.720 INFO:tasks.workunit.client.1.smithi192.stdout:4/838: dwrite d1/d15/d29/d3c/f112 [0,4194304] 0 2024-02-16T22:56:31.721 INFO:tasks.workunit.client.1.smithi192.stdout:9/908: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/feb 4 1 2024-02-16T22:56:31.722 INFO:tasks.workunit.client.1.smithi192.stdout:9/909: fdatasync d7/d19/f112 0 2024-02-16T22:56:31.724 INFO:tasks.workunit.client.0.smithi064.stdout:2/793: rmdir d7/d14/d1e/d34/d73/db5 39 2024-02-16T22:56:31.724 INFO:tasks.workunit.client.1.smithi192.stdout:1/747: link d3/de/d27/d34/d69/d85/d89/cc5 d3/de/d27/d34/d52/d63/cff 0 2024-02-16T22:56:31.724 INFO:tasks.workunit.client.0.smithi064.stdout:6/719: dwrite d0/d1b/f36 [0,4194304] 0 2024-02-16T22:56:31.726 INFO:tasks.workunit.client.0.smithi064.stdout:9/625: read d3/d9/d6a/db1/d2a/d2e/d31/d3f/f43 [1646822,82858] 0 2024-02-16T22:56:31.726 INFO:tasks.workunit.client.0.smithi064.stdout:2/794: dread - d7/d14/d25/d46/d55/fda zero size 2024-02-16T22:56:31.728 INFO:tasks.workunit.client.1.smithi192.stdout:5/787: dwrite d6/d12/d2d/f43 [0,4194304] 0 2024-02-16T22:56:31.729 INFO:tasks.workunit.client.0.smithi064.stdout:2/795: write d7/d14/d4b/d63/ffc [884975,69152] 0 2024-02-16T22:56:31.731 INFO:tasks.workunit.client.0.smithi064.stdout:2/796: chown d7/d14/d4b/fc6 50965608 1 2024-02-16T22:56:31.731 INFO:tasks.workunit.client.1.smithi192.stdout:1/748: write d3/de/d27/d34/d52/d63/d81/fdb [781832,120613] 0 2024-02-16T22:56:31.731 INFO:tasks.workunit.client.1.smithi192.stdout:3/725: write da/d11/f4b [2289725,101603] 0 2024-02-16T22:56:31.735 INFO:tasks.workunit.client.1.smithi192.stdout:7/892: dwrite d1/d2/de/d2c/d49/f80 [0,4194304] 0 2024-02-16T22:56:31.735 INFO:tasks.workunit.client.1.smithi192.stdout:9/910: fdatasync d7/d19/d37/f34 0 2024-02-16T22:56:31.736 INFO:tasks.workunit.client.1.smithi192.stdout:5/788: dread - d6/d12/d2d/d5c/df6/dbc/fd8 zero size 2024-02-16T22:56:31.736 INFO:tasks.workunit.client.0.smithi064.stdout:9/626: mkdir d3/d11/d15/dab/dcb 0 2024-02-16T22:56:31.737 INFO:tasks.workunit.client.1.smithi192.stdout:0/768: dwrite d1/d3/d29/f70 [0,4194304] 0 2024-02-16T22:56:31.738 INFO:tasks.workunit.client.0.smithi064.stdout:6/720: read d0/d6/da/f25 [954013,67510] 0 2024-02-16T22:56:31.740 INFO:tasks.workunit.client.1.smithi192.stdout:7/893: truncate d1/d2/de/d83/d9f/dfb/f12b 401537 0 2024-02-16T22:56:31.741 INFO:tasks.workunit.client.1.smithi192.stdout:4/839: fsync d1/d26/f36 0 2024-02-16T22:56:31.742 INFO:tasks.workunit.client.1.smithi192.stdout:1/749: creat d3/de/d27/f100 x:0 0 0 2024-02-16T22:56:31.743 INFO:tasks.workunit.client.0.smithi064.stdout:5/616: creat d6/d1c/fd7 x:0 0 0 2024-02-16T22:56:31.744 INFO:tasks.workunit.client.1.smithi192.stdout:8/780: write d5/d26/d83/d86/d5f/db2/fe7 [1680813,43583] 0 2024-02-16T22:56:31.744 INFO:tasks.workunit.client.1.smithi192.stdout:1/750: chown d3/de/f65 17491 1 2024-02-16T22:56:31.748 INFO:tasks.workunit.client.1.smithi192.stdout:8/781: write d5/db/de/df/f1b [1519349,43500] 0 2024-02-16T22:56:31.749 INFO:tasks.workunit.client.1.smithi192.stdout:9/911: dwrite d7/d19/d28/d43/fee [0,4194304] 0 2024-02-16T22:56:31.750 INFO:tasks.workunit.client.1.smithi192.stdout:7/894: dread d1/d2/de/d2c/f8b [0,4194304] 0 2024-02-16T22:56:31.751 INFO:tasks.workunit.client.1.smithi192.stdout:7/895: readlink d1/d2/d2d/d61/d8a/la6 0 2024-02-16T22:56:31.753 INFO:tasks.workunit.client.1.smithi192.stdout:7/896: write d1/d2/d2d/d61/d8a/f94 [3166984,109703] 0 2024-02-16T22:56:31.754 INFO:tasks.workunit.client.0.smithi064.stdout:4/672: dwrite dc/d4a/f4d [0,4194304] 0 2024-02-16T22:56:31.754 INFO:tasks.workunit.client.0.smithi064.stdout:5/617: creat d6/d1c/d43/db9/fd8 x:0 0 0 2024-02-16T22:56:31.755 INFO:tasks.workunit.client.1.smithi192.stdout:7/897: readlink d1/d2/de/d2c/d49/d89/dbd/lff 0 2024-02-16T22:56:31.757 INFO:tasks.workunit.client.1.smithi192.stdout:7/898: write d1/d2/d37/fe7 [2037439,91919] 0 2024-02-16T22:56:31.758 INFO:tasks.workunit.client.1.smithi192.stdout:5/789: mkdir d6/d8/d75/d8c/d106 0 2024-02-16T22:56:31.759 INFO:tasks.workunit.client.0.smithi064.stdout:6/721: symlink d0/d6/d10/d33/lf0 0 2024-02-16T22:56:31.759 INFO:tasks.workunit.client.1.smithi192.stdout:0/769: chown d1/d3/de/cef 23597143 1 2024-02-16T22:56:31.761 INFO:tasks.workunit.client.0.smithi064.stdout:6/722: chown d0/d6/da/d56/l6c 349678 1 2024-02-16T22:56:31.764 INFO:tasks.workunit.client.0.smithi064.stdout:4/673: symlink dc/d1d/d43/d7a/ld3 0 2024-02-16T22:56:31.766 INFO:tasks.workunit.client.0.smithi064.stdout:6/723: rename d0/d5/fab to d0/d6/d10/db2/ff1 0 2024-02-16T22:56:31.767 INFO:tasks.workunit.client.1.smithi192.stdout:9/912: read - d7/d15/fab zero size 2024-02-16T22:56:31.772 INFO:tasks.workunit.client.0.smithi064.stdout:5/618: creat d6/d18/d19/fd9 x:0 0 0 2024-02-16T22:56:31.772 INFO:tasks.workunit.client.0.smithi064.stdout:5/619: stat d6/d18/d90/fbc 0 2024-02-16T22:56:31.773 INFO:tasks.workunit.client.0.smithi064.stdout:6/724: creat d0/d5/df/d17/ff2 x:0 0 0 2024-02-16T22:56:31.775 INFO:tasks.workunit.client.1.smithi192.stdout:7/899: creat d1/d2/de/d83/f130 x:0 0 0 2024-02-16T22:56:31.776 INFO:tasks.workunit.client.1.smithi192.stdout:7/900: chown d1/d2/de/d2c/d49/d54/d9d/f11c 15 1 2024-02-16T22:56:31.777 INFO:tasks.workunit.client.1.smithi192.stdout:0/770: mkdir d1/d3/d39/df3 0 2024-02-16T22:56:31.777 INFO:tasks.workunit.client.1.smithi192.stdout:0/771: readlink d1/d3/de/d22/d37/d67/d34/l94 0 2024-02-16T22:56:31.780 INFO:tasks.workunit.client.0.smithi064.stdout:6/725: mknod d0/d89/cf3 0 2024-02-16T22:56:31.781 INFO:tasks.workunit.client.1.smithi192.stdout:5/790: truncate d6/d12/d1d/d56/fc8 1979589 0 2024-02-16T22:56:31.782 INFO:tasks.workunit.client.1.smithi192.stdout:0/772: dwrite d1/dc7/d9c/fe4 [0,4194304] 0 2024-02-16T22:56:31.783 INFO:tasks.workunit.client.0.smithi064.stdout:5/620: fsync d6/f34 0 2024-02-16T22:56:31.784 INFO:tasks.workunit.client.0.smithi064.stdout:2/797: write d7/d14/d1e/d34/d73/f77 [4499137,6051] 0 2024-02-16T22:56:31.788 INFO:tasks.workunit.client.0.smithi064.stdout:9/627: write d3/d9/d6a/db1/d2a/d2e/d31/f84 [4823981,102799] 0 2024-02-16T22:56:31.790 INFO:tasks.workunit.client.1.smithi192.stdout:8/782: mknod d5/cf2 0 2024-02-16T22:56:31.796 INFO:tasks.workunit.client.1.smithi192.stdout:7/901: symlink d1/d2/de/d2c/d49/d89/dbd/l131 0 2024-02-16T22:56:31.796 INFO:tasks.workunit.client.0.smithi064.stdout:4/674: link dc/dd/d16/l45 dc/dd/d16/d7c/ld4 0 2024-02-16T22:56:31.799 INFO:tasks.workunit.client.0.smithi064.stdout:9/628: chown d3/d9/d6a/db1/f52 50632579 1 2024-02-16T22:56:31.802 INFO:tasks.workunit.client.0.smithi064.stdout:2/798: mkdir d7/d14/d25/d46/d55/d102 0 2024-02-16T22:56:31.804 INFO:tasks.workunit.client.1.smithi192.stdout:4/840: rmdir d1/d26/d34/d11c 0 2024-02-16T22:56:31.804 INFO:tasks.workunit.client.1.smithi192.stdout:4/841: chown d1/d26/d66/d89/d11/d58/f90 14 1 2024-02-16T22:56:31.806 INFO:tasks.workunit.client.1.smithi192.stdout:1/751: creat d3/f101 x:0 0 0 2024-02-16T22:56:31.807 INFO:tasks.workunit.client.1.smithi192.stdout:7/902: dread d1/d2/d2d/d61/f74 [0,4194304] 0 2024-02-16T22:56:31.808 INFO:tasks.workunit.client.0.smithi064.stdout:9/629: creat d3/d11/d15/dab/fcc x:0 0 0 2024-02-16T22:56:31.812 INFO:tasks.workunit.client.0.smithi064.stdout:4/675: mknod dc/de/d28/d57/cd5 0 2024-02-16T22:56:31.812 INFO:tasks.workunit.client.1.smithi192.stdout:2/831: sync 2024-02-16T22:56:31.813 INFO:tasks.workunit.client.1.smithi192.stdout:2/832: stat d22/d24/d39/d3d/df2 0 2024-02-16T22:56:31.813 INFO:tasks.workunit.client.0.smithi064.stdout:6/726: write d0/d6/d10/f14 [1605304,55509] 0 2024-02-16T22:56:31.814 INFO:tasks.workunit.client.0.smithi064.stdout:9/630: creat d3/d11/fcd x:0 0 0 2024-02-16T22:56:31.816 INFO:tasks.workunit.client.0.smithi064.stdout:2/799: mkdir d7/d14/d1e/d34/d3f/d9a/dd9/d103 0 2024-02-16T22:56:31.818 INFO:tasks.workunit.client.0.smithi064.stdout:4/676: symlink dc/d1d/ld6 0 2024-02-16T22:56:31.820 INFO:tasks.workunit.client.0.smithi064.stdout:4/677: chown dc/dd/d5b/faf 242 1 2024-02-16T22:56:31.821 INFO:tasks.workunit.client.1.smithi192.stdout:4/842: mkdir d1/d26/d66/d89/da8/dc5/d111/d122 0 2024-02-16T22:56:31.821 INFO:tasks.workunit.client.1.smithi192.stdout:5/791: dwrite d6/d12/d4e/d58/d8f/fa2 [0,4194304] 0 2024-02-16T22:56:31.821 INFO:tasks.workunit.client.1.smithi192.stdout:4/843: dread - d1/d26/d34/f44 zero size 2024-02-16T22:56:31.821 INFO:tasks.workunit.client.0.smithi064.stdout:2/800: truncate d7/d14/d25/d46/d55/d7f/fe7 407937 0 2024-02-16T22:56:31.821 INFO:tasks.workunit.client.0.smithi064.stdout:5/621: truncate d6/d25/d48/d67/d91/fc4 350726 0 2024-02-16T22:56:31.822 INFO:tasks.workunit.client.0.smithi064.stdout:9/631: dwrite d3/d11/d15/dab/db6/fc8 [0,4194304] 0 2024-02-16T22:56:31.823 INFO:tasks.workunit.client.0.smithi064.stdout:6/727: readlink d0/d6/d10/d33/l6e 0 2024-02-16T22:56:31.824 INFO:tasks.workunit.client.1.smithi192.stdout:8/783: mknod d5/d26/d8f/dc0/dc3/cf3 0 2024-02-16T22:56:31.828 INFO:tasks.workunit.client.1.smithi192.stdout:7/903: creat d1/d2/d37/f132 x:0 0 0 2024-02-16T22:56:31.831 INFO:tasks.workunit.client.1.smithi192.stdout:9/913: link d7/d15/d36/d38/d3f/f59 d7/d15/d36/d38/d3f/d8a/def/d100/f137 0 2024-02-16T22:56:31.833 INFO:tasks.workunit.client.1.smithi192.stdout:3/726: creat da/d11/d63/d6b/d66/ff8 x:0 0 0 2024-02-16T22:56:31.834 INFO:tasks.workunit.client.1.smithi192.stdout:3/727: write da/d11/d63/d6b/d66/fe3 [118594,71460] 0 2024-02-16T22:56:31.834 INFO:tasks.workunit.client.0.smithi064.stdout:6/728: fsync d0/d6/da/f95 0 2024-02-16T22:56:31.836 INFO:tasks.workunit.client.0.smithi064.stdout:9/632: creat d3/d89/d9a/fce x:0 0 0 2024-02-16T22:56:31.836 INFO:tasks.workunit.client.1.smithi192.stdout:2/833: dread d22/d24/d39/d3d/f10d [0,4194304] 0 2024-02-16T22:56:31.839 INFO:tasks.workunit.client.0.smithi064.stdout:5/622: fsync d6/d25/d48/d67/fce 0 2024-02-16T22:56:31.839 INFO:tasks.workunit.client.1.smithi192.stdout:5/792: fsync d6/d12/d1d/fb5 0 2024-02-16T22:56:31.841 INFO:tasks.workunit.client.0.smithi064.stdout:2/801: dread d7/f13 [0,4194304] 0 2024-02-16T22:56:31.842 INFO:tasks.workunit.client.0.smithi064.stdout:4/678: link dc/dd/d5b/fc1 dc/dd/d32/d22/fd7 0 2024-02-16T22:56:31.842 INFO:tasks.workunit.client.1.smithi192.stdout:4/844: write d1/d26/fd7 [443619,22875] 0 2024-02-16T22:56:31.845 INFO:tasks.workunit.client.0.smithi064.stdout:6/729: creat d0/d85/ff4 x:0 0 0 2024-02-16T22:56:31.846 INFO:tasks.workunit.client.0.smithi064.stdout:6/730: chown d0/d3c/l87 9 1 2024-02-16T22:56:31.846 INFO:tasks.workunit.client.0.smithi064.stdout:6/731: dread - d0/d5/df/d17/ff2 zero size 2024-02-16T22:56:31.847 INFO:tasks.workunit.client.1.smithi192.stdout:8/784: fsync d5/db/d47/d43/d90/fbe 0 2024-02-16T22:56:31.849 INFO:tasks.workunit.client.1.smithi192.stdout:4/845: dwrite d1/d6b/f102 [0,4194304] 0 2024-02-16T22:56:31.850 INFO:tasks.workunit.client.0.smithi064.stdout:7/667: sync 2024-02-16T22:56:31.850 INFO:tasks.workunit.client.1.smithi192.stdout:4/846: chown d1/d6b/fa1 8 1 2024-02-16T22:56:31.851 INFO:tasks.workunit.client.1.smithi192.stdout:7/904: read d1/d2/d37/f68 [195361,91510] 0 2024-02-16T22:56:31.855 INFO:tasks.workunit.client.1.smithi192.stdout:2/834: dread d22/d24/d39/d42/d85/d98/dc4/ff1 [0,4194304] 0 2024-02-16T22:56:31.856 INFO:tasks.workunit.client.0.smithi064.stdout:5/623: rename d6/d18/d90/fbc to d6/d25/d48/d67/d80/fda 0 2024-02-16T22:56:31.859 INFO:tasks.workunit.client.1.smithi192.stdout:3/728: rename da/d11/d63/d6b/d66/fcd to da/d11/d35/ff9 0 2024-02-16T22:56:31.861 INFO:tasks.workunit.client.0.smithi064.stdout:2/802: write d7/d14/d4b/fc7 [8223718,51733] 0 2024-02-16T22:56:31.861 INFO:tasks.workunit.client.1.smithi192.stdout:0/773: getdents d1/d3/d29/dcd/dcf 0 2024-02-16T22:56:31.862 INFO:tasks.workunit.client.0.smithi064.stdout:2/803: fsync d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f49 0 2024-02-16T22:56:31.863 INFO:tasks.workunit.client.0.smithi064.stdout:4/679: truncate dc/dd/d32/d22/f9d 121546 0 2024-02-16T22:56:31.865 INFO:tasks.workunit.client.0.smithi064.stdout:6/732: rmdir d0/d6/d10/d33/d72 39 2024-02-16T22:56:31.868 INFO:tasks.workunit.client.1.smithi192.stdout:5/793: unlink d6/d12/d2d/lcb 0 2024-02-16T22:56:31.870 INFO:tasks.workunit.client.0.smithi064.stdout:6/733: dwrite d0/d5/df/fc2 [0,4194304] 0 2024-02-16T22:56:31.872 INFO:tasks.workunit.client.0.smithi064.stdout:7/668: creat d5/d6/d65/da7/dad/fd0 x:0 0 0 2024-02-16T22:56:31.872 INFO:tasks.workunit.client.0.smithi064.stdout:7/669: dread - d5/d33/d45/fb5 zero size 2024-02-16T22:56:31.877 INFO:tasks.workunit.client.1.smithi192.stdout:8/785: creat d5/d26/d8f/dc0/ff4 x:0 0 0 2024-02-16T22:56:31.880 INFO:tasks.workunit.client.1.smithi192.stdout:4/847: mknod d1/d26/d66/d89/da8/c123 0 2024-02-16T22:56:31.888 INFO:tasks.workunit.client.0.smithi064.stdout:9/633: rmdir d3/d89/da1 0 2024-02-16T22:56:31.890 INFO:tasks.workunit.client.0.smithi064.stdout:6/734: symlink d0/d3c/da9/lf5 0 2024-02-16T22:56:31.891 INFO:tasks.workunit.client.0.smithi064.stdout:5/624: write d6/d25/f32 [66337,65462] 0 2024-02-16T22:56:31.893 INFO:tasks.workunit.client.0.smithi064.stdout:4/680: write dc/f81 [133973,102301] 0 2024-02-16T22:56:31.896 INFO:tasks.workunit.client.0.smithi064.stdout:4/681: dread dc/fba [0,4194304] 0 2024-02-16T22:56:31.897 INFO:tasks.workunit.client.0.smithi064.stdout:4/682: chown dc/dd/d16/f3a 274 1 2024-02-16T22:56:31.901 INFO:tasks.workunit.client.1.smithi192.stdout:9/914: getdents d7/d15/d36/d38/d3f/d8a/d117 0 2024-02-16T22:56:31.901 INFO:tasks.workunit.client.1.smithi192.stdout:2/835: creat d22/d47/dca/dfa/f10f x:0 0 0 2024-02-16T22:56:31.902 INFO:tasks.workunit.client.1.smithi192.stdout:3/729: rename da/df/f8f to da/d11/d20/d39/dca/ffa 0 2024-02-16T22:56:31.903 INFO:tasks.workunit.client.0.smithi064.stdout:6/735: write d0/d89/f8e [1229459,27048] 0 2024-02-16T22:56:31.904 INFO:tasks.workunit.client.1.smithi192.stdout:3/730: write da/d6a/fe4 [46414,15552] 0 2024-02-16T22:56:31.906 INFO:tasks.workunit.client.0.smithi064.stdout:7/670: creat d5/d33/fd1 x:0 0 0 2024-02-16T22:56:31.906 INFO:tasks.workunit.client.0.smithi064.stdout:6/736: stat d0/d6/da/da2 0 2024-02-16T22:56:31.910 INFO:tasks.workunit.client.1.smithi192.stdout:0/774: dwrite d1/d3/d39/f77 [0,4194304] 0 2024-02-16T22:56:31.911 INFO:tasks.workunit.client.0.smithi064.stdout:9/634: dwrite d3/d11/d15/d23/f6e [0,4194304] 0 2024-02-16T22:56:31.911 INFO:tasks.workunit.client.0.smithi064.stdout:4/683: dwrite dc/dd/d16/fa6 [4194304,4194304] 0 2024-02-16T22:56:31.915 INFO:tasks.workunit.client.1.smithi192.stdout:4/848: link d1/d6b/fa1 d1/d15/d48/d8d/d9b/f124 0 2024-02-16T22:56:31.917 INFO:tasks.workunit.client.0.smithi064.stdout:5/625: truncate d6/f37 4102725 0 2024-02-16T22:56:31.921 INFO:tasks.workunit.client.0.smithi064.stdout:4/684: dwrite dc/f6c [0,4194304] 0 2024-02-16T22:56:31.922 INFO:tasks.workunit.client.1.smithi192.stdout:8/786: truncate d5/d16/dcf/fdc 2625919 0 2024-02-16T22:56:31.923 INFO:tasks.workunit.client.0.smithi064.stdout:7/671: mknod d5/d6/d65/da7/dad/cd2 0 2024-02-16T22:56:31.926 INFO:tasks.workunit.client.0.smithi064.stdout:9/635: mkdir d3/d89/d9a/dcf 0 2024-02-16T22:56:31.930 INFO:tasks.workunit.client.1.smithi192.stdout:2/836: creat d22/d47/dcb/dc9/d103/f110 x:0 0 0 2024-02-16T22:56:31.931 INFO:tasks.workunit.client.1.smithi192.stdout:9/915: dread d7/d15/d36/d38/f47 [0,4194304] 0 2024-02-16T22:56:31.933 INFO:tasks.workunit.client.1.smithi192.stdout:0/775: mknod d1/d3/d29/dcd/dcf/cf4 0 2024-02-16T22:56:31.939 INFO:tasks.workunit.client.1.smithi192.stdout:4/849: unlink d1/d15/d48/l100 0 2024-02-16T22:56:31.939 INFO:tasks.workunit.client.1.smithi192.stdout:3/731: dread da/d11/d63/d6b/d66/d75/f76 [0,4194304] 0 2024-02-16T22:56:31.940 INFO:tasks.workunit.client.0.smithi064.stdout:4/685: mknod dc/dd/d32/d46/d5a/cd8 0 2024-02-16T22:56:31.941 INFO:tasks.workunit.client.0.smithi064.stdout:7/672: unlink d5/d6/d54/d7b/fc1 0 2024-02-16T22:56:31.943 INFO:tasks.workunit.client.0.smithi064.stdout:9/636: dread - d3/d9/d1e/d3e/f87 zero size 2024-02-16T22:56:31.945 INFO:tasks.workunit.client.1.smithi192.stdout:5/794: getdents d6/d12/d4e/d5b 0 2024-02-16T22:56:31.945 INFO:tasks.workunit.client.0.smithi064.stdout:5/626: truncate d6/f3c 2989188 0 2024-02-16T22:56:31.946 INFO:tasks.workunit.client.0.smithi064.stdout:5/627: write d6/d1c/f53 [2138109,32392] 0 2024-02-16T22:56:31.947 INFO:tasks.workunit.client.1.smithi192.stdout:5/795: write d6/d12/d4e/d58/fb7 [1552392,96892] 0 2024-02-16T22:56:31.948 INFO:tasks.workunit.client.0.smithi064.stdout:6/737: rmdir d0/d5/df/d17/d4a/db4 0 2024-02-16T22:56:31.950 INFO:tasks.workunit.client.1.smithi192.stdout:0/776: creat d1/dc7/d9c/ff5 x:0 0 0 2024-02-16T22:56:31.951 INFO:tasks.workunit.client.1.smithi192.stdout:6/819: sync 2024-02-16T22:56:31.952 INFO:tasks.workunit.client.1.smithi192.stdout:6/820: truncate da/df/d12/d15/d1b/d70/fd3 397772 0 2024-02-16T22:56:31.952 INFO:tasks.workunit.client.1.smithi192.stdout:6/821: chown da/d40 711 1 2024-02-16T22:56:31.952 INFO:tasks.workunit.client.1.smithi192.stdout:4/850: mknod d1/d15/d48/d8d/c125 0 2024-02-16T22:56:31.953 INFO:tasks.workunit.client.1.smithi192.stdout:4/851: fsync d1/d26/d66/d89/d11/d58/f90 0 2024-02-16T22:56:31.954 INFO:tasks.workunit.client.1.smithi192.stdout:3/732: unlink da/d11/d63/d6b/d66/cde 0 2024-02-16T22:56:31.955 INFO:tasks.workunit.client.0.smithi064.stdout:7/673: symlink d5/d6/d54/d7b/ld3 0 2024-02-16T22:56:31.957 INFO:tasks.workunit.client.1.smithi192.stdout:2/837: dread d22/d47/dcb/f30 [0,4194304] 0 2024-02-16T22:56:31.958 INFO:tasks.workunit.client.1.smithi192.stdout:2/838: dread - d22/d47/dcb/d3f/dac/fe8 zero size 2024-02-16T22:56:31.961 INFO:tasks.workunit.client.1.smithi192.stdout:0/777: mknod d1/d3/de/d22/d37/d47/dd0/cf6 0 2024-02-16T22:56:31.962 INFO:tasks.workunit.client.1.smithi192.stdout:6/822: creat da/d40/f109 x:0 0 0 2024-02-16T22:56:31.963 INFO:tasks.workunit.client.0.smithi064.stdout:7/674: unlink d5/d6/l2e 0 2024-02-16T22:56:31.965 INFO:tasks.workunit.client.1.smithi192.stdout:0/778: dwrite d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:31.970 INFO:tasks.workunit.client.1.smithi192.stdout:1/752: sync 2024-02-16T22:56:31.970 INFO:tasks.workunit.client.1.smithi192.stdout:3/733: rmdir da/d11/d20/d39 39 2024-02-16T22:56:31.970 INFO:tasks.workunit.client.0.smithi064.stdout:6/738: dread d0/d1d/f61 [0,4194304] 0 2024-02-16T22:56:31.971 INFO:tasks.workunit.client.1.smithi192.stdout:1/753: write d3/de/d27/d34/d69/f6b [3486504,130483] 0 2024-02-16T22:56:31.974 INFO:tasks.workunit.client.1.smithi192.stdout:8/787: link d5/db/de/l8a d5/db/d47/d43/d90/lf5 0 2024-02-16T22:56:31.977 INFO:tasks.workunit.client.1.smithi192.stdout:9/916: rename d7/d15/d36/d77/c8f to d7/d19/d28/c138 0 2024-02-16T22:56:31.980 INFO:tasks.workunit.client.0.smithi064.stdout:7/675: dread d5/d33/d45/d73/d88/f90 [0,4194304] 0 2024-02-16T22:56:31.980 INFO:tasks.workunit.client.0.smithi064.stdout:6/739: read - d0/d6/d10/f9d zero size 2024-02-16T22:56:31.981 INFO:tasks.workunit.client.1.smithi192.stdout:2/839: creat d22/d24/d39/d42/d65/d69/db1/f111 x:0 0 0 2024-02-16T22:56:31.987 INFO:tasks.workunit.client.0.smithi064.stdout:6/740: truncate d0/d5/f27 4149585 0 2024-02-16T22:56:31.987 INFO:tasks.workunit.client.0.smithi064.stdout:4/686: dwrite dc/dd/d16/f51 [0,4194304] 0 2024-02-16T22:56:31.988 INFO:tasks.workunit.client.0.smithi064.stdout:6/741: write d0/d3c/fef [587783,120201] 0 2024-02-16T22:56:31.988 INFO:tasks.workunit.client.1.smithi192.stdout:0/779: creat d1/d3/de/d22/d37/d47/d53/ff7 x:0 0 0 2024-02-16T22:56:31.990 INFO:tasks.workunit.client.0.smithi064.stdout:9/637: write d3/d9/d6a/db1/f6d [799178,17140] 0 2024-02-16T22:56:31.991 INFO:tasks.workunit.client.0.smithi064.stdout:5/628: truncate d6/d25/d48/d67/f9e 143680 0 2024-02-16T22:56:31.993 INFO:tasks.workunit.client.0.smithi064.stdout:5/629: truncate d6/d25/da4/dc1/d85/fad 267801 0 2024-02-16T22:56:31.994 INFO:tasks.workunit.client.1.smithi192.stdout:0/780: dwrite d1/d3/d39/f5f [0,4194304] 0 2024-02-16T22:56:31.997 INFO:tasks.workunit.client.1.smithi192.stdout:0/781: write d1/d3/de/d22/d37/d47/fda [916964,11796] 0 2024-02-16T22:56:31.998 INFO:tasks.workunit.client.1.smithi192.stdout:1/754: unlink d3/de/d27/d34/d69/d85/d89/fbe 0 2024-02-16T22:56:32.000 INFO:tasks.workunit.client.1.smithi192.stdout:1/755: write d3/d5b/d77/f88 [1240622,123434] 0 2024-02-16T22:56:32.001 INFO:tasks.workunit.client.1.smithi192.stdout:0/782: chown d1/d3/de/d22/d37/d47/f81 729 1 2024-02-16T22:56:32.004 INFO:tasks.workunit.client.1.smithi192.stdout:1/756: read - d3/de/d27/d34/d69/d85/d89/daa/fc8 zero size 2024-02-16T22:56:32.008 INFO:tasks.workunit.client.1.smithi192.stdout:1/757: write d3/de/d27/d34/d69/f6b [4195957,74509] 0 2024-02-16T22:56:32.008 INFO:tasks.workunit.client.1.smithi192.stdout:6/823: dwrite da/d40/f82 [0,4194304] 0 2024-02-16T22:56:32.009 INFO:tasks.workunit.client.0.smithi064.stdout:5/630: unlink d6/d1c/l2d 0 2024-02-16T22:56:32.011 INFO:tasks.workunit.client.1.smithi192.stdout:8/788: dwrite d5/d26/d83/fdb [0,4194304] 0 2024-02-16T22:56:32.011 INFO:tasks.workunit.client.1.smithi192.stdout:4/852: creat d1/d15/d48/f126 x:0 0 0 2024-02-16T22:56:32.012 INFO:tasks.workunit.client.0.smithi064.stdout:6/742: truncate d0/d6/d10/d33/f55 1379092 0 2024-02-16T22:56:32.019 INFO:tasks.workunit.client.1.smithi192.stdout:3/734: symlink da/d11/d63/d6b/d66/lfb 0 2024-02-16T22:56:32.023 INFO:tasks.workunit.client.0.smithi064.stdout:4/687: getdents dc/de/d28 0 2024-02-16T22:56:32.024 INFO:tasks.workunit.client.0.smithi064.stdout:4/688: truncate dc/d1d/dbd/fc2 104259 0 2024-02-16T22:56:32.024 INFO:tasks.workunit.client.0.smithi064.stdout:5/631: mknod d6/d25/d48/d67/d91/da5/cdb 0 2024-02-16T22:56:32.026 INFO:tasks.workunit.client.1.smithi192.stdout:0/783: chown d1/d3/d39/ca4 113669 1 2024-02-16T22:56:32.029 INFO:tasks.workunit.client.0.smithi064.stdout:4/689: dread - dc/de/d28/fbf zero size 2024-02-16T22:56:32.029 INFO:tasks.workunit.client.1.smithi192.stdout:2/840: dread f10 [0,4194304] 0 2024-02-16T22:56:32.030 INFO:tasks.workunit.client.0.smithi064.stdout:9/638: creat d3/d9/d6a/db1/fd0 x:0 0 0 2024-02-16T22:56:32.031 INFO:tasks.workunit.client.0.smithi064.stdout:7/676: getdents d5/d6/d54/d41/da2 0 2024-02-16T22:56:32.033 INFO:tasks.workunit.client.1.smithi192.stdout:5/796: getdents d6/d8 0 2024-02-16T22:56:32.036 INFO:tasks.workunit.client.1.smithi192.stdout:6/824: rmdir da/df/d12/d15 39 2024-02-16T22:56:32.037 INFO:tasks.workunit.client.1.smithi192.stdout:6/825: chown da/df/d12/d57/d81/dfc/d101 11 1 2024-02-16T22:56:32.038 INFO:tasks.workunit.client.0.smithi064.stdout:0/593: sync 2024-02-16T22:56:32.038 INFO:tasks.workunit.client.0.smithi064.stdout:8/622: sync 2024-02-16T22:56:32.038 INFO:tasks.workunit.client.0.smithi064.stdout:1/604: sync 2024-02-16T22:56:32.038 INFO:tasks.workunit.client.0.smithi064.stdout:3/783: sync 2024-02-16T22:56:32.038 INFO:tasks.workunit.client.1.smithi192.stdout:5/797: dwrite d6/d12/d4e/d5b/fed [0,4194304] 0 2024-02-16T22:56:32.040 INFO:tasks.workunit.client.0.smithi064.stdout:1/605: write d2/df/d14/d18/d48/d84/fa4 [4688661,106167] 0 2024-02-16T22:56:32.040 INFO:tasks.workunit.client.1.smithi192.stdout:5/798: chown d6/d8/d75/l7c 22969 1 2024-02-16T22:56:32.041 INFO:tasks.workunit.client.1.smithi192.stdout:4/853: creat d1/d15/d48/db5/dd5/d104/f127 x:0 0 0 2024-02-16T22:56:32.042 INFO:tasks.workunit.client.1.smithi192.stdout:4/854: readlink d1/d15/d29/lbb 0 2024-02-16T22:56:32.046 INFO:tasks.workunit.client.0.smithi064.stdout:3/784: dwrite d4/d13/d1b/d4f/f103 [0,4194304] 0 2024-02-16T22:56:32.047 INFO:tasks.workunit.client.1.smithi192.stdout:3/735: read da/d6a/f84 [108996,67992] 0 2024-02-16T22:56:32.048 INFO:tasks.workunit.client.1.smithi192.stdout:4/855: dwrite d1/d26/d66/d89/d11/f98 [0,4194304] 0 2024-02-16T22:56:32.051 INFO:tasks.workunit.client.1.smithi192.stdout:7/905: sync 2024-02-16T22:56:32.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/906: fsync d1/d2/d2d/d61/d88/d90/d10f/f12f 0 2024-02-16T22:56:32.059 INFO:tasks.workunit.client.1.smithi192.stdout:5/799: creat d6/d8/d75/d8c/f107 x:0 0 0 2024-02-16T22:56:32.060 INFO:tasks.workunit.client.1.smithi192.stdout:1/758: write d3/de/d27/d34/d69/d85/d89/daa/fb3 [5140805,82933] 0 2024-02-16T22:56:32.060 INFO:tasks.workunit.client.1.smithi192.stdout:1/759: fdatasync d3/f101 0 2024-02-16T22:56:32.061 INFO:tasks.workunit.client.1.smithi192.stdout:1/760: chown d3/de/d27/f39 16182 1 2024-02-16T22:56:32.063 INFO:tasks.workunit.client.1.smithi192.stdout:8/789: write d5/db/faa [368483,19914] 0 2024-02-16T22:56:32.063 INFO:tasks.workunit.client.1.smithi192.stdout:8/790: stat d5/d16/dd5/d9d/fd9 0 2024-02-16T22:56:32.064 INFO:tasks.workunit.client.1.smithi192.stdout:3/736: fdatasync da/d11/dd4/deb/ff0 0 2024-02-16T22:56:32.067 INFO:tasks.workunit.client.1.smithi192.stdout:4/856: rename d1/d26/dbe/d107 to d1/d15/d48/d8d/d128 0 2024-02-16T22:56:32.067 INFO:tasks.workunit.client.1.smithi192.stdout:0/784: symlink d1/lf8 0 2024-02-16T22:56:32.069 INFO:tasks.workunit.client.1.smithi192.stdout:9/917: getdents d7/d15/d36/d77/db1 0 2024-02-16T22:56:32.083 INFO:tasks.workunit.client.1.smithi192.stdout:3/737: read da/d11/d63/d6b/f87 [1708617,42714] 0 2024-02-16T22:56:32.087 INFO:tasks.workunit.client.1.smithi192.stdout:1/761: rename d3/de/d27/d34/dbf/fcf to d3/de/d27/d34/d69/d85/d89/daa/dfb/f102 0 2024-02-16T22:56:32.088 INFO:tasks.workunit.client.0.smithi064.stdout:2/804: sync 2024-02-16T22:56:32.089 INFO:tasks.workunit.client.1.smithi192.stdout:2/841: creat d22/d24/d39/d3d/df2/f112 x:0 0 0 2024-02-16T22:56:32.089 INFO:tasks.workunit.client.1.smithi192.stdout:2/842: chown d22/d47/dcb/d78/lb2 30786258 1 2024-02-16T22:56:32.091 INFO:tasks.workunit.client.0.smithi064.stdout:5/632: creat d6/d25/d48/d67/d80/fdc x:0 0 0 2024-02-16T22:56:32.091 INFO:tasks.workunit.client.1.smithi192.stdout:7/907: write d1/f12 [51150,126885] 0 2024-02-16T22:56:32.092 INFO:tasks.workunit.client.1.smithi192.stdout:6/826: rmdir da/df/dfb 0 2024-02-16T22:56:32.093 INFO:tasks.workunit.client.0.smithi064.stdout:6/743: mknod d0/d6/d10/d33/d72/de8/dee/cf6 0 2024-02-16T22:56:32.095 INFO:tasks.workunit.client.1.smithi192.stdout:8/791: symlink d5/db/de/lf6 0 2024-02-16T22:56:32.096 INFO:tasks.workunit.client.0.smithi064.stdout:4/690: unlink dc/dd/d32/d46/d5a/l7f 0 2024-02-16T22:56:32.097 INFO:tasks.workunit.client.1.smithi192.stdout:8/792: read d5/db/f51 [1270602,24060] 0 2024-02-16T22:56:32.097 INFO:tasks.workunit.client.0.smithi064.stdout:4/691: fsync dc/d1d/d43/f5f 0 2024-02-16T22:56:32.098 INFO:tasks.workunit.client.0.smithi064.stdout:7/677: creat d5/d33/d42/d81/d82/fd4 x:0 0 0 2024-02-16T22:56:32.099 INFO:tasks.workunit.client.1.smithi192.stdout:5/800: dwrite d6/d12/d4e/f2e [0,4194304] 0 2024-02-16T22:56:32.099 INFO:tasks.workunit.client.1.smithi192.stdout:3/738: dread - da/d11/d63/d6b/f47 zero size 2024-02-16T22:56:32.103 INFO:tasks.workunit.client.1.smithi192.stdout:4/857: rename d1/d26/d66/d89/f68 to d1/d15/d48/d8d/def/f129 0 2024-02-16T22:56:32.104 INFO:tasks.workunit.client.0.smithi064.stdout:4/692: dwrite dc/dd/d16/fa6 [4194304,4194304] 0 2024-02-16T22:56:32.105 INFO:tasks.workunit.client.0.smithi064.stdout:4/693: write dc/de/d28/fbf [1046629,8737] 0 2024-02-16T22:56:32.110 INFO:tasks.workunit.client.1.smithi192.stdout:2/843: creat d22/d24/d39/d42/d85/d98/dc4/f113 x:0 0 0 2024-02-16T22:56:32.110 INFO:tasks.workunit.client.0.smithi064.stdout:3/785: write d4/d1f/d34/ff9 [605151,53906] 0 2024-02-16T22:56:32.111 INFO:tasks.workunit.client.1.smithi192.stdout:6/827: symlink da/db9/l10a 0 2024-02-16T22:56:32.115 INFO:tasks.workunit.client.0.smithi064.stdout:6/744: write d0/d89/f93 [954601,29334] 0 2024-02-16T22:56:32.117 INFO:tasks.workunit.client.0.smithi064.stdout:9/639: dread - d3/d9/d1e/d42/f7c zero size 2024-02-16T22:56:32.118 INFO:tasks.workunit.client.1.smithi192.stdout:8/793: creat d5/d16/dd5/ff7 x:0 0 0 2024-02-16T22:56:32.118 INFO:tasks.workunit.client.1.smithi192.stdout:2/844: dwrite d22/d24/d39/d42/d65/d69/f68 [0,4194304] 0 2024-02-16T22:56:32.118 INFO:tasks.workunit.client.1.smithi192.stdout:5/801: rmdir d6 39 2024-02-16T22:56:32.119 INFO:tasks.workunit.client.0.smithi064.stdout:9/640: fsync d3/d9/d6a/db1/d38/fca 0 2024-02-16T22:56:32.122 INFO:tasks.workunit.client.1.smithi192.stdout:3/739: chown da/d11/d2b/ff3 16225 1 2024-02-16T22:56:32.126 INFO:tasks.workunit.client.1.smithi192.stdout:1/762: mknod d3/de/d27/d34/d69/d85/df7/c103 0 2024-02-16T22:56:32.129 INFO:tasks.workunit.client.1.smithi192.stdout:9/918: rename d7/d19/d5b to d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/d139 0 2024-02-16T22:56:32.137 INFO:tasks.workunit.client.0.smithi064.stdout:2/805: mknod d7/d14/d1e/d34/d3f/d9a/dd9/d95/c104 0 2024-02-16T22:56:32.143 INFO:tasks.workunit.client.0.smithi064.stdout:6/745: creat d0/d1d/db1/ff7 x:0 0 0 2024-02-16T22:56:32.143 INFO:tasks.workunit.client.0.smithi064.stdout:9/641: creat d3/d11/fd1 x:0 0 0 2024-02-16T22:56:32.144 INFO:tasks.workunit.client.1.smithi192.stdout:5/802: write d6/d12/d2d/fd0 [738391,52533] 0 2024-02-16T22:56:32.146 INFO:tasks.workunit.client.0.smithi064.stdout:8/623: dwrite df/d31/d61/fa8 [0,4194304] 0 2024-02-16T22:56:32.147 INFO:tasks.workunit.client.0.smithi064.stdout:0/594: creat da/dc/de/fc8 x:0 0 0 2024-02-16T22:56:32.147 INFO:tasks.workunit.client.1.smithi192.stdout:4/858: dread d1/d26/fa5 [0,4194304] 0 2024-02-16T22:56:32.147 INFO:tasks.workunit.client.0.smithi064.stdout:9/642: chown d3/d9/d6a/db1/d2a/d2e/d31/d3f/f43 7 1 2024-02-16T22:56:32.148 INFO:tasks.workunit.client.1.smithi192.stdout:2/845: readlink d22/d47/dcb/dc9/l109 0 2024-02-16T22:56:32.148 INFO:tasks.workunit.client.1.smithi192.stdout:5/803: dwrite d6/d12/d4e/d58/d8f/fa2 [0,4194304] 0 2024-02-16T22:56:32.149 INFO:tasks.workunit.client.0.smithi064.stdout:8/624: dread df/f15 [0,4194304] 0 2024-02-16T22:56:32.151 INFO:tasks.workunit.client.1.smithi192.stdout:7/908: write d1/d2/de/d2c/d49/d54/dd8/d117/d9b/ffc [532812,129760] 0 2024-02-16T22:56:32.152 INFO:tasks.workunit.client.0.smithi064.stdout:1/606: creat d2/df/d14/d18/fd1 x:0 0 0 2024-02-16T22:56:32.155 INFO:tasks.workunit.client.0.smithi064.stdout:8/625: dread df/d31/d2f/d38/fba [0,4194304] 0 2024-02-16T22:56:32.155 INFO:tasks.workunit.client.0.smithi064.stdout:0/595: dwrite f9 [0,4194304] 0 2024-02-16T22:56:32.156 INFO:tasks.workunit.client.1.smithi192.stdout:1/763: mknod d3/de/d27/d34/d36/c104 0 2024-02-16T22:56:32.156 INFO:tasks.workunit.client.0.smithi064.stdout:8/626: chown df/d31/d61/cc2 3652678 1 2024-02-16T22:56:32.157 INFO:tasks.workunit.client.0.smithi064.stdout:5/633: write d6/d18/d19/f26 [2757555,108882] 0 2024-02-16T22:56:32.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/846: dwrite d22/d24/d39/d42/d84/f101 [0,4194304] 0 2024-02-16T22:56:32.162 INFO:tasks.workunit.client.1.smithi192.stdout:6/828: dread - da/df/d58/f9a zero size 2024-02-16T22:56:32.163 INFO:tasks.workunit.client.1.smithi192.stdout:0/785: sync 2024-02-16T22:56:32.166 INFO:tasks.workunit.client.0.smithi064.stdout:9/643: symlink d3/d9/d6a/db1/d2a/d2e/d31/ld2 0 2024-02-16T22:56:32.168 INFO:tasks.workunit.client.0.smithi064.stdout:3/786: write d4/f1e [3284323,34257] 0 2024-02-16T22:56:32.169 INFO:tasks.workunit.client.1.smithi192.stdout:0/786: dread d1/d3/d29/f8d [4194304,4194304] 0 2024-02-16T22:56:32.169 INFO:tasks.workunit.client.1.smithi192.stdout:0/787: fdatasync d1/f24 0 2024-02-16T22:56:32.170 INFO:tasks.workunit.client.1.smithi192.stdout:0/788: chown d1/dc7/lde 11257 1 2024-02-16T22:56:32.171 INFO:tasks.workunit.client.1.smithi192.stdout:5/804: truncate d6/d12/d1d/fb5 453915 0 2024-02-16T22:56:32.173 INFO:tasks.workunit.client.0.smithi064.stdout:0/596: mknod da/dc/de/d12/d2c/d2e/d38/dbe/cc9 0 2024-02-16T22:56:32.174 INFO:tasks.workunit.client.0.smithi064.stdout:0/597: readlink da/dc/de/d12/d2c/d2e/d38/l8a 0 2024-02-16T22:56:32.176 INFO:tasks.workunit.client.1.smithi192.stdout:7/909: fdatasync d1/d2/d2d/d61/d8a/de6/fe9 0 2024-02-16T22:56:32.179 INFO:tasks.workunit.client.0.smithi064.stdout:6/746: truncate d0/d5/f27 397316 0 2024-02-16T22:56:32.180 INFO:tasks.workunit.client.0.smithi064.stdout:6/747: truncate d0/d1d/dbc/fe4 160348 0 2024-02-16T22:56:32.181 INFO:tasks.workunit.client.0.smithi064.stdout:9/644: mkdir d3/d9/d6a/db1/d2a/d2e/d31/dd3 0 2024-02-16T22:56:32.187 INFO:tasks.workunit.client.1.smithi192.stdout:8/794: link d5/d16/f18 d5/db/ff8 0 2024-02-16T22:56:32.188 INFO:tasks.workunit.client.1.smithi192.stdout:8/795: dread - d5/d16/dd5/fd1 zero size 2024-02-16T22:56:32.190 INFO:tasks.workunit.client.1.smithi192.stdout:4/859: creat d1/d15/d48/d8d/d11b/f12a x:0 0 0 2024-02-16T22:56:32.191 INFO:tasks.workunit.client.0.smithi064.stdout:2/806: link d7/d14/d25/d46/d55/d7f/dde/lf8 d7/d14/d25/l105 0 2024-02-16T22:56:32.191 INFO:tasks.workunit.client.1.smithi192.stdout:1/764: dread d3/de/d87/fa9 [0,4194304] 0 2024-02-16T22:56:32.192 INFO:tasks.workunit.client.0.smithi064.stdout:2/807: readlink d7/d14/d25/d46/d55/lc2 0 2024-02-16T22:56:32.193 INFO:tasks.workunit.client.0.smithi064.stdout:6/748: creat d0/d6/d10/db2/ff8 x:0 0 0 2024-02-16T22:56:32.193 INFO:tasks.workunit.client.0.smithi064.stdout:6/749: chown d0/d5/df/d17 3479 1 2024-02-16T22:56:32.194 INFO:tasks.workunit.client.1.smithi192.stdout:0/789: mknod d1/d3/d18/cf9 0 2024-02-16T22:56:32.195 INFO:tasks.workunit.client.0.smithi064.stdout:4/694: truncate dc/dd/d32/d46/d5a/f66 3587619 0 2024-02-16T22:56:32.196 INFO:tasks.workunit.client.0.smithi064.stdout:4/695: stat dc/dd/d5b/c76 0 2024-02-16T22:56:32.196 INFO:tasks.workunit.client.0.smithi064.stdout:4/696: readlink dc/dd/d32/d22/l34 0 2024-02-16T22:56:32.196 INFO:tasks.workunit.client.1.smithi192.stdout:5/805: mkdir d6/d12/d1d/d56/d108 0 2024-02-16T22:56:32.198 INFO:tasks.workunit.client.1.smithi192.stdout:3/740: link da/df/lf6 da/d11/dd4/lfc 0 2024-02-16T22:56:32.199 INFO:tasks.workunit.client.0.smithi064.stdout:1/607: creat d2/df/d11/fd2 x:0 0 0 2024-02-16T22:56:32.199 INFO:tasks.workunit.client.1.smithi192.stdout:3/741: write da/f94 [161685,72141] 0 2024-02-16T22:56:32.201 INFO:tasks.workunit.client.0.smithi064.stdout:3/787: dread d4/f84 [0,4194304] 0 2024-02-16T22:56:32.203 INFO:tasks.workunit.client.0.smithi064.stdout:1/608: dread d2/df/d11/fc1 [0,4194304] 0 2024-02-16T22:56:32.203 INFO:tasks.workunit.client.1.smithi192.stdout:7/910: creat d1/d2/de/d2c/d49/d54/dd8/d117/dcd/f133 x:0 0 0 2024-02-16T22:56:32.204 INFO:tasks.workunit.client.0.smithi064.stdout:2/808: rmdir d7/d14/d25 39 2024-02-16T22:56:32.206 INFO:tasks.workunit.client.1.smithi192.stdout:1/765: dread d3/f46 [0,4194304] 0 2024-02-16T22:56:32.207 INFO:tasks.workunit.client.0.smithi064.stdout:6/750: unlink d0/d5/df/cdc 0 2024-02-16T22:56:32.208 INFO:tasks.workunit.client.0.smithi064.stdout:9/645: getdents d3/d89/d9a/dcf 0 2024-02-16T22:56:32.209 INFO:tasks.workunit.client.0.smithi064.stdout:9/646: chown d3/d9/d1e/d42/fb8 0 1 2024-02-16T22:56:32.209 INFO:tasks.workunit.client.1.smithi192.stdout:9/919: write d7/d15/d36/d38/d3f/d8a/def/d100/d10e/f119 [417631,93547] 0 2024-02-16T22:56:32.211 INFO:tasks.workunit.client.1.smithi192.stdout:6/829: chown da/df/d71/dab/fad 28843 1 2024-02-16T22:56:32.215 INFO:tasks.workunit.client.1.smithi192.stdout:8/796: symlink d5/d26/d8f/lf9 0 2024-02-16T22:56:32.215 INFO:tasks.workunit.client.1.smithi192.stdout:8/797: write d5/d16/dd5/fd1 [661434,80280] 0 2024-02-16T22:56:32.216 INFO:tasks.workunit.client.0.smithi064.stdout:3/788: rmdir d4/d1f/d34/d7b 39 2024-02-16T22:56:32.220 INFO:tasks.workunit.client.0.smithi064.stdout:8/627: dwrite df/d31/d2f/fc3 [0,4194304] 0 2024-02-16T22:56:32.222 INFO:tasks.workunit.client.0.smithi064.stdout:2/809: creat d7/d14/d4b/d63/f106 x:0 0 0 2024-02-16T22:56:32.222 INFO:tasks.workunit.client.0.smithi064.stdout:8/628: dread - df/d31/d61/d94/fa6 zero size 2024-02-16T22:56:32.224 INFO:tasks.workunit.client.0.smithi064.stdout:8/629: chown df/f32 651979 1 2024-02-16T22:56:32.224 INFO:tasks.workunit.client.1.smithi192.stdout:0/790: creat d1/d3/d29/dcd/dcf/ffa x:0 0 0 2024-02-16T22:56:32.224 INFO:tasks.workunit.client.0.smithi064.stdout:2/810: chown d7/d14/d1e/d34/d3f/d9a/dd9/lf9 1124789359 1 2024-02-16T22:56:32.226 INFO:tasks.workunit.client.0.smithi064.stdout:5/634: dwrite d6/d18/d19/f2e [0,4194304] 0 2024-02-16T22:56:32.227 INFO:tasks.workunit.client.1.smithi192.stdout:5/806: write d6/d12/d4e/d58/d8f/fa9 [461534,12815] 0 2024-02-16T22:56:32.235 INFO:tasks.workunit.client.1.smithi192.stdout:7/911: readlink d1/d2/d2d/d61/d88/d90/db2/ld5 0 2024-02-16T22:56:32.236 INFO:tasks.workunit.client.0.smithi064.stdout:1/609: dread - d2/df/d14/d18/d40/fa0 zero size 2024-02-16T22:56:32.239 INFO:tasks.workunit.client.1.smithi192.stdout:2/847: rename d22/d24/d39/d42/d52/cf8 to d22/d24/d39/c114 0 2024-02-16T22:56:32.239 INFO:tasks.workunit.client.0.smithi064.stdout:3/789: dwrite d4/d13/d1b/d27/d44/d97/fdc [0,4194304] 0 2024-02-16T22:56:32.240 INFO:tasks.workunit.client.1.smithi192.stdout:2/848: write d22/d24/d39/d42/d85/d98/db0/fc1 [784417,38540] 0 2024-02-16T22:56:32.240 INFO:tasks.workunit.client.0.smithi064.stdout:8/630: rmdir df 39 2024-02-16T22:56:32.242 INFO:tasks.workunit.client.1.smithi192.stdout:1/766: mkdir d3/de/d27/d34/d52/d5a/d105 0 2024-02-16T22:56:32.243 INFO:tasks.workunit.client.0.smithi064.stdout:2/811: rmdir d7/d14 39 2024-02-16T22:56:32.243 INFO:tasks.workunit.client.1.smithi192.stdout:1/767: stat d3/de/d27/d34/d36/d47/faf 0 2024-02-16T22:56:32.247 INFO:tasks.workunit.client.0.smithi064.stdout:5/635: fdatasync d6/d25/da4/dc1/fa1 0 2024-02-16T22:56:32.248 INFO:tasks.workunit.client.1.smithi192.stdout:8/798: fsync d5/d9/f79 0 2024-02-16T22:56:32.254 INFO:tasks.workunit.client.0.smithi064.stdout:9/647: dwrite d3/d9/f41 [0,4194304] 0 2024-02-16T22:56:32.255 INFO:tasks.workunit.client.1.smithi192.stdout:8/799: dwrite d5/d9/f33 [0,4194304] 0 2024-02-16T22:56:32.255 INFO:tasks.workunit.client.1.smithi192.stdout:0/791: mknod d1/d3/dac/cfb 0 2024-02-16T22:56:32.255 INFO:tasks.workunit.client.0.smithi064.stdout:9/648: chown d3/d9/d1e/f8d 0 1 2024-02-16T22:56:32.256 INFO:tasks.workunit.client.1.smithi192.stdout:0/792: write d1/d3/d29/dcd/dcf/feb [701560,69081] 0 2024-02-16T22:56:32.257 INFO:tasks.workunit.client.0.smithi064.stdout:9/649: read d3/fc [3607870,25672] 0 2024-02-16T22:56:32.257 INFO:tasks.workunit.client.0.smithi064.stdout:8/631: write df/d31/d8f/dc9/fd4 [130137,103681] 0 2024-02-16T22:56:32.258 INFO:tasks.workunit.client.1.smithi192.stdout:0/793: write d1/d3/de/d22/d37/fdc [549238,78251] 0 2024-02-16T22:56:32.259 INFO:tasks.workunit.client.0.smithi064.stdout:3/790: mkdir d4/d14/d2b/d10b 0 2024-02-16T22:56:32.262 INFO:tasks.workunit.client.0.smithi064.stdout:9/650: dread d3/d88/fc5 [0,4194304] 0 2024-02-16T22:56:32.266 INFO:tasks.workunit.client.0.smithi064.stdout:8/632: dwrite df/d31/d2f/d3e/f83 [0,4194304] 0 2024-02-16T22:56:32.267 INFO:tasks.workunit.client.1.smithi192.stdout:2/849: mkdir d22/d24/d39/d3d/df2/d99/d115 0 2024-02-16T22:56:32.269 INFO:tasks.workunit.client.0.smithi064.stdout:4/697: link dc/l6f dc/dd/d16/ld9 0 2024-02-16T22:56:32.270 INFO:tasks.workunit.client.1.smithi192.stdout:4/860: dwrite d1/f4 [0,4194304] 0 2024-02-16T22:56:32.270 INFO:tasks.workunit.client.0.smithi064.stdout:4/698: chown dc/dd/d32/d46/d5a/cd8 53532 1 2024-02-16T22:56:32.270 INFO:tasks.workunit.client.1.smithi192.stdout:9/920: symlink d7/d15/d36/d38/l13a 0 2024-02-16T22:56:32.271 INFO:tasks.workunit.client.0.smithi064.stdout:1/610: link d2/df/d11/d1a/d1d/fca d2/df/d11/d71/fd3 0 2024-02-16T22:56:32.278 INFO:tasks.workunit.client.1.smithi192.stdout:2/850: dwrite d22/d47/dca/dfa/f10f [0,4194304] 0 2024-02-16T22:56:32.279 INFO:tasks.workunit.client.0.smithi064.stdout:9/651: symlink d3/d89/ld4 0 2024-02-16T22:56:32.281 INFO:tasks.workunit.client.1.smithi192.stdout:5/807: creat d6/d12/d1d/d56/d108/f109 x:0 0 0 2024-02-16T22:56:32.281 INFO:tasks.workunit.client.0.smithi064.stdout:9/652: write d3/d9/d6a/db1/d38/fca [284517,126983] 0 2024-02-16T22:56:32.282 INFO:tasks.workunit.client.1.smithi192.stdout:0/794: read - d1/d3/de/d22/d37/d67/d75/db0/fc6 zero size 2024-02-16T22:56:32.282 INFO:tasks.workunit.client.0.smithi064.stdout:8/633: symlink df/d31/d61/d79/dc4/ld8 0 2024-02-16T22:56:32.283 INFO:tasks.workunit.client.0.smithi064.stdout:5/636: truncate d6/f3c 1140306 0 2024-02-16T22:56:32.284 INFO:tasks.workunit.client.1.smithi192.stdout:1/768: creat d3/d5b/d77/dd2/f106 x:0 0 0 2024-02-16T22:56:32.287 INFO:tasks.workunit.client.1.smithi192.stdout:9/921: unlink d7/d19/d37/d5a/f134 0 2024-02-16T22:56:32.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/922: chown d7/d19/d37/d5a/fdd 28584 1 2024-02-16T22:56:32.288 INFO:tasks.workunit.client.1.smithi192.stdout:3/742: sync 2024-02-16T22:56:32.289 INFO:tasks.workunit.client.0.smithi064.stdout:8/634: mknod df/d31/d2f/d3e/cd9 0 2024-02-16T22:56:32.293 INFO:tasks.workunit.client.1.smithi192.stdout:5/808: unlink d6/d12/d4e/d46/d4f/f6e 0 2024-02-16T22:56:32.293 INFO:tasks.workunit.client.1.smithi192.stdout:1/769: dwrite d3/de/d27/d34/d52/d63/d7a/dc9/fe1 [0,4194304] 0 2024-02-16T22:56:32.294 INFO:tasks.workunit.client.1.smithi192.stdout:0/795: rename d1/ca8 to d1/d3/d29/dcd/dcf/cfc 0 2024-02-16T22:56:32.295 INFO:tasks.workunit.client.1.smithi192.stdout:5/809: dread - d6/d12/d4e/d5b/fbf zero size 2024-02-16T22:56:32.296 INFO:tasks.workunit.client.1.smithi192.stdout:1/770: chown d3/de/d27/d34/d52/d63/d7a 0 1 2024-02-16T22:56:32.296 INFO:tasks.workunit.client.1.smithi192.stdout:9/923: mknod d7/d19/d5e/d6f/c13b 0 2024-02-16T22:56:32.297 INFO:tasks.workunit.client.1.smithi192.stdout:6/830: dwrite da/df/d12/d15/fdf [0,4194304] 0 2024-02-16T22:56:32.298 INFO:tasks.workunit.client.1.smithi192.stdout:8/800: creat d5/db/d47/d43/ffa x:0 0 0 2024-02-16T22:56:32.299 INFO:tasks.workunit.client.1.smithi192.stdout:7/912: getdents d1/d2/de/d83 0 2024-02-16T22:56:32.299 INFO:tasks.workunit.client.1.smithi192.stdout:6/831: fsync da/df/d12/d15/d1b/f1e 0 2024-02-16T22:56:32.303 INFO:tasks.workunit.client.1.smithi192.stdout:5/810: mknod d6/d12/d4e/d46/d4f/c10a 0 2024-02-16T22:56:32.304 INFO:tasks.workunit.client.1.smithi192.stdout:3/743: creat da/d11/d63/d6b/ffd x:0 0 0 2024-02-16T22:56:32.304 INFO:tasks.workunit.client.1.smithi192.stdout:8/801: mkdir d5/d26/d83/d86/d5f/dbf/dfb 0 2024-02-16T22:56:32.306 INFO:tasks.workunit.client.1.smithi192.stdout:0/796: mkdir d1/d3/d39/df3/dfd 0 2024-02-16T22:56:32.306 INFO:tasks.workunit.client.1.smithi192.stdout:8/802: stat d5/d9/f73 0 2024-02-16T22:56:32.307 INFO:tasks.workunit.client.1.smithi192.stdout:5/811: rename d6/d12/d1d/c23 to d6/dda/c10b 0 2024-02-16T22:56:32.308 INFO:tasks.workunit.client.1.smithi192.stdout:0/797: write d1/d3/de/d22/d37/d47/f81 [133763,66530] 0 2024-02-16T22:56:32.311 INFO:tasks.workunit.client.1.smithi192.stdout:7/913: link d1/d2/d2d/d61/d88/d90/f64 d1/d2/d2d/d61/d8a/f134 0 2024-02-16T22:56:32.312 INFO:tasks.workunit.client.1.smithi192.stdout:7/914: chown d1/d2/de/d2c/d49/d54/dd8/d117/d9b 13420917 1 2024-02-16T22:56:32.312 INFO:tasks.workunit.client.1.smithi192.stdout:0/798: creat d1/ffe x:0 0 0 2024-02-16T22:56:32.313 INFO:tasks.workunit.client.1.smithi192.stdout:7/915: symlink d1/d2/de/d2c/d49/d54/de3/l135 0 2024-02-16T22:56:32.315 INFO:tasks.workunit.client.1.smithi192.stdout:7/916: mkdir d1/d2/de/d2c/d49/d136 0 2024-02-16T22:56:32.317 INFO:tasks.workunit.client.1.smithi192.stdout:7/917: chown d1/c21 203 1 2024-02-16T22:56:32.318 INFO:tasks.workunit.client.1.smithi192.stdout:7/918: fdatasync d1/d2/fb4 0 2024-02-16T22:56:32.320 INFO:tasks.workunit.client.1.smithi192.stdout:7/919: creat d1/d2/d2d/d61/d88/d12e/f137 x:0 0 0 2024-02-16T22:56:32.320 INFO:tasks.workunit.client.1.smithi192.stdout:7/920: chown d1/d2/de/d2c/d49/d89 37365 1 2024-02-16T22:56:32.322 INFO:tasks.workunit.client.1.smithi192.stdout:7/921: mknod d1/d2/de/d2c/d49/d54/dd8/d117/dcd/c138 0 2024-02-16T22:56:32.325 INFO:tasks.workunit.client.1.smithi192.stdout:7/922: creat d1/d2/d2d/d61/d88/f139 x:0 0 0 2024-02-16T22:56:32.326 INFO:tasks.workunit.client.1.smithi192.stdout:2/851: dread d22/d47/dcb/d3f/dac/fde [0,4194304] 0 2024-02-16T22:56:32.330 INFO:tasks.workunit.client.0.smithi064.stdout:1/611: creat d2/df/d14/d18/d48/fd4 x:0 0 0 2024-02-16T22:56:32.331 INFO:tasks.workunit.client.0.smithi064.stdout:2/812: dread d7/d14/f2c [4194304,4194304] 0 2024-02-16T22:56:32.332 INFO:tasks.workunit.client.1.smithi192.stdout:2/852: rename d22/d47/dca/dfa/lfb to d22/d47/dcb/dda/l116 0 2024-02-16T22:56:32.332 INFO:tasks.workunit.client.1.smithi192.stdout:8/803: dread d5/d16/f4f [0,4194304] 0 2024-02-16T22:56:32.332 INFO:tasks.workunit.client.0.smithi064.stdout:5/637: mknod d6/d25/cdd 0 2024-02-16T22:56:32.333 INFO:tasks.workunit.client.0.smithi064.stdout:1/612: dread - d2/df/d11/d1a/d30/dbd/d91/f65 zero size 2024-02-16T22:56:32.334 INFO:tasks.workunit.client.0.smithi064.stdout:8/635: mknod df/d31/d2f/d3e/d5f/d93/dd2/cda 0 2024-02-16T22:56:32.335 INFO:tasks.workunit.client.0.smithi064.stdout:5/638: write d6/d25/d48/d67/d80/fdc [384357,66991] 0 2024-02-16T22:56:32.335 INFO:tasks.workunit.client.1.smithi192.stdout:2/853: truncate d22/d24/d39/d3d/df2/f8a 903901 0 2024-02-16T22:56:32.337 INFO:tasks.workunit.client.0.smithi064.stdout:5/639: chown d6/d25/da4/dc1/fa1 52927915 1 2024-02-16T22:56:32.337 INFO:tasks.workunit.client.1.smithi192.stdout:8/804: truncate d5/d26/d83/d86/f6e 693683 0 2024-02-16T22:56:32.339 INFO:tasks.workunit.client.1.smithi192.stdout:8/805: write d5/d26/d83/d86/d5f/dbf/fc2 [1499864,117772] 0 2024-02-16T22:56:32.340 INFO:tasks.workunit.client.0.smithi064.stdout:4/699: dread dc/d1d/f8b [0,4194304] 0 2024-02-16T22:56:32.340 INFO:tasks.workunit.client.0.smithi064.stdout:8/636: read df/d31/d2f/d38/d48/f49 [1291543,79726] 0 2024-02-16T22:56:32.342 INFO:tasks.workunit.client.0.smithi064.stdout:5/640: dread d6/f24 [0,4194304] 0 2024-02-16T22:56:32.342 INFO:tasks.workunit.client.0.smithi064.stdout:8/637: dread - df/d31/d8f/dab/fcd zero size 2024-02-16T22:56:32.345 INFO:tasks.workunit.client.1.smithi192.stdout:8/806: dwrite d5/d9/fc9 [0,4194304] 0 2024-02-16T22:56:32.346 INFO:tasks.workunit.client.1.smithi192.stdout:1/771: dread d3/de/f1a [0,4194304] 0 2024-02-16T22:56:32.349 INFO:tasks.workunit.client.0.smithi064.stdout:8/638: dwrite df/d31/d2f/d4f/f75 [0,4194304] 0 2024-02-16T22:56:32.350 INFO:tasks.workunit.client.0.smithi064.stdout:8/639: dread - df/d31/d2f/d7f/fd0 zero size 2024-02-16T22:56:32.359 INFO:tasks.workunit.client.0.smithi064.stdout:5/641: dread d6/f34 [0,4194304] 0 2024-02-16T22:56:32.363 INFO:tasks.workunit.client.0.smithi064.stdout:2/813: dread d7/d14/d1e/d34/d3f/d9a/dd9/f11 [0,4194304] 0 2024-02-16T22:56:32.366 INFO:tasks.workunit.client.0.smithi064.stdout:5/642: dwrite d6/d1c/d43/db9/fd8 [0,4194304] 0 2024-02-16T22:56:32.368 INFO:tasks.workunit.client.1.smithi192.stdout:2/854: dread f2 [0,4194304] 0 2024-02-16T22:56:32.390 INFO:tasks.workunit.client.0.smithi064.stdout:5/643: dread d6/d1c/d43/f4c [0,4194304] 0 2024-02-16T22:56:32.391 INFO:tasks.workunit.client.0.smithi064.stdout:5/644: dread - d6/d25/da4/dc1/d85/f9f zero size 2024-02-16T22:56:32.412 INFO:tasks.workunit.client.1.smithi192.stdout:0/799: sync 2024-02-16T22:56:32.412 INFO:tasks.workunit.client.1.smithi192.stdout:3/744: sync 2024-02-16T22:56:32.412 INFO:tasks.workunit.client.0.smithi064.stdout:1/613: fdatasync d2/df/d11/d1a/d30/dbd/d91/f64 0 2024-02-16T22:56:32.414 INFO:tasks.workunit.client.0.smithi064.stdout:1/614: chown d2/df/d11/d1a/d1d/f24 68372 1 2024-02-16T22:56:32.414 INFO:tasks.workunit.client.0.smithi064.stdout:4/700: dread - dc/dd/d16/d7c/f86 zero size 2024-02-16T22:56:32.415 INFO:tasks.workunit.client.0.smithi064.stdout:1/615: write d2/df/d14/d18/d48/d6b/fd0 [587881,3315] 0 2024-02-16T22:56:32.416 INFO:tasks.workunit.client.1.smithi192.stdout:8/807: mknod d5/db/d47/d43/d90/cfc 0 2024-02-16T22:56:32.418 INFO:tasks.workunit.client.1.smithi192.stdout:1/772: creat d3/de/d27/d34/d52/d72/f107 x:0 0 0 2024-02-16T22:56:32.420 INFO:tasks.workunit.client.0.smithi064.stdout:3/791: write d4/d13/d1b/d4f/f6f [709559,11035] 0 2024-02-16T22:56:32.423 INFO:tasks.workunit.client.0.smithi064.stdout:2/814: mknod d7/d14/d1e/d34/d3f/c107 0 2024-02-16T22:56:32.424 INFO:tasks.workunit.client.1.smithi192.stdout:3/745: read da/d11/d63/d6b/d66/f92 [3421271,14524] 0 2024-02-16T22:56:32.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/800: truncate d1/d3/de/d22/d37/d47/fa5 351945 0 2024-02-16T22:56:32.425 INFO:tasks.workunit.client.1.smithi192.stdout:1/773: dwrite d3/de/d27/f39 [4194304,4194304] 0 2024-02-16T22:56:32.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/801: chown d1/d3/de/d22/d37/d47/d53/ff7 119669943 1 2024-02-16T22:56:32.427 INFO:tasks.workunit.client.1.smithi192.stdout:8/808: creat d5/d16/ffd x:0 0 0 2024-02-16T22:56:32.427 INFO:tasks.workunit.client.1.smithi192.stdout:8/809: stat d5/d9/f73 0 2024-02-16T22:56:32.429 INFO:tasks.workunit.client.0.smithi064.stdout:5/645: fdatasync d6/d25/d48/d67/d91/da5/fcb 0 2024-02-16T22:56:32.469 INFO:tasks.workunit.client.1.smithi192.stdout:4/861: write d1/d26/d34/f76 [948231,22167] 0 2024-02-16T22:56:32.518 INFO:tasks.workunit.client.0.smithi064.stdout:9/653: truncate d3/d11/d15/f72 3889904 0 2024-02-16T22:56:32.524 INFO:tasks.workunit.client.0.smithi064.stdout:2/815: dread d7/d14/d25/d46/d55/d7f/fe7 [0,4194304] 0 2024-02-16T22:56:32.527 INFO:tasks.workunit.client.1.smithi192.stdout:4/862: mkdir d1/d15/d48/dc3/d115/d11d/d12b 0 2024-02-16T22:56:32.527 INFO:tasks.workunit.client.1.smithi192.stdout:4/863: chown d1/d15/l65 28663 1 2024-02-16T22:56:32.533 INFO:tasks.workunit.client.1.smithi192.stdout:2/855: creat d22/d47/dcb/f117 x:0 0 0 2024-02-16T22:56:32.534 INFO:tasks.workunit.client.1.smithi192.stdout:4/864: chown d1/d26/d66/d89/da8/fff 861532709 1 2024-02-16T22:56:32.536 INFO:tasks.workunit.client.0.smithi064.stdout:3/792: dread d4/d1f/d5b/d51/d7c/f49 [0,4194304] 0 2024-02-16T22:56:32.538 INFO:tasks.workunit.client.1.smithi192.stdout:9/924: dwrite d7/d15/f33 [0,4194304] 0 2024-02-16T22:56:32.539 INFO:tasks.workunit.client.0.smithi064.stdout:9/654: mkdir d3/d11/d15/dab/db6/dd5 0 2024-02-16T22:56:32.539 INFO:tasks.workunit.client.0.smithi064.stdout:2/816: dread d7/d14/d4b/f88 [0,4194304] 0 2024-02-16T22:56:32.541 INFO:tasks.workunit.client.1.smithi192.stdout:1/774: mknod d3/c108 0 2024-02-16T22:56:32.541 INFO:tasks.workunit.client.1.smithi192.stdout:1/775: readlink d3/l33 0 2024-02-16T22:56:32.547 INFO:tasks.workunit.client.1.smithi192.stdout:6/832: truncate da/df/d12/d57/f2f 2190437 0 2024-02-16T22:56:32.548 INFO:tasks.workunit.client.1.smithi192.stdout:0/802: mkdir d1/d3/de2/dff 0 2024-02-16T22:56:32.549 INFO:tasks.workunit.client.0.smithi064.stdout:9/655: dwrite d3/f8a [0,4194304] 0 2024-02-16T22:56:32.552 INFO:tasks.workunit.client.0.smithi064.stdout:4/701: link dc/dd/d32/d22/cb1 dc/dd/d32/d4b/cda 0 2024-02-16T22:56:32.557 INFO:tasks.workunit.client.1.smithi192.stdout:7/923: write d1/d2/de/d2c/d49/da4/d104/f114 [304925,42329] 0 2024-02-16T22:56:32.561 INFO:tasks.workunit.client.0.smithi064.stdout:3/793: dread d4/d13/d1b/fcb [0,4194304] 0 2024-02-16T22:56:32.567 INFO:tasks.workunit.client.0.smithi064.stdout:5/646: creat d6/d18/fde x:0 0 0 2024-02-16T22:56:32.571 INFO:tasks.workunit.client.0.smithi064.stdout:0/598: sync 2024-02-16T22:56:32.571 INFO:tasks.workunit.client.0.smithi064.stdout:7/678: sync 2024-02-16T22:56:32.581 INFO:tasks.workunit.client.0.smithi064.stdout:9/656: creat d3/d11/d9e/da5/fd6 x:0 0 0 2024-02-16T22:56:32.587 INFO:tasks.workunit.client.0.smithi064.stdout:0/599: rename da/dc/de/d12/d2c/d2e/d62/db6/fb8 to da/d71/da4/fca 0 2024-02-16T22:56:32.590 INFO:tasks.workunit.client.0.smithi064.stdout:4/702: mknod dc/dd/d32/d46/d5a/dcb/cdb 0 2024-02-16T22:56:32.592 INFO:tasks.workunit.client.0.smithi064.stdout:0/600: dwrite da/d26/f82 [0,4194304] 0 2024-02-16T22:56:32.594 INFO:tasks.workunit.client.0.smithi064.stdout:0/601: dread - da/d71/da4/fca zero size 2024-02-16T22:56:32.595 INFO:tasks.workunit.client.0.smithi064.stdout:0/602: chown da/dc/de/d12/d1d/c5e 3663469 1 2024-02-16T22:56:32.596 INFO:tasks.workunit.client.0.smithi064.stdout:0/603: dread - da/dc/de/d37/fa6 zero size 2024-02-16T22:56:32.615 INFO:tasks.workunit.client.0.smithi064.stdout:0/604: creat da/dc/d3b/d72/d7c/fcb x:0 0 0 2024-02-16T22:56:32.616 INFO:tasks.workunit.client.0.smithi064.stdout:0/605: write da/f1b [1367487,47300] 0 2024-02-16T22:56:32.631 INFO:tasks.workunit.client.1.smithi192.stdout:4/865: rmdir d1/d15/d48 39 2024-02-16T22:56:32.633 INFO:tasks.workunit.client.1.smithi192.stdout:2/856: rename d22/d24/l29 to d22/d24/d39/d3d/df2/d99/l118 0 2024-02-16T22:56:32.637 INFO:tasks.workunit.client.1.smithi192.stdout:8/810: link d5/d16/c6d d5/d16/d81/cfe 0 2024-02-16T22:56:32.638 INFO:tasks.workunit.client.1.smithi192.stdout:7/924: unlink d1/d2/d2d/d61/d88/d90/db2/ld5 0 2024-02-16T22:56:32.639 INFO:tasks.workunit.client.1.smithi192.stdout:7/925: chown d1/de1/c2b 3833312 1 2024-02-16T22:56:32.640 INFO:tasks.workunit.client.1.smithi192.stdout:5/812: dwrite d6/d12/d1d/ff5 [0,4194304] 0 2024-02-16T22:56:32.641 INFO:tasks.workunit.client.1.smithi192.stdout:4/866: stat d1/d15/d48/dc3/cd3 0 2024-02-16T22:56:32.645 INFO:tasks.workunit.client.1.smithi192.stdout:9/925: sync 2024-02-16T22:56:32.645 INFO:tasks.workunit.client.1.smithi192.stdout:1/776: sync 2024-02-16T22:56:32.650 INFO:tasks.workunit.client.1.smithi192.stdout:5/813: chown d6/d12/d1d/d56/d108/f109 2931265 1 2024-02-16T22:56:32.651 INFO:tasks.workunit.client.1.smithi192.stdout:5/814: truncate d6/d12/d1d/d56/d108/f109 153378 0 2024-02-16T22:56:32.654 INFO:tasks.workunit.client.1.smithi192.stdout:2/857: mknod d22/d47/dcb/c119 0 2024-02-16T22:56:32.659 INFO:tasks.workunit.client.1.smithi192.stdout:4/867: dread d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:32.660 INFO:tasks.workunit.client.1.smithi192.stdout:2/858: symlink d22/l11a 0 2024-02-16T22:56:32.663 INFO:tasks.workunit.client.1.smithi192.stdout:7/926: link d1/d2/d37/d86/lc6 d1/d2/d2d/d61/d8a/l13a 0 2024-02-16T22:56:32.665 INFO:tasks.workunit.client.1.smithi192.stdout:4/868: write d1/d26/d5e/f63 [1138848,111547] 0 2024-02-16T22:56:32.685 INFO:tasks.workunit.client.0.smithi064.stdout:8/640: mkdir df/d2a/ddb 0 2024-02-16T22:56:32.687 INFO:tasks.workunit.client.0.smithi064.stdout:8/641: symlink df/d31/d2f/d68/ldc 0 2024-02-16T22:56:32.689 INFO:tasks.workunit.client.0.smithi064.stdout:8/642: creat df/d31/dc1/fdd x:0 0 0 2024-02-16T22:56:32.694 INFO:tasks.workunit.client.0.smithi064.stdout:8/643: link df/d31/d2f/d4f/c9e df/d2a/d5c/cde 0 2024-02-16T22:56:32.695 INFO:tasks.workunit.client.0.smithi064.stdout:8/644: write df/d31/d8f/dab/dae/fb2 [1108284,53684] 0 2024-02-16T22:56:32.699 INFO:tasks.workunit.client.0.smithi064.stdout:8/645: mkdir df/d31/d2f/d3e/ddf 0 2024-02-16T22:56:32.703 INFO:tasks.workunit.client.1.smithi192.stdout:3/746: write da/d11/d63/d6b/d66/d75/f9f [359636,26459] 0 2024-02-16T22:56:32.721 INFO:tasks.workunit.client.1.smithi192.stdout:7/927: creat d1/d2/de/d2c/d49/d54/dd8/d117/f13b x:0 0 0 2024-02-16T22:56:32.723 INFO:tasks.workunit.client.0.smithi064.stdout:2/817: dwrite d7/d14/d25/f27 [0,4194304] 0 2024-02-16T22:56:32.725 INFO:tasks.workunit.client.0.smithi064.stdout:2/818: stat d7/d14/d1e/d34/d3f/d9a/dd9/c2a 0 2024-02-16T22:56:32.746 INFO:tasks.workunit.client.0.smithi064.stdout:1/616: sync 2024-02-16T22:56:32.746 INFO:tasks.workunit.client.0.smithi064.stdout:6/751: sync 2024-02-16T22:56:32.746 INFO:tasks.workunit.client.0.smithi064.stdout:3/794: truncate d4/d1f/d5b/d51/f6a 2730097 0 2024-02-16T22:56:32.748 INFO:tasks.workunit.client.0.smithi064.stdout:1/617: read d2/df/d11/d1a/f34 [1103693,62717] 0 2024-02-16T22:56:32.750 INFO:tasks.workunit.client.0.smithi064.stdout:3/795: creat d4/d13/d4d/d5c/f10c x:0 0 0 2024-02-16T22:56:32.753 INFO:tasks.workunit.client.0.smithi064.stdout:1/618: rmdir d2/df/d14/d2c/db5 39 2024-02-16T22:56:32.755 INFO:tasks.workunit.client.0.smithi064.stdout:6/752: creat d0/d1d/da8/d60/dbf/ff9 x:0 0 0 2024-02-16T22:56:32.755 INFO:tasks.workunit.client.0.smithi064.stdout:7/679: dwrite d5/d6/f5b [0,4194304] 0 2024-02-16T22:56:32.761 INFO:tasks.workunit.client.0.smithi064.stdout:6/753: mkdir d0/d1d/dfa 0 2024-02-16T22:56:32.761 INFO:tasks.workunit.client.0.smithi064.stdout:6/754: chown d0/d6/da/d56 2449 1 2024-02-16T22:56:32.762 INFO:tasks.workunit.client.0.smithi064.stdout:6/755: stat d0/d89/f8e 0 2024-02-16T22:56:32.764 INFO:tasks.workunit.client.0.smithi064.stdout:3/796: getdents d4/d13/d41 0 2024-02-16T22:56:32.765 INFO:tasks.workunit.client.0.smithi064.stdout:6/756: unlink d0/d6/da/l2e 0 2024-02-16T22:56:32.767 INFO:tasks.workunit.client.0.smithi064.stdout:3/797: creat d4/d13/d1b/d27/d81/f10d x:0 0 0 2024-02-16T22:56:32.768 INFO:tasks.workunit.client.0.smithi064.stdout:6/757: mkdir d0/d6/da/da2/dfb 0 2024-02-16T22:56:32.769 INFO:tasks.workunit.client.0.smithi064.stdout:3/798: truncate d4/d13/d1b/f1c 223458 0 2024-02-16T22:56:32.770 INFO:tasks.workunit.client.0.smithi064.stdout:6/758: creat d0/d6/d10/db2/ffc x:0 0 0 2024-02-16T22:56:32.774 INFO:tasks.workunit.client.0.smithi064.stdout:6/759: unlink d0/d5/fc7 0 2024-02-16T22:56:32.775 INFO:tasks.workunit.client.0.smithi064.stdout:6/760: dread d0/d3c/fb7 [0,4194304] 0 2024-02-16T22:56:32.776 INFO:tasks.workunit.client.0.smithi064.stdout:6/761: dread - d0/d5/df/d17/d18/fcf zero size 2024-02-16T22:56:32.782 INFO:tasks.workunit.client.0.smithi064.stdout:4/703: dwrite dc/d1d/dbd/fca [0,4194304] 0 2024-02-16T22:56:32.784 INFO:tasks.workunit.client.0.smithi064.stdout:9/657: sync 2024-02-16T22:56:32.788 INFO:tasks.workunit.client.1.smithi192.stdout:0/803: write d1/d3/de/d22/d37/d47/f93 [495191,5599] 0 2024-02-16T22:56:32.789 INFO:tasks.workunit.client.0.smithi064.stdout:6/762: dwrite d0/d5/df/f86 [0,4194304] 0 2024-02-16T22:56:32.791 INFO:tasks.workunit.client.0.smithi064.stdout:0/606: dwrite da/d26/f28 [0,4194304] 0 2024-02-16T22:56:32.793 INFO:tasks.workunit.client.0.smithi064.stdout:9/658: creat d3/d9/d1e/d42/d7b/d86/fd7 x:0 0 0 2024-02-16T22:56:32.794 INFO:tasks.workunit.client.1.smithi192.stdout:2/859: fsync d22/d24/d39/d42/f54 0 2024-02-16T22:56:32.795 INFO:tasks.workunit.client.0.smithi064.stdout:6/763: read d0/d3c/da9/f67 [833925,26388] 0 2024-02-16T22:56:32.796 INFO:tasks.workunit.client.0.smithi064.stdout:6/764: write d0/d3c/fef [540575,22670] 0 2024-02-16T22:56:32.796 INFO:tasks.workunit.client.0.smithi064.stdout:0/607: creat da/dc/de/d12/d2c/d2e/d38/d87/fcc x:0 0 0 2024-02-16T22:56:32.801 INFO:tasks.workunit.client.0.smithi064.stdout:5/647: unlink d6/d25/d48/d67/fb2 0 2024-02-16T22:56:32.802 INFO:tasks.workunit.client.0.smithi064.stdout:6/765: dread d0/d3c/fdf [0,4194304] 0 2024-02-16T22:56:32.804 INFO:tasks.workunit.client.1.smithi192.stdout:2/860: chown d22/d24/d39/c114 578 1 2024-02-16T22:56:32.804 INFO:tasks.workunit.client.0.smithi064.stdout:9/659: fdatasync d3/d9/f36 0 2024-02-16T22:56:32.806 INFO:tasks.workunit.client.0.smithi064.stdout:0/608: mkdir da/dc/de/d37/daa/dcd 0 2024-02-16T22:56:32.809 INFO:tasks.workunit.client.0.smithi064.stdout:5/648: mknod d6/d1c/d43/d7d/cdf 0 2024-02-16T22:56:32.810 INFO:tasks.workunit.client.1.smithi192.stdout:2/861: mkdir d22/d24/d39/d42/d84/d11b 0 2024-02-16T22:56:32.810 INFO:tasks.workunit.client.0.smithi064.stdout:5/649: chown d6/d1c/d43/d7d/c97 7856 1 2024-02-16T22:56:32.811 INFO:tasks.workunit.client.0.smithi064.stdout:6/766: dwrite d0/d6/d10/db2/ff8 [0,4194304] 0 2024-02-16T22:56:32.814 INFO:tasks.workunit.client.0.smithi064.stdout:9/660: creat d3/d89/fd8 x:0 0 0 2024-02-16T22:56:32.816 INFO:tasks.workunit.client.1.smithi192.stdout:9/926: write d7/d15/d36/d77/db1/ff8 [187998,78605] 0 2024-02-16T22:56:32.818 INFO:tasks.workunit.client.1.smithi192.stdout:9/927: readlink d7/d15/d36/d38/d3f/d8a/d98/l1f 0 2024-02-16T22:56:32.819 INFO:tasks.workunit.client.0.smithi064.stdout:5/650: dwrite d6/d25/da4/dc1/d85/f8b [0,4194304] 0 2024-02-16T22:56:32.820 INFO:tasks.workunit.client.1.smithi192.stdout:2/862: dread d22/d24/d39/d3d/fbb [0,4194304] 0 2024-02-16T22:56:32.823 INFO:tasks.workunit.client.1.smithi192.stdout:1/777: write d3/fb [4412263,90756] 0 2024-02-16T22:56:32.824 INFO:tasks.workunit.client.0.smithi064.stdout:5/651: creat d6/d25/d48/d67/d80/fe0 x:0 0 0 2024-02-16T22:56:32.826 INFO:tasks.workunit.client.0.smithi064.stdout:5/652: truncate d6/d25/d48/d67/f7f 1205483 0 2024-02-16T22:56:32.827 INFO:tasks.workunit.client.1.smithi192.stdout:2/863: read - d22/d24/d39/fe7 zero size 2024-02-16T22:56:32.828 INFO:tasks.workunit.client.0.smithi064.stdout:5/653: rmdir d6 39 2024-02-16T22:56:32.829 INFO:tasks.workunit.client.1.smithi192.stdout:5/815: dwrite d6/f10 [0,4194304] 0 2024-02-16T22:56:32.829 INFO:tasks.workunit.client.0.smithi064.stdout:8/646: rename df/d31/fb6 to df/d31/d2f/d3e/d5f/d93/fe0 0 2024-02-16T22:56:32.830 INFO:tasks.workunit.client.0.smithi064.stdout:9/661: dread d3/d9/d1e/d42/f9f [0,4194304] 0 2024-02-16T22:56:32.830 INFO:tasks.workunit.client.1.smithi192.stdout:9/928: link d7/d19/d5e/d6f/c13b d7/d15/d36/d77/db1/d120/c13c 0 2024-02-16T22:56:32.831 INFO:tasks.workunit.client.1.smithi192.stdout:9/929: chown l3 148 1 2024-02-16T22:56:32.833 INFO:tasks.workunit.client.0.smithi064.stdout:9/662: chown d3/d9/d6a/db1/d2a/c32 68288466 1 2024-02-16T22:56:32.836 INFO:tasks.workunit.client.0.smithi064.stdout:7/680: rename d5/d33/d45/d73/d88/f90 to d5/d33/d45/d73/dab/fd5 0 2024-02-16T22:56:32.837 INFO:tasks.workunit.client.0.smithi064.stdout:7/681: write d5/d33/d45/fb5 [56890,53922] 0 2024-02-16T22:56:32.840 INFO:tasks.workunit.client.0.smithi064.stdout:7/682: dread - d5/d33/d45/d73/dab/fbe zero size 2024-02-16T22:56:32.840 INFO:tasks.workunit.client.0.smithi064.stdout:8/647: mknod df/d31/d2f/d3e/d5f/d93/dd2/ce1 0 2024-02-16T22:56:32.844 INFO:tasks.workunit.client.0.smithi064.stdout:6/767: rename d0/d12/d7b/fa7 to d0/d1d/d21/ffd 0 2024-02-16T22:56:32.845 INFO:tasks.workunit.client.0.smithi064.stdout:7/683: fdatasync d5/d6/f22 0 2024-02-16T22:56:32.847 INFO:tasks.workunit.client.0.smithi064.stdout:5/654: unlink d6/d1c/c8c 0 2024-02-16T22:56:32.849 INFO:tasks.workunit.client.0.smithi064.stdout:9/663: link d3/d9/d6a/db1/d2a/fc9 d3/d11/d9e/da9/fd9 0 2024-02-16T22:56:32.852 INFO:tasks.workunit.client.0.smithi064.stdout:9/664: write d3/f8a [195683,116467] 0 2024-02-16T22:56:32.852 INFO:tasks.workunit.client.0.smithi064.stdout:4/704: rmdir dc/de/d6d 39 2024-02-16T22:56:32.853 INFO:tasks.workunit.client.0.smithi064.stdout:0/609: dread da/dc/f1e [0,4194304] 0 2024-02-16T22:56:32.854 INFO:tasks.workunit.client.0.smithi064.stdout:4/705: readlink dc/de/d28/l50 0 2024-02-16T22:56:32.855 INFO:tasks.workunit.client.0.smithi064.stdout:5/655: dwrite d6/d1c/fd7 [0,4194304] 0 2024-02-16T22:56:32.856 INFO:tasks.workunit.client.1.smithi192.stdout:2/864: dread d22/d24/f96 [0,4194304] 0 2024-02-16T22:56:32.857 INFO:tasks.workunit.client.1.smithi192.stdout:2/865: chown d22/d24/d39/l9a 18 1 2024-02-16T22:56:32.857 INFO:tasks.workunit.client.0.smithi064.stdout:9/665: chown d3/d9/d6a/db1/d2a/d2e/d31/d3f/faa 532 1 2024-02-16T22:56:32.857 INFO:tasks.workunit.client.0.smithi064.stdout:7/684: mkdir d5/d33/d42/dd6 0 2024-02-16T22:56:32.861 INFO:tasks.workunit.client.0.smithi064.stdout:6/768: creat d0/d1d/dbc/dec/ffe x:0 0 0 2024-02-16T22:56:32.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/866: symlink d22/d24/d39/l11c 0 2024-02-16T22:56:32.864 INFO:tasks.workunit.client.0.smithi064.stdout:5/656: dwrite d6/d25/d48/d67/d80/fe0 [0,4194304] 0 2024-02-16T22:56:32.866 INFO:tasks.workunit.client.0.smithi064.stdout:4/706: mkdir dc/ddc 0 2024-02-16T22:56:32.868 INFO:tasks.workunit.client.1.smithi192.stdout:2/867: mkdir d22/d47/dca/d11d 0 2024-02-16T22:56:32.869 INFO:tasks.workunit.client.1.smithi192.stdout:2/868: chown d22/d24/d39/d3d/df2/d99/d9b/l10a 697 1 2024-02-16T22:56:32.869 INFO:tasks.workunit.client.0.smithi064.stdout:7/685: mkdir d5/d33/d45/d73/dd7 0 2024-02-16T22:56:32.873 INFO:tasks.workunit.client.0.smithi064.stdout:6/769: mknod d0/d6/da/da2/cff 0 2024-02-16T22:56:32.878 INFO:tasks.workunit.client.0.smithi064.stdout:7/686: dwrite d5/d6/d54/d7b/fb1 [0,4194304] 0 2024-02-16T22:56:32.879 INFO:tasks.workunit.client.0.smithi064.stdout:9/666: truncate d3/d9/d6a/db1/f56 3908475 0 2024-02-16T22:56:32.880 INFO:tasks.workunit.client.0.smithi064.stdout:9/667: truncate d3/d9/ff 5347622 0 2024-02-16T22:56:32.885 INFO:tasks.workunit.client.0.smithi064.stdout:5/657: getdents d6/d18/d95/dbe 0 2024-02-16T22:56:32.889 INFO:tasks.workunit.client.0.smithi064.stdout:7/687: mknod d5/d33/d42/d81/d82/cd8 0 2024-02-16T22:56:32.890 INFO:tasks.workunit.client.0.smithi064.stdout:9/668: creat d3/d9/d1e/d3e/d76/fda x:0 0 0 2024-02-16T22:56:32.894 INFO:tasks.workunit.client.0.smithi064.stdout:9/669: creat d3/d9/d6a/db1/d2a/d2e/d31/d62/fdb x:0 0 0 2024-02-16T22:56:32.895 INFO:tasks.workunit.client.1.smithi192.stdout:7/928: write d1/de1/fb8 [1660844,70268] 0 2024-02-16T22:56:32.896 INFO:tasks.workunit.client.0.smithi064.stdout:5/658: creat d6/d18/d95/dbe/fe1 x:0 0 0 2024-02-16T22:56:32.897 INFO:tasks.workunit.client.1.smithi192.stdout:5/816: sync 2024-02-16T22:56:32.898 INFO:tasks.workunit.client.1.smithi192.stdout:5/817: write d6/d12/d2d/fc0 [702050,113421] 0 2024-02-16T22:56:32.898 INFO:tasks.workunit.client.1.smithi192.stdout:7/929: unlink d1/d2/d2d/d61/d8a/dbb/fc9 0 2024-02-16T22:56:32.899 INFO:tasks.workunit.client.0.smithi064.stdout:2/819: write d7/d14/d1e/d34/d3f/d9a/dd9/d95/f3a [512655,56753] 0 2024-02-16T22:56:32.901 INFO:tasks.workunit.client.0.smithi064.stdout:7/688: creat d5/d33/fd9 x:0 0 0 2024-02-16T22:56:32.902 INFO:tasks.workunit.client.0.smithi064.stdout:5/659: mkdir d6/d1c/d43/d7d/de2 0 2024-02-16T22:56:32.902 INFO:tasks.workunit.client.0.smithi064.stdout:9/670: dwrite d3/d9/d1e/d42/d7b/fa8 [0,4194304] 0 2024-02-16T22:56:32.903 INFO:tasks.workunit.client.0.smithi064.stdout:7/689: stat d5/d33/d45/d64/d56/d66 0 2024-02-16T22:56:32.904 INFO:tasks.workunit.client.0.smithi064.stdout:9/671: read - d3/d9/d1e/d42/fb8 zero size 2024-02-16T22:56:32.906 INFO:tasks.workunit.client.1.smithi192.stdout:7/930: unlink d1/d2/de/d83/d9f/feb 0 2024-02-16T22:56:32.908 INFO:tasks.workunit.client.1.smithi192.stdout:5/818: unlink d6/d8/d75/c9a 0 2024-02-16T22:56:32.909 INFO:tasks.workunit.client.0.smithi064.stdout:5/660: dwrite d6/d1c/f53 [4194304,4194304] 0 2024-02-16T22:56:32.910 INFO:tasks.workunit.client.0.smithi064.stdout:2/820: truncate d7/d14/d1e/d34/d3f/d9a/dd9/f56 1248355 0 2024-02-16T22:56:32.911 INFO:tasks.workunit.client.1.smithi192.stdout:7/931: creat d1/d2/d2d/d61/d8a/dbb/f13c x:0 0 0 2024-02-16T22:56:32.912 INFO:tasks.workunit.client.1.smithi192.stdout:7/932: write d1/d2/de/d83/f130 [278842,40151] 0 2024-02-16T22:56:32.913 INFO:tasks.workunit.client.1.smithi192.stdout:5/819: truncate d6/d12/d4e/d58/fbb 4955121 0 2024-02-16T22:56:32.915 INFO:tasks.workunit.client.0.smithi064.stdout:9/672: mknod d3/d11/d9e/da5/cdc 0 2024-02-16T22:56:32.916 INFO:tasks.workunit.client.0.smithi064.stdout:2/821: dread d7/d14/d25/f8a [0,4194304] 0 2024-02-16T22:56:32.919 INFO:tasks.workunit.client.1.smithi192.stdout:5/820: truncate d6/d12/d2d/d5c/df6/dbc/fd8 460705 0 2024-02-16T22:56:32.919 INFO:tasks.workunit.client.0.smithi064.stdout:5/661: dread - d6/d1c/d43/d7d/fa2 zero size 2024-02-16T22:56:32.920 INFO:tasks.workunit.client.0.smithi064.stdout:5/662: chown d6/d18/d95/dbe/fe1 46 1 2024-02-16T22:56:32.924 INFO:tasks.workunit.client.0.smithi064.stdout:9/673: mkdir d3/d11/d15/d23/ddd 0 2024-02-16T22:56:32.925 INFO:tasks.workunit.client.1.smithi192.stdout:5/821: mkdir d6/d12/d4e/d10c 0 2024-02-16T22:56:32.926 INFO:tasks.workunit.client.0.smithi064.stdout:9/674: write d3/d9/d1e/d42/d7b/d86/fd7 [161465,63226] 0 2024-02-16T22:56:32.928 INFO:tasks.workunit.client.1.smithi192.stdout:7/933: getdents d1/d2/de/d2c/d49/d89/dbd 0 2024-02-16T22:56:32.928 INFO:tasks.workunit.client.0.smithi064.stdout:1/619: write d2/df/d14/d2a/fbf [506730,88688] 0 2024-02-16T22:56:32.928 INFO:tasks.workunit.client.1.smithi192.stdout:7/934: readlink d1/d2/d2d/d61/d8a/la6 0 2024-02-16T22:56:32.932 INFO:tasks.workunit.client.1.smithi192.stdout:5/822: dwrite d6/d12/d4e/d5b/fed [0,4194304] 0 2024-02-16T22:56:32.935 INFO:tasks.workunit.client.0.smithi064.stdout:7/690: rmdir d5/d33/d42/dd6 0 2024-02-16T22:56:32.935 INFO:tasks.workunit.client.0.smithi064.stdout:1/620: dwrite d2/df/d14/f2d [0,4194304] 0 2024-02-16T22:56:32.937 INFO:tasks.workunit.client.0.smithi064.stdout:9/675: symlink d3/d9/d6a/db1/d2a/d2e/d31/d62/lde 0 2024-02-16T22:56:32.947 INFO:tasks.workunit.client.1.smithi192.stdout:6/833: rename da/la1 to da/df/d12/d15/l10b 0 2024-02-16T22:56:32.948 INFO:tasks.workunit.client.0.smithi064.stdout:3/799: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:56:32.951 INFO:tasks.workunit.client.1.smithi192.stdout:8/811: rename d5/d16/dd5/ff1 to d5/db/dcb/fff 0 2024-02-16T22:56:32.954 INFO:tasks.workunit.client.1.smithi192.stdout:6/834: dwrite da/df/d12/f90 [0,4194304] 0 2024-02-16T22:56:32.956 INFO:tasks.workunit.client.1.smithi192.stdout:6/835: write da/df/d12/d15/f31 [63379,31076] 0 2024-02-16T22:56:32.959 INFO:tasks.workunit.client.1.smithi192.stdout:4/869: rename d1/d15/c9f to d1/d6b/c12c 0 2024-02-16T22:56:32.960 INFO:tasks.workunit.client.0.smithi064.stdout:1/621: dread d2/df/d14/d18/f26 [0,4194304] 0 2024-02-16T22:56:32.964 INFO:tasks.workunit.client.0.smithi064.stdout:3/800: getdents d4/d1f/d5b/d51/dc1 0 2024-02-16T22:56:32.965 INFO:tasks.workunit.client.1.smithi192.stdout:8/812: mknod d5/db/c100 0 2024-02-16T22:56:32.966 INFO:tasks.workunit.client.0.smithi064.stdout:3/801: write d4/d13/d1b/d27/d44/d97/fdc [4707482,59821] 0 2024-02-16T22:56:32.966 INFO:tasks.workunit.client.0.smithi064.stdout:1/622: creat d2/df/d14/d18/d48/d6b/fd5 x:0 0 0 2024-02-16T22:56:32.969 INFO:tasks.workunit.client.1.smithi192.stdout:4/870: truncate d1/d26/d34/f50 2461890 0 2024-02-16T22:56:32.971 INFO:tasks.workunit.client.1.smithi192.stdout:6/836: symlink da/l10c 0 2024-02-16T22:56:32.973 INFO:tasks.workunit.client.1.smithi192.stdout:8/813: chown d5/db/d47/d43/d67 499093 1 2024-02-16T22:56:32.979 INFO:tasks.workunit.client.1.smithi192.stdout:9/930: write d7/d19/d28/f8b [5366122,108000] 0 2024-02-16T22:56:32.980 INFO:tasks.workunit.client.1.smithi192.stdout:8/814: fsync d5/d16/f18 0 2024-02-16T22:56:32.980 INFO:tasks.workunit.client.1.smithi192.stdout:8/815: fsync d5/d16/ffd 0 2024-02-16T22:56:32.983 INFO:tasks.workunit.client.0.smithi064.stdout:1/623: getdents d2/df/d14/d18 0 2024-02-16T22:56:32.984 INFO:tasks.workunit.client.1.smithi192.stdout:4/871: creat d1/d26/d34/db9/d110/f12d x:0 0 0 2024-02-16T22:56:32.985 INFO:tasks.workunit.client.1.smithi192.stdout:9/931: mknod d7/d15/d36/d38/d3f/c13d 0 2024-02-16T22:56:32.989 INFO:tasks.workunit.client.1.smithi192.stdout:3/747: rename da/d11/d2b/l65 to da/lfe 0 2024-02-16T22:56:32.990 INFO:tasks.workunit.client.1.smithi192.stdout:3/748: stat da/d11/d63/d6b/d66/d75/l8e 0 2024-02-16T22:56:32.990 INFO:tasks.workunit.client.0.smithi064.stdout:1/624: fsync d2/df/d14/d2c/f49 0 2024-02-16T22:56:32.991 INFO:tasks.workunit.client.1.smithi192.stdout:3/749: readlink da/d11/d63/d6b/d66/d75/l8e 0 2024-02-16T22:56:32.993 INFO:tasks.workunit.client.1.smithi192.stdout:6/837: mknod da/df/d18/d52/da9/dd8/c10d 0 2024-02-16T22:56:32.997 INFO:tasks.workunit.client.0.smithi064.stdout:1/625: dwrite d2/df/d14/d18/d48/d6b/fd0 [0,4194304] 0 2024-02-16T22:56:32.998 INFO:tasks.workunit.client.1.smithi192.stdout:0/804: rename d1/d3/de/d22/d37/d47/fd3 to d1/d3/de/d22/d37/d47/dd0/f100 0 2024-02-16T22:56:33.005 INFO:tasks.workunit.client.0.smithi064.stdout:1/626: fdatasync d2/df/d14/f88 0 2024-02-16T22:56:33.005 INFO:tasks.workunit.client.1.smithi192.stdout:4/872: dread d1/d26/fd7 [0,4194304] 0 2024-02-16T22:56:33.007 INFO:tasks.workunit.client.1.smithi192.stdout:2/869: write d22/d24/d39/d3d/df2/f38 [3997763,101647] 0 2024-02-16T22:56:33.009 INFO:tasks.workunit.client.0.smithi064.stdout:0/610: truncate da/dc/de/d12/f61 2560841 0 2024-02-16T22:56:33.011 INFO:tasks.workunit.client.0.smithi064.stdout:1/627: dwrite d2/df/d14/fa7 [0,4194304] 0 2024-02-16T22:56:33.011 INFO:tasks.workunit.client.0.smithi064.stdout:6/770: write d0/d1d/da8/fac [3751070,115427] 0 2024-02-16T22:56:33.013 INFO:tasks.workunit.client.1.smithi192.stdout:0/805: dread - d1/dc7/d9c/f9d zero size 2024-02-16T22:56:33.014 INFO:tasks.workunit.client.1.smithi192.stdout:0/806: chown d1/d3/de/d22/d37/d67/c3a 2 1 2024-02-16T22:56:33.016 INFO:tasks.workunit.client.0.smithi064.stdout:4/707: write dc/d1d/f2b [1214299,41682] 0 2024-02-16T22:56:33.017 INFO:tasks.workunit.client.0.smithi064.stdout:4/708: stat l9 0 2024-02-16T22:56:33.020 INFO:tasks.workunit.client.1.smithi192.stdout:3/750: symlink da/d11/d20/d39/d53/dc7/lff 0 2024-02-16T22:56:33.024 INFO:tasks.workunit.client.0.smithi064.stdout:0/611: truncate da/dc/de/d12/d2c/d2e/d38/f67 1523242 0 2024-02-16T22:56:33.030 INFO:tasks.workunit.client.1.smithi192.stdout:2/870: rmdir d22/d24/d39/d42/d65/d69/db1 39 2024-02-16T22:56:33.034 INFO:tasks.workunit.client.0.smithi064.stdout:5/663: rename d6/d1c to d6/d25/d48/d67/d91/da5/de3 0 2024-02-16T22:56:33.034 INFO:tasks.workunit.client.1.smithi192.stdout:1/778: rename d3/de/d27/d34/l41 to d3/de/d27/d34/d69/l109 0 2024-02-16T22:56:33.044 INFO:tasks.workunit.client.0.smithi064.stdout:0/612: dread da/dc/de/d12/f29 [0,4194304] 0 2024-02-16T22:56:33.046 INFO:tasks.workunit.client.1.smithi192.stdout:7/935: rename d1/d2/de/d2c/d49/da1 to d1/d2/de/d2c/d49/d54/d9d/ddd/d13d 0 2024-02-16T22:56:33.046 INFO:tasks.workunit.client.1.smithi192.stdout:7/936: readlink d1/d2/d2d/d61/d88/l123 0 2024-02-16T22:56:33.050 INFO:tasks.workunit.client.1.smithi192.stdout:3/751: fdatasync da/d6a/fd1 0 2024-02-16T22:56:33.053 INFO:tasks.workunit.client.0.smithi064.stdout:7/691: write d5/d7d/fa0 [442578,93055] 0 2024-02-16T22:56:33.054 INFO:tasks.workunit.client.1.smithi192.stdout:2/871: creat d22/d24/d39/d42/d85/d98/dcc/f11e x:0 0 0 2024-02-16T22:56:33.062 INFO:tasks.workunit.client.1.smithi192.stdout:7/937: dread d1/d69/f7f [0,4194304] 0 2024-02-16T22:56:33.062 INFO:tasks.workunit.client.1.smithi192.stdout:1/779: getdents d3/d40/d9f/df0 0 2024-02-16T22:56:33.064 INFO:tasks.workunit.client.0.smithi064.stdout:3/802: write d4/d13/d1b/d27/d44/f4b [1676413,56516] 0 2024-02-16T22:56:33.065 INFO:tasks.workunit.client.0.smithi064.stdout:3/803: chown d4/d13/l6d 93 1 2024-02-16T22:56:33.066 INFO:tasks.workunit.client.1.smithi192.stdout:9/932: sync 2024-02-16T22:56:33.069 INFO:tasks.workunit.client.1.smithi192.stdout:5/823: rename d6/d8/c63 to d6/d12/d4e/d10c/c10d 0 2024-02-16T22:56:33.069 INFO:tasks.workunit.client.1.smithi192.stdout:9/933: dread d7/d19/d28/d43/fee [0,4194304] 0 2024-02-16T22:56:33.070 INFO:tasks.workunit.client.1.smithi192.stdout:9/934: stat d7/d15/d36/d77/db1/cf9 0 2024-02-16T22:56:33.070 INFO:tasks.workunit.client.1.smithi192.stdout:9/935: chown d7/d15/c30 568 1 2024-02-16T22:56:33.071 INFO:tasks.workunit.client.1.smithi192.stdout:1/780: stat d3/de/d27/d34/d52/d63/cff 0 2024-02-16T22:56:33.072 INFO:tasks.workunit.client.1.smithi192.stdout:1/781: write d3/de/d27/f30 [5899817,94808] 0 2024-02-16T22:56:33.076 INFO:tasks.workunit.client.1.smithi192.stdout:8/816: write d5/f25 [1705514,69535] 0 2024-02-16T22:56:33.077 INFO:tasks.workunit.client.1.smithi192.stdout:8/817: truncate d5/db/dcb/fff 242199 0 2024-02-16T22:56:33.079 INFO:tasks.workunit.client.1.smithi192.stdout:6/838: write da/d40/d85/f94 [1754588,112911] 0 2024-02-16T22:56:33.079 INFO:tasks.workunit.client.1.smithi192.stdout:5/824: stat d6/d8/d80/c73 0 2024-02-16T22:56:33.082 INFO:tasks.workunit.client.1.smithi192.stdout:9/936: unlink d7/d15/d36/d38/d3f/d8a/lfd 0 2024-02-16T22:56:33.083 INFO:tasks.workunit.client.1.smithi192.stdout:9/937: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/ld7 28114734 1 2024-02-16T22:56:33.085 INFO:tasks.workunit.client.1.smithi192.stdout:1/782: mkdir d3/d40/d9f/d10a 0 2024-02-16T22:56:33.092 INFO:tasks.workunit.client.1.smithi192.stdout:8/818: dread d5/d26/fa0 [0,4194304] 0 2024-02-16T22:56:33.092 INFO:tasks.workunit.client.0.smithi064.stdout:6/771: dwrite d0/d1d/f61 [0,4194304] 0 2024-02-16T22:56:33.092 INFO:tasks.workunit.client.1.smithi192.stdout:8/819: fdatasync d5/d26/d8f/f72 0 2024-02-16T22:56:33.092 INFO:tasks.workunit.client.1.smithi192.stdout:5/825: creat d6/d8/d75/d8c/f10e x:0 0 0 2024-02-16T22:56:33.093 INFO:tasks.workunit.client.0.smithi064.stdout:1/628: write d2/df/d14/d18/d48/d6b/d75/f77 [256873,81789] 0 2024-02-16T22:56:33.095 INFO:tasks.workunit.client.0.smithi064.stdout:1/629: fdatasync d2/df/d14/d18/d48/d84/fa4 0 2024-02-16T22:56:33.096 INFO:tasks.workunit.client.0.smithi064.stdout:6/772: read d0/d1d/da8/d60/f7c [1844173,73156] 0 2024-02-16T22:56:33.096 INFO:tasks.workunit.client.1.smithi192.stdout:9/938: truncate d7/d15/d36/f99 1002941 0 2024-02-16T22:56:33.096 INFO:tasks.workunit.client.1.smithi192.stdout:4/873: write d1/d26/d6f/df4/f105 [609279,118496] 0 2024-02-16T22:56:33.097 INFO:tasks.workunit.client.0.smithi064.stdout:6/773: readlink d0/d5/lda 0 2024-02-16T22:56:33.098 INFO:tasks.workunit.client.1.smithi192.stdout:4/874: write d1/d26/d66/d89/d11/f98 [889343,27934] 0 2024-02-16T22:56:33.102 INFO:tasks.workunit.client.1.smithi192.stdout:4/875: dread d1/d26/d66/d89/d11/f98 [0,4194304] 0 2024-02-16T22:56:33.103 INFO:tasks.workunit.client.1.smithi192.stdout:3/752: rename da/df/f2e to da/f100 0 2024-02-16T22:56:33.108 INFO:tasks.workunit.client.1.smithi192.stdout:0/807: dwrite d1/f51 [0,4194304] 0 2024-02-16T22:56:33.109 INFO:tasks.workunit.client.1.smithi192.stdout:8/820: unlink d5/d16/dd5/fd1 0 2024-02-16T22:56:33.111 INFO:tasks.workunit.client.1.smithi192.stdout:9/939: mknod d7/d19/d28/d43/dea/c13e 0 2024-02-16T22:56:33.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/826: dwrite d6/d12/fee [0,4194304] 0 2024-02-16T22:56:33.120 INFO:tasks.workunit.client.0.smithi064.stdout:6/774: dread d0/d5/fcd [0,4194304] 0 2024-02-16T22:56:33.122 INFO:tasks.workunit.client.1.smithi192.stdout:0/808: creat d1/d3/dac/f101 x:0 0 0 2024-02-16T22:56:33.127 INFO:tasks.workunit.client.1.smithi192.stdout:8/821: rename d5/d16/d81 to d5/d16/dd5/d9d/d101 0 2024-02-16T22:56:33.127 INFO:tasks.workunit.client.1.smithi192.stdout:2/872: write d22/d24/d39/d42/d85/d98/fc8 [707366,58459] 0 2024-02-16T22:56:33.128 INFO:tasks.workunit.client.0.smithi064.stdout:0/613: dwrite da/dc/d3b/f5f [0,4194304] 0 2024-02-16T22:56:33.129 INFO:tasks.workunit.client.1.smithi192.stdout:2/873: dread - d22/d47/dcb/d78/ff6 zero size 2024-02-16T22:56:33.129 INFO:tasks.workunit.client.1.smithi192.stdout:9/940: dread - d7/d19/d37/d5a/fdd zero size 2024-02-16T22:56:33.137 INFO:tasks.workunit.client.0.smithi064.stdout:1/630: truncate d2/df/d14/d2c/f49 1127723 0 2024-02-16T22:56:33.143 INFO:tasks.workunit.client.1.smithi192.stdout:9/941: symlink d7/d19/d5e/l13f 0 2024-02-16T22:56:33.146 INFO:tasks.workunit.client.0.smithi064.stdout:2/822: rename d7/d14/d4b/d63/lcb to d7/d14/d1e/d34/d73/db5/l108 0 2024-02-16T22:56:33.148 INFO:tasks.workunit.client.0.smithi064.stdout:4/709: link dc/d1d/c68 dc/dd/d16/d9c/cdd 0 2024-02-16T22:56:33.149 INFO:tasks.workunit.client.0.smithi064.stdout:4/710: truncate dc/dd/d5b/fb2 712715 0 2024-02-16T22:56:33.151 INFO:tasks.workunit.client.0.smithi064.stdout:6/775: read d0/d5/f27 [382409,49476] 0 2024-02-16T22:56:33.155 INFO:tasks.workunit.client.1.smithi192.stdout:3/753: getdents da/d11/dd4/deb 0 2024-02-16T22:56:33.157 INFO:tasks.workunit.client.1.smithi192.stdout:1/783: write d3/f97 [374837,121167] 0 2024-02-16T22:56:33.158 INFO:tasks.workunit.client.0.smithi064.stdout:2/823: mkdir d7/d14/d25/d46/d55/d7f/d109 0 2024-02-16T22:56:33.159 INFO:tasks.workunit.client.1.smithi192.stdout:0/809: dread d1/d3/de/d22/d37/d67/d34/f8a [0,4194304] 0 2024-02-16T22:56:33.160 INFO:tasks.workunit.client.1.smithi192.stdout:3/754: mknod da/d11/d63/d6b/d66/d75/c101 0 2024-02-16T22:56:33.161 INFO:tasks.workunit.client.1.smithi192.stdout:8/822: dread d5/d9/f69 [0,4194304] 0 2024-02-16T22:56:33.165 INFO:tasks.workunit.client.0.smithi064.stdout:6/776: creat d0/d5/df/f100 x:0 0 0 2024-02-16T22:56:33.168 INFO:tasks.workunit.client.0.smithi064.stdout:0/614: mknod da/dc/de/d12/d2c/d2e/cce 0 2024-02-16T22:56:33.171 INFO:tasks.workunit.client.1.smithi192.stdout:9/942: dread d7/d15/f58 [0,4194304] 0 2024-02-16T22:56:33.172 INFO:tasks.workunit.client.0.smithi064.stdout:2/824: mknod d7/d14/d25/d46/d55/d7f/dde/c10a 0 2024-02-16T22:56:33.172 INFO:tasks.workunit.client.0.smithi064.stdout:2/825: write d7/d14/d1e/f81 [4182113,87617] 0 2024-02-16T22:56:33.173 INFO:tasks.workunit.client.1.smithi192.stdout:3/755: mkdir da/d11/d2b/ddc/d102 0 2024-02-16T22:56:33.173 INFO:tasks.workunit.client.0.smithi064.stdout:5/664: write d6/d25/da4/dc1/f60 [4280960,22783] 0 2024-02-16T22:56:33.173 INFO:tasks.workunit.client.1.smithi192.stdout:9/943: dread d7/d15/d36/d38/d3f/d8a/def/d100/f137 [0,4194304] 0 2024-02-16T22:56:33.174 INFO:tasks.workunit.client.0.smithi064.stdout:5/665: chown d6/d25/d48 1881073 1 2024-02-16T22:56:33.175 INFO:tasks.workunit.client.1.smithi192.stdout:0/810: dread d1/dc7/d9c/f72 [0,4194304] 0 2024-02-16T22:56:33.177 INFO:tasks.workunit.client.0.smithi064.stdout:5/666: chown d6/d25/d48/d67/d91/da5/de3/d4b/cd0 75620181 1 2024-02-16T22:56:33.183 INFO:tasks.workunit.client.1.smithi192.stdout:9/944: mkdir d7/d140 0 2024-02-16T22:56:33.186 INFO:tasks.workunit.client.1.smithi192.stdout:3/756: fdatasync da/d11/d63/f9e 0 2024-02-16T22:56:33.188 INFO:tasks.workunit.client.1.smithi192.stdout:7/938: sync 2024-02-16T22:56:33.188 INFO:tasks.workunit.client.1.smithi192.stdout:9/945: dwrite d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f116 [0,4194304] 0 2024-02-16T22:56:33.190 INFO:tasks.workunit.client.0.smithi064.stdout:9/676: rename d3/d9/d6a/db1/d38/la7 to d3/d9/d1e/ldf 0 2024-02-16T22:56:33.191 INFO:tasks.workunit.client.0.smithi064.stdout:0/615: unlink da/dc/d3b/ca9 0 2024-02-16T22:56:33.192 INFO:tasks.workunit.client.1.smithi192.stdout:3/757: rename da/d6a/fd1 to da/d11/d63/dbf/f103 0 2024-02-16T22:56:33.192 INFO:tasks.workunit.client.0.smithi064.stdout:9/677: readlink d3/d9/d91/lba 0 2024-02-16T22:56:33.193 INFO:tasks.workunit.client.0.smithi064.stdout:2/826: dwrite d7/d14/d25/f90 [0,4194304] 0 2024-02-16T22:56:33.193 INFO:tasks.workunit.client.1.smithi192.stdout:3/758: readlink da/d11/d20/d39/lc2 0 2024-02-16T22:56:33.194 INFO:tasks.workunit.client.0.smithi064.stdout:5/667: readlink d6/d25/d48/d67/d91/da5/de3/d43/d7d/l82 0 2024-02-16T22:56:33.194 INFO:tasks.workunit.client.1.smithi192.stdout:3/759: fsync da/d11/d20/d39/fad 0 2024-02-16T22:56:33.195 INFO:tasks.workunit.client.1.smithi192.stdout:5/827: write d6/d12/d1d/f34 [155699,7003] 0 2024-02-16T22:56:33.196 INFO:tasks.workunit.client.1.smithi192.stdout:3/760: fdatasync da/d11/fdf 0 2024-02-16T22:56:33.197 INFO:tasks.workunit.client.1.smithi192.stdout:7/939: creat d1/d2/de/d2c/d49/d54/d9d/ddd/d13d/f13e x:0 0 0 2024-02-16T22:56:33.198 INFO:tasks.workunit.client.0.smithi064.stdout:7/692: rename d5/d33/d45/d73/fb4 to d5/d33/d53/da3/fda 0 2024-02-16T22:56:33.200 INFO:tasks.workunit.client.0.smithi064.stdout:9/678: fsync d3/d89/d9a/f9b 0 2024-02-16T22:56:33.200 INFO:tasks.workunit.client.0.smithi064.stdout:9/679: chown f1 0 1 2024-02-16T22:56:33.202 INFO:tasks.workunit.client.0.smithi064.stdout:5/668: dwrite d6/d18/d19/fb0 [0,4194304] 0 2024-02-16T22:56:33.202 INFO:tasks.workunit.client.1.smithi192.stdout:2/874: dwrite d22/d24/f49 [8388608,4194304] 0 2024-02-16T22:56:33.204 INFO:tasks.workunit.client.0.smithi064.stdout:0/616: mkdir da/dc/de/d12/d2c/d2e/d38/dcf 0 2024-02-16T22:56:33.207 INFO:tasks.workunit.client.1.smithi192.stdout:0/811: getdents d1/d3/de/d22 0 2024-02-16T22:56:33.207 INFO:tasks.workunit.client.1.smithi192.stdout:1/784: write d3/de/d27/d34/d36/d47/d5d/fe5 [17019,17158] 0 2024-02-16T22:56:33.207 INFO:tasks.workunit.client.0.smithi064.stdout:4/711: write dc/fba [1314294,112581] 0 2024-02-16T22:56:33.208 INFO:tasks.workunit.client.0.smithi064.stdout:4/712: truncate dc/dd/d5b/fb7 661891 0 2024-02-16T22:56:33.211 INFO:tasks.workunit.client.0.smithi064.stdout:0/617: read da/dc/de/d12/d1d/f1f [7535602,51164] 0 2024-02-16T22:56:33.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/812: dwrite d1/ffe [0,4194304] 0 2024-02-16T22:56:33.217 INFO:tasks.workunit.client.1.smithi192.stdout:5/828: mkdir d6/d8/d86/d10f 0 2024-02-16T22:56:33.219 INFO:tasks.workunit.client.1.smithi192.stdout:7/940: rmdir d1/d2/d2d/d61/d88/d90/d4c/d60 39 2024-02-16T22:56:33.220 INFO:tasks.workunit.client.0.smithi064.stdout:6/777: dwrite d0/d5/df/f79 [0,4194304] 0 2024-02-16T22:56:33.220 INFO:tasks.workunit.client.1.smithi192.stdout:7/941: fsync d1/d2/d2d/d61/d8a/fde 0 2024-02-16T22:56:33.222 INFO:tasks.workunit.client.1.smithi192.stdout:7/942: dread - d1/d2/de/d2c/d49/d54/dd8/d117/dcd/f133 zero size 2024-02-16T22:56:33.223 INFO:tasks.workunit.client.1.smithi192.stdout:3/761: rename da/d11/d20/c17 to da/d11/d20/d39/dca/c104 0 2024-02-16T22:56:33.225 INFO:tasks.workunit.client.0.smithi064.stdout:9/680: symlink d3/d9/d1e/d3e/le0 0 2024-02-16T22:56:33.230 INFO:tasks.workunit.client.1.smithi192.stdout:2/875: truncate d22/d24/d39/d3d/f40 4197730 0 2024-02-16T22:56:33.236 INFO:tasks.workunit.client.1.smithi192.stdout:0/813: mknod d1/dc7/d9c/c102 0 2024-02-16T22:56:33.236 INFO:tasks.workunit.client.0.smithi064.stdout:2/827: write d7/d14/d1e/d34/d3f/f79 [490676,28372] 0 2024-02-16T22:56:33.237 INFO:tasks.workunit.client.1.smithi192.stdout:0/814: write d1/d3/de/d22/da7/fb5 [895066,79404] 0 2024-02-16T22:56:33.237 INFO:tasks.workunit.client.0.smithi064.stdout:2/828: chown d7/d14 0 1 2024-02-16T22:56:33.238 INFO:tasks.workunit.client.0.smithi064.stdout:2/829: chown d7/d14/d25/f8a 11306081 1 2024-02-16T22:56:33.239 INFO:tasks.workunit.client.1.smithi192.stdout:0/815: write d1/d3/de/d22/da7/fdf [484364,35160] 0 2024-02-16T22:56:33.239 INFO:tasks.workunit.client.1.smithi192.stdout:5/829: rmdir d6/d12/d4e/d46/d4f 39 2024-02-16T22:56:33.240 INFO:tasks.workunit.client.1.smithi192.stdout:0/816: write d1/d3/d39/f5f [1643535,101401] 0 2024-02-16T22:56:33.241 INFO:tasks.workunit.client.0.smithi064.stdout:4/713: dread dc/dd/d16/f5e [0,4194304] 0 2024-02-16T22:56:33.241 INFO:tasks.workunit.client.0.smithi064.stdout:1/631: rename d2/df/d14/d18/d48/d84/l90 to d2/df/d14/d2c/db5/dc6/ld6 0 2024-02-16T22:56:33.243 INFO:tasks.workunit.client.0.smithi064.stdout:1/632: chown d2/df/d14/d18/d40/l4c 675 1 2024-02-16T22:56:33.245 INFO:tasks.workunit.client.1.smithi192.stdout:7/943: creat d1/d69/f13f x:0 0 0 2024-02-16T22:56:33.246 INFO:tasks.workunit.client.1.smithi192.stdout:1/785: dread d3/de/f65 [0,4194304] 0 2024-02-16T22:56:33.247 INFO:tasks.workunit.client.0.smithi064.stdout:6/778: creat d0/d6/d10/d33/d72/de8/f101 x:0 0 0 2024-02-16T22:56:33.248 INFO:tasks.workunit.client.1.smithi192.stdout:9/946: link d7/d15/d36/d77/db1/lbb d7/d15/d36/d38/d3f/d8a/d131/l141 0 2024-02-16T22:56:33.249 INFO:tasks.workunit.client.1.smithi192.stdout:2/876: stat d22/d24/d39/d42/d65/d66/c107 0 2024-02-16T22:56:33.251 INFO:tasks.workunit.client.1.smithi192.stdout:2/877: read d22/d24/d39/d42/d65/d69/f7a [1358933,64276] 0 2024-02-16T22:56:33.254 INFO:tasks.workunit.client.1.smithi192.stdout:0/817: write d1/d3/de/d22/d37/d67/f9b [183601,108945] 0 2024-02-16T22:56:33.256 INFO:tasks.workunit.client.0.smithi064.stdout:2/830: creat d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/dd6/f10b x:0 0 0 2024-02-16T22:56:33.258 INFO:tasks.workunit.client.1.smithi192.stdout:7/944: symlink d1/d2/de/d2c/d49/d54/dd8/d117/d9b/l140 0 2024-02-16T22:56:33.258 INFO:tasks.workunit.client.1.smithi192.stdout:3/762: write da/d11/d63/fa6 [534927,58687] 0 2024-02-16T22:56:33.259 INFO:tasks.workunit.client.1.smithi192.stdout:1/786: creat d3/d5b/db1/f10b x:0 0 0 2024-02-16T22:56:33.262 INFO:tasks.workunit.client.0.smithi064.stdout:0/618: rename da/d26/d3f/f6b to da/d26/fd0 0 2024-02-16T22:56:33.264 INFO:tasks.workunit.client.1.smithi192.stdout:6/839: sync 2024-02-16T22:56:33.264 INFO:tasks.workunit.client.0.smithi064.stdout:7/693: creat d5/d33/fdb x:0 0 0 2024-02-16T22:56:33.266 INFO:tasks.workunit.client.1.smithi192.stdout:2/878: mknod d22/d24/d39/d42/d85/d98/c11f 0 2024-02-16T22:56:33.268 INFO:tasks.workunit.client.1.smithi192.stdout:5/830: dwrite d6/d8/d80/f8a [0,4194304] 0 2024-02-16T22:56:33.269 INFO:tasks.workunit.client.1.smithi192.stdout:0/818: dread d1/d3/f8c [0,4194304] 0 2024-02-16T22:56:33.270 INFO:tasks.workunit.client.0.smithi064.stdout:5/669: symlink d6/d25/da4/dc1/d8e/le4 0 2024-02-16T22:56:33.273 INFO:tasks.workunit.client.0.smithi064.stdout:6/779: dwrite d0/d6/d10/d33/da4/fd4 [4194304,4194304] 0 2024-02-16T22:56:33.274 INFO:tasks.workunit.client.0.smithi064.stdout:4/714: rename dc/dd/d5b to dc/dd/d16/d7c/dde 0 2024-02-16T22:56:33.274 INFO:tasks.workunit.client.1.smithi192.stdout:9/947: dwrite d7/d15/f58 [4194304,4194304] 0 2024-02-16T22:56:33.282 INFO:tasks.workunit.client.0.smithi064.stdout:4/715: dwrite dc/de/f99 [0,4194304] 0 2024-02-16T22:56:33.283 INFO:tasks.workunit.client.0.smithi064.stdout:1/633: link d2/df/d14/d18/d48/d6b/fd0 d2/df/d11/d1a/d30/dbd/d91/fd7 0 2024-02-16T22:56:33.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:32 smithi192 ceph-mon[36254]: pgmap v21: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 87 MiB/s rd, 209 MiB/s wr, 419 op/s 2024-02-16T22:56:33.288 INFO:tasks.workunit.client.0.smithi064.stdout:9/681: creat d3/d9/d1e/fe1 x:0 0 0 2024-02-16T22:56:33.290 INFO:tasks.workunit.client.0.smithi064.stdout:5/670: creat d6/d25/d48/d67/d80/fe5 x:0 0 0 2024-02-16T22:56:33.293 INFO:tasks.workunit.client.1.smithi192.stdout:2/879: creat d22/d24/d39/d42/d85/d98/dcc/f120 x:0 0 0 2024-02-16T22:56:33.295 INFO:tasks.workunit.client.0.smithi064.stdout:9/682: dwrite d3/d9/d6a/db1/f26 [0,4194304] 0 2024-02-16T22:56:33.296 INFO:tasks.workunit.client.0.smithi064.stdout:2/831: rename d7/d14/d1e/d34/d73/db5 to d7/d14/d1e/d34/dad/d10c 0 2024-02-16T22:56:33.296 INFO:tasks.workunit.client.0.smithi064.stdout:2/832: readlink d7/l30 0 2024-02-16T22:56:33.298 INFO:tasks.workunit.client.0.smithi064.stdout:2/833: write d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/dd6/f10b [535845,19644] 0 2024-02-16T22:56:33.299 INFO:tasks.workunit.client.1.smithi192.stdout:5/831: fsync d6/d12/d2d/fd5 0 2024-02-16T22:56:33.302 INFO:tasks.workunit.client.1.smithi192.stdout:0/819: creat d1/dc7/d9c/f103 x:0 0 0 2024-02-16T22:56:33.304 INFO:tasks.workunit.client.1.smithi192.stdout:0/820: readlink d1/d3/de/d22/d37/d67/l2d 0 2024-02-16T22:56:33.305 INFO:tasks.workunit.client.0.smithi064.stdout:4/716: symlink dc/dd/d32/d46/d5a/d7e/d9a/ldf 0 2024-02-16T22:56:33.306 INFO:tasks.workunit.client.0.smithi064.stdout:7/694: dwrite d5/d33/d53/f70 [0,4194304] 0 2024-02-16T22:56:33.306 INFO:tasks.workunit.client.1.smithi192.stdout:7/945: mknod d1/d2/de/d2c/d49/d54/d9d/c141 0 2024-02-16T22:56:33.312 INFO:tasks.workunit.client.0.smithi064.stdout:5/671: mkdir d6/d18/d95/de6 0 2024-02-16T22:56:33.313 INFO:tasks.workunit.client.1.smithi192.stdout:1/787: creat d3/d40/d9f/df0/f10c x:0 0 0 2024-02-16T22:56:33.313 INFO:tasks.workunit.client.0.smithi064.stdout:5/672: chown d6/d25/d48/d67/d91/da5 46686 1 2024-02-16T22:56:33.315 INFO:tasks.workunit.client.1.smithi192.stdout:4/876: sync 2024-02-16T22:56:33.315 INFO:tasks.workunit.client.1.smithi192.stdout:8/823: sync 2024-02-16T22:56:33.315 INFO:tasks.workunit.client.0.smithi064.stdout:5/673: truncate d6/d18/d95/dbe/fe1 466452 0 2024-02-16T22:56:33.316 INFO:tasks.workunit.client.1.smithi192.stdout:9/948: dwrite d7/d19/d28/d43/d70/df0/fb6 [4194304,4194304] 0 2024-02-16T22:56:33.316 INFO:tasks.workunit.client.1.smithi192.stdout:1/788: write d3/de/d27/d34/d52/d63/d7a/dc9/fe1 [1038602,81076] 0 2024-02-16T22:56:33.317 INFO:tasks.workunit.client.1.smithi192.stdout:9/949: chown d7/d19/c113 61 1 2024-02-16T22:56:33.317 INFO:tasks.workunit.client.1.smithi192.stdout:2/880: rename d22/d47/dcb/dda/dec to d22/d47/d121 0 2024-02-16T22:56:33.321 INFO:tasks.workunit.client.1.smithi192.stdout:9/950: fdatasync d7/d19/d37/d39/f44 0 2024-02-16T22:56:33.322 INFO:tasks.workunit.client.0.smithi064.stdout:2/834: mkdir d7/d14/d25/d46/d55/d7f/dde/d10d 0 2024-02-16T22:56:33.322 INFO:tasks.workunit.client.0.smithi064.stdout:6/780: write d0/d1d/f84 [3986754,126632] 0 2024-02-16T22:56:33.322 INFO:tasks.workunit.client.0.smithi064.stdout:2/835: readlink d7/d14/d1e/d34/d3f/d9a/dd9/lf9 0 2024-02-16T22:56:33.323 INFO:tasks.workunit.client.1.smithi192.stdout:3/763: dread da/f3d [0,4194304] 0 2024-02-16T22:56:33.323 INFO:tasks.workunit.client.1.smithi192.stdout:9/951: chown d7/d19/d28/d43/fc9 76 1 2024-02-16T22:56:33.323 INFO:tasks.workunit.client.1.smithi192.stdout:5/832: fsync d6/d20/f4d 0 2024-02-16T22:56:33.325 INFO:tasks.workunit.client.1.smithi192.stdout:0/821: fsync d1/f19 0 2024-02-16T22:56:33.329 INFO:tasks.workunit.client.0.smithi064.stdout:4/717: fsync dc/dd/d4f/fa0 0 2024-02-16T22:56:33.332 INFO:tasks.workunit.client.1.smithi192.stdout:6/840: rmdir da/df/d12/d57/d81/dfc/d101 0 2024-02-16T22:56:33.336 INFO:tasks.workunit.client.0.smithi064.stdout:9/683: symlink d3/d9/le2 0 2024-02-16T22:56:33.336 INFO:tasks.workunit.client.0.smithi064.stdout:4/718: dread dc/fba [0,4194304] 0 2024-02-16T22:56:33.337 INFO:tasks.workunit.client.1.smithi192.stdout:6/841: dread da/df/d71/dab/fcd [0,4194304] 0 2024-02-16T22:56:33.338 INFO:tasks.workunit.client.0.smithi064.stdout:5/674: write d6/d18/d19/fb3 [754151,81647] 0 2024-02-16T22:56:33.339 INFO:tasks.workunit.client.0.smithi064.stdout:4/719: dread dc/dd/d32/d46/f55 [0,4194304] 0 2024-02-16T22:56:33.339 INFO:tasks.workunit.client.0.smithi064.stdout:0/619: getdents da/dc/de/d12/d2c/d2e/d38/d7e/d89/d93 0 2024-02-16T22:56:33.342 INFO:tasks.workunit.client.0.smithi064.stdout:2/836: mkdir d7/d14/d25/d46/d55/d57/dbe/d10e 0 2024-02-16T22:56:33.345 INFO:tasks.workunit.client.1.smithi192.stdout:8/824: rename d5/d26/d83/d86/ld0 to d5/d26/d83/d86/d5f/db2/l102 0 2024-02-16T22:56:33.347 INFO:tasks.workunit.client.0.smithi064.stdout:1/634: link d2/df/d14/d2c/f3c d2/df/d11/d1a/d30/fd8 0 2024-02-16T22:56:33.349 INFO:tasks.workunit.client.0.smithi064.stdout:9/684: mknod d3/d9/d91/ce3 0 2024-02-16T22:56:33.350 INFO:tasks.workunit.client.1.smithi192.stdout:9/952: chown d7/d15/f6d 0 1 2024-02-16T22:56:33.354 INFO:tasks.workunit.client.1.smithi192.stdout:7/946: dwrite d1/de1/f7c [0,4194304] 0 2024-02-16T22:56:33.354 INFO:tasks.workunit.client.0.smithi064.stdout:5/675: rename d6/d18/f41 to d6/d25/d48/d67/d91/da5/de3/d43/d7d/de2/fe7 0 2024-02-16T22:56:33.356 INFO:tasks.workunit.client.0.smithi064.stdout:4/720: truncate dc/dd/d32/d46/d82/f9e 1484944 0 2024-02-16T22:56:33.356 INFO:tasks.workunit.client.1.smithi192.stdout:0/822: creat d1/d3/d39/f104 x:0 0 0 2024-02-16T22:56:33.358 INFO:tasks.workunit.client.0.smithi064.stdout:2/837: creat d7/d14/d1e/d34/dad/d10c/dea/f10f x:0 0 0 2024-02-16T22:56:33.363 INFO:tasks.workunit.client.1.smithi192.stdout:6/842: symlink da/df/d18/d52/da9/l10e 0 2024-02-16T22:56:33.364 INFO:tasks.workunit.client.0.smithi064.stdout:1/635: mknod d2/df/d11/dba/cd9 0 2024-02-16T22:56:33.365 INFO:tasks.workunit.client.1.smithi192.stdout:1/789: creat d3/d40/d9f/d10a/f10d x:0 0 0 2024-02-16T22:56:33.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:32 smithi064 ceph-mon[29226]: pgmap v21: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 87 MiB/s rd, 209 MiB/s wr, 419 op/s 2024-02-16T22:56:33.369 INFO:tasks.workunit.client.1.smithi192.stdout:6/843: dwrite da/df/d12/d15/d1b/f8a [0,4194304] 0 2024-02-16T22:56:33.369 INFO:tasks.workunit.client.1.smithi192.stdout:6/844: chown da/d76/dd5 1759143 1 2024-02-16T22:56:33.371 INFO:tasks.workunit.client.1.smithi192.stdout:9/953: mknod d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/d104/c142 0 2024-02-16T22:56:33.372 INFO:tasks.workunit.client.0.smithi064.stdout:2/838: fdatasync d7/d14/d1e/d34/f50 0 2024-02-16T22:56:33.372 INFO:tasks.workunit.client.1.smithi192.stdout:9/954: chown d7/d15/d36/d77/dbd/d101/f132 11023 1 2024-02-16T22:56:33.373 INFO:tasks.workunit.client.0.smithi064.stdout:7/695: link d5/l21 d5/d33/d45/d64/d56/d66/ldc 0 2024-02-16T22:56:33.379 INFO:tasks.workunit.client.1.smithi192.stdout:7/947: fsync d1/d2/de/d2c/d49/d54/d9d/ddd/d13d/fc8 0 2024-02-16T22:56:33.380 INFO:tasks.workunit.client.0.smithi064.stdout:2/839: dwrite d7/d14/d25/f4a [4194304,4194304] 0 2024-02-16T22:56:33.388 INFO:tasks.workunit.client.1.smithi192.stdout:7/948: dwrite d1/d2/d2d/d61/d88/d90/d10f/f12f [0,4194304] 0 2024-02-16T22:56:33.389 INFO:tasks.workunit.client.1.smithi192.stdout:1/790: dread d3/de/f82 [0,4194304] 0 2024-02-16T22:56:33.433 INFO:tasks.workunit.client.0.smithi064.stdout:8/648: sync 2024-02-16T22:56:33.438 INFO:tasks.workunit.client.1.smithi192.stdout:4/877: creat d1/d15/d29/f12e x:0 0 0 2024-02-16T22:56:33.439 INFO:tasks.workunit.client.0.smithi064.stdout:8/649: read df/d31/d2f/d38/d48/f49 [3810927,124006] 0 2024-02-16T22:56:33.440 INFO:tasks.workunit.client.1.smithi192.stdout:3/764: sync 2024-02-16T22:56:33.445 INFO:tasks.workunit.client.1.smithi192.stdout:5/833: rename d6/d12/d4e/l38 to d6/d12/d2d/d5c/l110 0 2024-02-16T22:56:33.446 INFO:tasks.workunit.client.1.smithi192.stdout:5/834: stat d6/d12/d4e 0 2024-02-16T22:56:33.447 INFO:tasks.workunit.client.1.smithi192.stdout:2/881: creat d22/d24/d39/d42/d85/d98/db0/f122 x:0 0 0 2024-02-16T22:56:33.450 INFO:tasks.workunit.client.0.smithi064.stdout:2/840: read d7/d14/d1e/fa4 [1038247,76703] 0 2024-02-16T22:56:33.450 INFO:tasks.workunit.client.0.smithi064.stdout:0/620: link da/dc/d3b/c9d da/dc/de/d12/d2c/d2e/d38/d7e/d89/d93/cd1 0 2024-02-16T22:56:33.451 INFO:tasks.workunit.client.1.smithi192.stdout:8/825: dwrite d5/d26/d83/d86/f41 [0,4194304] 0 2024-02-16T22:56:33.451 INFO:tasks.workunit.client.1.smithi192.stdout:8/826: readlink d5/d16/l30 0 2024-02-16T22:56:33.452 INFO:tasks.workunit.client.1.smithi192.stdout:8/827: stat d5/d26/d83/d86/f74 0 2024-02-16T22:56:33.453 INFO:tasks.workunit.client.1.smithi192.stdout:8/828: truncate d5/fc8 992764 0 2024-02-16T22:56:33.454 INFO:tasks.workunit.client.0.smithi064.stdout:9/685: write d3/d9/d6a/db1/f85 [711756,38419] 0 2024-02-16T22:56:33.455 INFO:tasks.workunit.client.0.smithi064.stdout:4/721: dwrite dc/d4a/f4d [0,4194304] 0 2024-02-16T22:56:33.456 INFO:tasks.workunit.client.0.smithi064.stdout:9/686: readlink d3/d11/l81 0 2024-02-16T22:56:33.457 INFO:tasks.workunit.client.0.smithi064.stdout:7/696: mkdir d5/d33/ddd 0 2024-02-16T22:56:33.460 INFO:tasks.workunit.client.0.smithi064.stdout:8/650: mkdir df/d31/d2f/d38/d48/da2/de2 0 2024-02-16T22:56:33.461 INFO:tasks.workunit.client.0.smithi064.stdout:5/676: truncate d6/d25/d48/d67/f7f 1098095 0 2024-02-16T22:56:33.464 INFO:tasks.workunit.client.0.smithi064.stdout:0/621: mkdir da/dc/de/d12/d1d/dd2 0 2024-02-16T22:56:33.469 INFO:tasks.workunit.client.0.smithi064.stdout:4/722: creat dc/dd/d32/d46/d5a/d7e/fe0 x:0 0 0 2024-02-16T22:56:33.469 INFO:tasks.workunit.client.1.smithi192.stdout:3/765: truncate da/d11/d63/f9e 1610447 0 2024-02-16T22:56:33.470 INFO:tasks.workunit.client.0.smithi064.stdout:4/723: write dc/dd/d32/d46/d82/fcf [1596571,126678] 0 2024-02-16T22:56:33.473 INFO:tasks.workunit.client.0.smithi064.stdout:7/697: readlink d5/d33/d45/d64/d56/d66/ldc 0 2024-02-16T22:56:33.475 INFO:tasks.workunit.client.1.smithi192.stdout:9/955: rename d7/d15/d36/d77/l129 to d7/d15/d36/l143 0 2024-02-16T22:56:33.476 INFO:tasks.workunit.client.1.smithi192.stdout:0/823: truncate d1/d3/d39/f77 911107 0 2024-02-16T22:56:33.478 INFO:tasks.workunit.client.0.smithi064.stdout:5/677: chown d6/l7c 330037537 1 2024-02-16T22:56:33.478 INFO:tasks.workunit.client.1.smithi192.stdout:6/845: mknod da/df/d18/c10f 0 2024-02-16T22:56:33.484 INFO:tasks.workunit.client.0.smithi064.stdout:2/841: creat d7/d14/d1e/d34/d3f/d9a/dd9/d103/f110 x:0 0 0 2024-02-16T22:56:33.484 INFO:tasks.workunit.client.1.smithi192.stdout:1/791: write d3/de/d27/d34/d52/ff8 [784053,3123] 0 2024-02-16T22:56:33.485 INFO:tasks.workunit.client.0.smithi064.stdout:2/842: truncate d7/d14/d4b/d63/f106 143272 0 2024-02-16T22:56:33.487 INFO:tasks.workunit.client.0.smithi064.stdout:9/687: mkdir d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/de4 0 2024-02-16T22:56:33.491 INFO:tasks.workunit.client.1.smithi192.stdout:4/878: rename d1/d15/d48/db5/dd5 to d1/d26/d66/d89/da8/dc5/d111/d12f 0 2024-02-16T22:56:33.493 INFO:tasks.workunit.client.0.smithi064.stdout:4/724: mknod dc/dd/d32/d46/d82/da1/ce1 0 2024-02-16T22:56:33.500 INFO:tasks.workunit.client.0.smithi064.stdout:0/622: mknod da/dc/cd3 0 2024-02-16T22:56:33.502 INFO:tasks.workunit.client.1.smithi192.stdout:8/829: symlink d5/l103 0 2024-02-16T22:56:33.506 INFO:tasks.workunit.client.0.smithi064.stdout:9/688: stat d3/d9/d6a/db1/d2a/d2e/d31/d3f/c55 0 2024-02-16T22:56:33.510 INFO:tasks.workunit.client.0.smithi064.stdout:7/698: dwrite d5/d33/d42/f8f [0,4194304] 0 2024-02-16T22:56:33.510 INFO:tasks.workunit.client.1.smithi192.stdout:2/882: dwrite d22/d24/d39/d42/d65/d69/db1/fbe [0,4194304] 0 2024-02-16T22:56:33.512 INFO:tasks.workunit.client.1.smithi192.stdout:5/835: rename d6/d12/fee to d6/d8/d86/d10f/f111 0 2024-02-16T22:56:33.515 INFO:tasks.workunit.client.0.smithi064.stdout:2/843: dwrite d7/f86 [0,4194304] 0 2024-02-16T22:56:33.515 INFO:tasks.workunit.client.1.smithi192.stdout:5/836: write d6/d12/d1d/ff5 [2750956,71729] 0 2024-02-16T22:56:33.516 INFO:tasks.workunit.client.1.smithi192.stdout:9/956: symlink d7/d19/d28/ded/l144 0 2024-02-16T22:56:33.516 INFO:tasks.workunit.client.1.smithi192.stdout:5/837: stat d6/d20/l88 0 2024-02-16T22:56:33.518 INFO:tasks.workunit.client.1.smithi192.stdout:0/824: creat d1/d3/de2/dff/f105 x:0 0 0 2024-02-16T22:56:33.519 INFO:tasks.workunit.client.1.smithi192.stdout:6/846: symlink da/df/d18/l110 0 2024-02-16T22:56:33.520 INFO:tasks.workunit.client.1.smithi192.stdout:3/766: dwrite da/d11/d63/d6b/d80/fb5 [0,4194304] 0 2024-02-16T22:56:33.521 INFO:tasks.workunit.client.0.smithi064.stdout:5/678: fsync d6/f37 0 2024-02-16T22:56:33.521 INFO:tasks.workunit.client.0.smithi064.stdout:9/689: rmdir d3/d9/d1e/d3e 39 2024-02-16T22:56:33.522 INFO:tasks.workunit.client.1.smithi192.stdout:3/767: chown da/d11/d20/f85 2 1 2024-02-16T22:56:33.522 INFO:tasks.workunit.client.0.smithi064.stdout:5/679: readlink d6/l7c 0 2024-02-16T22:56:33.523 INFO:tasks.workunit.client.0.smithi064.stdout:4/725: mknod dc/d1d/d43/ce2 0 2024-02-16T22:56:33.526 INFO:tasks.workunit.client.0.smithi064.stdout:7/699: mkdir d5/d33/d53/dde 0 2024-02-16T22:56:33.528 INFO:tasks.workunit.client.0.smithi064.stdout:8/651: getdents df/d31/d8f/dab 0 2024-02-16T22:56:33.529 INFO:tasks.workunit.client.1.smithi192.stdout:7/949: getdents d1/d2/de/d2c/d49/d89 0 2024-02-16T22:56:33.529 INFO:tasks.workunit.client.1.smithi192.stdout:3/768: write da/d11/d2b/ddc/ff2 [281679,23831] 0 2024-02-16T22:56:33.531 INFO:tasks.workunit.client.0.smithi064.stdout:0/623: symlink da/dc/de/d12/d1d/dd2/ld4 0 2024-02-16T22:56:33.531 INFO:tasks.workunit.client.0.smithi064.stdout:5/680: dwrite d6/d25/d48/d67/d80/fe5 [0,4194304] 0 2024-02-16T22:56:33.535 INFO:tasks.workunit.client.0.smithi064.stdout:2/844: read d7/d14/d1e/f21 [2558381,50957] 0 2024-02-16T22:56:33.535 INFO:tasks.workunit.client.0.smithi064.stdout:5/681: dread d6/d25/d48/d67/d91/da5/de3/d43/d7d/faa [0,4194304] 0 2024-02-16T22:56:33.537 INFO:tasks.workunit.client.1.smithi192.stdout:2/883: unlink d22/d24/d39/d3d/df2/lea 0 2024-02-16T22:56:33.537 INFO:tasks.workunit.client.1.smithi192.stdout:4/879: creat d1/d26/d34/db9/ddb/f130 x:0 0 0 2024-02-16T22:56:33.541 INFO:tasks.workunit.client.1.smithi192.stdout:1/792: rename d3/de/d27/d34/d69/d85/d89/daa/lf1 to d3/de/d27/d34/d36/d47/d5d/l10e 0 2024-02-16T22:56:33.542 INFO:tasks.workunit.client.0.smithi064.stdout:4/726: rename dc/dd/d16/d7c/dde to dc/dd/d32/d46/d79/de3 0 2024-02-16T22:56:33.544 INFO:tasks.workunit.client.0.smithi064.stdout:7/700: rmdir d5/d6/d54/d7b 39 2024-02-16T22:56:33.544 INFO:tasks.workunit.client.1.smithi192.stdout:1/793: write d3/de/d27/f39 [5794504,131061] 0 2024-02-16T22:56:33.545 INFO:tasks.workunit.client.0.smithi064.stdout:8/652: truncate df/d31/f41 1676878 0 2024-02-16T22:56:33.546 INFO:tasks.workunit.client.0.smithi064.stdout:8/653: readlink df/d31/d61/d79/d81/lb9 0 2024-02-16T22:56:33.546 INFO:tasks.workunit.client.1.smithi192.stdout:2/884: dwrite d22/fa9 [0,4194304] 0 2024-02-16T22:56:33.547 INFO:tasks.workunit.client.1.smithi192.stdout:6/847: fsync da/df/d12/d57/f50 0 2024-02-16T22:56:33.547 INFO:tasks.workunit.client.1.smithi192.stdout:4/880: dwrite d1/d26/d34/f60 [0,4194304] 0 2024-02-16T22:56:33.548 INFO:tasks.workunit.client.0.smithi064.stdout:0/624: truncate da/dc/de/d12/f42 160847 0 2024-02-16T22:56:33.548 INFO:tasks.workunit.client.1.smithi192.stdout:6/848: readlink da/df/d12/d15/l86 0 2024-02-16T22:56:33.551 INFO:tasks.workunit.client.1.smithi192.stdout:3/769: creat da/d11/d20/d39/dca/f105 x:0 0 0 2024-02-16T22:56:33.554 INFO:tasks.workunit.client.0.smithi064.stdout:5/682: creat d6/d25/d48/d67/fe8 x:0 0 0 2024-02-16T22:56:33.557 INFO:tasks.workunit.client.0.smithi064.stdout:4/727: mknod dc/dd/d32/d22/ce4 0 2024-02-16T22:56:33.558 INFO:tasks.workunit.client.0.smithi064.stdout:4/728: chown dc/d1d/d43/ce2 1883 1 2024-02-16T22:56:33.560 INFO:tasks.workunit.client.0.smithi064.stdout:7/701: unlink d5/d33/d45/d64/c5c 0 2024-02-16T22:56:33.561 INFO:tasks.workunit.client.0.smithi064.stdout:7/702: dread - d5/d33/d45/d64/d56/d5d/fca zero size 2024-02-16T22:56:33.565 INFO:tasks.workunit.client.0.smithi064.stdout:0/625: fdatasync da/dc/d3b/d72/d7c/fa5 0 2024-02-16T22:56:33.568 INFO:tasks.workunit.client.0.smithi064.stdout:5/683: creat d6/d25/da4/fe9 x:0 0 0 2024-02-16T22:56:33.568 INFO:tasks.workunit.client.0.smithi064.stdout:9/690: write d3/d9/d6a/db1/d38/f66 [86732,13658] 0 2024-02-16T22:56:33.571 INFO:tasks.workunit.client.0.smithi064.stdout:2/845: rename d7/d14/d1e/d34/dad/d10c/l108 to d7/d14/d25/d46/d55/d102/l111 0 2024-02-16T22:56:33.576 INFO:tasks.workunit.client.0.smithi064.stdout:8/654: creat df/d31/d2f/d3e/ddf/fe3 x:0 0 0 2024-02-16T22:56:33.581 INFO:tasks.workunit.client.0.smithi064.stdout:0/626: creat da/d26/fd5 x:0 0 0 2024-02-16T22:56:33.588 INFO:tasks.workunit.client.0.smithi064.stdout:2/846: symlink d7/d14/d25/d46/d55/d7f/l112 0 2024-02-16T22:56:33.590 INFO:tasks.workunit.client.0.smithi064.stdout:7/703: dread d5/d33/f57 [0,4194304] 0 2024-02-16T22:56:33.591 INFO:tasks.workunit.client.0.smithi064.stdout:7/704: chown d5/d7d/f93 764194 1 2024-02-16T22:56:33.592 INFO:tasks.workunit.client.0.smithi064.stdout:8/655: mkdir df/d31/d2f/d68/de4 0 2024-02-16T22:56:33.592 INFO:tasks.workunit.client.0.smithi064.stdout:8/656: stat df/d31/d2f/d68/lcf 0 2024-02-16T22:56:33.593 INFO:tasks.workunit.client.1.smithi192.stdout:8/830: dwrite d5/d26/f3f [4194304,4194304] 0 2024-02-16T22:56:33.600 INFO:tasks.workunit.client.0.smithi064.stdout:9/691: rename d3/d11/d15/d23/c3a to d3/d9/d6a/db1/d38/ce5 0 2024-02-16T22:56:33.601 INFO:tasks.workunit.client.1.smithi192.stdout:1/794: dread d3/fa6 [0,4194304] 0 2024-02-16T22:56:33.602 INFO:tasks.workunit.client.0.smithi064.stdout:9/692: chown d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/de4 44871 1 2024-02-16T22:56:33.602 INFO:tasks.workunit.client.0.smithi064.stdout:9/693: chown d3/d9/f12 237210 1 2024-02-16T22:56:33.606 INFO:tasks.workunit.client.0.smithi064.stdout:7/705: rmdir d5/d33/d53 39 2024-02-16T22:56:33.609 INFO:tasks.workunit.client.0.smithi064.stdout:8/657: write df/f1b [4786379,108714] 0 2024-02-16T22:56:33.609 INFO:tasks.workunit.client.0.smithi064.stdout:4/729: dwrite dc/dd/d16/f3a [4194304,4194304] 0 2024-02-16T22:56:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:2/847: creat d7/d14/d25/d46/d55/d57/dbe/d10e/f113 x:0 0 0 2024-02-16T22:56:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:9/694: chown d3/d9/d6a/db1/lc2 7821425 1 2024-02-16T22:56:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:0/627: link da/d71/f7d da/dc/de/fd6 0 2024-02-16T22:56:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:2/848: unlink d7/d14/d1e/d34/d73/c100 0 2024-02-16T22:56:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:9/695: read d3/f8f [92003,37519] 0 2024-02-16T22:56:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:4/730: symlink dc/d1d/d43/le5 0 2024-02-16T22:56:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:4/731: read dc/dd/d32/d46/d82/fcf [3530796,47373] 0 2024-02-16T22:56:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:4/732: chown dc/dd/d32/d46/d82 6 1 2024-02-16T22:56:33.637 INFO:tasks.workunit.client.0.smithi064.stdout:9/696: write d3/d9/d6a/db1/f27 [2321827,15455] 0 2024-02-16T22:56:33.638 INFO:tasks.workunit.client.1.smithi192.stdout:9/957: sync 2024-02-16T22:56:33.638 INFO:tasks.workunit.client.0.smithi064.stdout:9/697: readlink d3/d9/d6a/db1/d2a/d2e/d31/ld2 0 2024-02-16T22:56:33.642 INFO:tasks.workunit.client.0.smithi064.stdout:0/628: symlink da/dc/d3b/d72/db3/ld7 0 2024-02-16T22:56:33.646 INFO:tasks.workunit.client.0.smithi064.stdout:0/629: dread da/d26/f82 [0,4194304] 0 2024-02-16T22:56:33.647 INFO:tasks.workunit.client.0.smithi064.stdout:0/630: chown da/dc/d3b/f8e 6 1 2024-02-16T22:56:33.652 INFO:tasks.workunit.client.0.smithi064.stdout:9/698: rmdir d3/d9/d6a/db1/d2a/d2e/d31/d95 39 2024-02-16T22:56:33.653 INFO:tasks.workunit.client.1.smithi192.stdout:5/838: mknod d6/d12/d1d/d56/dcf/c112 0 2024-02-16T22:56:33.654 INFO:tasks.workunit.client.1.smithi192.stdout:5/839: dread - d6/d12/d1d/d56/ff1 zero size 2024-02-16T22:56:33.656 INFO:tasks.workunit.client.0.smithi064.stdout:2/849: dwrite d7/d14/d1e/f21 [4194304,4194304] 0 2024-02-16T22:56:33.658 INFO:tasks.workunit.client.1.smithi192.stdout:4/881: fsync d1/d26/d66/d89/f97 0 2024-02-16T22:56:33.660 INFO:tasks.workunit.client.0.smithi064.stdout:4/733: symlink dc/dd/d16/d6e/le6 0 2024-02-16T22:56:33.663 INFO:tasks.workunit.client.1.smithi192.stdout:6/849: truncate da/d40/db6/db8/fdd 990183 0 2024-02-16T22:56:33.663 INFO:tasks.workunit.client.0.smithi064.stdout:0/631: rmdir da/dc/de/d37 39 2024-02-16T22:56:33.663 INFO:tasks.workunit.client.1.smithi192.stdout:6/850: chown da/d40/d85/f87 61893 1 2024-02-16T22:56:33.665 INFO:tasks.workunit.client.0.smithi064.stdout:7/706: getdents d5/d33/d42/d81/d82 0 2024-02-16T22:56:33.666 INFO:tasks.workunit.client.1.smithi192.stdout:2/885: sync 2024-02-16T22:56:33.667 INFO:tasks.workunit.client.1.smithi192.stdout:8/831: chown d5/d26/d83/d86/f1c 14431017 1 2024-02-16T22:56:33.667 INFO:tasks.workunit.client.1.smithi192.stdout:2/886: fdatasync d22/d47/dcb/dc9/d103/f110 0 2024-02-16T22:56:33.667 INFO:tasks.workunit.client.0.smithi064.stdout:4/734: dwrite dc/dd/d16/f3a [4194304,4194304] 0 2024-02-16T22:56:33.670 INFO:tasks.workunit.client.0.smithi064.stdout:9/699: dread d3/f69 [0,4194304] 0 2024-02-16T22:56:33.672 INFO:tasks.workunit.client.1.smithi192.stdout:9/958: creat d7/d19/d37/d39/f145 x:0 0 0 2024-02-16T22:56:33.675 INFO:tasks.workunit.client.1.smithi192.stdout:0/825: rename d1/dc7/f68 to d1/d3/d29/dcd/dcf/dd1/f106 0 2024-02-16T22:56:33.677 INFO:tasks.workunit.client.1.smithi192.stdout:4/882: symlink d1/d26/d66/d89/da8/dc5/l131 0 2024-02-16T22:56:33.678 INFO:tasks.workunit.client.0.smithi064.stdout:7/707: stat d5/d6/d54/d7b/fb1 0 2024-02-16T22:56:33.680 INFO:tasks.workunit.client.1.smithi192.stdout:3/770: mkdir da/d106 0 2024-02-16T22:56:33.681 INFO:tasks.workunit.client.1.smithi192.stdout:9/959: dwrite d7/fe [4194304,4194304] 0 2024-02-16T22:56:33.682 INFO:tasks.workunit.client.1.smithi192.stdout:9/960: chown d7/d15/d36/l6b 112106836 1 2024-02-16T22:56:33.682 INFO:tasks.workunit.client.1.smithi192.stdout:6/851: mknod da/d40/d85/c111 0 2024-02-16T22:56:33.689 INFO:tasks.workunit.client.0.smithi064.stdout:9/700: rmdir d3/d9/d6a/db1 39 2024-02-16T22:56:33.692 INFO:tasks.workunit.client.0.smithi064.stdout:0/632: creat da/dc/de/d12/d2c/d2e/d38/dbe/dc6/fd8 x:0 0 0 2024-02-16T22:56:33.695 INFO:tasks.workunit.client.0.smithi064.stdout:9/701: chown d3/l6 4 1 2024-02-16T22:56:33.698 INFO:tasks.workunit.client.0.smithi064.stdout:2/850: link d7/d14/d25/d46/d55/d57/dbe/cc0 d7/d14/d1e/d34/c114 0 2024-02-16T22:56:33.698 INFO:tasks.workunit.client.1.smithi192.stdout:0/826: dread d1/d3/de/d22/d37/d67/d34/fc1 [0,4194304] 0 2024-02-16T22:56:33.699 INFO:tasks.workunit.client.1.smithi192.stdout:4/883: creat d1/d26/d6f/df4/f132 x:0 0 0 2024-02-16T22:56:33.702 INFO:tasks.workunit.client.0.smithi064.stdout:7/708: creat d5/d33/d53/dde/fdf x:0 0 0 2024-02-16T22:56:33.702 INFO:tasks.workunit.client.1.smithi192.stdout:4/884: truncate d1/d26/d6f/df4/f105 1778734 0 2024-02-16T22:56:33.702 INFO:tasks.workunit.client.1.smithi192.stdout:0/827: dread d1/d3/de/d22/d37/fdc [0,4194304] 0 2024-02-16T22:56:33.703 INFO:tasks.workunit.client.1.smithi192.stdout:0/828: stat d1/d3/de/d22/c96 0 2024-02-16T22:56:33.703 INFO:tasks.workunit.client.1.smithi192.stdout:0/829: write d1/d3/de/d22/d37/fdc [743680,53089] 0 2024-02-16T22:56:33.704 INFO:tasks.workunit.client.0.smithi064.stdout:9/702: link d3/d11/d15/dab/db6/fc8 d3/d11/d15/fe6 0 2024-02-16T22:56:33.705 INFO:tasks.workunit.client.0.smithi064.stdout:2/851: truncate d7/d14/d25/d46/d55/d57/f7b 3163507 0 2024-02-16T22:56:33.705 INFO:tasks.workunit.client.1.smithi192.stdout:6/852: truncate da/df/d6b/d74/ff8 505820 0 2024-02-16T22:56:33.707 INFO:tasks.workunit.client.0.smithi064.stdout:7/709: creat d5/d33/d45/d64/d56/d66/fe0 x:0 0 0 2024-02-16T22:56:33.708 INFO:tasks.workunit.client.1.smithi192.stdout:2/887: creat d22/d47/dca/d11d/f123 x:0 0 0 2024-02-16T22:56:33.709 INFO:tasks.workunit.client.1.smithi192.stdout:1/795: creat d3/de/d27/d34/d52/f10f x:0 0 0 2024-02-16T22:56:33.710 INFO:tasks.workunit.client.0.smithi064.stdout:2/852: dread d7/d14/d1e/d34/d3f/d9a/dd9/d95/f47 [0,4194304] 0 2024-02-16T22:56:33.712 INFO:tasks.workunit.client.0.smithi064.stdout:9/703: symlink d3/d9/d1e/le7 0 2024-02-16T22:56:33.714 INFO:tasks.workunit.client.1.smithi192.stdout:3/771: mkdir da/d107 0 2024-02-16T22:56:33.716 INFO:tasks.workunit.client.0.smithi064.stdout:2/853: mknod d7/d14/d25/d46/d55/d57/dbe/d10e/c115 0 2024-02-16T22:56:33.717 INFO:tasks.workunit.client.0.smithi064.stdout:4/735: write dc/d1d/d43/fae [643842,56677] 0 2024-02-16T22:56:33.719 INFO:tasks.workunit.client.0.smithi064.stdout:9/704: dwrite d3/d9/d1e/f8d [0,4194304] 0 2024-02-16T22:56:33.719 INFO:tasks.workunit.client.1.smithi192.stdout:9/961: dread d7/d15/d36/d38/d3f/fc8 [0,4194304] 0 2024-02-16T22:56:33.720 INFO:tasks.workunit.client.0.smithi064.stdout:4/736: chown dc/d1d/d43/d7a/ld3 50 1 2024-02-16T22:56:33.721 INFO:tasks.workunit.client.1.smithi192.stdout:8/832: write d5/d26/d83/db4/fba [3868556,13282] 0 2024-02-16T22:56:33.724 INFO:tasks.workunit.client.1.smithi192.stdout:7/950: rename d1/d2/d2d/d61/d8a/dbb to d1/d2/de/d142 0 2024-02-16T22:56:33.726 INFO:tasks.workunit.client.0.smithi064.stdout:2/854: creat d7/d14/d25/d46/d55/d102/f116 x:0 0 0 2024-02-16T22:56:33.727 INFO:tasks.workunit.client.1.smithi192.stdout:1/796: rmdir d3/de/d27/d34/d52/d63 39 2024-02-16T22:56:33.732 INFO:tasks.workunit.client.1.smithi192.stdout:3/772: chown da/d6a/lb1 2 1 2024-02-16T22:56:33.733 INFO:tasks.workunit.client.1.smithi192.stdout:4/885: dread d1/d15/d48/f95 [0,4194304] 0 2024-02-16T22:56:33.734 INFO:tasks.workunit.client.1.smithi192.stdout:9/962: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f9a 0 1 2024-02-16T22:56:33.734 INFO:tasks.workunit.client.1.smithi192.stdout:9/963: stat d7/d19/d37/ddf/cfb 0 2024-02-16T22:56:33.735 INFO:tasks.workunit.client.0.smithi064.stdout:4/737: read dc/d1d/fc4 [1457786,54134] 0 2024-02-16T22:56:33.739 INFO:tasks.workunit.client.0.smithi064.stdout:9/705: link d3/d9/d1e/d42/l99 d3/d11/d9e/da5/le8 0 2024-02-16T22:56:33.739 INFO:tasks.workunit.client.0.smithi064.stdout:7/710: write d5/d33/d42/d81/fae [895848,88348] 0 2024-02-16T22:56:33.740 INFO:tasks.workunit.client.0.smithi064.stdout:3/804: sync 2024-02-16T22:56:33.741 INFO:tasks.workunit.client.0.smithi064.stdout:4/738: truncate f3 1105914 0 2024-02-16T22:56:33.742 INFO:tasks.workunit.client.0.smithi064.stdout:4/739: readlink dc/dd/d16/d6e/l72 0 2024-02-16T22:56:33.746 INFO:tasks.workunit.client.1.smithi192.stdout:7/951: dread d1/d2/d2d/f40 [0,4194304] 0 2024-02-16T22:56:33.765 INFO:tasks.workunit.client.1.smithi192.stdout:7/952: dread d1/d2/de/d83/fbc [0,4194304] 0 2024-02-16T22:56:33.765 INFO:tasks.workunit.client.0.smithi064.stdout:4/740: dread dc/d1d/f4c [0,4194304] 0 2024-02-16T22:56:33.766 INFO:tasks.workunit.client.1.smithi192.stdout:7/953: fdatasync d1/d2/de/d2c/d49/d54/dd8/d117/d9b/ffc 0 2024-02-16T22:56:33.766 INFO:tasks.workunit.client.0.smithi064.stdout:4/741: stat dc/dd/d4f/fa0 0 2024-02-16T22:56:33.825 INFO:tasks.workunit.client.1.smithi192.stdout:4/886: creat d1/d26/d6f/df4/f133 x:0 0 0 2024-02-16T22:56:33.825 INFO:tasks.workunit.client.0.smithi064.stdout:3/805: symlink d4/d14/d2b/l10e 0 2024-02-16T22:56:33.826 INFO:tasks.workunit.client.1.smithi192.stdout:4/887: write d1/d26/d34/f76 [4918806,42438] 0 2024-02-16T22:56:33.827 INFO:tasks.workunit.client.0.smithi064.stdout:3/806: dread - d4/d13/d1b/d27/d61/d63/ffb zero size 2024-02-16T22:56:33.827 INFO:tasks.workunit.client.1.smithi192.stdout:9/964: mknod d7/d15/d36/d77/dbd/de0/c146 0 2024-02-16T22:56:33.828 INFO:tasks.workunit.client.1.smithi192.stdout:9/965: stat d7/d19/d37/d5a/l135 0 2024-02-16T22:56:33.828 INFO:tasks.workunit.client.0.smithi064.stdout:3/807: write d4/d1f/d34/ff1 [640467,116961] 0 2024-02-16T22:56:33.829 INFO:tasks.workunit.client.1.smithi192.stdout:9/966: truncate d7/d19/d28/d43/f12a 674143 0 2024-02-16T22:56:33.831 INFO:tasks.workunit.client.0.smithi064.stdout:2/855: link d7/d14/d25/d46/d55/d7f/dde/lf8 d7/d14/d25/dee/l117 0 2024-02-16T22:56:33.835 INFO:tasks.workunit.client.0.smithi064.stdout:4/742: mknod dc/de/d28/d57/d77/ce7 0 2024-02-16T22:56:33.837 INFO:tasks.workunit.client.1.smithi192.stdout:1/797: mknod d3/de/d27/d34/d69/d85/d89/c110 0 2024-02-16T22:56:33.838 INFO:tasks.workunit.client.0.smithi064.stdout:9/706: creat d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/fe9 x:0 0 0 2024-02-16T22:56:33.844 INFO:tasks.workunit.client.1.smithi192.stdout:4/888: dread d1/d26/d5e/f121 [0,4194304] 0 2024-02-16T22:56:33.848 INFO:tasks.workunit.client.0.smithi064.stdout:4/743: rename dc/dd/d32/d46/d82 to dc/ddc/de8 0 2024-02-16T22:56:33.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/853: link da/df/d18/f43 da/d40/db6/f112 0 2024-02-16T22:56:33.849 INFO:tasks.workunit.client.0.smithi064.stdout:2/856: dread d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f6d [0,4194304] 0 2024-02-16T22:56:33.849 INFO:tasks.workunit.client.0.smithi064.stdout:9/707: fdatasync d3/d11/f1f 0 2024-02-16T22:56:33.851 INFO:tasks.workunit.client.1.smithi192.stdout:9/967: creat d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/d139/f147 x:0 0 0 2024-02-16T22:56:33.852 INFO:tasks.workunit.client.0.smithi064.stdout:7/711: creat d5/d33/d45/d64/d56/fe1 x:0 0 0 2024-02-16T22:56:33.853 INFO:tasks.workunit.client.0.smithi064.stdout:7/712: fdatasync d5/d33/d53/f6e 0 2024-02-16T22:56:33.854 INFO:tasks.workunit.client.1.smithi192.stdout:5/840: rename d6/d12/d4e/l6d to d6/d12/d1d/l113 0 2024-02-16T22:56:33.854 INFO:tasks.workunit.client.0.smithi064.stdout:6/781: sync 2024-02-16T22:56:33.856 INFO:tasks.workunit.client.0.smithi064.stdout:3/808: dread d4/d13/d1b/d27/d61/d63/fd3 [0,4194304] 0 2024-02-16T22:56:33.857 INFO:tasks.workunit.client.1.smithi192.stdout:1/798: fsync d3/f46 0 2024-02-16T22:56:33.858 INFO:tasks.workunit.client.0.smithi064.stdout:3/809: truncate d4/d13/d1b/d27/fe7 673749 0 2024-02-16T22:56:33.859 INFO:tasks.workunit.client.0.smithi064.stdout:3/810: fsync d4/d1f/d5b/d51/fa4 0 2024-02-16T22:56:33.864 INFO:tasks.workunit.client.0.smithi064.stdout:4/744: truncate dc/dd/d16/f5e 360632 0 2024-02-16T22:56:33.866 INFO:tasks.workunit.client.0.smithi064.stdout:9/708: dread d3/d11/d15/d23/f73 [0,4194304] 0 2024-02-16T22:56:33.869 INFO:tasks.workunit.client.1.smithi192.stdout:9/968: mknod d7/d19/d5e/d81/c148 0 2024-02-16T22:56:33.869 INFO:tasks.workunit.client.0.smithi064.stdout:7/713: dread f0 [0,4194304] 0 2024-02-16T22:56:33.870 INFO:tasks.workunit.client.0.smithi064.stdout:7/714: chown d5/d6/d54/d41/l7a 126733 1 2024-02-16T22:56:33.873 INFO:tasks.workunit.client.1.smithi192.stdout:0/830: rename d1/d3/de/d22/d37/d47/f93 to d1/d3/dac/f107 0 2024-02-16T22:56:33.875 INFO:tasks.workunit.client.1.smithi192.stdout:3/773: write da/d11/d63/d6b/d4e/fa5 [3582945,7094] 0 2024-02-16T22:56:33.875 INFO:tasks.workunit.client.0.smithi064.stdout:6/782: creat d0/d3c/dc5/f102 x:0 0 0 2024-02-16T22:56:33.875 INFO:tasks.workunit.client.1.smithi192.stdout:5/841: symlink d6/d12/d2d/l114 0 2024-02-16T22:56:33.879 INFO:tasks.workunit.client.0.smithi064.stdout:2/857: symlink d7/d14/d1e/d34/d73/l118 0 2024-02-16T22:56:33.880 INFO:tasks.workunit.client.1.smithi192.stdout:1/799: fdatasync d3/de/d27/d34/d69/f8c 0 2024-02-16T22:56:33.881 INFO:tasks.workunit.client.1.smithi192.stdout:4/889: dread d1/d26/fa5 [0,4194304] 0 2024-02-16T22:56:33.889 INFO:tasks.workunit.client.1.smithi192.stdout:2/888: rename d22/d24/d39/d3d/df2/d99/d9b/cef to d22/d24/d39/d42/d65/c124 0 2024-02-16T22:56:33.889 INFO:tasks.workunit.client.1.smithi192.stdout:2/889: chown d22/d24/d39/d42/d84/fd4 58275 1 2024-02-16T22:56:33.890 INFO:tasks.workunit.client.1.smithi192.stdout:2/890: dread - d22/d24/d39/d42/d84/fc5 zero size 2024-02-16T22:56:33.890 INFO:tasks.workunit.client.1.smithi192.stdout:3/774: creat da/d11/d35/f108 x:0 0 0 2024-02-16T22:56:33.892 INFO:tasks.workunit.client.1.smithi192.stdout:3/775: write da/d11/d20/f69 [3302686,20276] 0 2024-02-16T22:56:33.892 INFO:tasks.workunit.client.1.smithi192.stdout:5/842: chown d6/c94 499 1 2024-02-16T22:56:33.892 INFO:tasks.workunit.client.1.smithi192.stdout:5/843: write d6/f9d [3756238,119943] 0 2024-02-16T22:56:33.893 INFO:tasks.workunit.client.0.smithi064.stdout:7/715: mknod d5/d6/ce2 0 2024-02-16T22:56:33.896 INFO:tasks.workunit.client.0.smithi064.stdout:6/783: symlink d0/d1d/dfa/l103 0 2024-02-16T22:56:33.897 INFO:tasks.workunit.client.1.smithi192.stdout:3/776: dwrite da/d11/dd4/deb/f77 [0,4194304] 0 2024-02-16T22:56:33.897 INFO:tasks.workunit.client.0.smithi064.stdout:6/784: chown d0/d1d/dbc/dec/ffe 16108 1 2024-02-16T22:56:33.898 INFO:tasks.workunit.client.1.smithi192.stdout:7/954: getdents d1/d2/de/d2c/d49/da4/d104 0 2024-02-16T22:56:33.898 INFO:tasks.workunit.client.0.smithi064.stdout:9/709: dwrite d3/d9/f48 [0,4194304] 0 2024-02-16T22:56:33.902 INFO:tasks.workunit.client.0.smithi064.stdout:4/745: rename dc/dd/d32/c25 to dc/de/d28/d57/ce9 0 2024-02-16T22:56:33.905 INFO:tasks.workunit.client.1.smithi192.stdout:8/833: rename d5/d26/f3f to d5/d26/dad/f104 0 2024-02-16T22:56:33.913 INFO:tasks.workunit.client.0.smithi064.stdout:2/858: rename d7/d14/d25/f8a to d7/d14/d1e/dfd/f119 0 2024-02-16T22:56:33.914 INFO:tasks.workunit.client.0.smithi064.stdout:2/859: chown d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f49 710513 1 2024-02-16T22:56:33.915 INFO:tasks.workunit.client.1.smithi192.stdout:4/890: write d1/d26/d66/d89/fd6 [148329,115056] 0 2024-02-16T22:56:33.916 INFO:tasks.workunit.client.1.smithi192.stdout:4/891: stat d1/d15/d48/dc3 0 2024-02-16T22:56:33.916 INFO:tasks.workunit.client.0.smithi064.stdout:2/860: chown d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/fbc 13 1 2024-02-16T22:56:33.918 INFO:tasks.workunit.client.0.smithi064.stdout:7/716: creat d5/d6/d54/fe3 x:0 0 0 2024-02-16T22:56:33.921 INFO:tasks.workunit.client.0.smithi064.stdout:4/746: creat dc/de/d28/d57/d77/dc6/fea x:0 0 0 2024-02-16T22:56:33.922 INFO:tasks.workunit.client.1.smithi192.stdout:3/777: creat da/d11/d2b/ddc/f109 x:0 0 0 2024-02-16T22:56:33.922 INFO:tasks.workunit.client.1.smithi192.stdout:3/778: dread - da/d11/d20/f41 zero size 2024-02-16T22:56:33.923 INFO:tasks.workunit.client.1.smithi192.stdout:3/779: dread - da/d11/d35/f108 zero size 2024-02-16T22:56:33.923 INFO:tasks.workunit.client.0.smithi064.stdout:7/717: unlink d5/f25 0 2024-02-16T22:56:33.927 INFO:tasks.workunit.client.0.smithi064.stdout:6/785: dread d0/d6/d10/f5e [4194304,4194304] 0 2024-02-16T22:56:33.928 INFO:tasks.workunit.client.0.smithi064.stdout:6/786: dread - d0/d1d/da8/d60/dbf/ff9 zero size 2024-02-16T22:56:33.929 INFO:tasks.workunit.client.1.smithi192.stdout:7/955: dread d1/d2/d2d/d61/f74 [0,4194304] 0 2024-02-16T22:56:33.932 INFO:tasks.workunit.client.1.smithi192.stdout:6/854: sync 2024-02-16T22:56:33.933 INFO:tasks.workunit.client.0.smithi064.stdout:7/718: mkdir d5/d33/d53/dde/de4 0 2024-02-16T22:56:33.934 INFO:tasks.workunit.client.0.smithi064.stdout:9/710: write d3/d9/f4a [3706194,47827] 0 2024-02-16T22:56:33.935 INFO:tasks.workunit.client.1.smithi192.stdout:5/844: symlink d6/d12/d4e/d46/l115 0 2024-02-16T22:56:33.935 INFO:tasks.workunit.client.0.smithi064.stdout:9/711: dread - d3/d11/d15/dab/fcc zero size 2024-02-16T22:56:33.937 INFO:tasks.workunit.client.0.smithi064.stdout:9/712: truncate d3/d11/d9e/da9/fd9 40766 0 2024-02-16T22:56:33.939 INFO:tasks.workunit.client.0.smithi064.stdout:7/719: dread d5/d33/f57 [0,4194304] 0 2024-02-16T22:56:33.939 INFO:tasks.workunit.client.1.smithi192.stdout:4/892: unlink d1/d26/d66/d89/d11/d58/lee 0 2024-02-16T22:56:33.944 INFO:tasks.workunit.client.1.smithi192.stdout:0/831: link d1/c74 d1/d3/d39/c108 0 2024-02-16T22:56:33.946 INFO:tasks.workunit.client.0.smithi064.stdout:2/861: getdents d7/d14/d25/d46/d55/d7f 0 2024-02-16T22:56:33.949 INFO:tasks.workunit.client.1.smithi192.stdout:7/956: rmdir d1/d2/d2d/d61/d8a/de6 39 2024-02-16T22:56:33.951 INFO:tasks.workunit.client.1.smithi192.stdout:7/957: dread - d1/d2/de/d2c/d49/d54/d9d/ddd/f12c zero size 2024-02-16T22:56:33.951 INFO:tasks.workunit.client.1.smithi192.stdout:3/780: dwrite da/d11/d20/f4a [0,4194304] 0 2024-02-16T22:56:33.952 INFO:tasks.workunit.client.0.smithi064.stdout:9/713: mkdir d3/d9/d6a/db1/d2a/d2e/da3/dea 0 2024-02-16T22:56:33.953 INFO:tasks.workunit.client.0.smithi064.stdout:7/720: creat d5/d6/d65/fe5 x:0 0 0 2024-02-16T22:56:33.955 INFO:tasks.workunit.client.0.smithi064.stdout:4/747: dread dc/de/d6d/f87 [0,4194304] 0 2024-02-16T22:56:33.955 INFO:tasks.workunit.client.0.smithi064.stdout:9/714: dread d3/d9/f36 [0,4194304] 0 2024-02-16T22:56:33.957 INFO:tasks.workunit.client.0.smithi064.stdout:9/715: chown d3/d11/d15/fe6 21 1 2024-02-16T22:56:33.958 INFO:tasks.workunit.client.1.smithi192.stdout:6/855: read da/df/f1c [2162472,52634] 0 2024-02-16T22:56:33.959 INFO:tasks.workunit.client.1.smithi192.stdout:2/891: dwrite d22/d24/d39/d42/d52/f71 [0,4194304] 0 2024-02-16T22:56:33.961 INFO:tasks.workunit.client.1.smithi192.stdout:1/800: getdents d3/d40/d9f 0 2024-02-16T22:56:33.963 INFO:tasks.workunit.client.1.smithi192.stdout:0/832: symlink d1/dc7/d9c/l109 0 2024-02-16T22:56:33.964 INFO:tasks.workunit.client.1.smithi192.stdout:0/833: write d1/d3/de2/dff/f105 [290992,49925] 0 2024-02-16T22:56:33.964 INFO:tasks.workunit.client.1.smithi192.stdout:9/969: rename d7/d15/d36/d38/d3f/c90 to d7/d15/c149 0 2024-02-16T22:56:33.966 INFO:tasks.workunit.client.0.smithi064.stdout:4/748: dread - dc/dd/d32/fa5 zero size 2024-02-16T22:56:33.969 INFO:tasks.workunit.client.0.smithi064.stdout:6/787: write d0/d6/d10/d33/f2a [4628745,18747] 0 2024-02-16T22:56:33.970 INFO:tasks.workunit.client.0.smithi064.stdout:7/721: dwrite d5/d33/d42/d81/f94 [0,4194304] 0 2024-02-16T22:56:33.971 INFO:tasks.workunit.client.1.smithi192.stdout:7/958: truncate d1/d2/d2d/f40 1377841 0 2024-02-16T22:56:33.972 INFO:tasks.workunit.client.0.smithi064.stdout:6/788: chown d0/d3c/dc5 3859 1 2024-02-16T22:56:33.973 INFO:tasks.workunit.client.1.smithi192.stdout:3/781: symlink da/d11/d20/d39/d53/dc7/l10a 0 2024-02-16T22:56:33.975 INFO:tasks.workunit.client.1.smithi192.stdout:5/845: dwrite d6/d20/fc5 [0,4194304] 0 2024-02-16T22:56:33.979 INFO:tasks.workunit.client.0.smithi064.stdout:2/862: dwrite d7/f54 [4194304,4194304] 0 2024-02-16T22:56:33.980 INFO:tasks.workunit.client.0.smithi064.stdout:9/716: mkdir d3/d9/d6a/db1/d2a/d2e/d31/deb 0 2024-02-16T22:56:33.982 INFO:tasks.workunit.client.0.smithi064.stdout:6/789: symlink d0/d6/d10/d33/d72/de8/l104 0 2024-02-16T22:56:33.983 INFO:tasks.workunit.client.1.smithi192.stdout:4/893: rename d1/d26/d66/d89/da8/dc5/d111/d11f to d1/d26/d66/d89/da8/da9/d134 0 2024-02-16T22:56:33.988 INFO:tasks.workunit.client.1.smithi192.stdout:9/970: creat d7/d15/d36/d77/db1/f14a x:0 0 0 2024-02-16T22:56:33.988 INFO:tasks.workunit.client.1.smithi192.stdout:8/834: getdents d5/db/d47/d43/d67 0 2024-02-16T22:56:33.988 INFO:tasks.workunit.client.0.smithi064.stdout:4/749: truncate dc/d1d/d43/f5f 41462 0 2024-02-16T22:56:33.993 INFO:tasks.workunit.client.0.smithi064.stdout:2/863: unlink d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f6d 0 2024-02-16T22:56:33.996 INFO:tasks.workunit.client.1.smithi192.stdout:3/782: rename da/d11/d63/d6b/d4e/l7f to da/d11/d63/d6b/d80/l10b 0 2024-02-16T22:56:33.997 INFO:tasks.workunit.client.0.smithi064.stdout:4/750: fsync dc/d1d/d43/fb5 0 2024-02-16T22:56:34.002 INFO:tasks.workunit.client.1.smithi192.stdout:0/834: dread d1/d3/de/d22/f30 [0,4194304] 0 2024-02-16T22:56:34.003 INFO:tasks.workunit.client.0.smithi064.stdout:7/722: rename d5/d33/f60 to d5/d6/d54/fe6 0 2024-02-16T22:56:34.004 INFO:tasks.workunit.client.1.smithi192.stdout:6/856: dwrite da/df/d18/fc0 [0,4194304] 0 2024-02-16T22:56:34.004 INFO:tasks.workunit.client.0.smithi064.stdout:7/723: dread - d5/d33/d45/d64/d56/d66/fe0 zero size 2024-02-16T22:56:34.005 INFO:tasks.workunit.client.1.smithi192.stdout:6/857: write da/d40/f109 [808537,58368] 0 2024-02-16T22:56:34.006 INFO:tasks.workunit.client.1.smithi192.stdout:9/971: unlink d7/d19/d5e/d81/fd0 0 2024-02-16T22:56:34.008 INFO:tasks.workunit.client.1.smithi192.stdout:8/835: fdatasync d5/d9/f17 0 2024-02-16T22:56:34.008 INFO:tasks.workunit.client.1.smithi192.stdout:9/972: chown d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/ldb 755377 1 2024-02-16T22:56:34.011 INFO:tasks.workunit.client.1.smithi192.stdout:2/892: creat d22/d24/d39/f125 x:0 0 0 2024-02-16T22:56:34.014 INFO:tasks.workunit.client.1.smithi192.stdout:1/801: rename d3/de/f1a to d3/de/d27/d34/d52/d63/d81/f111 0 2024-02-16T22:56:34.017 INFO:tasks.workunit.client.1.smithi192.stdout:3/783: fsync da/d11/dd4/deb/da1/fac 0 2024-02-16T22:56:34.023 INFO:tasks.workunit.client.0.smithi064.stdout:1/636: sync 2024-02-16T22:56:34.023 INFO:tasks.workunit.client.1.smithi192.stdout:0/835: symlink d1/d3/de/d22/d37/d47/d53/l10a 0 2024-02-16T22:56:34.025 INFO:tasks.workunit.client.0.smithi064.stdout:4/751: mkdir dc/dd/deb 0 2024-02-16T22:56:34.027 INFO:tasks.workunit.client.1.smithi192.stdout:5/846: write d6/d8/fa1 [1058905,63654] 0 2024-02-16T22:56:34.032 INFO:tasks.workunit.client.0.smithi064.stdout:9/717: dwrite d3/d11/d15/d23/f73 [0,4194304] 0 2024-02-16T22:56:34.034 INFO:tasks.workunit.client.1.smithi192.stdout:5/847: dwrite d6/d8/d75/d8c/f10e [0,4194304] 0 2024-02-16T22:56:34.041 INFO:tasks.workunit.client.1.smithi192.stdout:8/836: creat d5/d26/d83/db4/f105 x:0 0 0 2024-02-16T22:56:34.046 INFO:tasks.workunit.client.1.smithi192.stdout:2/893: chown d22/d24/d39/d3d/f40 34231189 1 2024-02-16T22:56:34.048 INFO:tasks.workunit.client.1.smithi192.stdout:4/894: dwrite d1/d15/d23/f5c [0,4194304] 0 2024-02-16T22:56:34.049 INFO:tasks.workunit.client.1.smithi192.stdout:1/802: fdatasync d3/de/d27/d34/d36/d47/faf 0 2024-02-16T22:56:34.051 INFO:tasks.workunit.client.1.smithi192.stdout:1/803: chown d3/de/da4 4 1 2024-02-16T22:56:34.051 INFO:tasks.workunit.client.0.smithi064.stdout:6/790: rename d0/d6/da/d56 to d0/d6/d10/d105 0 2024-02-16T22:56:34.052 INFO:tasks.workunit.client.0.smithi064.stdout:2/864: getdents d7/d14/d1e/d34/dad/d10c 0 2024-02-16T22:56:34.054 INFO:tasks.workunit.client.1.smithi192.stdout:3/784: creat da/d11/d2b/ddc/f10c x:0 0 0 2024-02-16T22:56:34.060 INFO:tasks.workunit.client.1.smithi192.stdout:5/848: chown d6/c3d 69 1 2024-02-16T22:56:34.061 INFO:tasks.workunit.client.0.smithi064.stdout:7/724: dread d5/d33/d45/d64/f91 [8388608,4194304] 0 2024-02-16T22:56:34.063 INFO:tasks.workunit.client.1.smithi192.stdout:8/837: mknod d5/d68/dee/c106 0 2024-02-16T22:56:34.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/894: mknod d22/d24/d39/d42/d85/d98/c126 0 2024-02-16T22:56:34.067 INFO:tasks.workunit.client.1.smithi192.stdout:5/849: dread d6/d8/d77/f85 [0,4194304] 0 2024-02-16T22:56:34.067 INFO:tasks.workunit.client.1.smithi192.stdout:4/895: rmdir d1/d15 39 2024-02-16T22:56:34.067 INFO:tasks.workunit.client.1.smithi192.stdout:5/850: stat d6/d12/d4e/d58/d8f 0 2024-02-16T22:56:34.069 INFO:tasks.workunit.client.1.smithi192.stdout:1/804: rmdir d3/de/d27/d34 39 2024-02-16T22:56:34.070 INFO:tasks.workunit.client.1.smithi192.stdout:3/785: write da/df/f49 [2690192,88258] 0 2024-02-16T22:56:34.073 INFO:tasks.workunit.client.1.smithi192.stdout:5/851: dwrite d6/d12/d4e/d5b/fed [0,4194304] 0 2024-02-16T22:56:34.074 INFO:tasks.workunit.client.1.smithi192.stdout:5/852: write d6/d12/f18 [4783072,13270] 0 2024-02-16T22:56:34.078 INFO:tasks.workunit.client.1.smithi192.stdout:8/838: read d5/d26/d83/d86/f6e [74775,68719] 0 2024-02-16T22:56:34.080 INFO:tasks.workunit.client.1.smithi192.stdout:7/959: sync 2024-02-16T22:56:34.080 INFO:tasks.workunit.client.1.smithi192.stdout:9/973: rename d7/d15/d36/d77/dbd/de0/l103 to d7/d19/d28/l14b 0 2024-02-16T22:56:34.090 INFO:tasks.workunit.client.0.smithi064.stdout:9/718: dwrite d3/d9/f36 [0,4194304] 0 2024-02-16T22:56:34.091 INFO:tasks.workunit.client.1.smithi192.stdout:6/858: getdents da/df/d12/d57 0 2024-02-16T22:56:34.094 INFO:tasks.workunit.client.0.smithi064.stdout:9/719: write d3/d9/d6a/db1/f26 [995699,121145] 0 2024-02-16T22:56:34.097 INFO:tasks.workunit.client.0.smithi064.stdout:9/720: dread d3/d9/d1e/d42/d7b/fa8 [0,4194304] 0 2024-02-16T22:56:34.098 INFO:tasks.workunit.client.1.smithi192.stdout:5/853: chown d6/d12/d4e/d46/d4f/c68 177826380 1 2024-02-16T22:56:34.101 INFO:tasks.workunit.client.1.smithi192.stdout:0/836: truncate d1/d3/de/d22/d37/d67/f66 134822 0 2024-02-16T22:56:34.103 INFO:tasks.workunit.client.1.smithi192.stdout:8/839: rmdir d5/db/de/df 39 2024-02-16T22:56:34.107 INFO:tasks.workunit.client.1.smithi192.stdout:0/837: dwrite d1/dc7/f8b [0,4194304] 0 2024-02-16T22:56:34.108 INFO:tasks.workunit.client.1.smithi192.stdout:0/838: stat d1/d3/de/d22/fe5 0 2024-02-16T22:56:34.109 INFO:tasks.workunit.client.1.smithi192.stdout:9/974: symlink d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/d139/l14c 0 2024-02-16T22:56:34.111 INFO:tasks.workunit.client.1.smithi192.stdout:8/840: dread d5/db/de/ff0 [0,4194304] 0 2024-02-16T22:56:34.112 INFO:tasks.workunit.client.1.smithi192.stdout:3/786: mknod da/d11/dd4/deb/da1/db9/ded/c10d 0 2024-02-16T22:56:34.115 INFO:tasks.workunit.client.1.smithi192.stdout:1/805: dwrite d3/d5b/fc0 [0,4194304] 0 2024-02-16T22:56:34.116 INFO:tasks.workunit.client.1.smithi192.stdout:5/854: stat d6/d12/d1d/c97 0 2024-02-16T22:56:34.125 INFO:tasks.workunit.client.1.smithi192.stdout:7/960: creat d1/d2/d2d/ddf/d113/f143 x:0 0 0 2024-02-16T22:56:34.127 INFO:tasks.workunit.client.0.smithi064.stdout:4/752: creat dc/dd/d32/d46/d5a/d7e/d9a/da2/fec x:0 0 0 2024-02-16T22:56:34.129 INFO:tasks.workunit.client.1.smithi192.stdout:0/839: rename d1/d3/de/d22/d37/d67/c3a to d1/d3/d18/c10b 0 2024-02-16T22:56:34.130 INFO:tasks.workunit.client.0.smithi064.stdout:7/725: read d5/d6/d65/d7f/f9c [67744,24791] 0 2024-02-16T22:56:34.132 INFO:tasks.workunit.client.1.smithi192.stdout:4/896: dwrite d1/d15/d48/d8d/d9b/f108 [0,4194304] 0 2024-02-16T22:56:34.133 INFO:tasks.workunit.client.1.smithi192.stdout:6/859: symlink da/df/l113 0 2024-02-16T22:56:34.136 INFO:tasks.workunit.client.0.smithi064.stdout:9/721: truncate d3/fc 3231361 0 2024-02-16T22:56:34.136 INFO:tasks.workunit.client.1.smithi192.stdout:5/855: write d6/d12/d1d/fb8 [26788,70976] 0 2024-02-16T22:56:34.137 INFO:tasks.workunit.client.1.smithi192.stdout:2/895: getdents d22/d24/d39/d3d/df2/d67 0 2024-02-16T22:56:34.137 INFO:tasks.workunit.client.0.smithi064.stdout:9/722: write d3/d11/f1f [3850327,65285] 0 2024-02-16T22:56:34.139 INFO:tasks.workunit.client.0.smithi064.stdout:6/791: mkdir d0/d5/df/d17/ded/d106 0 2024-02-16T22:56:34.139 INFO:tasks.workunit.client.1.smithi192.stdout:1/806: read d3/de/d27/d34/d36/d47/faf [3776673,53754] 0 2024-02-16T22:56:34.140 INFO:tasks.workunit.client.0.smithi064.stdout:9/723: chown d3/d9/d1e/fad 0 1 2024-02-16T22:56:34.141 INFO:tasks.workunit.client.1.smithi192.stdout:6/860: dwrite da/df/d12/d15/d1b/f97 [4194304,4194304] 0 2024-02-16T22:56:34.143 INFO:tasks.workunit.client.0.smithi064.stdout:4/753: truncate dc/dd/d32/d58/f8e 9907 0 2024-02-16T22:56:34.144 INFO:tasks.workunit.client.1.smithi192.stdout:9/975: mkdir d7/d140/d14d 0 2024-02-16T22:56:34.149 INFO:tasks.workunit.client.0.smithi064.stdout:7/726: mkdir d5/d6/d54/d7b/de7 0 2024-02-16T22:56:34.151 INFO:tasks.workunit.client.0.smithi064.stdout:1/637: truncate d2/df/d11/d1a/d1d/f4d 8324152 0 2024-02-16T22:56:34.151 INFO:tasks.workunit.client.0.smithi064.stdout:1/638: chown d2/l7 192417 1 2024-02-16T22:56:34.152 INFO:tasks.workunit.client.1.smithi192.stdout:3/787: dread da/d11/d63/d6b/d66/f92 [0,4194304] 0 2024-02-16T22:56:34.152 INFO:tasks.workunit.client.0.smithi064.stdout:1/639: readlink d2/df/d11/d25/l9d 0 2024-02-16T22:56:34.155 INFO:tasks.workunit.client.1.smithi192.stdout:0/840: symlink d1/d3/d18/l10c 0 2024-02-16T22:56:34.158 INFO:tasks.workunit.client.1.smithi192.stdout:4/897: fsync d1/d26/d66/d89/f97 0 2024-02-16T22:56:34.159 INFO:tasks.workunit.client.1.smithi192.stdout:4/898: stat d1/d26/dbe/fc4 0 2024-02-16T22:56:34.160 INFO:tasks.workunit.client.0.smithi064.stdout:2/865: link d7/d14/d1e/d34/d73/fba d7/d14/d25/daf/de9/f11a 0 2024-02-16T22:56:34.162 INFO:tasks.workunit.client.0.smithi064.stdout:7/727: mknod d5/d6/d54/d41/da2/ce8 0 2024-02-16T22:56:34.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/856: readlink d6/d8/d77/l101 0 2024-02-16T22:56:34.163 INFO:tasks.workunit.client.0.smithi064.stdout:1/640: symlink d2/df/d11/d1a/d30/dbd/d91/lda 0 2024-02-16T22:56:34.164 INFO:tasks.workunit.client.1.smithi192.stdout:2/896: creat d22/d24/d39/d42/d85/d98/dc4/f127 x:0 0 0 2024-02-16T22:56:34.165 INFO:tasks.workunit.client.0.smithi064.stdout:6/792: creat d0/d1d/d21/d98/f107 x:0 0 0 2024-02-16T22:56:34.166 INFO:tasks.workunit.client.0.smithi064.stdout:2/866: dwrite d7/d14/d1e/f81 [0,4194304] 0 2024-02-16T22:56:34.168 INFO:tasks.workunit.client.0.smithi064.stdout:6/793: write d0/d5/df/d17/ff2 [812661,47653] 0 2024-02-16T22:56:34.168 INFO:tasks.workunit.client.0.smithi064.stdout:9/724: mkdir d3/d9/dec 0 2024-02-16T22:56:34.168 INFO:tasks.workunit.client.0.smithi064.stdout:2/867: dread - d7/d14/d1e/d34/d3f/d9a/ffe zero size 2024-02-16T22:56:34.169 INFO:tasks.workunit.client.1.smithi192.stdout:6/861: rename da/db9/ff9 to da/df/d12/d57/daf/dcf/f114 0 2024-02-16T22:56:34.169 INFO:tasks.workunit.client.0.smithi064.stdout:9/725: stat d3/d11/d9e/da9/cc1 0 2024-02-16T22:56:34.171 INFO:tasks.workunit.client.0.smithi064.stdout:9/726: dread - d3/d9/d6a/db1/fd0 zero size 2024-02-16T22:56:34.171 INFO:tasks.workunit.client.0.smithi064.stdout:9/727: stat d3/d9/d6a/db1 0 2024-02-16T22:56:34.172 INFO:tasks.workunit.client.0.smithi064.stdout:2/868: write d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/dd6/f10b [1409601,104475] 0 2024-02-16T22:56:34.177 INFO:tasks.workunit.client.0.smithi064.stdout:1/641: fdatasync d2/df/d11/d1a/d1d/f24 0 2024-02-16T22:56:34.179 INFO:tasks.workunit.client.0.smithi064.stdout:4/754: link dc/dd/d32/d46/d5a/f66 dc/dd/deb/fed 0 2024-02-16T22:56:34.181 INFO:tasks.workunit.client.1.smithi192.stdout:3/788: creat da/d11/d20/d39/dca/f10e x:0 0 0 2024-02-16T22:56:34.186 INFO:tasks.workunit.client.1.smithi192.stdout:3/789: dread da/d11/d35/f70 [0,4194304] 0 2024-02-16T22:56:34.187 INFO:tasks.workunit.client.0.smithi064.stdout:9/728: creat d3/d9/d6a/db1/d2a/fed x:0 0 0 2024-02-16T22:56:34.188 INFO:tasks.workunit.client.1.smithi192.stdout:8/841: write d5/db/d47/fd4 [322607,22473] 0 2024-02-16T22:56:34.191 INFO:tasks.workunit.client.1.smithi192.stdout:8/842: stat d5/d9/fc9 0 2024-02-16T22:56:34.191 INFO:tasks.workunit.client.1.smithi192.stdout:0/841: dwrite d1/d3/dac/f107 [0,4194304] 0 2024-02-16T22:56:34.192 INFO:tasks.workunit.client.0.smithi064.stdout:5/684: sync 2024-02-16T22:56:34.192 INFO:tasks.workunit.client.0.smithi064.stdout:1/642: stat d2/df/d11/d1a/d30/dbd/lc0 0 2024-02-16T22:56:34.200 INFO:tasks.workunit.client.1.smithi192.stdout:2/897: dread d22/d24/fa0 [0,4194304] 0 2024-02-16T22:56:34.200 INFO:tasks.workunit.client.0.smithi064.stdout:9/729: mkdir d3/d11/d15/d23/dee 0 2024-02-16T22:56:34.201 INFO:tasks.workunit.client.0.smithi064.stdout:9/730: truncate d3/d9/d1e/fad 842911 0 2024-02-16T22:56:34.203 INFO:tasks.workunit.client.0.smithi064.stdout:2/869: mkdir d7/d14/d1e/d34/d3f/d9a/dd9/df7/d11b 0 2024-02-16T22:56:34.204 INFO:tasks.workunit.client.0.smithi064.stdout:2/870: fsync d7/d14/d1e/d34/d3f/d9a/ffe 0 2024-02-16T22:56:34.205 INFO:tasks.workunit.client.0.smithi064.stdout:5/685: fdatasync d6/d25/d48/d67/d91/da5/de3/f54 0 2024-02-16T22:56:34.206 INFO:tasks.workunit.client.1.smithi192.stdout:6/862: dread da/d40/d85/f87 [0,4194304] 0 2024-02-16T22:56:34.207 INFO:tasks.workunit.client.1.smithi192.stdout:7/961: link d1/faf d1/d2/de/d2c/d49/d54/dd8/d117/dcd/f144 0 2024-02-16T22:56:34.211 INFO:tasks.workunit.client.1.smithi192.stdout:3/790: read da/f12 [1456086,119420] 0 2024-02-16T22:56:34.216 INFO:tasks.workunit.client.1.smithi192.stdout:5/857: mknod d6/d12/d2d/d5c/df6/dbc/d102/c116 0 2024-02-16T22:56:34.218 INFO:tasks.workunit.client.1.smithi192.stdout:8/843: mkdir d5/d16/dd5/d9d/d107 0 2024-02-16T22:56:34.219 INFO:tasks.workunit.client.1.smithi192.stdout:1/807: creat d3/de/d27/d34/f112 x:0 0 0 2024-02-16T22:56:34.221 INFO:tasks.workunit.client.1.smithi192.stdout:2/898: unlink d22/d24/d39/d3d/ffd 0 2024-02-16T22:56:34.222 INFO:tasks.workunit.client.1.smithi192.stdout:9/976: creat d7/d15/d36/d38/f14e x:0 0 0 2024-02-16T22:56:34.222 INFO:tasks.workunit.client.1.smithi192.stdout:3/791: dread da/f12 [0,4194304] 0 2024-02-16T22:56:34.224 INFO:tasks.workunit.client.1.smithi192.stdout:9/977: readlink d7/d15/d36/d38/d3f/d8a/d98/d6e/l115 0 2024-02-16T22:56:34.226 INFO:tasks.workunit.client.1.smithi192.stdout:5/858: fdatasync d6/d12/d4e/d5b/daf/fbe 0 2024-02-16T22:56:34.226 INFO:tasks.workunit.client.1.smithi192.stdout:8/844: dread d5/f25 [0,4194304] 0 2024-02-16T22:56:34.230 INFO:tasks.workunit.client.1.smithi192.stdout:1/808: dread - d3/de/d27/d34/d36/d47/d5d/fd3 zero size 2024-02-16T22:56:34.230 INFO:tasks.workunit.client.1.smithi192.stdout:3/792: dwrite da/d11/d63/d6b/d66/f60 [0,4194304] 0 2024-02-16T22:56:34.232 INFO:tasks.workunit.client.1.smithi192.stdout:2/899: creat d22/d24/d39/d42/d84/f128 x:0 0 0 2024-02-16T22:56:34.233 INFO:tasks.workunit.client.1.smithi192.stdout:4/899: getdents d1/d26/d66/d89/da8/dc5 0 2024-02-16T22:56:34.234 INFO:tasks.workunit.client.1.smithi192.stdout:6/863: dread da/df/d12/d15/d1b/faa [0,4194304] 0 2024-02-16T22:56:34.235 INFO:tasks.workunit.client.1.smithi192.stdout:9/978: fsync d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/f9a 0 2024-02-16T22:56:34.236 INFO:tasks.workunit.client.1.smithi192.stdout:8/845: chown d5/le3 155 1 2024-02-16T22:56:34.237 INFO:tasks.workunit.client.1.smithi192.stdout:0/842: truncate d1/d3/dac/f107 2660992 0 2024-02-16T22:56:34.238 INFO:tasks.workunit.client.1.smithi192.stdout:1/809: mkdir d3/de/d27/d34/d52/d63/d7a/dc9/d113 0 2024-02-16T22:56:34.238 INFO:tasks.workunit.client.1.smithi192.stdout:4/900: read - d1/d26/d66/ff7 zero size 2024-02-16T22:56:34.239 INFO:tasks.workunit.client.1.smithi192.stdout:1/810: stat d3/de/d27/d34/le4 0 2024-02-16T22:56:34.240 INFO:tasks.workunit.client.1.smithi192.stdout:2/900: fsync d22/d24/d39/d42/d65/d69/db1/fdd 0 2024-02-16T22:56:34.241 INFO:tasks.workunit.client.1.smithi192.stdout:6/864: read da/d40/f54 [520639,52211] 0 2024-02-16T22:56:34.243 INFO:tasks.workunit.client.1.smithi192.stdout:0/843: mkdir d1/d3/de/d22/da7/d10d 0 2024-02-16T22:56:34.244 INFO:tasks.workunit.client.1.smithi192.stdout:4/901: truncate d1/d26/d5e/fc2 806452 0 2024-02-16T22:56:34.245 INFO:tasks.workunit.client.1.smithi192.stdout:2/901: truncate f4 3864566 0 2024-02-16T22:56:34.246 INFO:tasks.workunit.client.1.smithi192.stdout:2/902: write d22/d24/d39/d42/d65/d69/fa1 [279507,102516] 0 2024-02-16T22:56:34.246 INFO:tasks.workunit.client.1.smithi192.stdout:5/859: rename d6/d8/la to d6/d12/d1d/l117 0 2024-02-16T22:56:34.248 INFO:tasks.workunit.client.1.smithi192.stdout:0/844: creat d1/d3/de/d22/d37/d47/f10e x:0 0 0 2024-02-16T22:56:34.250 INFO:tasks.workunit.client.1.smithi192.stdout:5/860: mknod d6/d8/d75/d8c/dc4/c118 0 2024-02-16T22:56:34.251 INFO:tasks.workunit.client.1.smithi192.stdout:2/903: truncate d22/d24/d39/d42/d65/d66/fd6 452900 0 2024-02-16T22:56:34.252 INFO:tasks.workunit.client.1.smithi192.stdout:5/861: creat d6/d12/d4e/f119 x:0 0 0 2024-02-16T22:56:34.254 INFO:tasks.workunit.client.1.smithi192.stdout:2/904: symlink d22/d24/d39/d42/d85/d98/db0/db6/l129 0 2024-02-16T22:56:34.254 INFO:tasks.workunit.client.1.smithi192.stdout:5/862: dread d6/d12/d4e/faa [0,4194304] 0 2024-02-16T22:56:34.263 INFO:tasks.workunit.client.1.smithi192.stdout:1/811: write d3/de/f35 [7331644,90663] 0 2024-02-16T22:56:34.267 INFO:tasks.workunit.client.1.smithi192.stdout:1/812: creat d3/d5b/db1/f114 x:0 0 0 2024-02-16T22:56:34.267 INFO:tasks.workunit.client.1.smithi192.stdout:0/845: dread d1/d3/f12 [0,4194304] 0 2024-02-16T22:56:34.268 INFO:tasks.workunit.client.1.smithi192.stdout:4/902: write d1/d26/d66/d89/fc [496528,80528] 0 2024-02-16T22:56:34.270 INFO:tasks.workunit.client.1.smithi192.stdout:9/979: dwrite d7/f8 [0,4194304] 0 2024-02-16T22:56:34.271 INFO:tasks.workunit.client.1.smithi192.stdout:2/905: write d22/d47/f88 [738698,11325] 0 2024-02-16T22:56:34.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/863: write d6/d12/d2d/d5c/df6/dbc/fd8 [1423612,70493] 0 2024-02-16T22:56:34.278 INFO:tasks.workunit.client.1.smithi192.stdout:0/846: getdents d1/d3/de2 0 2024-02-16T22:56:34.278 INFO:tasks.workunit.client.1.smithi192.stdout:6/865: dwrite da/df/d58/d72/fbe [0,4194304] 0 2024-02-16T22:56:34.279 INFO:tasks.workunit.client.1.smithi192.stdout:0/847: readlink d1/d3/d18/l5a 0 2024-02-16T22:56:34.280 INFO:tasks.workunit.client.1.smithi192.stdout:1/813: dread d3/de/f65 [0,4194304] 0 2024-02-16T22:56:34.280 INFO:tasks.workunit.client.1.smithi192.stdout:2/906: mkdir d22/d24/d39/d3d/df2/d12a 0 2024-02-16T22:56:34.284 INFO:tasks.workunit.client.1.smithi192.stdout:6/866: truncate da/df/d12/d15/f102 447952 0 2024-02-16T22:56:34.284 INFO:tasks.workunit.client.1.smithi192.stdout:2/907: chown d22/d24/d39/d42/d84/fd4 0 1 2024-02-16T22:56:34.287 INFO:tasks.workunit.client.1.smithi192.stdout:0/848: creat d1/d3/de/d22/d37/f10f x:0 0 0 2024-02-16T22:56:34.287 INFO:tasks.workunit.client.1.smithi192.stdout:5/864: getdents d6/d12/d2d/d5c/df6 0 2024-02-16T22:56:34.287 INFO:tasks.workunit.client.1.smithi192.stdout:6/867: rmdir da/df/d18/d52/da9 39 2024-02-16T22:56:34.287 INFO:tasks.workunit.client.1.smithi192.stdout:9/980: dwrite d7/d15/d36/d38/d3f/d8a/d98/d6e/d92/fa1 [4194304,4194304] 0 2024-02-16T22:56:34.288 INFO:tasks.workunit.client.1.smithi192.stdout:6/868: chown da/lb4 7321 1 2024-02-16T22:56:34.291 INFO:tasks.workunit.client.1.smithi192.stdout:5/865: mkdir d6/d8/d77/d11a 0 2024-02-16T22:56:34.291 INFO:tasks.workunit.client.1.smithi192.stdout:5/866: chown d6/d12/d4e/d10c 7343346 1 2024-02-16T22:56:34.293 INFO:tasks.workunit.client.1.smithi192.stdout:2/908: rename d22/d24/d39/d3d/df2/f38 to d22/d24/d39/d42/d65/d66/f12b 0 2024-02-16T22:56:34.294 INFO:tasks.workunit.client.1.smithi192.stdout:6/869: chown da/d76/lac 196026472 1 2024-02-16T22:56:34.295 INFO:tasks.workunit.client.1.smithi192.stdout:2/909: write d22/d24/d39/d42/d85/d98/fdb [146759,82257] 0 2024-02-16T22:56:34.296 INFO:tasks.workunit.client.1.smithi192.stdout:9/981: creat d7/d19/d37/dbf/d10b/f14f x:0 0 0 2024-02-16T22:56:34.296 INFO:tasks.workunit.client.1.smithi192.stdout:9/982: dread - d7/d15/d36/d77/db1/fc4 zero size 2024-02-16T22:56:34.297 INFO:tasks.workunit.client.0.smithi064.stdout:1/643: rmdir d2/df/d11/d1a/d30/d3d 39 2024-02-16T22:56:34.298 INFO:tasks.workunit.client.0.smithi064.stdout:6/794: link d0/d6/d10/d33/l6e d0/d1d/d21/l108 0 2024-02-16T22:56:34.298 INFO:tasks.workunit.client.0.smithi064.stdout:4/755: dread dc/d1d/f36 [0,4194304] 0 2024-02-16T22:56:34.300 INFO:tasks.workunit.client.1.smithi192.stdout:0/849: rename d1/d3/l55 to d1/d3/de2/l110 0 2024-02-16T22:56:34.301 INFO:tasks.workunit.client.0.smithi064.stdout:2/871: mkdir d7/d14/d25/d11c 0 2024-02-16T22:56:34.303 INFO:tasks.workunit.client.0.smithi064.stdout:9/731: write d3/d9/d1e/d42/f58 [3604053,105814] 0 2024-02-16T22:56:34.305 INFO:tasks.workunit.client.0.smithi064.stdout:5/686: truncate d6/d25/d48/d67/d91/da5/de3/d43/db9/fd8 4589509 0 2024-02-16T22:56:34.306 INFO:tasks.workunit.client.1.smithi192.stdout:4/903: truncate d1/d15/d48/d8d/def/f129 551025 0 2024-02-16T22:56:34.307 INFO:tasks.workunit.client.0.smithi064.stdout:1/644: dwrite d2/df/d11/d25/f85 [0,4194304] 0 2024-02-16T22:56:34.312 INFO:tasks.workunit.client.1.smithi192.stdout:9/983: mknod d7/d140/d14d/c150 0 2024-02-16T22:56:34.314 INFO:tasks.workunit.client.0.smithi064.stdout:4/756: rename dc/dd/deb to dc/de/d6d/dee 0 2024-02-16T22:56:34.315 INFO:tasks.workunit.client.0.smithi064.stdout:4/757: chown dc/d1d/d43/c91 8520 1 2024-02-16T22:56:34.316 INFO:tasks.workunit.client.0.smithi064.stdout:2/872: dread - d7/d14/d1e/d34/dad/fc1 zero size 2024-02-16T22:56:34.317 INFO:tasks.workunit.client.0.smithi064.stdout:2/873: fsync d7/d14/d1e/f81 0 2024-02-16T22:56:34.317 INFO:tasks.workunit.client.1.smithi192.stdout:9/984: truncate d7/d19/d37/d5a/fdd 1018992 0 2024-02-16T22:56:34.318 INFO:tasks.workunit.client.0.smithi064.stdout:5/687: creat d6/d18/d95/fea x:0 0 0 2024-02-16T22:56:34.319 INFO:tasks.workunit.client.0.smithi064.stdout:2/874: chown d7/d14/d1e/d34/d3f/d9a/dd9/led 2441400 1 2024-02-16T22:56:34.320 INFO:tasks.workunit.client.0.smithi064.stdout:1/645: creat d2/df/d11/d1a/d1d/fdb x:0 0 0 2024-02-16T22:56:34.320 INFO:tasks.workunit.client.1.smithi192.stdout:2/910: dread d22/d47/dca/dfa/f10f [0,4194304] 0 2024-02-16T22:56:34.322 INFO:tasks.workunit.client.1.smithi192.stdout:2/911: fsync d22/d24/d39/d42/d85/d98/db0/fc1 0 2024-02-16T22:56:34.324 INFO:tasks.workunit.client.0.smithi064.stdout:4/758: dread dc/dd/d32/d22/f93 [0,4194304] 0 2024-02-16T22:56:34.325 INFO:tasks.workunit.client.1.smithi192.stdout:6/870: write da/d40/f54 [4776646,125622] 0 2024-02-16T22:56:34.327 INFO:tasks.workunit.client.0.smithi064.stdout:9/732: symlink d3/d9/d6a/db1/d2a/d2e/d31/lef 0 2024-02-16T22:56:34.328 INFO:tasks.workunit.client.0.smithi064.stdout:9/733: chown d3/d9/d6a/db1/fd0 53 1 2024-02-16T22:56:34.328 INFO:tasks.workunit.client.1.smithi192.stdout:5/867: dwrite d6/d12/d2d/d5c/f99 [0,4194304] 0 2024-02-16T22:56:34.330 INFO:tasks.workunit.client.1.smithi192.stdout:2/912: dwrite d22/d24/d39/d3d/f10d [0,4194304] 0 2024-02-16T22:56:34.331 INFO:tasks.workunit.client.0.smithi064.stdout:6/795: write d0/d5/df/d17/f62 [1208293,83948] 0 2024-02-16T22:56:34.332 INFO:tasks.workunit.client.0.smithi064.stdout:1/646: dread - d2/df/d14/d18/fb6 zero size 2024-02-16T22:56:34.334 INFO:tasks.workunit.client.1.smithi192.stdout:6/871: creat da/d40/d85/f115 x:0 0 0 2024-02-16T22:56:34.338 INFO:tasks.workunit.client.1.smithi192.stdout:2/913: creat d22/d24/d39/d42/d65/d66/f12c x:0 0 0 2024-02-16T22:56:34.340 INFO:tasks.workunit.client.1.smithi192.stdout:5/868: dread - d6/d12/d1d/d56/fde zero size 2024-02-16T22:56:34.342 INFO:tasks.workunit.client.0.smithi064.stdout:2/875: mknod d7/d14/d1e/d34/dad/d10c/df1/c11d 0 2024-02-16T22:56:34.345 INFO:tasks.workunit.client.0.smithi064.stdout:1/647: symlink d2/df/d14/d18/d48/d6b/ldc 0 2024-02-16T22:56:34.346 INFO:tasks.workunit.client.0.smithi064.stdout:4/759: truncate dc/dd/d32/d46/d79/fc7 3326886 0 2024-02-16T22:56:34.347 INFO:tasks.workunit.client.0.smithi064.stdout:2/876: mknod d7/d14/d1e/d34/d3f/d9a/dd9/df7/c11e 0 2024-02-16T22:56:34.350 INFO:tasks.workunit.client.0.smithi064.stdout:6/796: dwrite d0/d3c/fdf [0,4194304] 0 2024-02-16T22:56:34.350 INFO:tasks.workunit.client.0.smithi064.stdout:5/688: creat d6/d25/da4/dc1/d8e/feb x:0 0 0 2024-02-16T22:56:34.357 INFO:tasks.workunit.client.0.smithi064.stdout:4/760: dread dc/dd/d32/d58/f38 [0,4194304] 0 2024-02-16T22:56:34.360 INFO:tasks.workunit.client.0.smithi064.stdout:2/877: link d7/d14/d1e/d34/d73/fba d7/f11f 0 2024-02-16T22:56:34.363 INFO:tasks.workunit.client.0.smithi064.stdout:2/878: creat d7/d14/d25/d46/d55/d7f/dde/d10d/f120 x:0 0 0 2024-02-16T22:56:34.365 INFO:tasks.workunit.client.0.smithi064.stdout:2/879: mkdir d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/d121 0 2024-02-16T22:56:34.365 INFO:tasks.workunit.client.0.smithi064.stdout:2/880: chown d7/l6e 52977 1 2024-02-16T22:56:34.370 INFO:tasks.workunit.client.0.smithi064.stdout:2/881: dwrite d7/d14/d25/d46/d55/d7f/faa [0,4194304] 0 2024-02-16T22:56:34.374 INFO:tasks.workunit.client.0.smithi064.stdout:2/882: mkdir d7/d14/d25/d46/d55/d57/dbe/d122 0 2024-02-16T22:56:34.375 INFO:tasks.workunit.client.0.smithi064.stdout:2/883: write d7/f10 [9089358,40152] 0 2024-02-16T22:56:34.376 INFO:tasks.workunit.client.0.smithi064.stdout:9/734: write d3/d9/d6a/db1/d2a/d2e/f5f [888753,37817] 0 2024-02-16T22:56:34.379 INFO:tasks.workunit.client.1.smithi192.stdout:9/985: write d7/d19/d37/d39/f10c [1047736,63653] 0 2024-02-16T22:56:34.380 INFO:tasks.workunit.client.1.smithi192.stdout:3/793: sync 2024-02-16T22:56:34.380 INFO:tasks.workunit.client.1.smithi192.stdout:8/846: sync 2024-02-16T22:56:34.380 INFO:tasks.workunit.client.1.smithi192.stdout:7/962: sync 2024-02-16T22:56:34.382 INFO:tasks.workunit.client.0.smithi064.stdout:9/735: read d3/d9/d1e/d3e/d76/f79 [50026,95313] 0 2024-02-16T22:56:34.384 INFO:tasks.workunit.client.0.smithi064.stdout:9/736: dread d3/d9/d6a/db1/d2a/fc9 [0,4194304] 0 2024-02-16T22:56:34.384 INFO:tasks.workunit.client.1.smithi192.stdout:0/850: dwrite d1/d3/de/d22/d37/d67/f66 [0,4194304] 0 2024-02-16T22:56:34.384 INFO:tasks.workunit.client.1.smithi192.stdout:2/914: dwrite d22/d47/dcb/d3f/dac/fe8 [0,4194304] 0 2024-02-16T22:56:34.385 INFO:tasks.workunit.client.0.smithi064.stdout:3/811: sync 2024-02-16T22:56:34.385 INFO:tasks.workunit.client.0.smithi064.stdout:8/658: sync 2024-02-16T22:56:34.385 INFO:tasks.workunit.client.0.smithi064.stdout:0/633: sync 2024-02-16T22:56:34.387 INFO:tasks.workunit.client.1.smithi192.stdout:9/986: write d7/d19/d28/d43/d70/df0/fb6 [2579932,86744] 0 2024-02-16T22:56:34.388 INFO:tasks.workunit.client.0.smithi064.stdout:0/634: fdatasync da/dc/de/fc8 0 2024-02-16T22:56:34.391 INFO:tasks.workunit.client.1.smithi192.stdout:0/851: chown d1/d3/de/d22/da7/ff1 25589054 1 2024-02-16T22:56:34.396 INFO:tasks.workunit.client.1.smithi192.stdout:7/963: dwrite d1/d2/de/d2c/d49/d89/ffa [0,4194304] 0 2024-02-16T22:56:34.399 INFO:tasks.workunit.client.1.smithi192.stdout:7/964: chown d1/de1/fb8 18578 1 2024-02-16T22:56:34.400 INFO:tasks.workunit.client.0.smithi064.stdout:1/648: dwrite d2/df/d11/d1a/d1d/fca [0,4194304] 0 2024-02-16T22:56:34.402 INFO:tasks.workunit.client.0.smithi064.stdout:8/659: creat df/d31/d8f/d80/fe5 x:0 0 0 2024-02-16T22:56:34.405 INFO:tasks.workunit.client.0.smithi064.stdout:1/649: write d2/df/d14/d2a/fbf [1532306,27840] 0 2024-02-16T22:56:34.406 INFO:tasks.workunit.client.1.smithi192.stdout:7/965: dwrite d1/d2/de/d2c/d49/fd7 [0,4194304] 0 2024-02-16T22:56:34.406 INFO:tasks.workunit.client.0.smithi064.stdout:2/884: write d7/d14/d1e/d34/d3f/f4e [4644169,12850] 0 2024-02-16T22:56:34.407 INFO:tasks.workunit.client.0.smithi064.stdout:9/737: symlink d3/d9/d1e/d3e/d76/lf0 0 2024-02-16T22:56:34.407 INFO:tasks.workunit.client.0.smithi064.stdout:6/797: dwrite d0/d1b/f23 [0,4194304] 0 2024-02-16T22:56:34.407 INFO:tasks.workunit.client.1.smithi192.stdout:2/915: unlink d22/d24/d39/d42/d65/d69/d4f/fae 0 2024-02-16T22:56:34.411 INFO:tasks.workunit.client.0.smithi064.stdout:5/689: dwrite d6/d25/d48/d67/d91/da5/de3/d4b/fb5 [4194304,4194304] 0 2024-02-16T22:56:34.414 INFO:tasks.workunit.client.1.smithi192.stdout:6/872: dread da/df/d12/f75 [0,4194304] 0 2024-02-16T22:56:34.415 INFO:tasks.workunit.client.0.smithi064.stdout:0/635: mkdir da/dc/d3b/dd9 0 2024-02-16T22:56:34.416 INFO:tasks.workunit.client.0.smithi064.stdout:3/812: unlink d4/d13/d1b/d27/d61/d63/f71 0 2024-02-16T22:56:34.418 INFO:tasks.workunit.client.1.smithi192.stdout:0/852: mkdir d1/d3/de/d22/d111 0 2024-02-16T22:56:34.421 INFO:tasks.workunit.client.0.smithi064.stdout:4/761: dwrite dc/d1d/d43/d7a/f9b [4194304,4194304] 0 2024-02-16T22:56:34.421 INFO:tasks.workunit.client.0.smithi064.stdout:3/813: readlink d4/d13/d1b/d27/d82/db0/lc4 0 2024-02-16T22:56:34.423 INFO:tasks.workunit.client.0.smithi064.stdout:1/650: write d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce [124314,57159] 0 2024-02-16T22:56:34.426 INFO:tasks.workunit.client.1.smithi192.stdout:6/873: symlink da/df/d6b/d83/l116 0 2024-02-16T22:56:34.427 INFO:tasks.workunit.client.0.smithi064.stdout:6/798: creat d0/d3c/da9/d76/f109 x:0 0 0 2024-02-16T22:56:34.427 INFO:tasks.workunit.client.0.smithi064.stdout:1/651: chown d2/df/d14/d18/d48/d6b/d75/f77 1 1 2024-02-16T22:56:34.427 INFO:tasks.workunit.client.1.smithi192.stdout:8/847: dread d5/d16/f7d [0,4194304] 0 2024-02-16T22:56:34.428 INFO:tasks.workunit.client.0.smithi064.stdout:1/652: write d2/df/d11/d71/fd3 [1601782,20269] 0 2024-02-16T22:56:34.428 INFO:tasks.workunit.client.0.smithi064.stdout:6/799: dread - d0/d1d/d21/d98/f107 zero size 2024-02-16T22:56:34.428 INFO:tasks.workunit.client.0.smithi064.stdout:5/690: chown d6/l14 3 1 2024-02-16T22:56:34.429 INFO:tasks.workunit.client.1.smithi192.stdout:0/853: mkdir d1/d3/de/d22/da7/d112 0 2024-02-16T22:56:34.430 INFO:tasks.workunit.client.0.smithi064.stdout:8/660: read df/d31/d2f/d3e/f83 [1809537,129191] 0 2024-02-16T22:56:34.430 INFO:tasks.workunit.client.0.smithi064.stdout:8/661: stat df/d2a/f57 0 2024-02-16T22:56:34.431 INFO:tasks.workunit.client.1.smithi192.stdout:9/987: dread d7/d15/d36/d38/d3f/d8a/def/df6/f110 [0,4194304] 0 2024-02-16T22:56:34.432 INFO:tasks.workunit.client.1.smithi192.stdout:2/916: mknod d22/d24/d39/d42/d85/d98/db0/c12d 0 2024-02-16T22:56:34.434 INFO:tasks.workunit.client.0.smithi064.stdout:2/885: rename d7/d14/d25/d46/d55/d102 to d7/d14/d25/daf/de9/d123 0 2024-02-16T22:56:34.435 INFO:tasks.workunit.client.1.smithi192.stdout:6/874: chown da/df/d71/dab/cc7 31546125 1 2024-02-16T22:56:34.436 INFO:tasks.workunit.client.1.smithi192.stdout:6/875: readlink da/d40/l93 0 2024-02-16T22:56:34.436 INFO:tasks.workunit.client.0.smithi064.stdout:3/814: symlink d4/d13/d4d/l10f 0 2024-02-16T22:56:34.438 INFO:tasks.workunit.client.1.smithi192.stdout:7/966: link d1/d2/d2d/d61/d8a/f94 d1/d2/d2d/d61/d8a/de6/f145 0 2024-02-16T22:56:34.438 INFO:tasks.workunit.client.1.smithi192.stdout:2/917: dwrite d22/d47/dcb/dc9/d103/f110 [0,4194304] 0 2024-02-16T22:56:34.440 INFO:tasks.workunit.client.1.smithi192.stdout:0/854: creat d1/d3/de/d22/d37/d67/f113 x:0 0 0 2024-02-16T22:56:34.442 INFO:tasks.workunit.client.0.smithi064.stdout:6/800: chown d0/d5/lb0 591 1 2024-02-16T22:56:34.445 INFO:tasks.workunit.client.0.smithi064.stdout:5/691: unlink d6/d25/d48/d67/cba 0 2024-02-16T22:56:34.446 INFO:tasks.workunit.client.0.smithi064.stdout:5/692: readlink d6/d25/d48/d67/d91/da5/de3/l2b 0 2024-02-16T22:56:34.449 INFO:tasks.workunit.client.1.smithi192.stdout:8/848: link d5/d16/dd5/d9d/d101/fc6 d5/d16/de1/f108 0 2024-02-16T22:56:34.450 INFO:tasks.workunit.client.1.smithi192.stdout:8/849: dread - d5/fae zero size 2024-02-16T22:56:34.451 INFO:tasks.workunit.client.1.smithi192.stdout:8/850: write d5/db/d47/d43/d67/fd6 [85004,688] 0 2024-02-16T22:56:34.451 INFO:tasks.workunit.client.0.smithi064.stdout:2/886: unlink d7/d14/d1e/d34/d3f/f43 0 2024-02-16T22:56:34.452 INFO:tasks.workunit.client.1.smithi192.stdout:0/855: readlink d1/d3/de2/l110 0 2024-02-16T22:56:34.453 INFO:tasks.workunit.client.1.smithi192.stdout:0/856: dread - d1/d3/f83 zero size 2024-02-16T22:56:34.455 INFO:tasks.workunit.client.1.smithi192.stdout:6/876: link da/df/d12/d57/fc6 da/df/d6b/d74/f117 0 2024-02-16T22:56:34.459 INFO:tasks.workunit.client.0.smithi064.stdout:6/801: symlink d0/d1d/dfa/l10a 0 2024-02-16T22:56:34.461 INFO:tasks.workunit.client.1.smithi192.stdout:7/967: link d1/d2/de/d2c/d49/da4/d104/f114 d1/d2/d2d/d61/d8a/de6/f146 0 2024-02-16T22:56:34.463 INFO:tasks.workunit.client.1.smithi192.stdout:8/851: unlink d5/db/de/f99 0 2024-02-16T22:56:34.465 INFO:tasks.workunit.client.0.smithi064.stdout:6/802: dwrite d0/d5/df/d17/d4a/fe2 [0,4194304] 0 2024-02-16T22:56:34.467 INFO:tasks.workunit.client.0.smithi064.stdout:2/887: mknod d7/d14/d25/d46/c124 0 2024-02-16T22:56:34.468 INFO:tasks.workunit.client.1.smithi192.stdout:6/877: creat da/d40/db6/db8/f118 x:0 0 0 2024-02-16T22:56:34.470 INFO:tasks.workunit.client.0.smithi064.stdout:3/815: link d4/f84 d4/d13/d1b/d27/d99/d7d/f110 0 2024-02-16T22:56:34.474 INFO:tasks.workunit.client.1.smithi192.stdout:0/857: mknod d1/d3/de/d22/d37/d67/d34/c114 0 2024-02-16T22:56:34.476 INFO:tasks.workunit.client.1.smithi192.stdout:5/869: sync 2024-02-16T22:56:34.477 INFO:tasks.workunit.client.1.smithi192.stdout:4/904: sync 2024-02-16T22:56:34.477 INFO:tasks.workunit.client.1.smithi192.stdout:1/814: sync 2024-02-16T22:56:34.479 INFO:tasks.workunit.client.0.smithi064.stdout:6/803: creat d0/d1d/dbc/dec/f10b x:0 0 0 2024-02-16T22:56:34.479 INFO:tasks.workunit.client.1.smithi192.stdout:3/794: dwrite da/d11/d35/ff9 [0,4194304] 0 2024-02-16T22:56:34.480 INFO:tasks.workunit.client.0.smithi064.stdout:6/804: truncate d0/d3c/da9/d76/f109 752816 0 2024-02-16T22:56:34.480 INFO:tasks.workunit.client.0.smithi064.stdout:2/888: symlink d7/d14/d3e/l125 0 2024-02-16T22:56:34.482 INFO:tasks.workunit.client.0.smithi064.stdout:9/738: write d3/d9/d1e/d42/d7b/fa8 [1495937,129824] 0 2024-02-16T22:56:34.486 INFO:tasks.workunit.client.1.smithi192.stdout:4/905: creat d1/d15/d48/d8d/f135 x:0 0 0 2024-02-16T22:56:34.489 INFO:tasks.workunit.client.1.smithi192.stdout:1/815: symlink d3/de/d27/d34/d36/d47/l115 0 2024-02-16T22:56:34.492 INFO:tasks.workunit.client.0.smithi064.stdout:2/889: mkdir d7/d14/d25/d46/d55/d57/dbe/d10e/d126 0 2024-02-16T22:56:34.493 INFO:tasks.workunit.client.0.smithi064.stdout:2/890: chown d7/fb1 94024953 1 2024-02-16T22:56:34.494 INFO:tasks.workunit.client.1.smithi192.stdout:3/795: creat da/d11/dd4/deb/da1/f10f x:0 0 0 2024-02-16T22:56:34.495 INFO:tasks.workunit.client.1.smithi192.stdout:4/906: truncate d1/d26/d5e/f70 1360736 0 2024-02-16T22:56:34.496 INFO:tasks.workunit.client.0.smithi064.stdout:9/739: creat d3/d9/d6a/db1/d2a/ff1 x:0 0 0 2024-02-16T22:56:34.496 INFO:tasks.workunit.client.0.smithi064.stdout:8/662: write df/d31/d2f/d4f/f8d [1383886,101989] 0 2024-02-16T22:56:34.498 INFO:tasks.workunit.client.1.smithi192.stdout:1/816: mkdir d3/de/d27/d34/d52/d63/d81/d116 0 2024-02-16T22:56:34.499 INFO:tasks.workunit.client.0.smithi064.stdout:8/663: dread df/f15 [0,4194304] 0 2024-02-16T22:56:34.499 INFO:tasks.workunit.client.0.smithi064.stdout:5/693: link d6/d25/d48/d67/f9e d6/d25/d48/d67/d80/fec 0 2024-02-16T22:56:34.499 INFO:tasks.workunit.client.0.smithi064.stdout:8/664: readlink df/l10 0 2024-02-16T22:56:34.500 INFO:tasks.workunit.client.0.smithi064.stdout:5/694: chown d6/d25/da4/dc1/f60 1 1 2024-02-16T22:56:34.502 INFO:tasks.workunit.client.1.smithi192.stdout:9/988: dwrite d7/d15/d36/fb4 [0,4194304] 0 2024-02-16T22:56:34.507 INFO:tasks.workunit.client.1.smithi192.stdout:3/796: dwrite da/d11/d2b/ff3 [0,4194304] 0 2024-02-16T22:56:34.508 INFO:tasks.workunit.client.0.smithi064.stdout:4/762: dwrite dc/dd/d32/d46/d5a/f65 [0,4194304] 0 2024-02-16T22:56:34.508 INFO:tasks.workunit.client.0.smithi064.stdout:1/653: write d2/df/d11/f3b [391675,23502] 0 2024-02-16T22:56:34.509 INFO:tasks.workunit.client.1.smithi192.stdout:0/858: dread d1/d3/f7c [0,4194304] 0 2024-02-16T22:56:34.509 INFO:tasks.workunit.client.1.smithi192.stdout:2/918: dwrite d22/d47/f7d [4194304,4194304] 0 2024-02-16T22:56:34.510 INFO:tasks.workunit.client.0.smithi064.stdout:1/654: chown d2/f5c 0 1 2024-02-16T22:56:34.511 INFO:tasks.workunit.client.0.smithi064.stdout:2/891: mknod d7/d14/d25/d46/d55/d57/dbe/d10e/d126/c127 0 2024-02-16T22:56:34.513 INFO:tasks.workunit.client.1.smithi192.stdout:6/878: link da/d40/db6/f106 da/df/d12/d57/d81/dfc/f119 0 2024-02-16T22:56:34.514 INFO:tasks.workunit.client.1.smithi192.stdout:9/989: truncate d7/d19/d28/d43/d70/df0/f96 4699841 0 2024-02-16T22:56:34.519 INFO:tasks.workunit.client.1.smithi192.stdout:9/990: dread d7/fe [4194304,4194304] 0 2024-02-16T22:56:34.523 INFO:tasks.workunit.client.0.smithi064.stdout:1/655: dread d2/df/d14/d2a/fbf [0,4194304] 0 2024-02-16T22:56:34.523 INFO:tasks.workunit.client.0.smithi064.stdout:6/805: dread d0/d5/f42 [0,4194304] 0 2024-02-16T22:56:34.523 INFO:tasks.workunit.client.0.smithi064.stdout:2/892: dwrite d7/d14/d25/d46/d55/d7f/f99 [4194304,4194304] 0 2024-02-16T22:56:34.527 INFO:tasks.workunit.client.0.smithi064.stdout:4/763: symlink dc/dd/d32/d46/d5a/lef 0 2024-02-16T22:56:34.529 INFO:tasks.workunit.client.1.smithi192.stdout:0/859: dread d1/d3/de/d22/d37/d67/d34/f7f [0,4194304] 0 2024-02-16T22:56:34.529 INFO:tasks.workunit.client.0.smithi064.stdout:6/806: dread d0/d5/df/f79 [0,4194304] 0 2024-02-16T22:56:34.532 INFO:tasks.workunit.client.1.smithi192.stdout:2/919: creat d22/d24/d39/d42/d85/d98/db0/db6/f12e x:0 0 0 2024-02-16T22:56:34.534 INFO:tasks.workunit.client.1.smithi192.stdout:7/968: dwrite d1/d2/de/d2c/d49/d54/de3/f100 [0,4194304] 0 2024-02-16T22:56:34.535 INFO:tasks.workunit.client.1.smithi192.stdout:2/920: fsync d22/d47/dcb/dc9/fcf 0 2024-02-16T22:56:34.535 INFO:tasks.workunit.client.1.smithi192.stdout:7/969: readlink d1/d69/l6d 0 2024-02-16T22:56:34.536 INFO:tasks.workunit.client.1.smithi192.stdout:8/852: dwrite d5/db/de/f80 [0,4194304] 0 2024-02-16T22:56:34.539 INFO:tasks.workunit.client.0.smithi064.stdout:9/740: rename d3/l33 to d3/d9/d6a/db1/d2a/d2e/d31/d95/lf2 0 2024-02-16T22:56:34.540 INFO:tasks.workunit.client.0.smithi064.stdout:3/816: dwrite d4/d13/d1b/d27/d99/d7d/fc8 [4194304,4194304] 0 2024-02-16T22:56:34.544 INFO:tasks.workunit.client.1.smithi192.stdout:1/817: creat d3/de/d27/d34/de7/f117 x:0 0 0 2024-02-16T22:56:34.545 INFO:tasks.workunit.client.1.smithi192.stdout:7/970: chown d1/d2/de/d2c/d49/d54/de3/l135 367797506 1 2024-02-16T22:56:34.545 INFO:tasks.workunit.client.1.smithi192.stdout:1/818: write d3/de/d27/f3b [3267478,64609] 0 2024-02-16T22:56:34.547 INFO:tasks.workunit.client.1.smithi192.stdout:7/971: chown d1/d2/de/d2c/d49/da4/fa3 89 1 2024-02-16T22:56:34.547 INFO:tasks.workunit.client.1.smithi192.stdout:1/819: chown d3/d40/f94 0 1 2024-02-16T22:56:34.551 INFO:tasks.workunit.client.0.smithi064.stdout:9/741: dwrite d3/d9/d6a/db1/d2a/d2e/f9d [0,4194304] 0 2024-02-16T22:56:34.552 INFO:tasks.workunit.client.0.smithi064.stdout:9/742: dread - d3/d9/d1e/d42/f7c zero size 2024-02-16T22:56:34.554 INFO:tasks.workunit.client.0.smithi064.stdout:9/743: fsync d3/d9/d6a/db1/d2a/d2e/fc0 0 2024-02-16T22:56:34.556 INFO:tasks.workunit.client.0.smithi064.stdout:1/656: creat d2/df/d11/d71/fdd x:0 0 0 2024-02-16T22:56:34.559 INFO:tasks.workunit.client.0.smithi064.stdout:4/764: mkdir dc/ddc/df0 0 2024-02-16T22:56:34.560 INFO:tasks.workunit.client.1.smithi192.stdout:4/907: write d1/d6b/fa1 [194591,41488] 0 2024-02-16T22:56:34.562 INFO:tasks.workunit.client.1.smithi192.stdout:4/908: write d1/d15/d48/db5/f106 [260005,20242] 0 2024-02-16T22:56:34.563 INFO:tasks.workunit.client.1.smithi192.stdout:0/860: chown d1/d3/d29/f27 1 1 2024-02-16T22:56:34.567 INFO:tasks.workunit.client.0.smithi064.stdout:5/695: link d6/d25/d48/d67/d91/da5/de3/c8d d6/d18/dc5/ced 0 2024-02-16T22:56:34.567 INFO:tasks.workunit.client.1.smithi192.stdout:8/853: chown d5/d9/c5b 641349 1 2024-02-16T22:56:34.571 INFO:tasks.workunit.client.0.smithi064.stdout:8/665: getdents df/d31/d61/d79/dc4 0 2024-02-16T22:56:34.573 INFO:tasks.workunit.client.0.smithi064.stdout:3/817: creat d4/d13/d1b/d27/d61/dd8/df5/f111 x:0 0 0 2024-02-16T22:56:34.573 INFO:tasks.workunit.client.0.smithi064.stdout:5/696: dwrite d6/d25/d48/d67/d91/da5/de3/d4b/f94 [0,4194304] 0 2024-02-16T22:56:34.575 INFO:tasks.workunit.client.0.smithi064.stdout:5/697: fsync d6/d18/fde 0 2024-02-16T22:56:34.580 INFO:tasks.workunit.client.0.smithi064.stdout:1/657: rmdir d2/df/d14/d18 39 2024-02-16T22:56:34.582 INFO:tasks.workunit.client.1.smithi192.stdout:6/879: dread da/df/d6b/f88 [0,4194304] 0 2024-02-16T22:56:34.583 INFO:tasks.workunit.client.0.smithi064.stdout:9/744: mkdir d3/d11/d9e/da5/df3 0 2024-02-16T22:56:34.588 INFO:tasks.workunit.client.1.smithi192.stdout:4/909: dread d1/d26/d66/d89/d11/fa6 [0,4194304] 0 2024-02-16T22:56:34.589 INFO:tasks.workunit.client.0.smithi064.stdout:6/807: symlink d0/d1b/l10c 0 2024-02-16T22:56:34.591 INFO:tasks.workunit.client.1.smithi192.stdout:2/921: write d22/d24/d39/d42/d84/fd4 [2589923,60360] 0 2024-02-16T22:56:34.592 INFO:tasks.workunit.client.1.smithi192.stdout:5/870: sync 2024-02-16T22:56:34.603 INFO:tasks.workunit.client.0.smithi064.stdout:2/893: link d7/d14/d25/ce4 d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/dd6/c128 0 2024-02-16T22:56:34.603 INFO:tasks.workunit.client.0.smithi064.stdout:6/808: symlink d0/d3c/da9/d76/l10d 0 2024-02-16T22:56:34.604 INFO:tasks.workunit.client.1.smithi192.stdout:2/922: dread d22/d47/dcb/f2d [0,4194304] 0 2024-02-16T22:56:34.607 INFO:tasks.workunit.client.0.smithi064.stdout:4/765: dwrite dc/de/f75 [0,4194304] 0 2024-02-16T22:56:34.607 INFO:tasks.workunit.client.1.smithi192.stdout:2/923: dwrite d22/fa9 [0,4194304] 0 2024-02-16T22:56:34.608 INFO:tasks.workunit.client.0.smithi064.stdout:4/766: dread - dc/dd/d16/d7c/f86 zero size 2024-02-16T22:56:34.615 INFO:tasks.workunit.client.0.smithi064.stdout:1/658: mknod d2/df/d14/d18/d40/cde 0 2024-02-16T22:56:34.620 INFO:tasks.workunit.client.0.smithi064.stdout:2/894: creat d7/d14/d1e/d34/d3f/d9a/dd9/d60/f129 x:0 0 0 2024-02-16T22:56:34.622 INFO:tasks.workunit.client.0.smithi064.stdout:9/745: dwrite d3/d89/d9a/f9b [0,4194304] 0 2024-02-16T22:56:34.628 INFO:tasks.workunit.client.0.smithi064.stdout:3/818: creat d4/d13/d35/f112 x:0 0 0 2024-02-16T22:56:34.629 INFO:tasks.workunit.client.0.smithi064.stdout:6/809: dread d0/d6/d10/db2/ff8 [0,4194304] 0 2024-02-16T22:56:34.630 INFO:tasks.workunit.client.0.smithi064.stdout:4/767: fsync dc/d1d/dbd/fca 0 2024-02-16T22:56:34.632 INFO:tasks.workunit.client.0.smithi064.stdout:2/895: dwrite d7/d14/d1e/d34/d3f/f79 [0,4194304] 0 2024-02-16T22:56:34.634 INFO:tasks.workunit.client.0.smithi064.stdout:5/698: rename d6/fc to d6/d18/fee 0 2024-02-16T22:56:34.636 INFO:tasks.workunit.client.0.smithi064.stdout:1/659: truncate d2/df/d11/d1a/d1d/f24 2543511 0 2024-02-16T22:56:34.636 INFO:tasks.workunit.client.0.smithi064.stdout:8/666: dwrite f9 [0,4194304] 0 2024-02-16T22:56:34.642 INFO:tasks.workunit.client.0.smithi064.stdout:7/728: sync 2024-02-16T22:56:34.647 INFO:tasks.workunit.client.0.smithi064.stdout:1/660: dread d2/df/d11/d1a/d30/dbd/fb7 [0,4194304] 0 2024-02-16T22:56:34.647 INFO:tasks.workunit.client.0.smithi064.stdout:7/729: dread d5/d33/d45/fb5 [0,4194304] 0 2024-02-16T22:56:34.650 INFO:tasks.workunit.client.0.smithi064.stdout:6/810: symlink d0/d3c/dc5/l10e 0 2024-02-16T22:56:34.651 INFO:tasks.workunit.client.0.smithi064.stdout:3/819: mknod d4/d13/d1b/d27/d99/c113 0 2024-02-16T22:56:34.653 INFO:tasks.workunit.client.0.smithi064.stdout:2/896: dread - d7/d14/d25/daf/de9/f11a zero size 2024-02-16T22:56:34.656 INFO:tasks.workunit.client.0.smithi064.stdout:6/811: dwrite d0/d6/d10/f9f [0,4194304] 0 2024-02-16T22:56:34.664 INFO:tasks.workunit.client.0.smithi064.stdout:7/730: symlink d5/d33/d42/le9 0 2024-02-16T22:56:34.665 INFO:tasks.workunit.client.0.smithi064.stdout:7/731: write d5/d7d/fa0 [133841,115204] 0 2024-02-16T22:56:34.667 INFO:tasks.workunit.client.0.smithi064.stdout:1/661: rename d2/df/d14/d18/d48/d6b/dc7 to d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf 0 2024-02-16T22:56:34.671 INFO:tasks.workunit.client.0.smithi064.stdout:3/820: mknod d4/d13/d1b/d27/d81/c114 0 2024-02-16T22:56:34.682 INFO:tasks.workunit.client.0.smithi064.stdout:6/812: dread d0/d6/d10/d33/f55 [0,4194304] 0 2024-02-16T22:56:34.684 INFO:tasks.workunit.client.1.smithi192.stdout:9/991: rename d7/d15/fce to d7/d15/d36/d38/d3f/f151 0 2024-02-16T22:56:34.687 INFO:tasks.workunit.client.1.smithi192.stdout:1/820: mknod d3/de/d27/d34/d69/d85/df7/c118 0 2024-02-16T22:56:34.688 INFO:tasks.workunit.client.0.smithi064.stdout:0/636: sync 2024-02-16T22:56:34.688 INFO:tasks.workunit.client.1.smithi192.stdout:9/992: dread d7/d19/d37/d39/f10c [0,4194304] 0 2024-02-16T22:56:34.689 INFO:tasks.workunit.client.1.smithi192.stdout:3/797: creat da/d11/d2b/f110 x:0 0 0 2024-02-16T22:56:34.691 INFO:tasks.workunit.client.0.smithi064.stdout:9/746: creat d3/ff4 x:0 0 0 2024-02-16T22:56:34.693 INFO:tasks.workunit.client.0.smithi064.stdout:5/699: dwrite d6/d18/d19/f1f [0,4194304] 0 2024-02-16T22:56:34.697 INFO:tasks.workunit.client.0.smithi064.stdout:7/732: creat d5/d6/d54/d41/fea x:0 0 0 2024-02-16T22:56:34.702 INFO:tasks.workunit.client.0.smithi064.stdout:4/768: link dc/de/d28/d57/cd5 dc/de/d6d/cf1 0 2024-02-16T22:56:34.703 INFO:tasks.workunit.client.0.smithi064.stdout:7/733: dwrite d5/d6/d65/da7/dad/fd0 [0,4194304] 0 2024-02-16T22:56:34.707 INFO:tasks.workunit.client.0.smithi064.stdout:6/813: creat d0/d5/df/d17/da1/f10f x:0 0 0 2024-02-16T22:56:34.709 INFO:tasks.workunit.client.0.smithi064.stdout:0/637: creat da/dc/d3b/d72/db3/fda x:0 0 0 2024-02-16T22:56:34.711 INFO:tasks.workunit.client.0.smithi064.stdout:0/638: fdatasync da/dc/de/d12/d2c/d2e/d38/d87/fcc 0 2024-02-16T22:56:34.711 INFO:tasks.workunit.client.0.smithi064.stdout:8/667: truncate df/d31/d2f/d4f/f75 2761306 0 2024-02-16T22:56:34.712 INFO:tasks.workunit.client.0.smithi064.stdout:0/639: fdatasync da/dc/de/d12/d2c/d2e/d38/d7e/fc4 0 2024-02-16T22:56:34.714 INFO:tasks.workunit.client.0.smithi064.stdout:0/640: write da/d26/f28 [4190742,71008] 0 2024-02-16T22:56:34.718 INFO:tasks.workunit.client.0.smithi064.stdout:1/662: unlink d2/df/lcb 0 2024-02-16T22:56:34.720 INFO:tasks.workunit.client.1.smithi192.stdout:7/972: mkdir d1/d2/d2d/d61/d88/d90/d147 0 2024-02-16T22:56:34.720 INFO:tasks.workunit.client.0.smithi064.stdout:1/663: chown d2/df/d11/d1a/d30/dbd/d91/lda 25046 1 2024-02-16T22:56:34.722 INFO:tasks.workunit.client.1.smithi192.stdout:1/821: symlink d3/de/d27/d34/d36/d47/dcd/l119 0 2024-02-16T22:56:34.723 INFO:tasks.workunit.client.0.smithi064.stdout:3/821: truncate d4/d13/d4d/d5c/d78/fed 598519 0 2024-02-16T22:56:34.724 INFO:tasks.workunit.client.1.smithi192.stdout:9/993: mkdir d7/d19/d28/d43/d152 0 2024-02-16T22:56:34.725 INFO:tasks.workunit.client.1.smithi192.stdout:3/798: fsync da/df/f79 0 2024-02-16T22:56:34.726 INFO:tasks.workunit.client.0.smithi064.stdout:2/897: truncate d7/d14/d4b/d63/ffc 103156 0 2024-02-16T22:56:34.727 INFO:tasks.workunit.client.1.smithi192.stdout:0/861: dwrite d1/d3/de/d22/d37/f87 [0,4194304] 0 2024-02-16T22:56:34.733 INFO:tasks.workunit.client.1.smithi192.stdout:3/799: dwrite da/d11/d63/d6b/d4e/f56 [0,4194304] 0 2024-02-16T22:56:34.735 INFO:tasks.workunit.client.0.smithi064.stdout:6/814: readlink d0/d6/d10/d33/l20 0 2024-02-16T22:56:34.736 INFO:tasks.workunit.client.0.smithi064.stdout:5/700: write d6/d25/d48/d67/d91/da5/de3/d43/f58 [158753,83015] 0 2024-02-16T22:56:34.737 INFO:tasks.workunit.client.1.smithi192.stdout:6/880: mknod da/df/d18/d46/dff/c11a 0 2024-02-16T22:56:34.737 INFO:tasks.workunit.client.0.smithi064.stdout:4/769: dread dc/fba [0,4194304] 0 2024-02-16T22:56:34.740 INFO:tasks.workunit.client.1.smithi192.stdout:4/910: getdents d1/d26/d66/d89/da8/da9/d134 0 2024-02-16T22:56:34.746 INFO:tasks.workunit.client.1.smithi192.stdout:5/871: link d6/d20/fc5 d6/d12/d4e/d58/d8f/f11b 0 2024-02-16T22:56:34.752 INFO:tasks.workunit.client.0.smithi064.stdout:8/668: creat df/d31/d61/d79/dc4/fe6 x:0 0 0 2024-02-16T22:56:34.752 INFO:tasks.workunit.client.0.smithi064.stdout:6/815: write d0/d5/df/f100 [70871,118269] 0 2024-02-16T22:56:34.755 INFO:tasks.workunit.client.0.smithi064.stdout:3/822: unlink d4/d13/d1b/d27/fd6 0 2024-02-16T22:56:34.760 INFO:tasks.workunit.client.1.smithi192.stdout:9/994: creat d7/d19/d5e/d81/f153 x:0 0 0 2024-02-16T22:56:34.765 INFO:tasks.workunit.client.1.smithi192.stdout:0/862: creat d1/d3/d18/f115 x:0 0 0 2024-02-16T22:56:34.767 INFO:tasks.workunit.client.1.smithi192.stdout:8/854: creat d5/db/f109 x:0 0 0 2024-02-16T22:56:34.770 INFO:tasks.workunit.client.1.smithi192.stdout:7/973: dread d1/d2/d2d/d61/d88/d90/f78 [0,4194304] 0 2024-02-16T22:56:34.775 INFO:tasks.workunit.client.1.smithi192.stdout:1/822: write d3/de/d27/f42 [3115463,97127] 0 2024-02-16T22:56:34.778 INFO:tasks.workunit.client.1.smithi192.stdout:0/863: creat d1/d3/de/d22/d37/d67/d75/db0/f116 x:0 0 0 2024-02-16T22:56:34.779 INFO:tasks.workunit.client.1.smithi192.stdout:8/855: rmdir d5/d26/d83/d86/d5f/db2 39 2024-02-16T22:56:34.782 INFO:tasks.workunit.client.1.smithi192.stdout:7/974: mkdir d1/d2/de/d2c/d49/d54/dd8/d117/dcd/d148 0 2024-02-16T22:56:34.784 INFO:tasks.workunit.client.1.smithi192.stdout:2/924: getdents d22/d47 0 2024-02-16T22:56:34.785 INFO:tasks.workunit.client.1.smithi192.stdout:2/925: truncate d22/d24/d39/d42/d65/d66/f12c 659188 0 2024-02-16T22:56:34.785 INFO:tasks.workunit.client.1.smithi192.stdout:1/823: rename d3/de/d27/d34/cb8 to d3/d5b/db1/c11a 0 2024-02-16T22:56:34.786 INFO:tasks.workunit.client.1.smithi192.stdout:4/911: write d1/d26/d66/d89/d11/d58/f5d [1750991,32581] 0 2024-02-16T22:56:34.787 INFO:tasks.workunit.client.1.smithi192.stdout:9/995: mkdir d7/d15/d36/d77/dff/d154 0 2024-02-16T22:56:34.787 INFO:tasks.workunit.client.1.smithi192.stdout:4/912: readlink d1/d26/d34/lf9 0 2024-02-16T22:56:34.788 INFO:tasks.workunit.client.1.smithi192.stdout:0/864: fsync d1/d3/d29/f27 0 2024-02-16T22:56:34.789 INFO:tasks.workunit.client.1.smithi192.stdout:9/996: fsync d7/d15/d36/d77/dbd/de0/f11e 0 2024-02-16T22:56:34.789 INFO:tasks.workunit.client.1.smithi192.stdout:8/856: mknod d5/d16/dcf/c10a 0 2024-02-16T22:56:34.791 INFO:tasks.workunit.client.1.smithi192.stdout:2/926: truncate d22/d24/d39/ff5 364228 0 2024-02-16T22:56:34.793 INFO:tasks.workunit.client.1.smithi192.stdout:9/997: write d7/d19/d37/d39/f44 [4598427,81063] 0 2024-02-16T22:56:34.796 INFO:tasks.workunit.client.1.smithi192.stdout:5/872: getdents d6/d8/d75/d8c/dc4 0 2024-02-16T22:56:34.798 INFO:tasks.workunit.client.1.smithi192.stdout:1/824: dread d3/fa6 [0,4194304] 0 2024-02-16T22:56:34.799 INFO:tasks.workunit.client.1.smithi192.stdout:4/913: write d1/d26/d66/d89/f1e [1414943,24542] 0 2024-02-16T22:56:34.799 INFO:tasks.workunit.client.1.smithi192.stdout:8/857: mkdir d5/db/de/d10b 0 2024-02-16T22:56:34.799 INFO:tasks.workunit.client.1.smithi192.stdout:3/800: dwrite da/d6a/fc6 [0,4194304] 0 2024-02-16T22:56:34.801 INFO:tasks.workunit.client.1.smithi192.stdout:6/881: dwrite da/df/d12/d15/d1b/f37 [0,4194304] 0 2024-02-16T22:56:34.801 INFO:tasks.workunit.client.1.smithi192.stdout:2/927: creat d22/d24/d39/d3d/df2/d99/f12f x:0 0 0 2024-02-16T22:56:34.801 INFO:tasks.workunit.client.1.smithi192.stdout:0/865: write d1/d3/d29/f8d [6215663,78996] 0 2024-02-16T22:56:34.803 INFO:tasks.workunit.client.1.smithi192.stdout:5/873: truncate d6/d12/d1d/d56/d108/f109 473287 0 2024-02-16T22:56:34.805 INFO:tasks.workunit.client.1.smithi192.stdout:8/858: readlink d5/db/d47/l7c 0 2024-02-16T22:56:34.805 INFO:tasks.workunit.client.1.smithi192.stdout:4/914: rmdir d1 39 2024-02-16T22:56:34.806 INFO:tasks.workunit.client.1.smithi192.stdout:5/874: dread - d6/d12/d1d/fb0 zero size 2024-02-16T22:56:34.807 INFO:tasks.workunit.client.1.smithi192.stdout:1/825: chown d3/de/d27/d34/d52/d5a/d105 78805233 1 2024-02-16T22:56:34.809 INFO:tasks.workunit.client.1.smithi192.stdout:9/998: truncate d7/d19/d28/f74 7097190 0 2024-02-16T22:56:34.812 INFO:tasks.workunit.client.1.smithi192.stdout:0/866: rename d1/d3/d18/cf9 to d1/d3/d39/c117 0 2024-02-16T22:56:34.814 INFO:tasks.workunit.client.1.smithi192.stdout:5/875: creat d6/d20/f11c x:0 0 0 2024-02-16T22:56:34.815 INFO:tasks.workunit.client.1.smithi192.stdout:3/801: read da/d11/d63/d6b/d66/fe3 [115613,62498] 0 2024-02-16T22:56:34.816 INFO:tasks.workunit.client.1.smithi192.stdout:9/999: rmdir d7/d19/d28/d43/d70 39 2024-02-16T22:56:34.817 INFO:tasks.workunit.client.1.smithi192.stdout:6/882: dwrite da/df/d12/d15/f33 [0,4194304] 0 2024-02-16T22:56:34.817 INFO:tasks.workunit.client.1.smithi192.stdout:2/928: dwrite d22/d47/f88 [0,4194304] 0 2024-02-16T22:56:34.820 INFO:tasks.workunit.client.1.smithi192.stdout:4/915: mknod d1/d15/d29/d3c/c136 0 2024-02-16T22:56:34.821 INFO:tasks.workunit.client.1.smithi192.stdout:0/867: symlink d1/d3/dac/l118 0 2024-02-16T22:56:34.821 INFO:tasks.workunit.client.1.smithi192.stdout:4/916: chown d1/d15/l65 215 1 2024-02-16T22:56:34.823 INFO:tasks.workunit.client.1.smithi192.stdout:5/876: symlink d6/d12/d4e/d5b/d67/l11d 0 2024-02-16T22:56:34.823 INFO:tasks.workunit.client.1.smithi192.stdout:6/883: symlink da/df/d12/d57/daf/dcf/l11b 0 2024-02-16T22:56:34.827 INFO:tasks.workunit.client.1.smithi192.stdout:3/802: truncate da/d11/d35/f70 527960 0 2024-02-16T22:56:34.829 INFO:tasks.workunit.client.1.smithi192.stdout:4/917: dwrite d1/d15/d48/f55 [0,4194304] 0 2024-02-16T22:56:34.829 INFO:tasks.workunit.client.1.smithi192.stdout:0/868: rmdir d1/d3/de/d22/d37/d47 39 2024-02-16T22:56:34.831 INFO:tasks.workunit.client.1.smithi192.stdout:6/884: mknod da/df/d12/d57/daf/c11c 0 2024-02-16T22:56:34.833 INFO:tasks.workunit.client.1.smithi192.stdout:8/859: link d5/db/de/l8a d5/d16/l10c 0 2024-02-16T22:56:34.834 INFO:tasks.workunit.client.1.smithi192.stdout:0/869: rmdir d1/d3/de/d22/da7 39 2024-02-16T22:56:34.834 INFO:tasks.workunit.client.1.smithi192.stdout:2/929: creat d22/d24/d39/f130 x:0 0 0 2024-02-16T22:56:34.834 INFO:tasks.workunit.client.1.smithi192.stdout:4/918: truncate d1/d26/f31 931771 0 2024-02-16T22:56:34.839 INFO:tasks.workunit.client.1.smithi192.stdout:2/930: dread d22/d24/d39/d42/d84/f8c [0,4194304] 0 2024-02-16T22:56:34.840 INFO:tasks.workunit.client.1.smithi192.stdout:2/931: stat d22/d24/d39/d42/d85 0 2024-02-16T22:56:34.840 INFO:tasks.workunit.client.1.smithi192.stdout:8/860: rename d5/db/d47/d43/d67/c50 to d5/d26/d83/db4/c10d 0 2024-02-16T22:56:34.841 INFO:tasks.workunit.client.1.smithi192.stdout:0/870: creat d1/d3/de/d22/d37/f119 x:0 0 0 2024-02-16T22:56:34.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/932: creat d22/d47/dcb/d78/f131 x:0 0 0 2024-02-16T22:56:34.842 INFO:tasks.workunit.client.1.smithi192.stdout:0/871: dread - d1/dc7/d9c/f9d zero size 2024-02-16T22:56:34.842 INFO:tasks.workunit.client.1.smithi192.stdout:8/861: mknod d5/d26/d83/d86/c10e 0 2024-02-16T22:56:34.843 INFO:tasks.workunit.client.1.smithi192.stdout:2/933: readlink lf 0 2024-02-16T22:56:34.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/934: stat d22/d24/d39/d42/d85/d98/c11f 0 2024-02-16T22:56:34.845 INFO:tasks.workunit.client.1.smithi192.stdout:0/872: creat d1/d3/de/d22/d37/f11a x:0 0 0 2024-02-16T22:56:34.847 INFO:tasks.workunit.client.1.smithi192.stdout:8/862: getdents d5/db/de 0 2024-02-16T22:56:34.849 INFO:tasks.workunit.client.1.smithi192.stdout:0/873: symlink d1/d3/de/d22/da7/l11b 0 2024-02-16T22:56:34.850 INFO:tasks.workunit.client.1.smithi192.stdout:2/935: dwrite d22/d47/f88 [0,4194304] 0 2024-02-16T22:56:34.850 INFO:tasks.workunit.client.1.smithi192.stdout:0/874: chown d1/d3/d39/df3 621 1 2024-02-16T22:56:34.851 INFO:tasks.workunit.client.1.smithi192.stdout:2/936: dread d22/d24/d39/d42/d84/f8c [0,4194304] 0 2024-02-16T22:56:34.855 INFO:tasks.workunit.client.1.smithi192.stdout:2/937: fdatasync d22/d24/d39/d42/d84/f8c 0 2024-02-16T22:56:34.858 INFO:tasks.workunit.client.0.smithi064.stdout:9/747: dwrite d3/d9/d6a/db1/d2a/fc9 [0,4194304] 0 2024-02-16T22:56:34.858 INFO:tasks.workunit.client.0.smithi064.stdout:2/898: mkdir d7/d14/d1e/d34/d3f/d12a 0 2024-02-16T22:56:34.859 INFO:tasks.workunit.client.1.smithi192.stdout:2/938: mkdir d22/d24/d39/d3d/df2/d99/d132 0 2024-02-16T22:56:34.859 INFO:tasks.workunit.client.0.smithi064.stdout:7/734: mknod d5/d6/d65/d7f/dcd/ceb 0 2024-02-16T22:56:34.861 INFO:tasks.workunit.client.0.smithi064.stdout:4/770: fdatasync dc/d1d/dbd/fca 0 2024-02-16T22:56:34.861 INFO:tasks.workunit.client.1.smithi192.stdout:2/939: write d22/d24/d39/d42/d85/d98/db0/fc1 [302405,45278] 0 2024-02-16T22:56:34.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/940: chown d22/d24/d39/d42/d85/d98/dc4/f113 22119717 1 2024-02-16T22:56:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:0/641: dread da/dc/de/d12/d1d/f1f [4194304,4194304] 0 2024-02-16T22:56:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:5/701: chown d6/d25/d48/d67/d91/da5/de3/l75 1832506 1 2024-02-16T22:56:34.867 INFO:tasks.workunit.client.1.smithi192.stdout:1/826: dwrite d3/d5b/f71 [0,4194304] 0 2024-02-16T22:56:34.870 INFO:tasks.workunit.client.0.smithi064.stdout:8/669: dwrite df/d31/d8f/dc9/fd4 [0,4194304] 0 2024-02-16T22:56:34.873 INFO:tasks.workunit.client.0.smithi064.stdout:4/771: mkdir dc/ddc/de8/df2 0 2024-02-16T22:56:34.873 INFO:tasks.workunit.client.0.smithi064.stdout:0/642: chown da/dc/de/d12/d2c/fc5 2515 1 2024-02-16T22:56:34.874 INFO:tasks.workunit.client.0.smithi064.stdout:6/816: dread d0/d5/df/d17/d4a/fe2 [0,4194304] 0 2024-02-16T22:56:34.875 INFO:tasks.workunit.client.1.smithi192.stdout:1/827: dread d3/f97 [0,4194304] 0 2024-02-16T22:56:34.876 INFO:tasks.workunit.client.0.smithi064.stdout:7/735: truncate d5/d33/d45/d64/d56/faa 888014 0 2024-02-16T22:56:34.877 INFO:tasks.workunit.client.1.smithi192.stdout:2/941: rename d22/d24/d39/d3d/df2/d99/l105 to d22/d24/d39/d42/d84/l133 0 2024-02-16T22:56:34.878 INFO:tasks.workunit.client.1.smithi192.stdout:2/942: write d22/d24/d39/d42/d65/d69/f68 [2975014,129560] 0 2024-02-16T22:56:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:2/899: creat d7/d14/d1e/d34/d3f/d9a/dcf/f12b x:0 0 0 2024-02-16T22:56:34.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/885: write da/df/d12/d15/f68 [621854,76699] 0 2024-02-16T22:56:34.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/803: truncate da/d11/d2b/ddc/ff2 29728 0 2024-02-16T22:56:34.883 INFO:tasks.workunit.client.1.smithi192.stdout:4/919: write d1/d15/d23/f4a [761728,28380] 0 2024-02-16T22:56:34.884 INFO:tasks.workunit.client.1.smithi192.stdout:1/828: creat d3/de/d27/d34/d36/d47/dcd/f11b x:0 0 0 2024-02-16T22:56:34.886 INFO:tasks.workunit.client.0.smithi064.stdout:9/748: rename d3/d9/d6a/db1/d2a/d2e/l70 to d3/d89/d9a/lf5 0 2024-02-16T22:56:34.893 INFO:tasks.workunit.client.1.smithi192.stdout:8/863: write d5/d26/d83/db4/f4b [1773766,60921] 0 2024-02-16T22:56:34.893 INFO:tasks.workunit.client.1.smithi192.stdout:0/875: write d1/f1f [1988399,67383] 0 2024-02-16T22:56:34.894 INFO:tasks.workunit.client.1.smithi192.stdout:0/876: readlink d1/d3/d18/l10c 0 2024-02-16T22:56:34.895 INFO:tasks.workunit.client.1.smithi192.stdout:4/920: mknod d1/d26/d66/d89/da8/dc5/d111/d12f/c137 0 2024-02-16T22:56:34.896 INFO:tasks.workunit.client.1.smithi192.stdout:4/921: write d1/d15/d23/f4a [2197521,111392] 0 2024-02-16T22:56:34.897 INFO:tasks.workunit.client.1.smithi192.stdout:4/922: write d1/d15/d48/db5/f106 [2429063,82125] 0 2024-02-16T22:56:34.898 INFO:tasks.workunit.client.1.smithi192.stdout:1/829: mkdir d3/de/d27/d34/d52/d72/d11c 0 2024-02-16T22:56:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:4/772: mknod dc/dd/d32/d46/d5a/d7e/d9a/da2/cf3 0 2024-02-16T22:56:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:7/736: mknod d5/d33/d45/d73/cec 0 2024-02-16T22:56:34.903 INFO:tasks.workunit.client.1.smithi192.stdout:3/804: creat da/d11/d2b/ddc/d102/f111 x:0 0 0 2024-02-16T22:56:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:2/900: mknod d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/c12c 0 2024-02-16T22:56:34.904 INFO:tasks.workunit.client.1.smithi192.stdout:3/805: write da/d11/d63/d6b/d4e/fa5 [2937930,40339] 0 2024-02-16T22:56:34.905 INFO:tasks.workunit.client.1.smithi192.stdout:8/864: symlink d5/d26/d8f/dc0/dc3/l10f 0 2024-02-16T22:56:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:3/823: creat d4/d14/f115 x:0 0 0 2024-02-16T22:56:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:9/749: mkdir d3/d9/d6a/db1/d2a/d2e/d31/d62/df6 0 2024-02-16T22:56:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:9/750: dread - d3/ff4 zero size 2024-02-16T22:56:34.910 INFO:tasks.workunit.client.0.smithi064.stdout:7/737: dread d5/d6/d54/f78 [0,4194304] 0 2024-02-16T22:56:34.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/806: dwrite da/f94 [0,4194304] 0 2024-02-16T22:56:34.912 INFO:tasks.workunit.client.0.smithi064.stdout:5/702: fdatasync d6/d25/d48/d67/d80/fec 0 2024-02-16T22:56:34.914 INFO:tasks.workunit.client.1.smithi192.stdout:4/923: unlink d1/d15/l8e 0 2024-02-16T22:56:34.917 INFO:tasks.workunit.client.1.smithi192.stdout:6/886: dread da/df/d18/f29 [0,4194304] 0 2024-02-16T22:56:34.917 INFO:tasks.workunit.client.1.smithi192.stdout:6/887: readlink da/ld 0 2024-02-16T22:56:34.919 INFO:tasks.workunit.client.1.smithi192.stdout:7/975: sync 2024-02-16T22:56:34.920 INFO:tasks.workunit.client.0.smithi064.stdout:5/703: dwrite d6/d25/d48/d67/d91/da5/de3/fd7 [4194304,4194304] 0 2024-02-16T22:56:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:6/817: dread d0/d5/fce [0,4194304] 0 2024-02-16T22:56:34.931 INFO:tasks.workunit.client.1.smithi192.stdout:3/807: mknod da/d11/d63/dbf/c112 0 2024-02-16T22:56:34.933 INFO:tasks.workunit.client.1.smithi192.stdout:2/943: getdents d22/d47/dcb/d3f 0 2024-02-16T22:56:34.935 INFO:tasks.workunit.client.0.smithi064.stdout:8/670: creat df/fe7 x:0 0 0 2024-02-16T22:56:34.937 INFO:tasks.workunit.client.0.smithi064.stdout:0/643: write da/dc/d3b/d72/d7c/fad [967661,353] 0 2024-02-16T22:56:34.937 INFO:tasks.workunit.client.0.smithi064.stdout:8/671: write df/d31/d2f/d3e/ddf/fe3 [639452,58996] 0 2024-02-16T22:56:34.938 INFO:tasks.workunit.client.0.smithi064.stdout:8/672: readlink df/d31/dc1/lce 0 2024-02-16T22:56:34.943 INFO:tasks.workunit.client.1.smithi192.stdout:7/976: rmdir d1/d2/de/d2c/d49/d54/dd8/d117 39 2024-02-16T22:56:34.947 INFO:tasks.workunit.client.1.smithi192.stdout:8/865: symlink d5/d16/dd5/d9d/d107/l110 0 2024-02-16T22:56:34.948 INFO:tasks.workunit.client.0.smithi064.stdout:6/818: dread - d0/d6/d10/d33/da5/fc1 zero size 2024-02-16T22:56:34.949 INFO:tasks.workunit.client.1.smithi192.stdout:3/808: chown da/d11/l29 381545 1 2024-02-16T22:56:34.949 INFO:tasks.workunit.client.0.smithi064.stdout:6/819: read - d0/d6/d10/d33/d72/de8/f101 zero size 2024-02-16T22:56:34.950 INFO:tasks.workunit.client.1.smithi192.stdout:2/944: truncate d22/d24/d39/d42/d85/d98/db0/db6/de2/ff0 866544 0 2024-02-16T22:56:34.953 INFO:tasks.workunit.client.1.smithi192.stdout:4/924: mknod d1/d26/d66/d89/da8/da9/d114/c138 0 2024-02-16T22:56:34.956 INFO:tasks.workunit.client.1.smithi192.stdout:0/877: truncate d1/d3/de/d22/d37/d67/d34/f92 1905208 0 2024-02-16T22:56:34.956 INFO:tasks.workunit.client.1.smithi192.stdout:7/977: creat d1/d2/d2d/d61/d88/d12e/f149 x:0 0 0 2024-02-16T22:56:34.958 INFO:tasks.workunit.client.0.smithi064.stdout:8/673: mknod df/d31/d8f/dc9/d73/ce8 0 2024-02-16T22:56:34.958 INFO:tasks.workunit.client.0.smithi064.stdout:3/824: dwrite d4/f11 [0,4194304] 0 2024-02-16T22:56:34.960 INFO:tasks.workunit.client.1.smithi192.stdout:8/866: rename d5/db/c63 to d5/d9/c111 0 2024-02-16T22:56:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:9/751: dwrite d3/f63 [4194304,4194304] 0 2024-02-16T22:56:34.965 INFO:tasks.workunit.client.1.smithi192.stdout:2/945: creat d22/d47/f134 x:0 0 0 2024-02-16T22:56:34.966 INFO:tasks.workunit.client.0.smithi064.stdout:6/820: rmdir d0/d1d/da8/d60 39 2024-02-16T22:56:34.968 INFO:tasks.workunit.client.1.smithi192.stdout:4/925: symlink d1/d15/d29/dd4/l139 0 2024-02-16T22:56:34.969 INFO:tasks.workunit.client.1.smithi192.stdout:4/926: readlink d1/d26/d34/dc7/le4 0 2024-02-16T22:56:34.971 INFO:tasks.workunit.client.1.smithi192.stdout:6/888: creat da/df/d18/d52/f11d x:0 0 0 2024-02-16T22:56:34.972 INFO:tasks.workunit.client.0.smithi064.stdout:4/773: dwrite dc/d1d/d43/f5f [0,4194304] 0 2024-02-16T22:56:34.975 INFO:tasks.workunit.client.1.smithi192.stdout:0/878: symlink d1/d3/d39/l11c 0 2024-02-16T22:56:34.977 INFO:tasks.workunit.client.1.smithi192.stdout:7/978: truncate d1/d2/de/d2c/d49/d54/f67 356737 0 2024-02-16T22:56:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:7/738: dwrite d5/d33/d53/da3/fda [0,4194304] 0 2024-02-16T22:56:34.981 INFO:tasks.workunit.client.1.smithi192.stdout:3/809: creat da/d107/f113 x:0 0 0 2024-02-16T22:56:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:2/901: link d7/d14/d4b/d63/lfa d7/d14/d1e/d34/l12d 0 2024-02-16T22:56:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:9/752: rename d3/d9/d6a/db1/d2a/d2e/d31/d62/l75 to d3/d9/d6a/db1/d2a/d2e/d31/d3f/lf7 0 2024-02-16T22:56:34.985 INFO:tasks.workunit.client.0.smithi064.stdout:6/821: read - d0/d5/df/f96 zero size 2024-02-16T22:56:34.985 INFO:tasks.workunit.client.0.smithi064.stdout:9/753: chown d3/d11/d15/dab 774082 1 2024-02-16T22:56:34.985 INFO:tasks.workunit.client.1.smithi192.stdout:6/889: symlink da/d76/l11e 0 2024-02-16T22:56:34.986 INFO:tasks.workunit.client.1.smithi192.stdout:0/879: dread - d1/dc7/d9c/fad zero size 2024-02-16T22:56:34.987 INFO:tasks.workunit.client.0.smithi064.stdout:9/754: chown d3/d9/d6a/db1/d2a/d2e/d31/d95/c96 232 1 2024-02-16T22:56:34.987 INFO:tasks.workunit.client.0.smithi064.stdout:4/774: symlink dc/dd/db8/lf4 0 2024-02-16T22:56:34.988 INFO:tasks.workunit.client.0.smithi064.stdout:9/755: readlink d3/d9/d6a/db1/d2a/d2e/l7a 0 2024-02-16T22:56:34.988 INFO:tasks.workunit.client.1.smithi192.stdout:7/979: mknod d1/d2/d2d/d61/d88/d90/db2/c14a 0 2024-02-16T22:56:34.990 INFO:tasks.workunit.client.0.smithi064.stdout:4/775: truncate dc/dd/d32/d46/f55 1413624 0 2024-02-16T22:56:34.990 INFO:tasks.workunit.client.0.smithi064.stdout:4/776: readlink l9 0 2024-02-16T22:56:34.991 INFO:tasks.workunit.client.1.smithi192.stdout:6/890: read da/df/d12/d15/fdf [3741670,83536] 0 2024-02-16T22:56:34.991 INFO:tasks.workunit.client.1.smithi192.stdout:2/946: creat d22/d24/d39/d42/d52/d7c/f135 x:0 0 0 2024-02-16T22:56:34.993 INFO:tasks.workunit.client.0.smithi064.stdout:7/739: creat d5/d33/d42/fed x:0 0 0 2024-02-16T22:56:34.994 INFO:tasks.workunit.client.0.smithi064.stdout:5/704: getdents d6/d25/d48/d67/d91/da5/de3 0 2024-02-16T22:56:34.998 INFO:tasks.workunit.client.0.smithi064.stdout:6/822: mkdir d0/d12/d7b/d110 0 2024-02-16T22:56:34.998 INFO:tasks.workunit.client.1.smithi192.stdout:0/880: dread - d1/d3/f6d zero size 2024-02-16T22:56:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:8/674: link df/d31/d2f/ca4 df/d31/d61/d79/dc4/ce9 0 2024-02-16T22:56:35.001 INFO:tasks.workunit.client.1.smithi192.stdout:2/947: creat d22/d24/d39/d42/d65/f136 x:0 0 0 2024-02-16T22:56:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:4/777: creat dc/de/d6d/dee/ff5 x:0 0 0 2024-02-16T22:56:35.003 INFO:tasks.workunit.client.1.smithi192.stdout:4/927: creat d1/f13a x:0 0 0 2024-02-16T22:56:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:7/740: creat d5/d6/d65/da7/dad/fee x:0 0 0 2024-02-16T22:56:35.007 INFO:tasks.workunit.client.1.smithi192.stdout:5/877: sync 2024-02-16T22:56:35.008 INFO:tasks.workunit.client.1.smithi192.stdout:6/891: truncate da/d40/db6/f112 1800318 0 2024-02-16T22:56:35.011 INFO:tasks.workunit.client.0.smithi064.stdout:3/825: dread d4/d13/d1b/d4f/f5d [4194304,4194304] 0 2024-02-16T22:56:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:9/756: symlink d3/d9/d6a/lf8 0 2024-02-16T22:56:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:0/644: truncate da/dc/de/d12/f49 875613 0 2024-02-16T22:56:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:8/675: mkdir df/d31/d2f/d3e/ddf/dea 0 2024-02-16T22:56:35.021 INFO:tasks.workunit.client.1.smithi192.stdout:2/948: symlink d22/d24/d39/d3d/df2/d99/l137 0 2024-02-16T22:56:35.025 INFO:tasks.workunit.client.1.smithi192.stdout:4/928: unlink d1/d26/d66/d89/d11/d58/c8a 0 2024-02-16T22:56:35.029 INFO:tasks.workunit.client.1.smithi192.stdout:6/892: fsync da/df/d12/f5a 0 2024-02-16T22:56:35.031 INFO:tasks.workunit.client.1.smithi192.stdout:3/810: link da/d11/d20/d39/d53/dc7/cbc da/d11/dd4/deb/c114 0 2024-02-16T22:56:35.031 INFO:tasks.workunit.client.1.smithi192.stdout:2/949: dwrite d22/d24/d39/d42/d65/d66/f12c [0,4194304] 0 2024-02-16T22:56:35.032 INFO:tasks.workunit.client.0.smithi064.stdout:6/823: mknod d0/c111 0 2024-02-16T22:56:35.034 INFO:tasks.workunit.client.1.smithi192.stdout:8/867: dwrite d5/fa7 [0,4194304] 0 2024-02-16T22:56:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:9/757: creat d3/d11/d9e/da5/ff9 x:0 0 0 2024-02-16T22:56:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:9/758: chown d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/c92 1550350 1 2024-02-16T22:56:35.039 INFO:tasks.workunit.client.1.smithi192.stdout:4/929: creat d1/d26/d5e/f13b x:0 0 0 2024-02-16T22:56:35.042 INFO:tasks.workunit.client.1.smithi192.stdout:1/830: sync 2024-02-16T22:56:35.042 INFO:tasks.workunit.client.1.smithi192.stdout:5/878: getdents d6/d8/d75/d8c/d106 0 2024-02-16T22:56:35.044 INFO:tasks.workunit.client.1.smithi192.stdout:7/980: write d1/d2/de/d83/d9f/fad [620641,117928] 0 2024-02-16T22:56:35.044 INFO:tasks.workunit.client.1.smithi192.stdout:5/879: readlink d6/d12/d4e/d5b/d67/l98 0 2024-02-16T22:56:35.046 INFO:tasks.workunit.client.0.smithi064.stdout:8/676: creat df/d31/d2f/d3e/d5f/d93/feb x:0 0 0 2024-02-16T22:56:35.053 INFO:tasks.workunit.client.0.smithi064.stdout:6/824: rename d0/d1d/d21/d98/f107 to d0/d6/d10/d33/da4/ddd/f112 0 2024-02-16T22:56:35.053 INFO:tasks.workunit.client.0.smithi064.stdout:6/825: dread - d0/d6/d10/f9d zero size 2024-02-16T22:56:35.054 INFO:tasks.workunit.client.0.smithi064.stdout:6/826: chown d0/d3c/le3 113 1 2024-02-16T22:56:35.055 INFO:tasks.workunit.client.1.smithi192.stdout:6/893: unlink da/df/d6b/d83/cf3 0 2024-02-16T22:56:35.057 INFO:tasks.workunit.client.1.smithi192.stdout:3/811: chown da/d11/d63/d6b/c97 1 1 2024-02-16T22:56:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:9/759: symlink d3/d9/d1e/d42/d7b/lfa 0 2024-02-16T22:56:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:5/705: dwrite d6/d25/d48/d67/d91/da5/de3/f59 [0,4194304] 0 2024-02-16T22:56:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:5/706: fsync d6/d18/d19/f2e 0 2024-02-16T22:56:35.062 INFO:tasks.workunit.client.1.smithi192.stdout:6/894: dwrite da/d40/f109 [0,4194304] 0 2024-02-16T22:56:35.065 INFO:tasks.workunit.client.1.smithi192.stdout:1/831: creat d3/d5b/d77/f11d x:0 0 0 2024-02-16T22:56:35.069 INFO:tasks.workunit.client.1.smithi192.stdout:3/812: dwrite da/d11/d35/ff9 [0,4194304] 0 2024-02-16T22:56:35.072 INFO:tasks.workunit.client.0.smithi064.stdout:1/664: sync 2024-02-16T22:56:35.073 INFO:tasks.workunit.client.1.smithi192.stdout:1/832: write d3/d40/d9f/d10a/f10d [294399,100711] 0 2024-02-16T22:56:35.078 INFO:tasks.workunit.client.1.smithi192.stdout:6/895: dwrite da/df/d6b/ffd [0,4194304] 0 2024-02-16T22:56:35.078 INFO:tasks.workunit.client.1.smithi192.stdout:0/881: dwrite d1/d3/de/d22/d37/d67/d34/da1/fb9 [0,4194304] 0 2024-02-16T22:56:35.079 INFO:tasks.workunit.client.1.smithi192.stdout:1/833: write d3/de/d27/d34/d36/d47/dcd/ffd [986465,40079] 0 2024-02-16T22:56:35.081 INFO:tasks.workunit.client.1.smithi192.stdout:6/896: dread da/d40/d85/f87 [0,4194304] 0 2024-02-16T22:56:35.083 INFO:tasks.workunit.client.1.smithi192.stdout:6/897: chown da/df/d18/l110 59142329 1 2024-02-16T22:56:35.091 INFO:tasks.workunit.client.0.smithi064.stdout:2/902: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/f56 [0,4194304] 0 2024-02-16T22:56:35.092 INFO:tasks.workunit.client.0.smithi064.stdout:9/760: dwrite d3/d9/d6a/db1/d2a/d2e/d31/d3f/f43 [4194304,4194304] 0 2024-02-16T22:56:35.093 INFO:tasks.workunit.client.0.smithi064.stdout:9/761: read d3/d9/ff [7346964,91389] 0 2024-02-16T22:56:35.102 INFO:tasks.workunit.client.0.smithi064.stdout:2/903: dwrite d7/d14/d4b/d63/f106 [0,4194304] 0 2024-02-16T22:56:35.104 INFO:tasks.workunit.client.0.smithi064.stdout:4/778: write dc/dd/f59 [850031,28351] 0 2024-02-16T22:56:35.107 INFO:tasks.workunit.client.0.smithi064.stdout:9/762: read d3/d9/d6a/db1/d2a/f4b [205893,78768] 0 2024-02-16T22:56:35.108 INFO:tasks.workunit.client.1.smithi192.stdout:4/930: fsync d1/d26/d6f/fab 0 2024-02-16T22:56:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:4/779: dwrite dc/dd/f59 [0,4194304] 0 2024-02-16T22:56:35.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/880: symlink d6/d12/d4e/l11e 0 2024-02-16T22:56:35.113 INFO:tasks.workunit.client.1.smithi192.stdout:5/881: write d6/d12/d2d/fd0 [763503,21178] 0 2024-02-16T22:56:35.115 INFO:tasks.workunit.client.1.smithi192.stdout:5/882: fdatasync d6/d8/fa1 0 2024-02-16T22:56:35.116 INFO:tasks.workunit.client.1.smithi192.stdout:3/813: dread da/d11/dd4/deb/da1/fb0 [0,4194304] 0 2024-02-16T22:56:35.123 INFO:tasks.workunit.client.1.smithi192.stdout:0/882: mkdir d1/d3/d18/d11d 0 2024-02-16T22:56:35.123 INFO:tasks.workunit.client.1.smithi192.stdout:0/883: chown d1/d3/c2b 10895377 1 2024-02-16T22:56:35.127 INFO:tasks.workunit.client.1.smithi192.stdout:1/834: truncate d3/f50 1629751 0 2024-02-16T22:56:35.128 INFO:tasks.workunit.client.0.smithi064.stdout:3/826: dwrite d4/d13/d1b/f68 [0,4194304] 0 2024-02-16T22:56:35.129 INFO:tasks.workunit.client.0.smithi064.stdout:3/827: truncate d4/fd9 1134668 0 2024-02-16T22:56:35.131 INFO:tasks.workunit.client.1.smithi192.stdout:2/950: creat d22/d24/d39/d42/f138 x:0 0 0 2024-02-16T22:56:35.134 INFO:tasks.workunit.client.1.smithi192.stdout:4/931: symlink d1/d15/d48/dc3/l13c 0 2024-02-16T22:56:35.136 INFO:tasks.workunit.client.1.smithi192.stdout:6/898: dwrite da/d40/db6/db8/ff0 [4194304,4194304] 0 2024-02-16T22:56:35.141 INFO:tasks.workunit.client.1.smithi192.stdout:3/814: mknod da/d11/dd4/deb/da1/c115 0 2024-02-16T22:56:35.143 INFO:tasks.workunit.client.1.smithi192.stdout:0/884: mknod d1/d3/de2/dff/c11e 0 2024-02-16T22:56:35.143 INFO:tasks.workunit.client.1.smithi192.stdout:8/868: dwrite d5/d16/f4f [0,4194304] 0 2024-02-16T22:56:35.144 INFO:tasks.workunit.client.1.smithi192.stdout:0/885: chown d1/d3/de/d22/d37 721116 1 2024-02-16T22:56:35.146 INFO:tasks.workunit.client.1.smithi192.stdout:0/886: write d1/d3/dac/f101 [663400,121295] 0 2024-02-16T22:56:35.156 INFO:tasks.workunit.client.1.smithi192.stdout:1/835: mknod d3/de/d27/d34/d52/d72/d11c/c11e 0 2024-02-16T22:56:35.158 INFO:tasks.workunit.client.1.smithi192.stdout:6/899: truncate da/df/f8d 2604222 0 2024-02-16T22:56:35.158 INFO:tasks.workunit.client.0.smithi064.stdout:0/645: link da/dc/de/d12/d2c/d2e/l5b da/dc/d3b/d72/db3/ldb 0 2024-02-16T22:56:35.161 INFO:tasks.workunit.client.1.smithi192.stdout:3/815: mkdir da/d11/d20/d39/d116 0 2024-02-16T22:56:35.163 INFO:tasks.workunit.client.0.smithi064.stdout:5/707: write d6/d18/f56 [2934875,82802] 0 2024-02-16T22:56:35.167 INFO:tasks.workunit.client.1.smithi192.stdout:1/836: rename d3/de/d27/d34/dbf to d3/de/d27/d34/d69/d85/d11f 0 2024-02-16T22:56:35.168 INFO:tasks.workunit.client.1.smithi192.stdout:8/869: dread d5/d26/d83/db4/fba [0,4194304] 0 2024-02-16T22:56:35.170 INFO:tasks.workunit.client.1.smithi192.stdout:3/816: mknod da/df/c117 0 2024-02-16T22:56:35.172 INFO:tasks.workunit.client.1.smithi192.stdout:0/887: unlink d1/d3/de/d22/d37/d47/fa5 0 2024-02-16T22:56:35.176 INFO:tasks.workunit.client.1.smithi192.stdout:5/883: link d6/d12/d1d/f3a d6/d12/d1d/f11f 0 2024-02-16T22:56:35.176 INFO:tasks.workunit.client.1.smithi192.stdout:5/884: stat d6/f9d 0 2024-02-16T22:56:35.177 INFO:tasks.workunit.client.1.smithi192.stdout:8/870: mkdir d5/db/d47/d43/d67/d112 0 2024-02-16T22:56:35.178 INFO:tasks.workunit.client.1.smithi192.stdout:1/837: dwrite d3/de/d27/d34/d36/d47/d5d/fce [0,4194304] 0 2024-02-16T22:56:35.182 INFO:tasks.workunit.client.0.smithi064.stdout:3/828: creat d4/d1f/d5b/d51/dc1/f116 x:0 0 0 2024-02-16T22:56:35.183 INFO:tasks.workunit.client.1.smithi192.stdout:3/817: creat da/d11/d63/d6b/d66/d75/f118 x:0 0 0 2024-02-16T22:56:35.184 INFO:tasks.workunit.client.1.smithi192.stdout:1/838: dread d3/de/d27/d34/d69/fef [0,4194304] 0 2024-02-16T22:56:35.185 INFO:tasks.workunit.client.0.smithi064.stdout:7/741: getdents d5/d6 0 2024-02-16T22:56:35.185 INFO:tasks.workunit.client.1.smithi192.stdout:1/839: write d3/d5b/d77/f11d [351950,39939] 0 2024-02-16T22:56:35.186 INFO:tasks.workunit.client.0.smithi064.stdout:7/742: dread - d5/d33/d45/d73/d88/fc5 zero size 2024-02-16T22:56:35.186 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:34 smithi192 ceph-mon[36254]: pgmap v22: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 122 MiB/s rd, 283 MiB/s wr, 609 op/s 2024-02-16T22:56:35.187 INFO:tasks.workunit.client.1.smithi192.stdout:5/885: mkdir d6/d12/d2d/d5c/df6/d120 0 2024-02-16T22:56:35.190 INFO:tasks.workunit.client.0.smithi064.stdout:5/708: truncate d6/f17 1418860 0 2024-02-16T22:56:35.190 INFO:tasks.workunit.client.0.smithi064.stdout:5/709: stat d6/d18/d19/fb0 0 2024-02-16T22:56:35.192 INFO:tasks.workunit.client.0.smithi064.stdout:5/710: fsync d6/d25/d48/d67/d91/da5/de3/fd7 0 2024-02-16T22:56:35.192 INFO:tasks.workunit.client.1.smithi192.stdout:6/900: link da/df/d12/d57/fc6 da/d76/dd5/de2/f11f 0 2024-02-16T22:56:35.194 INFO:tasks.workunit.client.1.smithi192.stdout:8/871: dwrite d5/d26/d83/d86/f74 [0,4194304] 0 2024-02-16T22:56:35.195 INFO:tasks.workunit.client.1.smithi192.stdout:3/818: chown da/d11/d20/d39/d53/fce 1792 1 2024-02-16T22:56:35.196 INFO:tasks.workunit.client.1.smithi192.stdout:3/819: fsync da/d107/f113 0 2024-02-16T22:56:35.196 INFO:tasks.workunit.client.0.smithi064.stdout:2/904: mknod d7/d14/d1e/d34/d3f/d9a/dd9/d95/c12e 0 2024-02-16T22:56:35.200 INFO:tasks.workunit.client.1.smithi192.stdout:7/981: dwrite d1/d2/de/d2c/d49/da4/fcf [0,4194304] 0 2024-02-16T22:56:35.202 INFO:tasks.workunit.client.1.smithi192.stdout:5/886: rmdir d6/d12/d4e/de6 39 2024-02-16T22:56:35.204 INFO:tasks.workunit.client.0.smithi064.stdout:3/829: mkdir d4/d13/d4d/d5c/d78/d117 0 2024-02-16T22:56:35.205 INFO:tasks.workunit.client.0.smithi064.stdout:3/830: fsync d4/d1f/f20 0 2024-02-16T22:56:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:3/831: fsync d4/d1f/f67 0 2024-02-16T22:56:35.209 INFO:tasks.workunit.client.1.smithi192.stdout:6/901: mkdir da/df/d12/d15/d1b/d70/d120 0 2024-02-16T22:56:35.216 INFO:tasks.workunit.client.0.smithi064.stdout:5/711: symlink d6/d25/d48/d67/d91/da5/de3/d4b/lef 0 2024-02-16T22:56:35.217 INFO:tasks.workunit.client.0.smithi064.stdout:5/712: stat d6/d25/d48/d67/f8f 0 2024-02-16T22:56:35.217 INFO:tasks.workunit.client.1.smithi192.stdout:3/820: creat da/df/f119 x:0 0 0 2024-02-16T22:56:35.218 INFO:tasks.workunit.client.0.smithi064.stdout:1/665: rmdir d2/df/da8 0 2024-02-16T22:56:35.220 INFO:tasks.workunit.client.1.smithi192.stdout:7/982: symlink d1/d2/de/d2c/d49/d54/d9d/ddd/d13d/l14b 0 2024-02-16T22:56:35.221 INFO:tasks.workunit.client.1.smithi192.stdout:5/887: dread - d6/d12/d4e/d5b/fdf zero size 2024-02-16T22:56:35.222 INFO:tasks.workunit.client.0.smithi064.stdout:2/905: dread d7/d14/d25/d46/f97 [0,4194304] 0 2024-02-16T22:56:35.223 INFO:tasks.workunit.client.1.smithi192.stdout:6/902: creat da/d76/dd5/f121 x:0 0 0 2024-02-16T22:56:35.224 INFO:tasks.workunit.client.1.smithi192.stdout:6/903: chown da/df/d12/d57/f35 12829453 1 2024-02-16T22:56:35.225 INFO:tasks.workunit.client.0.smithi064.stdout:9/763: rename d3/d9/d91 to d3/dfb 0 2024-02-16T22:56:35.228 INFO:tasks.workunit.client.1.smithi192.stdout:6/904: read da/d40/f4e [3478502,2118] 0 2024-02-16T22:56:35.229 INFO:tasks.workunit.client.1.smithi192.stdout:3/821: unlink da/d11/d35/fd8 0 2024-02-16T22:56:35.230 INFO:tasks.workunit.client.1.smithi192.stdout:3/822: truncate da/d11/d2b/ddc/f109 749920 0 2024-02-16T22:56:35.230 INFO:tasks.workunit.client.1.smithi192.stdout:7/983: write d1/d2/d2d/d61/d88/d90/f1d [8986801,13974] 0 2024-02-16T22:56:35.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/984: write d1/de1/dd0/fd2 [268114,50576] 0 2024-02-16T22:56:35.233 INFO:tasks.workunit.client.1.smithi192.stdout:5/888: unlink d6/d12/d4e/d5b/daf/fbe 0 2024-02-16T22:56:35.239 INFO:tasks.workunit.client.1.smithi192.stdout:7/985: creat d1/d2/de/d2c/d49/da4/d104/f14c x:0 0 0 2024-02-16T22:56:35.240 INFO:tasks.workunit.client.0.smithi064.stdout:2/906: mkdir d7/d14/d1e/d34/d3f/d9a/dd9/d103/d12f 0 2024-02-16T22:56:35.240 INFO:tasks.workunit.client.0.smithi064.stdout:2/907: chown d7/d14/d25/c3c 11265987 1 2024-02-16T22:56:35.241 INFO:tasks.workunit.client.1.smithi192.stdout:6/905: dwrite da/df/d18/f29 [0,4194304] 0 2024-02-16T22:56:35.243 INFO:tasks.workunit.client.0.smithi064.stdout:2/908: read d7/d14/d1e/d34/d73/f77 [3292413,120265] 0 2024-02-16T22:56:35.244 INFO:tasks.workunit.client.0.smithi064.stdout:7/743: link d5/d6/f19 d5/d6/d54/d41/da2/fef 0 2024-02-16T22:56:35.249 INFO:tasks.workunit.client.1.smithi192.stdout:1/840: dread d3/d40/f94 [0,4194304] 0 2024-02-16T22:56:35.249 INFO:tasks.workunit.client.0.smithi064.stdout:3/832: creat d4/d13/d4d/d5c/d78/d104/f118 x:0 0 0 2024-02-16T22:56:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:7/744: dread d5/d33/d42/d81/f94 [0,4194304] 0 2024-02-16T22:56:35.251 INFO:tasks.workunit.client.1.smithi192.stdout:7/986: unlink d1/d2/de/f99 0 2024-02-16T22:56:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:7/745: truncate d5/d6/d65/fe5 902681 0 2024-02-16T22:56:35.252 INFO:tasks.workunit.client.1.smithi192.stdout:2/951: write d22/d24/d39/fe7 [519528,84981] 0 2024-02-16T22:56:35.253 INFO:tasks.workunit.client.1.smithi192.stdout:6/906: symlink da/df/d6b/d83/l122 0 2024-02-16T22:56:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:7/746: write d5/d6/d65/da7/fce [977926,95415] 0 2024-02-16T22:56:35.256 INFO:tasks.workunit.client.1.smithi192.stdout:1/841: rename d3/fb to d3/de/d27/d34/de7/f120 0 2024-02-16T22:56:35.258 INFO:tasks.workunit.client.1.smithi192.stdout:1/842: write d3/d5b/d77/f11d [1361760,5887] 0 2024-02-16T22:56:35.259 INFO:tasks.workunit.client.1.smithi192.stdout:1/843: chown d3/d40/d9f/df0 3558035 1 2024-02-16T22:56:35.260 INFO:tasks.workunit.client.1.smithi192.stdout:4/932: dwrite d1/d15/d29/d3c/f43 [4194304,4194304] 0 2024-02-16T22:56:35.261 INFO:tasks.workunit.client.0.smithi064.stdout:3/833: unlink d4/d13/d1b/d27/d61/d63/ldd 0 2024-02-16T22:56:35.261 INFO:tasks.workunit.client.1.smithi192.stdout:7/987: read d1/d2/de/d2c/d49/d54/de3/fd3 [1121335,6965] 0 2024-02-16T22:56:35.262 INFO:tasks.workunit.client.1.smithi192.stdout:2/952: symlink d22/d47/d121/l139 0 2024-02-16T22:56:35.263 INFO:tasks.workunit.client.0.smithi064.stdout:5/713: creat d6/ff0 x:0 0 0 2024-02-16T22:56:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:7/747: mkdir d5/d7d/df0 0 2024-02-16T22:56:35.273 INFO:tasks.workunit.client.0.smithi064.stdout:8/677: dwrite df/d31/d2f/f7d [0,4194304] 0 2024-02-16T22:56:35.274 INFO:tasks.workunit.client.0.smithi064.stdout:5/714: dwrite d6/d25/d48/d67/d91/fcd [0,4194304] 0 2024-02-16T22:56:35.276 INFO:tasks.workunit.client.0.smithi064.stdout:4/780: rename dc/dd/d16/d9c/fad to dc/d1d/d43/ff6 0 2024-02-16T22:56:35.276 INFO:tasks.workunit.client.0.smithi064.stdout:1/666: dread d2/df/d11/d1a/d30/dbd/d91/fd7 [0,4194304] 0 2024-02-16T22:56:35.279 INFO:tasks.workunit.client.0.smithi064.stdout:1/667: chown d2/df/d14/d18/fb6 38353817 1 2024-02-16T22:56:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:1/668: dwrite d2/df/d11/f3b [0,4194304] 0 2024-02-16T22:56:35.287 INFO:tasks.workunit.client.1.smithi192.stdout:7/988: rename d1/d2/de/d83/d9f/c11f to d1/d2/d2d/ddf/d113/c14d 0 2024-02-16T22:56:35.287 INFO:tasks.workunit.client.1.smithi192.stdout:5/889: dread d6/d12/d4e/d5b/f5e [0,4194304] 0 2024-02-16T22:56:35.288 INFO:tasks.workunit.client.1.smithi192.stdout:7/989: chown d1/d2/de/d142/f13c 54 1 2024-02-16T22:56:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:7/748: fsync d5/d33/d45/d73/d88/fc5 0 2024-02-16T22:56:35.293 INFO:tasks.workunit.client.1.smithi192.stdout:5/890: dwrite d6/d12/d4e/d5b/fed [0,4194304] 0 2024-02-16T22:56:35.297 INFO:tasks.workunit.client.1.smithi192.stdout:2/953: rename d22/d47/dcb/d78/f131 to d22/d47/d121/f13a 0 2024-02-16T22:56:35.302 INFO:tasks.workunit.client.0.smithi064.stdout:2/909: rename d7/d14/d1e/d34/d3f/la0 to d7/d14/d1e/d34/d3f/d9a/dd9/d103/d12f/l130 0 2024-02-16T22:56:35.305 INFO:tasks.workunit.client.1.smithi192.stdout:0/888: write d1/d3/de/d22/d37/fa0 [1001063,98566] 0 2024-02-16T22:56:35.305 INFO:tasks.workunit.client.1.smithi192.stdout:1/844: link d3/de/d27/d34/f4e d3/d40/d9f/df0/f121 0 2024-02-16T22:56:35.308 INFO:tasks.workunit.client.1.smithi192.stdout:5/891: write d6/fd [1653161,56686] 0 2024-02-16T22:56:35.313 INFO:tasks.workunit.client.1.smithi192.stdout:2/954: mkdir d22/d24/d39/d42/d85/d98/db0/db6/d13b 0 2024-02-16T22:56:35.315 INFO:tasks.workunit.client.1.smithi192.stdout:2/955: write d22/d24/d39/d42/d65/d69/db1/fbe [4979334,59306] 0 2024-02-16T22:56:35.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/889: creat d1/d3/d39/df3/f11f x:0 0 0 2024-02-16T22:56:35.316 INFO:tasks.workunit.client.0.smithi064.stdout:0/646: dwrite da/dc/de/d12/d1d/f2b [0,4194304] 0 2024-02-16T22:56:35.318 INFO:tasks.workunit.client.1.smithi192.stdout:0/890: chown d1/d3/d18/l10c 26104472 1 2024-02-16T22:56:35.319 INFO:tasks.workunit.client.1.smithi192.stdout:8/872: dwrite d5/db/de/ff0 [0,4194304] 0 2024-02-16T22:56:35.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/933: getdents d1/d26/d6f/df4 0 2024-02-16T22:56:35.324 INFO:tasks.workunit.client.1.smithi192.stdout:5/892: fdatasync d6/d12/d4e/d5b/d67/d78/feb 0 2024-02-16T22:56:35.324 INFO:tasks.workunit.client.0.smithi064.stdout:4/781: dread dc/ddc/de8/f9e [0,4194304] 0 2024-02-16T22:56:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:7/749: dread d5/d6/d54/d7b/fb1 [0,4194304] 0 2024-02-16T22:56:35.331 INFO:tasks.workunit.client.1.smithi192.stdout:2/956: creat d22/d47/dcb/dda/f13c x:0 0 0 2024-02-16T22:56:35.331 INFO:tasks.workunit.client.1.smithi192.stdout:2/957: stat d22/d24/d39/d3d/df2/d67/fd3 0 2024-02-16T22:56:35.333 INFO:tasks.workunit.client.1.smithi192.stdout:2/958: read d22/d24/d39/fe7 [331486,52304] 0 2024-02-16T22:56:35.334 INFO:tasks.workunit.client.1.smithi192.stdout:0/891: symlink d1/d3/de2/dff/l120 0 2024-02-16T22:56:35.335 INFO:tasks.workunit.client.1.smithi192.stdout:8/873: readlink d5/d16/l46 0 2024-02-16T22:56:35.335 INFO:tasks.workunit.client.1.smithi192.stdout:2/959: truncate d22/d24/d39/d42/d85/d98/db0/f122 712711 0 2024-02-16T22:56:35.336 INFO:tasks.workunit.client.1.smithi192.stdout:2/960: dread - d22/d24/d39/d42/d65/f136 zero size 2024-02-16T22:56:35.337 INFO:tasks.workunit.client.0.smithi064.stdout:2/910: symlink d7/d14/d1e/d34/d3f/d9a/dd9/l131 0 2024-02-16T22:56:35.338 INFO:tasks.workunit.client.0.smithi064.stdout:2/911: fsync d7/d14/d25/d46/d55/d7f/fdf 0 2024-02-16T22:56:35.338 INFO:tasks.workunit.client.1.smithi192.stdout:4/934: creat d1/d26/d66/d89/d11/f13d x:0 0 0 2024-02-16T22:56:35.338 INFO:tasks.workunit.client.1.smithi192.stdout:3/823: write da/f1a [680937,1589] 0 2024-02-16T22:56:35.339 INFO:tasks.workunit.client.1.smithi192.stdout:3/824: read da/d11/d2b/ff3 [8766,115039] 0 2024-02-16T22:56:35.340 INFO:tasks.workunit.client.0.smithi064.stdout:1/669: creat d2/df/d14/d2c/fe0 x:0 0 0 2024-02-16T22:56:35.342 INFO:tasks.workunit.client.1.smithi192.stdout:5/893: rmdir d6/d8/d75/d8c 39 2024-02-16T22:56:35.342 INFO:tasks.workunit.client.1.smithi192.stdout:5/894: chown d6/d8/c24 0 1 2024-02-16T22:56:35.343 INFO:tasks.workunit.client.0.smithi064.stdout:4/782: read dc/ddc/de8/fcf [1208355,99557] 0 2024-02-16T22:56:35.343 INFO:tasks.workunit.client.0.smithi064.stdout:7/750: rmdir d5/d33/d45/d73/d88 39 2024-02-16T22:56:35.344 INFO:tasks.workunit.client.1.smithi192.stdout:5/895: write d6/d12/d4e/d5b/daf/fcd [4574366,9957] 0 2024-02-16T22:56:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:0/647: fdatasync da/dc/de/d12/f42 0 2024-02-16T22:56:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:5/715: rename d6/d25/d48/d67 to d6/d25/df1 0 2024-02-16T22:56:35.353 INFO:tasks.workunit.client.1.smithi192.stdout:0/892: mkdir d1/d3/d39/df3/d121 0 2024-02-16T22:56:35.354 INFO:tasks.workunit.client.1.smithi192.stdout:6/907: write da/df/d6b/d74/f9d [568354,73490] 0 2024-02-16T22:56:35.355 INFO:tasks.workunit.client.1.smithi192.stdout:8/874: symlink d5/d16/dd5/l113 0 2024-02-16T22:56:35.356 INFO:tasks.workunit.client.0.smithi064.stdout:2/912: mkdir d7/d14/d4b/d132 0 2024-02-16T22:56:35.357 INFO:tasks.workunit.client.1.smithi192.stdout:8/875: write d5/fae [340436,70738] 0 2024-02-16T22:56:35.357 INFO:tasks.workunit.client.1.smithi192.stdout:4/935: dread d1/d15/f4f [0,4194304] 0 2024-02-16T22:56:35.358 INFO:tasks.workunit.client.1.smithi192.stdout:2/961: creat d22/d47/dcb/dc9/f13d x:0 0 0 2024-02-16T22:56:35.360 INFO:tasks.workunit.client.0.smithi064.stdout:7/751: rmdir d5/d6/d54/d7b 39 2024-02-16T22:56:35.361 INFO:tasks.workunit.client.1.smithi192.stdout:0/893: dwrite d1/d3/de/d22/d37/d67/d75/db0/f116 [0,4194304] 0 2024-02-16T22:56:35.362 INFO:tasks.workunit.client.0.smithi064.stdout:3/834: write d4/d13/d35/f3e [811262,60092] 0 2024-02-16T22:56:35.363 INFO:tasks.workunit.client.1.smithi192.stdout:3/825: chown da/d11/d20/d39/d53/cf1 38075172 1 2024-02-16T22:56:35.364 INFO:tasks.workunit.client.1.smithi192.stdout:7/990: stat d1/d2/d2d/ddf/d113/c14d 0 2024-02-16T22:56:35.367 INFO:tasks.workunit.client.1.smithi192.stdout:2/962: dwrite d22/d24/f49 [8388608,4194304] 0 2024-02-16T22:56:35.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:34 smithi064 ceph-mon[29226]: pgmap v22: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 122 MiB/s rd, 283 MiB/s wr, 609 op/s 2024-02-16T22:56:35.376 INFO:tasks.workunit.client.1.smithi192.stdout:0/894: dread d1/ffe [0,4194304] 0 2024-02-16T22:56:35.387 INFO:tasks.workunit.client.0.smithi064.stdout:6/827: sync 2024-02-16T22:56:35.389 INFO:tasks.workunit.client.0.smithi064.stdout:6/828: read d0/d5/df/f86 [4629625,99723] 0 2024-02-16T22:56:35.391 INFO:tasks.workunit.client.0.smithi064.stdout:6/829: write d0/d1b/f23 [3981961,27091] 0 2024-02-16T22:56:35.393 INFO:tasks.workunit.client.0.smithi064.stdout:4/783: rename dc/f81 to dc/dd/d32/d46/d5a/d7e/d9a/da2/ff7 0 2024-02-16T22:56:35.395 INFO:tasks.workunit.client.0.smithi064.stdout:8/678: write f8 [484551,117158] 0 2024-02-16T22:56:35.395 INFO:tasks.workunit.client.0.smithi064.stdout:2/913: mkdir d7/d14/d1e/d133 0 2024-02-16T22:56:35.396 INFO:tasks.workunit.client.0.smithi064.stdout:2/914: chown d7/c45 212463 1 2024-02-16T22:56:35.402 INFO:tasks.workunit.client.0.smithi064.stdout:3/835: symlink d4/d13/d1b/d27/d82/l119 0 2024-02-16T22:56:35.407 INFO:tasks.workunit.client.1.smithi192.stdout:4/936: rmdir d1/d15/d48/d8d 39 2024-02-16T22:56:35.408 INFO:tasks.workunit.client.0.smithi064.stdout:0/648: read - da/dc/de/d37/daa/db4/fba zero size 2024-02-16T22:56:35.409 INFO:tasks.workunit.client.1.smithi192.stdout:8/876: creat d5/db/d47/f114 x:0 0 0 2024-02-16T22:56:35.414 INFO:tasks.workunit.client.0.smithi064.stdout:6/830: creat d0/d6/d10/d33/d72/de8/dee/f113 x:0 0 0 2024-02-16T22:56:35.415 INFO:tasks.workunit.client.0.smithi064.stdout:4/784: rmdir dc/d4a 39 2024-02-16T22:56:35.428 INFO:tasks.workunit.client.0.smithi064.stdout:3/836: symlink d4/d13/d1b/d27/d44/l11a 0 2024-02-16T22:56:35.428 INFO:tasks.workunit.client.0.smithi064.stdout:0/649: symlink da/dc/de/d37/d88/ldc 0 2024-02-16T22:56:35.431 INFO:tasks.workunit.client.0.smithi064.stdout:6/831: creat d0/d6/d10/d33/da5/f114 x:0 0 0 2024-02-16T22:56:35.433 INFO:tasks.workunit.client.0.smithi064.stdout:4/785: mkdir dc/dd/d32/d46/d5a/d7e/d9a/df8 0 2024-02-16T22:56:35.434 INFO:tasks.workunit.client.0.smithi064.stdout:8/679: unlink df/d31/d2f/c33 0 2024-02-16T22:56:35.435 INFO:tasks.workunit.client.0.smithi064.stdout:1/670: write d2/df/d14/d2c/f5f [2883761,7200] 0 2024-02-16T22:56:35.435 INFO:tasks.workunit.client.0.smithi064.stdout:7/752: link d5/d6/f22 d5/d6/ff1 0 2024-02-16T22:56:35.435 INFO:tasks.workunit.client.0.smithi064.stdout:1/671: chown d2/l7 0 1 2024-02-16T22:56:35.436 INFO:tasks.workunit.client.1.smithi192.stdout:3/826: unlink da/d11/d20/d39/fc4 0 2024-02-16T22:56:35.437 INFO:tasks.workunit.client.1.smithi192.stdout:2/963: mknod d22/d47/dcb/d3f/dac/c13e 0 2024-02-16T22:56:35.437 INFO:tasks.workunit.client.1.smithi192.stdout:0/895: mknod d1/d3/de/d22/d37/d67/c122 0 2024-02-16T22:56:35.437 INFO:tasks.workunit.client.0.smithi064.stdout:3/837: creat d4/d1f/f11b x:0 0 0 2024-02-16T22:56:35.438 INFO:tasks.workunit.client.0.smithi064.stdout:0/650: fsync da/dc/d3b/d72/d7c/fa5 0 2024-02-16T22:56:35.438 INFO:tasks.workunit.client.1.smithi192.stdout:8/877: unlink d5/d26/d8f/c56 0 2024-02-16T22:56:35.438 INFO:tasks.workunit.client.1.smithi192.stdout:5/896: write d6/d8/f9e [492460,8215] 0 2024-02-16T22:56:35.439 INFO:tasks.workunit.client.1.smithi192.stdout:8/878: dread - d5/db/de/fe9 zero size 2024-02-16T22:56:35.441 INFO:tasks.workunit.client.0.smithi064.stdout:6/832: creat d0/d6/d10/d33/da4/f115 x:0 0 0 2024-02-16T22:56:35.443 INFO:tasks.workunit.client.0.smithi064.stdout:4/786: mknod dc/de/d28/d57/d77/cf9 0 2024-02-16T22:56:35.443 INFO:tasks.workunit.client.0.smithi064.stdout:3/838: dwrite d4/d1f/d34/ff9 [0,4194304] 0 2024-02-16T22:56:35.445 INFO:tasks.workunit.client.1.smithi192.stdout:8/879: dread d5/d26/fa0 [0,4194304] 0 2024-02-16T22:56:35.448 INFO:tasks.workunit.client.1.smithi192.stdout:4/937: unlink d1/d26/d66/d89/c22 0 2024-02-16T22:56:35.449 INFO:tasks.workunit.client.1.smithi192.stdout:4/938: write d1/d26/d5e/f63 [5171472,29034] 0 2024-02-16T22:56:35.450 INFO:tasks.workunit.client.1.smithi192.stdout:3/827: mkdir da/d11/d63/d6b/d4e/d11a 0 2024-02-16T22:56:35.452 INFO:tasks.workunit.client.0.smithi064.stdout:2/915: write d7/d14/d25/d46/f97 [537476,56631] 0 2024-02-16T22:56:35.453 INFO:tasks.workunit.client.1.smithi192.stdout:3/828: truncate da/d11/d63/d6b/d66/fe3 1115429 0 2024-02-16T22:56:35.454 INFO:tasks.workunit.client.0.smithi064.stdout:2/916: dread - d7/d14/d1e/d34/dad/d10c/dea/f67 zero size 2024-02-16T22:56:35.457 INFO:tasks.workunit.client.1.smithi192.stdout:5/897: truncate d6/d12/d1d/f34 853594 0 2024-02-16T22:56:35.458 INFO:tasks.workunit.client.1.smithi192.stdout:6/908: write da/d40/db6/db8/fdd [244356,63759] 0 2024-02-16T22:56:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:3/839: creat d4/d13/d1b/d27/d99/f11c x:0 0 0 2024-02-16T22:56:35.464 INFO:tasks.workunit.client.0.smithi064.stdout:1/672: mkdir d2/df/d14/d2c/de1 0 2024-02-16T22:56:35.466 INFO:tasks.workunit.client.1.smithi192.stdout:2/964: symlink d22/d24/d39/d42/d65/d69/l13f 0 2024-02-16T22:56:35.466 INFO:tasks.workunit.client.1.smithi192.stdout:0/896: dread d1/dc7/d9c/fc4 [0,4194304] 0 2024-02-16T22:56:35.466 INFO:tasks.workunit.client.1.smithi192.stdout:2/965: stat d22/d47/dca 0 2024-02-16T22:56:35.467 INFO:tasks.workunit.client.1.smithi192.stdout:3/829: mkdir da/d11/d20/d39/d53/dc7/d11b 0 2024-02-16T22:56:35.471 INFO:tasks.workunit.client.0.smithi064.stdout:8/680: dread df/f3d [0,4194304] 0 2024-02-16T22:56:35.472 INFO:tasks.workunit.client.0.smithi064.stdout:8/681: dread - df/d31/d8f/dc9/d73/fc8 zero size 2024-02-16T22:56:35.472 INFO:tasks.workunit.client.1.smithi192.stdout:0/897: dwrite d1/d3/d39/df3/f11f [0,4194304] 0 2024-02-16T22:56:35.472 INFO:tasks.workunit.client.0.smithi064.stdout:8/682: stat df/d31/d8f/d80 0 2024-02-16T22:56:35.473 INFO:tasks.workunit.client.1.smithi192.stdout:0/898: stat d1/dc7/d9c/f72 0 2024-02-16T22:56:35.473 INFO:tasks.workunit.client.0.smithi064.stdout:1/673: rename d2/df/d11/d1a/d1d/f61 to d2/df/d11/d1a/d30/dbd/fe2 0 2024-02-16T22:56:35.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/845: sync 2024-02-16T22:56:35.477 INFO:tasks.workunit.client.0.smithi064.stdout:0/651: creat da/dc/fdd x:0 0 0 2024-02-16T22:56:35.478 INFO:tasks.workunit.client.1.smithi192.stdout:1/846: read - d3/de/d27/d34/d52/db7/ff2 zero size 2024-02-16T22:56:35.478 INFO:tasks.workunit.client.1.smithi192.stdout:8/880: dread d5/d26/d83/fdb [0,4194304] 0 2024-02-16T22:56:35.478 INFO:tasks.workunit.client.0.smithi064.stdout:4/787: creat dc/d1d/d43/ffa x:0 0 0 2024-02-16T22:56:35.479 INFO:tasks.workunit.client.0.smithi064.stdout:4/788: stat dc/ddc/de8/df2 0 2024-02-16T22:56:35.482 INFO:tasks.workunit.client.1.smithi192.stdout:4/939: write d1/d26/dbe/ff6 [912663,2093] 0 2024-02-16T22:56:35.482 INFO:tasks.workunit.client.1.smithi192.stdout:3/830: read da/df/f1f [2402475,67901] 0 2024-02-16T22:56:35.487 INFO:tasks.workunit.client.0.smithi064.stdout:0/652: dwrite da/dc/d3b/d72/db3/fda [0,4194304] 0 2024-02-16T22:56:35.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/898: write d6/d12/d2d/d5c/df6/dbc/d102/f104 [236541,35972] 0 2024-02-16T22:56:35.488 INFO:tasks.workunit.client.1.smithi192.stdout:5/899: stat d6/d12/f18 0 2024-02-16T22:56:35.491 INFO:tasks.workunit.client.1.smithi192.stdout:0/899: creat d1/d3/de/d22/da7/f123 x:0 0 0 2024-02-16T22:56:35.492 INFO:tasks.workunit.client.0.smithi064.stdout:8/683: dread df/d31/d2f/d38/f5d [0,4194304] 0 2024-02-16T22:56:35.494 INFO:tasks.workunit.client.1.smithi192.stdout:1/847: creat d3/de/d87/f122 x:0 0 0 2024-02-16T22:56:35.495 INFO:tasks.workunit.client.1.smithi192.stdout:1/848: dread - d3/d5b/d77/dd2/f106 zero size 2024-02-16T22:56:35.495 INFO:tasks.workunit.client.0.smithi064.stdout:4/789: mkdir dc/de/dfb 0 2024-02-16T22:56:35.496 INFO:tasks.workunit.client.0.smithi064.stdout:3/840: getdents d4/d13/d1b/d4f 0 2024-02-16T22:56:35.496 INFO:tasks.workunit.client.1.smithi192.stdout:4/940: mknod d1/d26/d66/d89/d11/c13e 0 2024-02-16T22:56:35.497 INFO:tasks.workunit.client.1.smithi192.stdout:4/941: chown d1/d15/d29/dd4/cf8 190 1 2024-02-16T22:56:35.498 INFO:tasks.workunit.client.1.smithi192.stdout:3/831: truncate da/d11/d63/d6b/d66/f92 2356941 0 2024-02-16T22:56:35.499 INFO:tasks.workunit.client.0.smithi064.stdout:8/684: mkdir df/d31/d8f/d80/dec 0 2024-02-16T22:56:35.500 INFO:tasks.workunit.client.1.smithi192.stdout:3/832: truncate da/d11/d63/fa6 990136 0 2024-02-16T22:56:35.500 INFO:tasks.workunit.client.1.smithi192.stdout:5/900: mknod d6/d12/d4e/d5b/daf/c121 0 2024-02-16T22:56:35.502 INFO:tasks.workunit.client.0.smithi064.stdout:3/841: dread - d4/d14/d2b/fe9 zero size 2024-02-16T22:56:35.503 INFO:tasks.workunit.client.0.smithi064.stdout:3/842: readlink d4/d13/d1b/d48/lc6 0 2024-02-16T22:56:35.503 INFO:tasks.workunit.client.1.smithi192.stdout:6/909: creat da/df/d18/f123 x:0 0 0 2024-02-16T22:56:35.504 INFO:tasks.workunit.client.1.smithi192.stdout:1/849: symlink d3/de/d27/d34/d69/d85/d89/daa/de6/l123 0 2024-02-16T22:56:35.504 INFO:tasks.workunit.client.1.smithi192.stdout:4/942: dwrite d1/f13a [0,4194304] 0 2024-02-16T22:56:35.506 INFO:tasks.workunit.client.1.smithi192.stdout:2/966: dread d22/d24/d39/d42/d84/fd4 [0,4194304] 0 2024-02-16T22:56:35.508 INFO:tasks.workunit.client.0.smithi064.stdout:3/843: read d4/d1f/d5b/fd5 [4094953,58451] 0 2024-02-16T22:56:35.515 INFO:tasks.workunit.client.1.smithi192.stdout:6/910: creat da/df/d71/dab/f124 x:0 0 0 2024-02-16T22:56:35.516 INFO:tasks.workunit.client.0.smithi064.stdout:3/844: creat d4/d13/d1b/d27/d99/d7d/f11d x:0 0 0 2024-02-16T22:56:35.517 INFO:tasks.workunit.client.1.smithi192.stdout:8/881: dread d5/f61 [0,4194304] 0 2024-02-16T22:56:35.518 INFO:tasks.workunit.client.1.smithi192.stdout:1/850: rename d3/de/d87/fa9 to d3/de/d27/d34/d52/d63/d7a/db0/f124 0 2024-02-16T22:56:35.518 INFO:tasks.workunit.client.1.smithi192.stdout:8/882: write d5/d16/f4f [2476382,45238] 0 2024-02-16T22:56:35.521 INFO:tasks.workunit.client.0.smithi064.stdout:3/845: creat d4/d13/d4d/d5c/d78/d117/f11e x:0 0 0 2024-02-16T22:56:35.523 INFO:tasks.workunit.client.0.smithi064.stdout:1/674: dwrite d2/df/d14/d18/d48/fb8 [0,4194304] 0 2024-02-16T22:56:35.524 INFO:tasks.workunit.client.0.smithi064.stdout:1/675: readlink d2/df/d11/d25/l9d 0 2024-02-16T22:56:35.531 INFO:tasks.workunit.client.0.smithi064.stdout:8/685: write df/d31/f7c [400717,39068] 0 2024-02-16T22:56:35.532 INFO:tasks.workunit.client.0.smithi064.stdout:0/653: dwrite da/dc/d3b/f3c [0,4194304] 0 2024-02-16T22:56:35.533 INFO:tasks.workunit.client.0.smithi064.stdout:1/676: getdents d2/df/d11 0 2024-02-16T22:56:35.535 INFO:tasks.workunit.client.0.smithi064.stdout:8/686: creat df/d2a/d5c/fed x:0 0 0 2024-02-16T22:56:35.536 INFO:tasks.workunit.client.0.smithi064.stdout:0/654: mknod da/d71/cde 0 2024-02-16T22:56:35.538 INFO:tasks.workunit.client.0.smithi064.stdout:4/790: truncate dc/d4a/f4d 1882033 0 2024-02-16T22:56:35.540 INFO:tasks.workunit.client.0.smithi064.stdout:8/687: mkdir df/d31/d2f/d3e/d5f/dee 0 2024-02-16T22:56:35.544 INFO:tasks.workunit.client.0.smithi064.stdout:0/655: mknod da/dc/d3b/dd9/cdf 0 2024-02-16T22:56:35.548 INFO:tasks.workunit.client.0.smithi064.stdout:4/791: rename dc/dd/d16/d7c/ld4 to dc/dd/d32/d46/d79/de3/lfc 0 2024-02-16T22:56:35.548 INFO:tasks.workunit.client.0.smithi064.stdout:8/688: dwrite df/d31/d2f/d3e/ddf/fe3 [0,4194304] 0 2024-02-16T22:56:35.552 INFO:tasks.workunit.client.1.smithi192.stdout:3/833: creat da/d11/d20/d39/d53/dc7/d11b/f11c x:0 0 0 2024-02-16T22:56:35.553 INFO:tasks.workunit.client.0.smithi064.stdout:1/677: dread d2/f43 [0,4194304] 0 2024-02-16T22:56:35.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/900: truncate d1/d3/de/d22/da7/fdf 108003 0 2024-02-16T22:56:35.554 INFO:tasks.workunit.client.0.smithi064.stdout:8/689: creat df/d31/d8f/d80/dec/fef x:0 0 0 2024-02-16T22:56:35.555 INFO:tasks.workunit.client.1.smithi192.stdout:2/967: link d22/d24/d39/d42/d85/d98/db0/fc1 d22/d47/dca/f140 0 2024-02-16T22:56:35.555 INFO:tasks.workunit.client.0.smithi064.stdout:8/690: chown df/d31/d2f/d3e/d5f/d93/fe0 2 1 2024-02-16T22:56:35.559 INFO:tasks.workunit.client.1.smithi192.stdout:5/901: dwrite d6/d12/d1d/d56/fde [0,4194304] 0 2024-02-16T22:56:35.560 INFO:tasks.workunit.client.1.smithi192.stdout:5/902: write d6/d12/d4e/d5b/daf/fd6 [1113333,47443] 0 2024-02-16T22:56:35.561 INFO:tasks.workunit.client.1.smithi192.stdout:5/903: fdatasync d6/d12/d2d/fc0 0 2024-02-16T22:56:35.572 INFO:tasks.workunit.client.0.smithi064.stdout:8/691: link df/d2a/d5c/l95 df/d31/d2f/d38/d48/db5/lf0 0 2024-02-16T22:56:35.575 INFO:tasks.workunit.client.0.smithi064.stdout:8/692: symlink df/d31/d2f/d38/d48/lf1 0 2024-02-16T22:56:35.576 INFO:tasks.workunit.client.0.smithi064.stdout:8/693: dread - df/d31/d2f/d7f/fd0 zero size 2024-02-16T22:56:35.576 INFO:tasks.workunit.client.0.smithi064.stdout:8/694: stat df/d31/d2f/d7f 0 2024-02-16T22:56:35.581 INFO:tasks.workunit.client.0.smithi064.stdout:1/678: truncate d2/df/d11/d25/f85 3908678 0 2024-02-16T22:56:35.583 INFO:tasks.workunit.client.0.smithi064.stdout:8/695: dwrite df/d31/d2f/d3e/d5f/fc7 [0,4194304] 0 2024-02-16T22:56:35.588 INFO:tasks.workunit.client.0.smithi064.stdout:8/696: symlink df/d31/lf2 0 2024-02-16T22:56:35.591 INFO:tasks.workunit.client.1.smithi192.stdout:7/991: sync 2024-02-16T22:56:35.592 INFO:tasks.workunit.client.1.smithi192.stdout:7/992: stat d1/d2/de/f24 0 2024-02-16T22:56:35.603 INFO:tasks.workunit.client.1.smithi192.stdout:6/911: creat da/df/d18/d52/da9/f125 x:0 0 0 2024-02-16T22:56:35.603 INFO:tasks.workunit.client.1.smithi192.stdout:6/912: stat da/df/d71/dab/cc7 0 2024-02-16T22:56:35.604 INFO:tasks.workunit.client.1.smithi192.stdout:1/851: symlink d3/de/d27/d34/d69/d85/l125 0 2024-02-16T22:56:35.605 INFO:tasks.workunit.client.1.smithi192.stdout:0/901: mkdir d1/d3/dac/d124 0 2024-02-16T22:56:35.605 INFO:tasks.workunit.client.0.smithi064.stdout:9/764: sync 2024-02-16T22:56:35.606 INFO:tasks.workunit.client.1.smithi192.stdout:0/902: read d1/d3/de/f10 [5450180,102319] 0 2024-02-16T22:56:35.613 INFO:tasks.workunit.client.1.smithi192.stdout:3/834: symlink da/l11d 0 2024-02-16T22:56:35.615 INFO:tasks.workunit.client.1.smithi192.stdout:6/913: unlink da/df/d12/d15/d1b/c96 0 2024-02-16T22:56:35.618 INFO:tasks.workunit.client.1.smithi192.stdout:1/852: chown d3/l43 98 1 2024-02-16T22:56:35.620 INFO:tasks.workunit.client.1.smithi192.stdout:7/993: mknod d1/d2/de/d2c/d49/d54/dd8/d117/d9b/c14e 0 2024-02-16T22:56:35.621 INFO:tasks.workunit.client.1.smithi192.stdout:6/914: mkdir da/db9/d126 0 2024-02-16T22:56:35.622 INFO:tasks.workunit.client.1.smithi192.stdout:7/994: truncate d1/d2/d2d/ddf/d113/f143 172460 0 2024-02-16T22:56:35.623 INFO:tasks.workunit.client.1.smithi192.stdout:1/853: mkdir d3/de/d27/d34/d69/d85/d89/daa/d126 0 2024-02-16T22:56:35.623 INFO:tasks.workunit.client.1.smithi192.stdout:7/995: stat d1/d2/de/d2c/d49/d54/d9d/c10c 0 2024-02-16T22:56:35.624 INFO:tasks.workunit.client.1.smithi192.stdout:5/904: rename d6/d12/f1e to d6/d8/d75/f122 0 2024-02-16T22:56:35.625 INFO:tasks.workunit.client.1.smithi192.stdout:5/905: stat d6/d12/d2d/fc0 0 2024-02-16T22:56:35.625 INFO:tasks.workunit.client.1.smithi192.stdout:2/968: getdents d22/d24/d39/d42/d85 0 2024-02-16T22:56:35.626 INFO:tasks.workunit.client.1.smithi192.stdout:5/906: write d6/fd [4934619,93459] 0 2024-02-16T22:56:35.627 INFO:tasks.workunit.client.1.smithi192.stdout:2/969: read - d22/d47/f134 zero size 2024-02-16T22:56:35.629 INFO:tasks.workunit.client.1.smithi192.stdout:4/943: sync 2024-02-16T22:56:35.631 INFO:tasks.workunit.client.1.smithi192.stdout:4/944: fdatasync d1/d26/d5e/f63 0 2024-02-16T22:56:35.632 INFO:tasks.workunit.client.1.smithi192.stdout:7/996: mkdir d1/de1/d14f 0 2024-02-16T22:56:35.634 INFO:tasks.workunit.client.1.smithi192.stdout:5/907: symlink d6/d12/d4e/d58/d8f/l123 0 2024-02-16T22:56:35.635 INFO:tasks.workunit.client.1.smithi192.stdout:0/903: dread d1/d3/de/d22/d37/d47/dd0/f100 [0,4194304] 0 2024-02-16T22:56:35.635 INFO:tasks.workunit.client.1.smithi192.stdout:2/970: read d22/d24/d39/d42/d65/d66/fa4 [47488,15584] 0 2024-02-16T22:56:35.636 INFO:tasks.workunit.client.1.smithi192.stdout:3/835: rename da/df/l2d to da/d11/d2b/ddc/d102/l11e 0 2024-02-16T22:56:35.637 INFO:tasks.workunit.client.1.smithi192.stdout:8/883: dwrite d5/d16/fcc [0,4194304] 0 2024-02-16T22:56:35.641 INFO:tasks.workunit.client.1.smithi192.stdout:5/908: creat d6/d20/f124 x:0 0 0 2024-02-16T22:56:35.643 INFO:tasks.workunit.client.1.smithi192.stdout:7/997: dwrite d1/d2/d2d/d61/d88/d12e/f149 [0,4194304] 0 2024-02-16T22:56:35.643 INFO:tasks.workunit.client.1.smithi192.stdout:0/904: chown d1/c20 801070924 1 2024-02-16T22:56:35.644 INFO:tasks.workunit.client.1.smithi192.stdout:1/854: dread d3/de/d27/d34/d52/d63/d7a/dc9/fe1 [0,4194304] 0 2024-02-16T22:56:35.645 INFO:tasks.workunit.client.1.smithi192.stdout:0/905: chown d1/d3/de/d22/d37/d47/le9 218 1 2024-02-16T22:56:35.645 INFO:tasks.workunit.client.0.smithi064.stdout:8/697: dwrite df/d31/d2f/f59 [0,4194304] 0 2024-02-16T22:56:35.647 INFO:tasks.workunit.client.1.smithi192.stdout:3/836: mkdir da/d11/dd4/d11f 0 2024-02-16T22:56:35.651 INFO:tasks.workunit.client.0.smithi064.stdout:9/765: dwrite d3/d9/d6a/db1/fb9 [0,4194304] 0 2024-02-16T22:56:35.652 INFO:tasks.workunit.client.1.smithi192.stdout:8/884: mkdir d5/d9/d115 0 2024-02-16T22:56:35.658 INFO:tasks.workunit.client.0.smithi064.stdout:9/766: fdatasync d3/d9/d6a/db1/f29 0 2024-02-16T22:56:35.658 INFO:tasks.workunit.client.1.smithi192.stdout:4/945: creat d1/d26/d66/d89/da8/dc5/d111/d122/f13f x:0 0 0 2024-02-16T22:56:35.664 INFO:tasks.workunit.client.1.smithi192.stdout:6/915: dwrite da/df/d12/f51 [0,4194304] 0 2024-02-16T22:56:35.665 INFO:tasks.workunit.client.1.smithi192.stdout:8/885: dwrite d5/db/d47/f4a [0,4194304] 0 2024-02-16T22:56:35.665 INFO:tasks.workunit.client.1.smithi192.stdout:2/971: truncate f4 3245073 0 2024-02-16T22:56:35.667 INFO:tasks.workunit.client.0.smithi064.stdout:9/767: mknod d3/cfc 0 2024-02-16T22:56:35.668 INFO:tasks.workunit.client.1.smithi192.stdout:1/855: truncate d3/de/d27/d34/d36/d47/d5d/fd3 631780 0 2024-02-16T22:56:35.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/886: unlink d5/d16/dd5/l113 0 2024-02-16T22:56:35.672 INFO:tasks.workunit.client.1.smithi192.stdout:2/972: write d22/d47/dcb/f37 [77460,1082] 0 2024-02-16T22:56:35.674 INFO:tasks.workunit.client.1.smithi192.stdout:8/887: readlink d5/db/d47/l76 0 2024-02-16T22:56:35.675 INFO:tasks.workunit.client.1.smithi192.stdout:8/888: dread - d5/db/f109 zero size 2024-02-16T22:56:35.677 INFO:tasks.workunit.client.0.smithi064.stdout:9/768: link d3/d9/d6a/db1/d2a/d2e/d31/d3f/c93 d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/de4/cfd 0 2024-02-16T22:56:35.679 INFO:tasks.workunit.client.1.smithi192.stdout:3/837: link da/d11/l29 da/d11/d35/l120 0 2024-02-16T22:56:35.681 INFO:tasks.workunit.client.0.smithi064.stdout:9/769: read d3/f8f [576706,77308] 0 2024-02-16T22:56:35.684 INFO:tasks.workunit.client.1.smithi192.stdout:0/906: dread d1/d3/fc [0,4194304] 0 2024-02-16T22:56:35.684 INFO:tasks.workunit.client.1.smithi192.stdout:0/907: stat d1/d3/de/d22/d37/d47/f48 0 2024-02-16T22:56:35.685 INFO:tasks.workunit.client.0.smithi064.stdout:8/698: dwrite df/d31/d8f/d80/f84 [0,4194304] 0 2024-02-16T22:56:35.685 INFO:tasks.workunit.client.1.smithi192.stdout:0/908: stat d1/d3/de/d22/d37/d47/d53/ff7 0 2024-02-16T22:56:35.689 INFO:tasks.workunit.client.0.smithi064.stdout:9/770: dwrite d3/d9/f21 [0,4194304] 0 2024-02-16T22:56:35.696 INFO:tasks.workunit.client.0.smithi064.stdout:9/771: dwrite d3/d11/d15/f51 [0,4194304] 0 2024-02-16T22:56:35.699 INFO:tasks.workunit.client.1.smithi192.stdout:6/916: dread da/d40/d85/f94 [0,4194304] 0 2024-02-16T22:56:35.707 INFO:tasks.workunit.client.1.smithi192.stdout:6/917: dwrite da/d40/db6/db8/dfa/f105 [0,4194304] 0 2024-02-16T22:56:35.726 INFO:tasks.workunit.client.0.smithi064.stdout:5/716: sync 2024-02-16T22:56:35.781 INFO:tasks.workunit.client.1.smithi192.stdout:2/973: symlink d22/d24/d39/d42/d85/d98/db0/dcd/l141 0 2024-02-16T22:56:35.782 INFO:tasks.workunit.client.1.smithi192.stdout:8/889: unlink d5/db/d47/d43/d67/c53 0 2024-02-16T22:56:35.782 INFO:tasks.workunit.client.1.smithi192.stdout:7/998: write d1/d2/de/d2c/d49/d54/de3/f119 [498897,90418] 0 2024-02-16T22:56:35.796 INFO:tasks.workunit.client.1.smithi192.stdout:5/909: sync 2024-02-16T22:56:35.803 INFO:tasks.workunit.client.1.smithi192.stdout:2/974: mkdir d22/d47/dcb/d78/d142 0 2024-02-16T22:56:35.804 INFO:tasks.workunit.client.0.smithi064.stdout:9/772: mknod d3/d9/d6a/db1/d2a/d2e/d31/cfe 0 2024-02-16T22:56:35.805 INFO:tasks.workunit.client.1.smithi192.stdout:4/946: truncate d1/d15/d48/f55 972849 0 2024-02-16T22:56:35.806 INFO:tasks.workunit.client.1.smithi192.stdout:1/856: dwrite d3/de/d27/f6a [4194304,4194304] 0 2024-02-16T22:56:35.807 INFO:tasks.workunit.client.1.smithi192.stdout:8/890: symlink d5/d26/d83/d86/d5f/dbf/l116 0 2024-02-16T22:56:35.811 INFO:tasks.workunit.client.1.smithi192.stdout:4/947: dwrite d1/d26/d6f/f8f [0,4194304] 0 2024-02-16T22:56:35.814 INFO:tasks.workunit.client.0.smithi064.stdout:5/717: dread d6/d25/df1/d91/da5/de3/d43/d7d/de2/fe7 [0,4194304] 0 2024-02-16T22:56:35.815 INFO:tasks.workunit.client.1.smithi192.stdout:5/910: dread d6/d12/fc7 [0,4194304] 0 2024-02-16T22:56:35.817 INFO:tasks.workunit.client.0.smithi064.stdout:9/773: rename d3/d9/d1e/d42/c77 to d3/d9/cff 0 2024-02-16T22:56:35.819 INFO:tasks.workunit.client.1.smithi192.stdout:2/975: symlink d22/d24/d39/d42/d85/d98/dc4/l143 0 2024-02-16T22:56:35.822 INFO:tasks.workunit.client.0.smithi064.stdout:5/718: mkdir d6/d25/df2 0 2024-02-16T22:56:35.822 INFO:tasks.workunit.client.1.smithi192.stdout:6/918: dwrite da/df/d58/d72/fea [0,4194304] 0 2024-02-16T22:56:35.825 INFO:tasks.workunit.client.0.smithi064.stdout:9/774: rename d3/d9/d6a/db1/f26 to d3/d89/f100 0 2024-02-16T22:56:35.827 INFO:tasks.workunit.client.1.smithi192.stdout:3/838: creat da/d11/d63/d6b/d66/f121 x:0 0 0 2024-02-16T22:56:35.828 INFO:tasks.workunit.client.1.smithi192.stdout:1/857: creat d3/de/d27/d34/d69/d85/f127 x:0 0 0 2024-02-16T22:56:35.829 INFO:tasks.workunit.client.1.smithi192.stdout:8/891: creat d5/d26/d83/d86/f117 x:0 0 0 2024-02-16T22:56:35.834 INFO:tasks.workunit.client.1.smithi192.stdout:0/909: creat d1/f125 x:0 0 0 2024-02-16T22:56:35.834 INFO:tasks.workunit.client.0.smithi064.stdout:5/719: dwrite d6/d25/df1/d91/da5/de3/d43/f4d [4194304,4194304] 0 2024-02-16T22:56:35.835 INFO:tasks.workunit.client.0.smithi064.stdout:5/720: fsync d6/d18/d19/f2e 0 2024-02-16T22:56:35.840 INFO:tasks.workunit.client.1.smithi192.stdout:4/948: mkdir d1/d15/d48/dc3/deb/d140 0 2024-02-16T22:56:35.841 INFO:tasks.workunit.client.1.smithi192.stdout:4/949: fsync d1/d26/d66/d89/da8/dc5/d111/d122/f13f 0 2024-02-16T22:56:35.844 INFO:tasks.workunit.client.1.smithi192.stdout:5/911: symlink d6/d12/d1d/d56/l125 0 2024-02-16T22:56:35.845 INFO:tasks.workunit.client.1.smithi192.stdout:2/976: fsync d22/d24/d39/d42/d85/d98/db0/db6/fce 0 2024-02-16T22:56:35.847 INFO:tasks.workunit.client.0.smithi064.stdout:5/721: link d6/d25/da4/fe9 d6/d25/df1/d91/da5/de3/ff3 0 2024-02-16T22:56:35.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/919: rmdir da/df/d71/dde 39 2024-02-16T22:56:35.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/920: chown da/df/d71/dab/cc7 58439347 1 2024-02-16T22:56:35.851 INFO:tasks.workunit.client.0.smithi064.stdout:5/722: write d6/d18/fee [2287353,126287] 0 2024-02-16T22:56:35.852 INFO:tasks.workunit.client.1.smithi192.stdout:1/858: write d3/de/d27/d34/d52/d63/d7a/dc9/fe1 [841949,7414] 0 2024-02-16T22:56:35.852 INFO:tasks.workunit.client.0.smithi064.stdout:5/723: fsync d6/d18/d95/fea 0 2024-02-16T22:56:35.852 INFO:tasks.workunit.client.1.smithi192.stdout:1/859: fdatasync d3/de/d27/d34/d52/f10f 0 2024-02-16T22:56:35.853 INFO:tasks.workunit.client.1.smithi192.stdout:8/892: symlink d5/d26/l118 0 2024-02-16T22:56:35.856 INFO:tasks.workunit.client.1.smithi192.stdout:0/910: creat d1/dc7/d9c/f126 x:0 0 0 2024-02-16T22:56:35.859 INFO:tasks.workunit.client.1.smithi192.stdout:7/999: dwrite d1/d2/d2d/d61/d88/d90/d4c/d60/ffd [0,4194304] 0 2024-02-16T22:56:35.862 INFO:tasks.workunit.client.0.smithi064.stdout:9/775: dwrite d3/d9/d6a/db1/d2a/d2e/d31/f4e [4194304,4194304] 0 2024-02-16T22:56:35.866 INFO:tasks.workunit.client.0.smithi064.stdout:5/724: getdents d6/d18 0 2024-02-16T22:56:35.867 INFO:tasks.workunit.client.1.smithi192.stdout:1/860: read d3/de/d27/d34/f4e [514534,123806] 0 2024-02-16T22:56:35.868 INFO:tasks.workunit.client.1.smithi192.stdout:8/893: creat d5/db/de/f119 x:0 0 0 2024-02-16T22:56:35.869 INFO:tasks.workunit.client.1.smithi192.stdout:1/861: stat d3/de/d27/d34/d69/fef 0 2024-02-16T22:56:35.872 INFO:tasks.workunit.client.1.smithi192.stdout:1/862: write d3/de/d27/d34/d36/d47/dcd/f11b [85824,13365] 0 2024-02-16T22:56:35.872 INFO:tasks.workunit.client.0.smithi064.stdout:9/776: dwrite d3/d9/d6a/db1/d38/fa6 [0,4194304] 0 2024-02-16T22:56:35.872 INFO:tasks.workunit.client.1.smithi192.stdout:1/863: chown d3/de/d87/ffc 0 1 2024-02-16T22:56:35.874 INFO:tasks.workunit.client.1.smithi192.stdout:5/912: mkdir d6/d12/d1d/d126 0 2024-02-16T22:56:35.875 INFO:tasks.workunit.client.1.smithi192.stdout:5/913: write d6/d12/d4e/d5b/fed [4427378,56326] 0 2024-02-16T22:56:35.877 INFO:tasks.workunit.client.1.smithi192.stdout:5/914: write d6/d12/d4e/d5b/fed [983130,37871] 0 2024-02-16T22:56:35.880 INFO:tasks.workunit.client.1.smithi192.stdout:4/950: dread d1/f19 [0,4194304] 0 2024-02-16T22:56:35.882 INFO:tasks.workunit.client.1.smithi192.stdout:6/921: symlink da/df/d18/l127 0 2024-02-16T22:56:35.883 INFO:tasks.workunit.client.1.smithi192.stdout:6/922: chown da/d76/dc8/ff4 2648 1 2024-02-16T22:56:35.884 INFO:tasks.workunit.client.1.smithi192.stdout:8/894: creat d5/d26/d8f/dc0/f11a x:0 0 0 2024-02-16T22:56:35.887 INFO:tasks.workunit.client.0.smithi064.stdout:9/777: rmdir d3/d9/d6a/db1/d2a/d2e/d31/deb 0 2024-02-16T22:56:35.888 INFO:tasks.workunit.client.1.smithi192.stdout:1/864: fdatasync d3/de/d27/d34/d52/ff8 0 2024-02-16T22:56:35.889 INFO:tasks.workunit.client.1.smithi192.stdout:0/911: dwrite d1/d3/de/d22/d37/d47/f48 [0,4194304] 0 2024-02-16T22:56:35.890 INFO:tasks.workunit.client.1.smithi192.stdout:2/977: truncate d22/d24/d39/d42/d84/f101 225015 0 2024-02-16T22:56:35.892 INFO:tasks.workunit.client.1.smithi192.stdout:1/865: truncate d3/de/d27/d34/d69/d85/d89/daa/dfb/f102 541780 0 2024-02-16T22:56:35.899 INFO:tasks.workunit.client.0.smithi064.stdout:9/778: dread d3/d11/f28 [0,4194304] 0 2024-02-16T22:56:35.899 INFO:tasks.workunit.client.0.smithi064.stdout:9/779: fsync d3/d9/f41 0 2024-02-16T22:56:35.899 INFO:tasks.workunit.client.0.smithi064.stdout:9/780: fsync d3/f63 0 2024-02-16T22:56:35.900 INFO:tasks.workunit.client.1.smithi192.stdout:6/923: unlink da/df/d71/dab/cc7 0 2024-02-16T22:56:35.900 INFO:tasks.workunit.client.1.smithi192.stdout:6/924: fdatasync da/d40/f109 0 2024-02-16T22:56:35.902 INFO:tasks.workunit.client.0.smithi064.stdout:5/725: dread d6/d25/df1/d91/da5/de3/f93 [0,4194304] 0 2024-02-16T22:56:35.907 INFO:tasks.workunit.client.1.smithi192.stdout:1/866: creat d3/d40/d9f/df0/f128 x:0 0 0 2024-02-16T22:56:35.909 INFO:tasks.workunit.client.1.smithi192.stdout:4/951: mknod d1/d15/d48/d8d/d128/c141 0 2024-02-16T22:56:35.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/839: link da/d11/d20/d39/d53/dc7/la3 da/d11/d20/d39/l122 0 2024-02-16T22:56:35.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/840: chown da/d11/d20/d39/d53/dc7 13179 1 2024-02-16T22:56:35.915 INFO:tasks.workunit.client.0.smithi064.stdout:9/781: rename d3/d9/d6a/db1/l6c to d3/d9/d1e/l101 0 2024-02-16T22:56:35.918 INFO:tasks.workunit.client.1.smithi192.stdout:6/925: truncate da/df/d12/d57/fbd 1973475 0 2024-02-16T22:56:35.919 INFO:tasks.workunit.client.1.smithi192.stdout:0/912: creat d1/dd6/f127 x:0 0 0 2024-02-16T22:56:35.919 INFO:tasks.workunit.client.0.smithi064.stdout:5/726: write d6/f17 [1198645,107713] 0 2024-02-16T22:56:35.919 INFO:tasks.workunit.client.1.smithi192.stdout:0/913: dread - d1/d3/de/d22/d37/d47/f10e zero size 2024-02-16T22:56:35.921 INFO:tasks.workunit.client.0.smithi064.stdout:9/782: rename d3/d9/d6a/db1/d2a/d2e/d31/dd3 to d3/d11/d9e/da5/d102 0 2024-02-16T22:56:35.924 INFO:tasks.workunit.client.0.smithi064.stdout:5/727: mknod d6/d25/da4/cf4 0 2024-02-16T22:56:35.926 INFO:tasks.workunit.client.1.smithi192.stdout:3/841: fsync da/d11/f96 0 2024-02-16T22:56:35.927 INFO:tasks.workunit.client.1.smithi192.stdout:1/867: dread d3/de/d27/d34/d52/d63/d81/f111 [0,4194304] 0 2024-02-16T22:56:35.927 INFO:tasks.workunit.client.0.smithi064.stdout:5/728: chown d6/d25/da4/dc1/d8e 203831 1 2024-02-16T22:56:35.928 INFO:tasks.workunit.client.0.smithi064.stdout:5/729: chown d6/d18/dc5/fc7 0 1 2024-02-16T22:56:35.933 INFO:tasks.workunit.client.1.smithi192.stdout:1/868: dwrite d3/de/d27/d34/d36/feb [0,4194304] 0 2024-02-16T22:56:35.934 INFO:tasks.workunit.client.1.smithi192.stdout:1/869: dread - d3/d40/d9f/df0/f10c zero size 2024-02-16T22:56:35.971 INFO:tasks.workunit.client.1.smithi192.stdout:5/915: sync 2024-02-16T22:56:35.972 INFO:tasks.workunit.client.1.smithi192.stdout:5/916: dread - d6/d12/d4e/d58/ffe zero size 2024-02-16T22:56:35.994 INFO:tasks.workunit.client.0.smithi064.stdout:5/730: mknod d6/d25/df1/cf5 0 2024-02-16T22:56:35.996 INFO:tasks.workunit.client.1.smithi192.stdout:6/926: rename da/d76/lac to da/df/d6b/d74/de0/l128 0 2024-02-16T22:56:35.996 INFO:tasks.workunit.client.0.smithi064.stdout:9/783: link d3/d9/d1e/d42/d7b/d86/l90 d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/l103 0 2024-02-16T22:56:35.999 INFO:tasks.workunit.client.0.smithi064.stdout:5/731: creat d6/d18/ff6 x:0 0 0 2024-02-16T22:56:36.000 INFO:tasks.workunit.client.1.smithi192.stdout:2/978: link d22/d24/d39/d42/d52/d7c/cab d22/d24/d39/d3d/df2/d12a/c144 0 2024-02-16T22:56:36.001 INFO:tasks.workunit.client.1.smithi192.stdout:2/979: chown d22/d24/d39/d42/d65/f8e 10885157 1 2024-02-16T22:56:36.002 INFO:tasks.workunit.client.1.smithi192.stdout:2/980: chown d22/d24/d39/d42/d52/d7c/c81 38080048 1 2024-02-16T22:56:36.004 INFO:tasks.workunit.client.1.smithi192.stdout:4/952: dwrite d1/d26/d34/db9/fdd [0,4194304] 0 2024-02-16T22:56:36.004 INFO:tasks.workunit.client.0.smithi064.stdout:5/732: stat d6/d25/df1/d91/da5/de3/d43/d7d/cd2 0 2024-02-16T22:56:36.004 INFO:tasks.workunit.client.0.smithi064.stdout:9/784: dwrite d3/d9/d1e/d42/d7b/d86/fd7 [0,4194304] 0 2024-02-16T22:56:36.005 INFO:tasks.workunit.client.1.smithi192.stdout:3/842: readlink da/d11/dd4/lfc 0 2024-02-16T22:56:36.008 INFO:tasks.workunit.client.1.smithi192.stdout:1/870: rmdir d3/de/d87 39 2024-02-16T22:56:36.011 INFO:tasks.workunit.client.1.smithi192.stdout:6/927: symlink da/d76/dc8/l129 0 2024-02-16T22:56:36.012 INFO:tasks.workunit.client.1.smithi192.stdout:0/914: mknod d1/d3/d18/d11d/c128 0 2024-02-16T22:56:36.012 INFO:tasks.workunit.client.1.smithi192.stdout:6/928: dread - da/df/d18/d52/f11d zero size 2024-02-16T22:56:36.017 INFO:tasks.workunit.client.0.smithi064.stdout:5/733: dread - d6/d18/d90/fc2 zero size 2024-02-16T22:56:36.019 INFO:tasks.workunit.client.1.smithi192.stdout:8/895: creat d5/db/de/df/f11b x:0 0 0 2024-02-16T22:56:36.020 INFO:tasks.workunit.client.1.smithi192.stdout:8/896: truncate d5/db/faa 622356 0 2024-02-16T22:56:36.021 INFO:tasks.workunit.client.1.smithi192.stdout:1/871: creat d3/d40/d9f/df0/f129 x:0 0 0 2024-02-16T22:56:36.022 INFO:tasks.workunit.client.1.smithi192.stdout:1/872: truncate d3/de/f65 4719288 0 2024-02-16T22:56:36.026 INFO:tasks.workunit.client.0.smithi064.stdout:5/734: creat d6/d18/ff7 x:0 0 0 2024-02-16T22:56:36.026 INFO:tasks.workunit.client.1.smithi192.stdout:3/843: symlink da/d11/d20/d39/d116/l123 0 2024-02-16T22:56:36.027 INFO:tasks.workunit.client.0.smithi064.stdout:5/735: truncate d6/d25/da4/dc1/d8e/feb 764577 0 2024-02-16T22:56:36.028 INFO:tasks.workunit.client.1.smithi192.stdout:8/897: symlink d5/d26/d8f/dc0/l11c 0 2024-02-16T22:56:36.030 INFO:tasks.workunit.client.1.smithi192.stdout:1/873: creat d3/d5b/d77/dd2/f12a x:0 0 0 2024-02-16T22:56:36.031 INFO:tasks.workunit.client.1.smithi192.stdout:2/981: dwrite d22/f4a [4194304,4194304] 0 2024-02-16T22:56:36.034 INFO:tasks.workunit.client.0.smithi064.stdout:5/736: mkdir d6/d18/df8 0 2024-02-16T22:56:36.038 INFO:tasks.workunit.client.0.smithi064.stdout:5/737: creat d6/d25/df1/d91/da5/de3/d43/ff9 x:0 0 0 2024-02-16T22:56:36.043 INFO:tasks.workunit.client.1.smithi192.stdout:0/915: dwrite d1/d3/de/d22/d37/d67/d34/fbc [4194304,4194304] 0 2024-02-16T22:56:36.044 INFO:tasks.workunit.client.0.smithi064.stdout:5/738: chown d6/d25/da4/dc1/d8e/l9b 695074606 1 2024-02-16T22:56:36.044 INFO:tasks.workunit.client.1.smithi192.stdout:2/982: mkdir d22/d145 0 2024-02-16T22:56:36.050 INFO:tasks.workunit.client.0.smithi064.stdout:5/739: fdatasync d6/f31 0 2024-02-16T22:56:36.050 INFO:tasks.workunit.client.1.smithi192.stdout:2/983: dread d22/fc7 [0,4194304] 0 2024-02-16T22:56:36.050 INFO:tasks.workunit.client.1.smithi192.stdout:6/929: dwrite da/df/d18/f43 [0,4194304] 0 2024-02-16T22:56:36.051 INFO:tasks.workunit.client.1.smithi192.stdout:0/916: creat d1/d3/d29/dcd/f129 x:0 0 0 2024-02-16T22:56:36.055 INFO:tasks.workunit.client.1.smithi192.stdout:8/898: write d5/d16/dcf/fe2 [294716,127224] 0 2024-02-16T22:56:36.060 INFO:tasks.workunit.client.0.smithi064.stdout:7/753: sync 2024-02-16T22:56:36.063 INFO:tasks.workunit.client.1.smithi192.stdout:0/917: fsync d1/dc7/d9c/fc4 0 2024-02-16T22:56:36.064 INFO:tasks.workunit.client.1.smithi192.stdout:1/874: getdents d3/d40/d9f 0 2024-02-16T22:56:36.065 INFO:tasks.workunit.client.1.smithi192.stdout:3/844: dwrite da/d11/d2b/f5e [0,4194304] 0 2024-02-16T22:56:36.065 INFO:tasks.workunit.client.1.smithi192.stdout:0/918: dread - d1/d3/de/d22/d37/d47/d53/ff7 zero size 2024-02-16T22:56:36.066 INFO:tasks.workunit.client.1.smithi192.stdout:8/899: mkdir d5/d9/d11d 0 2024-02-16T22:56:36.066 INFO:tasks.workunit.client.1.smithi192.stdout:3/845: readlink da/d6a/lb1 0 2024-02-16T22:56:36.068 INFO:tasks.workunit.client.1.smithi192.stdout:3/846: dread - da/d107/f113 zero size 2024-02-16T22:56:36.069 INFO:tasks.workunit.client.1.smithi192.stdout:2/984: write f5 [475116,124300] 0 2024-02-16T22:56:36.070 INFO:tasks.workunit.client.1.smithi192.stdout:2/985: chown d22/d24/d39/d42/f54 2 1 2024-02-16T22:56:36.074 INFO:tasks.workunit.client.1.smithi192.stdout:3/847: mknod da/d11/dd4/deb/da1/c124 0 2024-02-16T22:56:36.076 INFO:tasks.workunit.client.1.smithi192.stdout:2/986: truncate d22/d24/d39/d42/d65/f8e 1390359 0 2024-02-16T22:56:36.078 INFO:tasks.workunit.client.1.smithi192.stdout:0/919: mknod d1/d3/de/d22/c12a 0 2024-02-16T22:56:36.081 INFO:tasks.workunit.client.1.smithi192.stdout:6/930: write da/df/d12/f9c [1670708,20064] 0 2024-02-16T22:56:36.083 INFO:tasks.workunit.client.1.smithi192.stdout:0/920: unlink d1/d3/de/d22/f30 0 2024-02-16T22:56:36.083 INFO:tasks.workunit.client.1.smithi192.stdout:3/848: dwrite da/d11/d63/d6b/ffd [0,4194304] 0 2024-02-16T22:56:36.085 INFO:tasks.workunit.client.1.smithi192.stdout:2/987: symlink d22/d24/d39/d42/d65/d69/l146 0 2024-02-16T22:56:36.090 INFO:tasks.workunit.client.1.smithi192.stdout:3/849: creat da/d11/d2b/ddc/f125 x:0 0 0 2024-02-16T22:56:36.092 INFO:tasks.workunit.client.1.smithi192.stdout:1/875: dread d3/fa6 [0,4194304] 0 2024-02-16T22:56:36.094 INFO:tasks.workunit.client.1.smithi192.stdout:0/921: mknod d1/d3/de/d22/d37/c12b 0 2024-02-16T22:56:36.096 INFO:tasks.workunit.client.1.smithi192.stdout:3/850: dread - da/d11/dd4/fe1 zero size 2024-02-16T22:56:36.096 INFO:tasks.workunit.client.1.smithi192.stdout:0/922: chown d1/d3/de/d22/fd5 4184330 1 2024-02-16T22:56:36.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/988: dwrite d22/d24/d39/d42/d85/d98/dc4/ff1 [4194304,4194304] 0 2024-02-16T22:56:36.100 INFO:tasks.workunit.client.1.smithi192.stdout:0/923: read d1/d3/de2/dff/f105 [121222,86157] 0 2024-02-16T22:56:36.103 INFO:tasks.workunit.client.1.smithi192.stdout:1/876: dwrite d3/de/d27/f42 [0,4194304] 0 2024-02-16T22:56:36.103 INFO:tasks.workunit.client.1.smithi192.stdout:1/877: read - d3/d5b/db1/f10b zero size 2024-02-16T22:56:36.115 INFO:tasks.workunit.client.0.smithi064.stdout:5/740: readlink d6/ld 0 2024-02-16T22:56:36.118 INFO:tasks.workunit.client.0.smithi064.stdout:5/741: fdatasync d6/d18/d95/fea 0 2024-02-16T22:56:36.121 INFO:tasks.workunit.client.0.smithi064.stdout:5/742: mknod d6/d18/cfa 0 2024-02-16T22:56:36.122 INFO:tasks.workunit.client.0.smithi064.stdout:5/743: truncate d6/d25/da4/dc1/d85/f9f 98192 0 2024-02-16T22:56:36.125 INFO:tasks.workunit.client.0.smithi064.stdout:5/744: unlink d6/d25/df1/d91/da5/de3/d43/d7d/f88 0 2024-02-16T22:56:36.131 INFO:tasks.workunit.client.0.smithi064.stdout:5/745: dwrite d6/d25/df1/d91/da5/de3/d43/db9/fd8 [4194304,4194304] 0 2024-02-16T22:56:36.132 INFO:tasks.workunit.client.0.smithi064.stdout:5/746: chown d6/d18 14 1 2024-02-16T22:56:36.132 INFO:tasks.workunit.client.0.smithi064.stdout:7/754: dwrite f0 [0,4194304] 0 2024-02-16T22:56:36.141 INFO:tasks.workunit.client.0.smithi064.stdout:7/755: rename d5/d6 to d5/d33/d53/df2 0 2024-02-16T22:56:36.146 INFO:tasks.workunit.client.0.smithi064.stdout:7/756: dwrite d5/d7d/fa0 [0,4194304] 0 2024-02-16T22:56:36.149 INFO:tasks.workunit.client.0.smithi064.stdout:7/757: fdatasync d5/d33/f8a 0 2024-02-16T22:56:36.153 INFO:tasks.workunit.client.0.smithi064.stdout:7/758: rename d5/d33/d45/d73/dab to d5/d33/d42/d81/d82/df3 0 2024-02-16T22:56:36.155 INFO:tasks.workunit.client.1.smithi192.stdout:5/917: sync 2024-02-16T22:56:36.156 INFO:tasks.workunit.client.1.smithi192.stdout:5/918: stat d6/d8/d80 0 2024-02-16T22:56:36.158 INFO:tasks.workunit.client.1.smithi192.stdout:6/931: dread da/df/d12/d15/d1b/f8a [0,4194304] 0 2024-02-16T22:56:36.158 INFO:tasks.workunit.client.1.smithi192.stdout:2/989: write d22/d24/d39/d42/d84/fc5 [508342,122431] 0 2024-02-16T22:56:36.159 INFO:tasks.workunit.client.1.smithi192.stdout:1/878: rmdir d3/de/d27/d34/d52/d72 39 2024-02-16T22:56:36.160 INFO:tasks.workunit.client.1.smithi192.stdout:5/919: mkdir d6/d12/d2d/d5c/d127 0 2024-02-16T22:56:36.161 INFO:tasks.workunit.client.1.smithi192.stdout:6/932: dread - da/df/d18/fe4 zero size 2024-02-16T22:56:36.161 INFO:tasks.workunit.client.1.smithi192.stdout:2/990: mknod d22/d47/dca/dfa/c147 0 2024-02-16T22:56:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:7/759: write d5/d33/d45/fb5 [248548,118754] 0 2024-02-16T22:56:36.164 INFO:tasks.workunit.client.1.smithi192.stdout:0/924: dwrite d1/d3/de/d22/d37/d67/d34/f6e [0,4194304] 0 2024-02-16T22:56:36.166 INFO:tasks.workunit.client.1.smithi192.stdout:2/991: dread d22/d24/d39/f4c [0,4194304] 0 2024-02-16T22:56:36.167 INFO:tasks.workunit.client.1.smithi192.stdout:5/920: rename d6/d8/d77/l101 to d6/d8/d86/l128 0 2024-02-16T22:56:36.168 INFO:tasks.workunit.client.1.smithi192.stdout:6/933: getdents da/db9/d126 0 2024-02-16T22:56:36.169 INFO:tasks.workunit.client.1.smithi192.stdout:1/879: getdents d3/de/d27/d34/d36 0 2024-02-16T22:56:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:7/760: dread d5/d33/d53/df2/d65/da7/fce [0,4194304] 0 2024-02-16T22:56:36.178 INFO:tasks.workunit.client.0.smithi064.stdout:7/761: mkdir d5/d33/d53/df2/d54/df4 0 2024-02-16T22:56:36.180 INFO:tasks.workunit.client.1.smithi192.stdout:5/921: link d6/d12/c19 d6/d12/d1d/c129 0 2024-02-16T22:56:36.180 INFO:tasks.workunit.client.1.smithi192.stdout:5/922: dread - d6/d12/d1d/fb0 zero size 2024-02-16T22:56:36.181 INFO:tasks.workunit.client.0.smithi064.stdout:7/762: truncate d5/d33/d45/d64/f91 8847979 0 2024-02-16T22:56:36.182 INFO:tasks.workunit.client.1.smithi192.stdout:5/923: creat d6/d12/d2d/d5c/df6/f12a x:0 0 0 2024-02-16T22:56:36.183 INFO:tasks.workunit.client.1.smithi192.stdout:5/924: fdatasync d6/d20/d93/fc3 0 2024-02-16T22:56:36.184 INFO:tasks.workunit.client.1.smithi192.stdout:5/925: dread - d6/d12/d4e/d46/d4f/f62 zero size 2024-02-16T22:56:36.186 INFO:tasks.workunit.client.1.smithi192.stdout:5/926: creat d6/d8/d75/d8c/d106/f12b x:0 0 0 2024-02-16T22:56:36.188 INFO:tasks.workunit.client.1.smithi192.stdout:5/927: rmdir d6/d12/d4e/d5b 39 2024-02-16T22:56:36.190 INFO:tasks.workunit.client.1.smithi192.stdout:5/928: mkdir d6/d12/d4e/d12c 0 2024-02-16T22:56:36.191 INFO:tasks.workunit.client.1.smithi192.stdout:6/934: write da/df/f84 [1616950,54924] 0 2024-02-16T22:56:36.192 INFO:tasks.workunit.client.1.smithi192.stdout:2/992: write d22/d24/d39/ff5 [835316,23100] 0 2024-02-16T22:56:36.194 INFO:tasks.workunit.client.1.smithi192.stdout:5/929: symlink d6/d12/l12d 0 2024-02-16T22:56:36.195 INFO:tasks.workunit.client.1.smithi192.stdout:2/993: chown d22/d24/d39/d42/d85 35210 1 2024-02-16T22:56:36.199 INFO:tasks.workunit.client.1.smithi192.stdout:0/925: dwrite d1/dc7/d9c/fee [0,4194304] 0 2024-02-16T22:56:36.200 INFO:tasks.workunit.client.1.smithi192.stdout:1/880: dwrite d3/de/ff [0,4194304] 0 2024-02-16T22:56:36.200 INFO:tasks.workunit.client.0.smithi064.stdout:7/763: dread d5/d33/d45/f46 [0,4194304] 0 2024-02-16T22:56:36.200 INFO:tasks.workunit.client.1.smithi192.stdout:2/994: dread d22/d24/d39/d42/d65/d66/fa4 [0,4194304] 0 2024-02-16T22:56:36.201 INFO:tasks.workunit.client.1.smithi192.stdout:5/930: rename d6/d12/d4e/d46/c49 to d6/d20/d93/c12e 0 2024-02-16T22:56:36.201 INFO:tasks.workunit.client.1.smithi192.stdout:0/926: stat d1/d3/de/d22/d37/d67/d34/cd8 0 2024-02-16T22:56:36.203 INFO:tasks.workunit.client.0.smithi064.stdout:7/764: truncate d5/d33/d53/df2/d54/d7b/fb1 4865306 0 2024-02-16T22:56:36.203 INFO:tasks.workunit.client.1.smithi192.stdout:1/881: mkdir d3/de/d27/d34/d52/d63/d12b 0 2024-02-16T22:56:36.205 INFO:tasks.workunit.client.1.smithi192.stdout:2/995: mkdir d22/d24/d39/d42/d65/d69/d4f/d148 0 2024-02-16T22:56:36.205 INFO:tasks.workunit.client.1.smithi192.stdout:6/935: write da/df/d18/d46/f80 [912427,87200] 0 2024-02-16T22:56:36.209 INFO:tasks.workunit.client.0.smithi064.stdout:7/765: stat d5/d33/d53/c71 0 2024-02-16T22:56:36.211 INFO:tasks.workunit.client.1.smithi192.stdout:0/927: rename d1/d3/d39/df3 to d1/dc7/d9c/d12c 0 2024-02-16T22:56:36.212 INFO:tasks.workunit.client.1.smithi192.stdout:8/900: sync 2024-02-16T22:56:36.212 INFO:tasks.workunit.client.1.smithi192.stdout:4/953: sync 2024-02-16T22:56:36.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/882: chown d3/de/d27/d34/d52/d72/d11c/c11e 1 1 2024-02-16T22:56:36.214 INFO:tasks.workunit.client.1.smithi192.stdout:6/936: rename da/df/d6b/d83/l122 to da/df/d12/d57/daf/l12a 0 2024-02-16T22:56:36.214 INFO:tasks.workunit.client.1.smithi192.stdout:1/883: stat d3/de/d27/d34/d36/d47/l115 0 2024-02-16T22:56:36.214 INFO:tasks.workunit.client.1.smithi192.stdout:6/937: chown da/df/d18/d52 144792478 1 2024-02-16T22:56:36.215 INFO:tasks.workunit.client.1.smithi192.stdout:2/996: mknod d22/d24/d39/d42/c149 0 2024-02-16T22:56:36.216 INFO:tasks.workunit.client.1.smithi192.stdout:4/954: mknod d1/d15/d29/dd4/c142 0 2024-02-16T22:56:36.218 INFO:tasks.workunit.client.1.smithi192.stdout:4/955: write d1/f4 [2003350,103542] 0 2024-02-16T22:56:36.218 INFO:tasks.workunit.client.1.smithi192.stdout:1/884: symlink d3/de/d27/d34/d36/d47/d5d/l12c 0 2024-02-16T22:56:36.220 INFO:tasks.workunit.client.1.smithi192.stdout:0/928: rmdir d1/d3/de/d22/d111 0 2024-02-16T22:56:36.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/956: fsync d1/d26/d66/d89/da8/dc5/d111/d122/f13f 0 2024-02-16T22:56:36.221 INFO:tasks.workunit.client.1.smithi192.stdout:2/997: mknod d22/d24/d39/d42/d65/d69/d4f/d148/c14a 0 2024-02-16T22:56:36.224 INFO:tasks.workunit.client.1.smithi192.stdout:1/885: symlink d3/de/d27/d34/d52/d63/d7a/dc9/l12d 0 2024-02-16T22:56:36.226 INFO:tasks.workunit.client.1.smithi192.stdout:4/957: mknod d1/d26/d66/d89/da8/dc5/c143 0 2024-02-16T22:56:36.230 INFO:tasks.workunit.client.1.smithi192.stdout:2/998: dwrite d22/d47/dca/dfa/f10f [0,4194304] 0 2024-02-16T22:56:36.231 INFO:tasks.workunit.client.1.smithi192.stdout:2/999: write d22/d47/dcb/dc9/f13d [922032,363] 0 2024-02-16T22:56:36.232 INFO:tasks.workunit.client.1.smithi192.stdout:4/958: rmdir d1/d26/d6f 39 2024-02-16T22:56:36.233 INFO:tasks.workunit.client.0.smithi064.stdout:7/766: dread d5/d33/d42/f4f [0,4194304] 0 2024-02-16T22:56:36.235 INFO:tasks.workunit.client.1.smithi192.stdout:1/886: symlink d3/de/d27/d34/d36/d47/dd9/l12e 0 2024-02-16T22:56:36.238 INFO:tasks.workunit.client.1.smithi192.stdout:4/959: creat d1/d15/d48/d8d/d9b/f144 x:0 0 0 2024-02-16T22:56:36.239 INFO:tasks.workunit.client.1.smithi192.stdout:1/887: chown d3/d5b/f71 1 1 2024-02-16T22:56:36.239 INFO:tasks.workunit.client.1.smithi192.stdout:5/931: dread d6/d12/d4e/d58/fbb [0,4194304] 0 2024-02-16T22:56:36.240 INFO:tasks.workunit.client.1.smithi192.stdout:8/901: dwrite d5/db/d47/f92 [0,4194304] 0 2024-02-16T22:56:36.241 INFO:tasks.workunit.client.1.smithi192.stdout:0/929: truncate d1/d3/de/d22/d37/d47/f48 723697 0 2024-02-16T22:56:36.242 INFO:tasks.workunit.client.1.smithi192.stdout:4/960: chown d1/d26/d66/d89/da8/da9 134461605 1 2024-02-16T22:56:36.245 INFO:tasks.workunit.client.1.smithi192.stdout:5/932: dread - d6/d12/d4e/d5b/d67/d78/feb zero size 2024-02-16T22:56:36.248 INFO:tasks.workunit.client.1.smithi192.stdout:8/902: read d5/f25 [1486891,36863] 0 2024-02-16T22:56:36.252 INFO:tasks.workunit.client.1.smithi192.stdout:0/930: read d1/d3/de/d22/d37/d67/d34/f8a [3267314,4244] 0 2024-02-16T22:56:36.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/938: dwrite da/df/d12/d57/fc6 [4194304,4194304] 0 2024-02-16T22:56:36.252 INFO:tasks.workunit.client.1.smithi192.stdout:5/933: mkdir d6/d12/d2d/d5c/df6/d12f 0 2024-02-16T22:56:36.254 INFO:tasks.workunit.client.1.smithi192.stdout:6/939: fsync da/df/d12/d15/d1b/fbf 0 2024-02-16T22:56:36.255 INFO:tasks.workunit.client.1.smithi192.stdout:0/931: symlink d1/d3/de/d22/d37/d47/l12d 0 2024-02-16T22:56:36.257 INFO:tasks.workunit.client.1.smithi192.stdout:5/934: mknod d6/d20/d93/c130 0 2024-02-16T22:56:36.260 INFO:tasks.workunit.client.1.smithi192.stdout:5/935: rmdir d6/d12/d4e/d5b/d67 39 2024-02-16T22:56:36.262 INFO:tasks.workunit.client.1.smithi192.stdout:6/940: chown da/d76/dd5/de2/f100 1500723 1 2024-02-16T22:56:36.266 INFO:tasks.workunit.client.1.smithi192.stdout:6/941: fsync da/df/d71/fd0 0 2024-02-16T22:56:36.268 INFO:tasks.workunit.client.1.smithi192.stdout:0/932: dwrite d1/d3/d29/dcd/dcf/ffa [0,4194304] 0 2024-02-16T22:56:36.270 INFO:tasks.workunit.client.1.smithi192.stdout:1/888: dwrite d3/de/d27/d34/d52/f70 [0,4194304] 0 2024-02-16T22:56:36.270 INFO:tasks.workunit.client.1.smithi192.stdout:3/851: sync 2024-02-16T22:56:36.271 INFO:tasks.workunit.client.0.smithi064.stdout:7/767: dread d5/f23 [0,4194304] 0 2024-02-16T22:56:36.274 INFO:tasks.workunit.client.1.smithi192.stdout:8/903: dread d5/f25 [0,4194304] 0 2024-02-16T22:56:36.275 INFO:tasks.workunit.client.0.smithi064.stdout:7/768: unlink d5/d33/d45/d73/fa4 0 2024-02-16T22:56:36.275 INFO:tasks.workunit.client.1.smithi192.stdout:6/942: chown da/df/d18/d46/f80 23717 1 2024-02-16T22:56:36.275 INFO:tasks.workunit.client.1.smithi192.stdout:0/933: chown d1/d3/de/d22/d37/d67/d75/db0/fe3 1 1 2024-02-16T22:56:36.276 INFO:tasks.workunit.client.1.smithi192.stdout:0/934: chown d1/d3/de/d22/f85 4826 1 2024-02-16T22:56:36.277 INFO:tasks.workunit.client.1.smithi192.stdout:1/889: fsync d3/de/d27/d34/d52/ff8 0 2024-02-16T22:56:36.280 INFO:tasks.workunit.client.1.smithi192.stdout:4/961: dread d1/d15/d29/fdf [0,4194304] 0 2024-02-16T22:56:36.281 INFO:tasks.workunit.client.1.smithi192.stdout:6/943: creat da/d76/f12b x:0 0 0 2024-02-16T22:56:36.283 INFO:tasks.workunit.client.1.smithi192.stdout:5/936: truncate d6/d12/d2d/fd0 404819 0 2024-02-16T22:56:36.284 INFO:tasks.workunit.client.0.smithi064.stdout:7/769: mkdir d5/d33/d53/dde/de4/df5 0 2024-02-16T22:56:36.288 INFO:tasks.workunit.client.1.smithi192.stdout:5/937: dwrite d6/d12/d2d/f43 [0,4194304] 0 2024-02-16T22:56:36.289 INFO:tasks.workunit.client.0.smithi064.stdout:7/770: creat d5/d33/d53/df2/d65/ff6 x:0 0 0 2024-02-16T22:56:36.289 INFO:tasks.workunit.client.1.smithi192.stdout:1/890: truncate d3/de/fc6 5798371 0 2024-02-16T22:56:36.290 INFO:tasks.workunit.client.1.smithi192.stdout:4/962: chown d1/d26/d66/l83 18733742 1 2024-02-16T22:56:36.291 INFO:tasks.workunit.client.1.smithi192.stdout:8/904: mknod d5/d16/de1/c11e 0 2024-02-16T22:56:36.295 INFO:tasks.workunit.client.0.smithi064.stdout:2/917: sync 2024-02-16T22:56:36.295 INFO:tasks.workunit.client.0.smithi064.stdout:3/846: sync 2024-02-16T22:56:36.295 INFO:tasks.workunit.client.0.smithi064.stdout:6/833: sync 2024-02-16T22:56:36.296 INFO:tasks.workunit.client.0.smithi064.stdout:7/771: mkdir d5/d33/d53/dde/de4/df5/df7 0 2024-02-16T22:56:36.297 INFO:tasks.workunit.client.0.smithi064.stdout:6/834: write d0/d6/d10/f14 [1997297,39283] 0 2024-02-16T22:56:36.298 INFO:tasks.workunit.client.0.smithi064.stdout:7/772: readlink d5/d33/d53/df2/d65/da7/lbb 0 2024-02-16T22:56:36.299 INFO:tasks.workunit.client.1.smithi192.stdout:5/938: fdatasync d6/d12/d4e/d58/fd3 0 2024-02-16T22:56:36.302 INFO:tasks.workunit.client.0.smithi064.stdout:3/847: dread d4/f11 [0,4194304] 0 2024-02-16T22:56:36.302 INFO:tasks.workunit.client.1.smithi192.stdout:4/963: creat d1/d26/d34/dc7/f145 x:0 0 0 2024-02-16T22:56:36.303 INFO:tasks.workunit.client.0.smithi064.stdout:7/773: read d5/d33/d53/df2/d65/da7/dad/fd0 [1929662,108457] 0 2024-02-16T22:56:36.305 INFO:tasks.workunit.client.1.smithi192.stdout:8/905: rename d5/d26/d83/db4/f4b to d5/d9/d115/f11f 0 2024-02-16T22:56:36.305 INFO:tasks.workunit.client.0.smithi064.stdout:6/835: dread d0/d5/df/d17/d4a/fe2 [0,4194304] 0 2024-02-16T22:56:36.308 INFO:tasks.workunit.client.0.smithi064.stdout:3/848: creat d4/d13/d4d/d5c/d78/d117/f11f x:0 0 0 2024-02-16T22:56:36.311 INFO:tasks.workunit.client.1.smithi192.stdout:8/906: dwrite d5/d26/d83/d86/f74 [4194304,4194304] 0 2024-02-16T22:56:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:7/774: mkdir d5/d33/d53/df2/d54/d7b/df8 0 2024-02-16T22:56:36.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/935: dwrite d1/d3/d39/f76 [4194304,4194304] 0 2024-02-16T22:56:36.316 INFO:tasks.workunit.client.0.smithi064.stdout:6/836: creat d0/d6/d10/d33/da4/f116 x:0 0 0 2024-02-16T22:56:36.321 INFO:tasks.workunit.client.0.smithi064.stdout:3/849: creat d4/d1f/d5b/f120 x:0 0 0 2024-02-16T22:56:36.321 INFO:tasks.workunit.client.1.smithi192.stdout:3/852: truncate da/df/f49 679333 0 2024-02-16T22:56:36.323 INFO:tasks.workunit.client.0.smithi064.stdout:1/679: sync 2024-02-16T22:56:36.323 INFO:tasks.workunit.client.0.smithi064.stdout:0/656: sync 2024-02-16T22:56:36.323 INFO:tasks.workunit.client.0.smithi064.stdout:8/699: sync 2024-02-16T22:56:36.323 INFO:tasks.workunit.client.0.smithi064.stdout:4/792: sync 2024-02-16T22:56:36.327 INFO:tasks.workunit.client.0.smithi064.stdout:0/657: fdatasync da/dc/de/d12/d2c/d2e/d38/dbe/dc6/fd8 0 2024-02-16T22:56:36.328 INFO:tasks.workunit.client.0.smithi064.stdout:0/658: fdatasync da/dc/de/d12/f61 0 2024-02-16T22:56:36.328 INFO:tasks.workunit.client.1.smithi192.stdout:6/944: creat da/df/d18/f12c x:0 0 0 2024-02-16T22:56:36.329 INFO:tasks.workunit.client.0.smithi064.stdout:7/775: dread d5/d33/d45/d64/f9e [0,4194304] 0 2024-02-16T22:56:36.334 INFO:tasks.workunit.client.1.smithi192.stdout:8/907: unlink d5/db/f51 0 2024-02-16T22:56:36.335 INFO:tasks.workunit.client.1.smithi192.stdout:8/908: truncate d5/d16/ffd 905763 0 2024-02-16T22:56:36.335 INFO:tasks.workunit.client.0.smithi064.stdout:2/918: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/d95/f66 [0,4194304] 0 2024-02-16T22:56:36.338 INFO:tasks.workunit.client.0.smithi064.stdout:1/680: mkdir d2/df/d14/d2c/db5/de3 0 2024-02-16T22:56:36.343 INFO:tasks.workunit.client.1.smithi192.stdout:4/964: write d1/d26/f77 [419383,26769] 0 2024-02-16T22:56:36.345 INFO:tasks.workunit.client.1.smithi192.stdout:3/853: rename da/d11/d20/f41 to da/d11/d63/d6b/d66/d75/f126 0 2024-02-16T22:56:36.346 INFO:tasks.workunit.client.1.smithi192.stdout:0/936: write d1/d3/d29/f1a [3933325,63698] 0 2024-02-16T22:56:36.346 INFO:tasks.workunit.client.1.smithi192.stdout:4/965: chown d1/d26/d66/d89/d11/fa6 32005 1 2024-02-16T22:56:36.348 INFO:tasks.workunit.client.0.smithi064.stdout:0/659: rename da/dc/de/d12/d2c/d2e/d38/d7e/d89 to da/d26/de0 0 2024-02-16T22:56:36.348 INFO:tasks.workunit.client.0.smithi064.stdout:6/837: rmdir d0/d6/d10/d33/da4 39 2024-02-16T22:56:36.349 INFO:tasks.workunit.client.1.smithi192.stdout:1/891: write d3/de/fc6 [6446084,15419] 0 2024-02-16T22:56:36.351 INFO:tasks.workunit.client.1.smithi192.stdout:8/909: creat d5/d26/d8f/dc0/dc3/f120 x:0 0 0 2024-02-16T22:56:36.351 INFO:tasks.workunit.client.1.smithi192.stdout:8/910: readlink d5/l103 0 2024-02-16T22:56:36.353 INFO:tasks.workunit.client.0.smithi064.stdout:7/776: unlink d5/d33/d42/f74 0 2024-02-16T22:56:36.355 INFO:tasks.workunit.client.1.smithi192.stdout:1/892: dwrite d3/d5b/fc0 [0,4194304] 0 2024-02-16T22:56:36.356 INFO:tasks.workunit.client.1.smithi192.stdout:5/939: fdatasync d6/d12/d2d/fd0 0 2024-02-16T22:56:36.356 INFO:tasks.workunit.client.0.smithi064.stdout:3/850: write d4/d13/d1b/d27/d61/d63/fd3 [1748708,57091] 0 2024-02-16T22:56:36.357 INFO:tasks.workunit.client.1.smithi192.stdout:1/893: readlink d3/de/d27/d34/d52/lca 0 2024-02-16T22:56:36.359 INFO:tasks.workunit.client.0.smithi064.stdout:2/919: mknod d7/d14/d1e/d34/d3f/d9a/dd9/d60/dce/c134 0 2024-02-16T22:56:36.364 INFO:tasks.workunit.client.0.smithi064.stdout:1/681: creat d2/df/d11/d71/fe4 x:0 0 0 2024-02-16T22:56:36.368 INFO:tasks.workunit.client.1.smithi192.stdout:0/937: truncate d1/d3/de/d22/fc0 931940 0 2024-02-16T22:56:36.368 INFO:tasks.workunit.client.0.smithi064.stdout:8/700: dwrite df/d31/d47/f67 [0,4194304] 0 2024-02-16T22:56:36.370 INFO:tasks.workunit.client.0.smithi064.stdout:4/793: dwrite dc/d1d/f36 [0,4194304] 0 2024-02-16T22:56:36.371 INFO:tasks.workunit.client.1.smithi192.stdout:3/854: truncate da/d11/dd4/deb/da1/db9/fdd 1412524 0 2024-02-16T22:56:36.376 INFO:tasks.workunit.client.0.smithi064.stdout:2/920: mknod d7/d14/d25/d46/d55/d57/c135 0 2024-02-16T22:56:36.377 INFO:tasks.workunit.client.1.smithi192.stdout:3/855: dwrite da/d11/f4b [0,4194304] 0 2024-02-16T22:56:36.377 INFO:tasks.workunit.client.0.smithi064.stdout:2/921: read - d7/f11f zero size 2024-02-16T22:56:36.377 INFO:tasks.workunit.client.1.smithi192.stdout:3/856: write da/f1a [335950,87025] 0 2024-02-16T22:56:36.382 INFO:tasks.workunit.client.1.smithi192.stdout:8/911: mknod d5/d16/dd5/c121 0 2024-02-16T22:56:36.387 INFO:tasks.workunit.client.0.smithi064.stdout:0/660: unlink da/dc/d3b/d72/d7c/cb1 0 2024-02-16T22:56:36.387 INFO:tasks.workunit.client.0.smithi064.stdout:3/851: dread d4/d1f/d34/fda [8388608,4194304] 0 2024-02-16T22:56:36.388 INFO:tasks.workunit.client.0.smithi064.stdout:0/661: read - da/dc/fdd zero size 2024-02-16T22:56:36.393 INFO:tasks.workunit.client.0.smithi064.stdout:9/785: sync 2024-02-16T22:56:36.393 INFO:tasks.workunit.client.0.smithi064.stdout:5/747: sync 2024-02-16T22:56:36.395 INFO:tasks.workunit.client.0.smithi064.stdout:9/786: stat d3/d9/d1e/d3e/d76/lf0 0 2024-02-16T22:56:36.395 INFO:tasks.workunit.client.0.smithi064.stdout:6/838: rename d0/d5/df/f8d to d0/d6/d10/f117 0 2024-02-16T22:56:36.396 INFO:tasks.workunit.client.0.smithi064.stdout:3/852: dread - d4/f42 zero size 2024-02-16T22:56:36.398 INFO:tasks.workunit.client.0.smithi064.stdout:8/701: unlink df/d31/d8f/c8b 0 2024-02-16T22:56:36.401 INFO:tasks.workunit.client.1.smithi192.stdout:1/894: mknod d3/de/d27/d34/d52/d63/d81/d116/c12f 0 2024-02-16T22:56:36.401 INFO:tasks.workunit.client.0.smithi064.stdout:5/748: creat d6/d18/d95/dbe/ffb x:0 0 0 2024-02-16T22:56:36.403 INFO:tasks.workunit.client.1.smithi192.stdout:6/945: rename da/df/d6b/d7d/ced to da/df/d6b/c12d 0 2024-02-16T22:56:36.405 INFO:tasks.workunit.client.0.smithi064.stdout:4/794: dread dc/de/f99 [0,4194304] 0 2024-02-16T22:56:36.406 INFO:tasks.workunit.client.1.smithi192.stdout:0/938: creat d1/d3/de/d22/da7/d10d/f12e x:0 0 0 2024-02-16T22:56:36.407 INFO:tasks.workunit.client.1.smithi192.stdout:3/857: mknod da/d106/c127 0 2024-02-16T22:56:36.408 INFO:tasks.workunit.client.0.smithi064.stdout:9/787: mknod d3/d9/d6a/db1/d2a/d2e/d31/d95/c104 0 2024-02-16T22:56:36.411 INFO:tasks.workunit.client.1.smithi192.stdout:8/912: creat d5/db/d47/d43/d67/d112/f122 x:0 0 0 2024-02-16T22:56:36.411 INFO:tasks.workunit.client.1.smithi192.stdout:6/946: dwrite da/df/d71/dab/f124 [0,4194304] 0 2024-02-16T22:56:36.412 INFO:tasks.workunit.client.1.smithi192.stdout:0/939: write d1/d3/de/d22/da7/fb5 [484723,43012] 0 2024-02-16T22:56:36.412 INFO:tasks.workunit.client.1.smithi192.stdout:1/895: mknod d3/d40/d9f/c130 0 2024-02-16T22:56:36.413 INFO:tasks.workunit.client.0.smithi064.stdout:8/702: symlink df/d31/d47/lf3 0 2024-02-16T22:56:36.414 INFO:tasks.workunit.client.1.smithi192.stdout:8/913: chown d5/d26/d8f/dc0/ff4 7 1 2024-02-16T22:56:36.415 INFO:tasks.workunit.client.0.smithi064.stdout:1/682: getdents d2 0 2024-02-16T22:56:36.416 INFO:tasks.workunit.client.0.smithi064.stdout:1/683: write d2/df/d11/d71/fdd [426978,97728] 0 2024-02-16T22:56:36.416 INFO:tasks.workunit.client.1.smithi192.stdout:4/966: dwrite d1/d26/d66/fac [0,4194304] 0 2024-02-16T22:56:36.417 INFO:tasks.workunit.client.0.smithi064.stdout:5/749: mkdir d6/d25/df1/d91/da5/de3/d43/db9/dfc 0 2024-02-16T22:56:36.427 INFO:tasks.workunit.client.1.smithi192.stdout:0/940: dwrite d1/d3/d39/f79 [0,4194304] 0 2024-02-16T22:56:36.429 INFO:tasks.workunit.client.1.smithi192.stdout:0/941: readlink d1/l8f 0 2024-02-16T22:56:36.429 INFO:tasks.workunit.client.1.smithi192.stdout:5/940: dwrite d6/d8/fa3 [0,4194304] 0 2024-02-16T22:56:36.430 INFO:tasks.workunit.client.0.smithi064.stdout:6/839: dread d0/d5/df/f79 [0,4194304] 0 2024-02-16T22:56:36.431 INFO:tasks.workunit.client.0.smithi064.stdout:2/922: dwrite d7/d14/d25/d46/d55/f71 [0,4194304] 0 2024-02-16T22:56:36.431 INFO:tasks.workunit.client.1.smithi192.stdout:1/896: creat d3/de/d27/d34/d36/d47/dcd/f131 x:0 0 0 2024-02-16T22:56:36.432 INFO:tasks.workunit.client.1.smithi192.stdout:6/947: dwrite da/d76/dd5/de2/f11f [8388608,4194304] 0 2024-02-16T22:56:36.432 INFO:tasks.workunit.client.1.smithi192.stdout:5/941: stat d6/d20 0 2024-02-16T22:56:36.433 INFO:tasks.workunit.client.0.smithi064.stdout:8/703: mknod df/d31/d2f/d7f/cf4 0 2024-02-16T22:56:36.434 INFO:tasks.workunit.client.0.smithi064.stdout:1/684: dwrite d2/df/d11/fd2 [0,4194304] 0 2024-02-16T22:56:36.435 INFO:tasks.workunit.client.1.smithi192.stdout:5/942: chown d6/ld4 0 1 2024-02-16T22:56:36.436 INFO:tasks.workunit.client.1.smithi192.stdout:5/943: chown d6/d12/d2d/d5c/d127 3 1 2024-02-16T22:56:36.436 INFO:tasks.workunit.client.0.smithi064.stdout:7/777: rename d5/d33/d53/dde to d5/df9 0 2024-02-16T22:56:36.444 INFO:tasks.workunit.client.0.smithi064.stdout:7/778: dread d5/d33/d53/da3/fb7 [0,4194304] 0 2024-02-16T22:56:36.451 INFO:tasks.workunit.client.0.smithi064.stdout:3/853: write d4/d13/d1b/d27/d44/f5f [4155115,64643] 0 2024-02-16T22:56:36.456 INFO:tasks.workunit.client.0.smithi064.stdout:9/788: mknod d3/d9/d1e/c105 0 2024-02-16T22:56:36.456 INFO:tasks.workunit.client.1.smithi192.stdout:6/948: mkdir da/d40/db6/d12e 0 2024-02-16T22:56:36.457 INFO:tasks.workunit.client.0.smithi064.stdout:6/840: unlink d0/d3c/cb5 0 2024-02-16T22:56:36.459 INFO:tasks.workunit.client.0.smithi064.stdout:2/923: mkdir d7/d14/d4b/d136 0 2024-02-16T22:56:36.461 INFO:tasks.workunit.client.1.smithi192.stdout:1/897: mkdir d3/d5b/db1/db6/d132 0 2024-02-16T22:56:36.461 INFO:tasks.workunit.client.1.smithi192.stdout:4/967: dread d1/d15/d48/f55 [0,4194304] 0 2024-02-16T22:56:36.461 INFO:tasks.workunit.client.0.smithi064.stdout:8/704: mknod df/d31/dc1/cf5 0 2024-02-16T22:56:36.462 INFO:tasks.workunit.client.0.smithi064.stdout:8/705: fsync df/d31/d2f/f59 0 2024-02-16T22:56:36.463 INFO:tasks.workunit.client.1.smithi192.stdout:8/914: creat d5/d26/d83/d86/d5f/db2/f123 x:0 0 0 2024-02-16T22:56:36.464 INFO:tasks.workunit.client.0.smithi064.stdout:0/662: rename da/dc/de/d12/d2c/l59 to da/dc/de/d12/d2c/d2e/d38/dbe/dc6/le1 0 2024-02-16T22:56:36.465 INFO:tasks.workunit.client.1.smithi192.stdout:3/858: dwrite da/d11/dd4/deb/da1/fb0 [0,4194304] 0 2024-02-16T22:56:36.465 INFO:tasks.workunit.client.1.smithi192.stdout:1/898: truncate d3/de/d27/d34/d52/db7/ffa 2298695 0 2024-02-16T22:56:36.465 INFO:tasks.workunit.client.1.smithi192.stdout:8/915: dread d5/db/dcb/fff [0,4194304] 0 2024-02-16T22:56:36.466 INFO:tasks.workunit.client.1.smithi192.stdout:0/942: mkdir d1/dc7/d9c/d12c/d121/d12f 0 2024-02-16T22:56:36.467 INFO:tasks.workunit.client.1.smithi192.stdout:1/899: chown d3/de/d27/d34/d36/d47/dcd/f11b 42945251 1 2024-02-16T22:56:36.474 INFO:tasks.workunit.client.0.smithi064.stdout:7/779: dread d5/f8b [0,4194304] 0 2024-02-16T22:56:36.475 INFO:tasks.workunit.client.0.smithi064.stdout:3/854: truncate d4/d13/d1b/d27/d99/d7d/f9c 2153747 0 2024-02-16T22:56:36.475 INFO:tasks.workunit.client.0.smithi064.stdout:7/780: stat d5/d33/d45/d64/f9e 0 2024-02-16T22:56:36.478 INFO:tasks.workunit.client.0.smithi064.stdout:9/789: creat d3/d88/f106 x:0 0 0 2024-02-16T22:56:36.479 INFO:tasks.workunit.client.1.smithi192.stdout:6/949: creat da/db9/d126/f12f x:0 0 0 2024-02-16T22:56:36.482 INFO:tasks.workunit.client.0.smithi064.stdout:2/924: symlink d7/d14/d1e/d34/d3f/l137 0 2024-02-16T22:56:36.482 INFO:tasks.workunit.client.0.smithi064.stdout:2/925: stat l3 0 2024-02-16T22:56:36.486 INFO:tasks.workunit.client.0.smithi064.stdout:6/841: dwrite d0/d6/d10/db2/ff1 [0,4194304] 0 2024-02-16T22:56:36.486 INFO:tasks.workunit.client.1.smithi192.stdout:0/943: symlink d1/dc7/d9c/d12c/l130 0 2024-02-16T22:56:36.488 INFO:tasks.workunit.client.1.smithi192.stdout:5/944: dread d6/d12/d4e/d5b/daf/fcd [0,4194304] 0 2024-02-16T22:56:36.489 INFO:tasks.workunit.client.0.smithi064.stdout:8/706: dread df/f32 [0,4194304] 0 2024-02-16T22:56:36.490 INFO:tasks.workunit.client.0.smithi064.stdout:6/842: write d0/d89/f93 [268100,118644] 0 2024-02-16T22:56:36.491 INFO:tasks.workunit.client.0.smithi064.stdout:8/707: readlink df/d31/d2f/d68/lcf 0 2024-02-16T22:56:36.494 INFO:tasks.workunit.client.0.smithi064.stdout:4/795: getdents dc/d1d 0 2024-02-16T22:56:36.495 INFO:tasks.workunit.client.1.smithi192.stdout:0/944: dwrite d1/d3/de/d22/f28 [0,4194304] 0 2024-02-16T22:56:36.495 INFO:tasks.workunit.client.0.smithi064.stdout:4/796: chown dc/de/l2a 1740443015 1 2024-02-16T22:56:36.496 INFO:tasks.workunit.client.0.smithi064.stdout:5/750: getdents d6/d25/df1/d91/da5/de3 0 2024-02-16T22:56:36.497 INFO:tasks.workunit.client.1.smithi192.stdout:4/968: write d1/d26/d34/f44 [660932,2220] 0 2024-02-16T22:56:36.497 INFO:tasks.workunit.client.1.smithi192.stdout:6/950: truncate da/d76/dc8/ff4 989375 0 2024-02-16T22:56:36.499 INFO:tasks.workunit.client.0.smithi064.stdout:5/751: truncate d6/d25/df1/d80/fdc 1104503 0 2024-02-16T22:56:36.502 INFO:tasks.workunit.client.0.smithi064.stdout:3/855: mknod d4/d13/d1b/d27/d82/db0/c121 0 2024-02-16T22:56:36.508 INFO:tasks.workunit.client.0.smithi064.stdout:1/685: creat d2/df/d11/d25/fe5 x:0 0 0 2024-02-16T22:56:36.513 INFO:tasks.workunit.client.0.smithi064.stdout:4/797: creat dc/dd/d32/d46/d5a/d7e/d9a/da2/ffd x:0 0 0 2024-02-16T22:56:36.514 INFO:tasks.workunit.client.1.smithi192.stdout:8/916: creat d5/f124 x:0 0 0 2024-02-16T22:56:36.515 INFO:tasks.workunit.client.0.smithi064.stdout:5/752: fdatasync d6/d25/df1/d91/da5/de3/d43/d7d/f9d 0 2024-02-16T22:56:36.515 INFO:tasks.workunit.client.1.smithi192.stdout:5/945: getdents d6/d12/d2d/d5c/d127 0 2024-02-16T22:56:36.517 INFO:tasks.workunit.client.1.smithi192.stdout:5/946: write d6/d8/d75/d8c/d106/f12b [791783,119318] 0 2024-02-16T22:56:36.517 INFO:tasks.workunit.client.0.smithi064.stdout:7/781: mkdir d5/d33/d53/df2/d54/dfa 0 2024-02-16T22:56:36.519 INFO:tasks.workunit.client.1.smithi192.stdout:1/900: rename d3/de/da4/fda to d3/de/d87/f133 0 2024-02-16T22:56:36.521 INFO:tasks.workunit.client.1.smithi192.stdout:0/945: mknod d1/d3/de/d22/c131 0 2024-02-16T22:56:36.522 INFO:tasks.workunit.client.0.smithi064.stdout:9/790: symlink d3/d11/d9e/da5/df3/l107 0 2024-02-16T22:56:36.524 INFO:tasks.workunit.client.1.smithi192.stdout:0/946: chown d1/d3/de/d22/da7/f123 62070 1 2024-02-16T22:56:36.524 INFO:tasks.workunit.client.1.smithi192.stdout:4/969: dread d1/f30 [0,4194304] 0 2024-02-16T22:56:36.524 INFO:tasks.workunit.client.0.smithi064.stdout:9/791: dread d3/d9/d6a/db1/d38/f66 [0,4194304] 0 2024-02-16T22:56:36.525 INFO:tasks.workunit.client.1.smithi192.stdout:3/859: link da/d11/d2b/cb3 da/d11/d63/d6b/d66/d75/c128 0 2024-02-16T22:56:36.526 INFO:tasks.workunit.client.1.smithi192.stdout:1/901: dwrite d3/de/d27/d34/f112 [0,4194304] 0 2024-02-16T22:56:36.528 INFO:tasks.workunit.client.0.smithi064.stdout:6/843: truncate d0/d12/f74 2535988 0 2024-02-16T22:56:36.530 INFO:tasks.workunit.client.1.smithi192.stdout:6/951: dwrite da/df/d18/f63 [0,4194304] 0 2024-02-16T22:56:36.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/947: creat d6/d8/d86/d10f/f131 x:0 0 0 2024-02-16T22:56:36.533 INFO:tasks.workunit.client.1.smithi192.stdout:0/947: creat d1/dd6/f132 x:0 0 0 2024-02-16T22:56:36.535 INFO:tasks.workunit.client.1.smithi192.stdout:4/970: rmdir d1/d26/d66/d89/d11/d58 39 2024-02-16T22:56:36.535 INFO:tasks.workunit.client.1.smithi192.stdout:0/948: read d1/dc7/d9c/d12c/f11f [4033790,13746] 0 2024-02-16T22:56:36.538 INFO:tasks.workunit.client.1.smithi192.stdout:8/917: symlink d5/db/de/l125 0 2024-02-16T22:56:36.542 INFO:tasks.workunit.client.1.smithi192.stdout:1/902: unlink d3/d5b/d77/f11d 0 2024-02-16T22:56:36.542 INFO:tasks.workunit.client.1.smithi192.stdout:1/903: readlink d3/de/l58 0 2024-02-16T22:56:36.542 INFO:tasks.workunit.client.1.smithi192.stdout:0/949: dwrite d1/f125 [0,4194304] 0 2024-02-16T22:56:36.545 INFO:tasks.workunit.client.0.smithi064.stdout:2/926: link d7/d14/d1e/d34/dad/d10c/dea/c9e d7/d14/d4b/d136/c138 0 2024-02-16T22:56:36.548 INFO:tasks.workunit.client.0.smithi064.stdout:1/686: mkdir d2/df/d14/d2c/db5/de3/de6 0 2024-02-16T22:56:36.550 INFO:tasks.workunit.client.0.smithi064.stdout:4/798: write dc/dd/d32/f8d [1352971,114831] 0 2024-02-16T22:56:36.552 INFO:tasks.workunit.client.0.smithi064.stdout:1/687: dread - d2/df/d14/d18/d48/d6b/d75/fcf zero size 2024-02-16T22:56:36.552 INFO:tasks.workunit.client.0.smithi064.stdout:4/799: write dc/d1d/d43/fae [2376170,112142] 0 2024-02-16T22:56:36.553 INFO:tasks.workunit.client.0.smithi064.stdout:9/792: read d3/d9/d6a/db1/d2a/d2e/d31/d3f/f5b [310562,90794] 0 2024-02-16T22:56:36.554 INFO:tasks.workunit.client.0.smithi064.stdout:9/793: readlink d3/d88/la0 0 2024-02-16T22:56:36.554 INFO:tasks.workunit.client.0.smithi064.stdout:6/844: truncate d0/d6/f15 338407 0 2024-02-16T22:56:36.556 INFO:tasks.workunit.client.0.smithi064.stdout:6/845: stat d0/d5/df/d17/d4a/lad 0 2024-02-16T22:56:36.557 INFO:tasks.workunit.client.0.smithi064.stdout:6/846: chown d0/d1d/d21/d98 2388 1 2024-02-16T22:56:36.560 INFO:tasks.workunit.client.1.smithi192.stdout:4/971: mknod d1/d26/d34/dc7/c146 0 2024-02-16T22:56:36.560 INFO:tasks.workunit.client.0.smithi064.stdout:5/753: dwrite d6/d25/da4/dc1/f69 [0,4194304] 0 2024-02-16T22:56:36.562 INFO:tasks.workunit.client.1.smithi192.stdout:5/948: write d6/d12/d4e/d46/d4f/fec [2846281,127449] 0 2024-02-16T22:56:36.566 INFO:tasks.workunit.client.0.smithi064.stdout:2/927: dread d7/d14/d1e/d34/f50 [0,4194304] 0 2024-02-16T22:56:36.567 INFO:tasks.workunit.client.1.smithi192.stdout:0/950: unlink d1/c74 0 2024-02-16T22:56:36.568 INFO:tasks.workunit.client.0.smithi064.stdout:7/782: chown d5/d33/d42/d81/d82/ca6 782782 1 2024-02-16T22:56:36.569 INFO:tasks.workunit.client.0.smithi064.stdout:7/783: write d5/d33/fd1 [636474,31143] 0 2024-02-16T22:56:36.572 INFO:tasks.workunit.client.1.smithi192.stdout:6/952: getdents da/df/d18/d52/de6 0 2024-02-16T22:56:36.575 INFO:tasks.workunit.client.1.smithi192.stdout:3/860: dwrite da/d11/d63/f9e [0,4194304] 0 2024-02-16T22:56:36.575 INFO:tasks.workunit.client.0.smithi064.stdout:4/800: mknod dc/d1d/cfe 0 2024-02-16T22:56:36.578 INFO:tasks.workunit.client.0.smithi064.stdout:9/794: creat d3/d11/d9e/f108 x:0 0 0 2024-02-16T22:56:36.578 INFO:tasks.workunit.client.1.smithi192.stdout:8/918: dwrite d5/d16/dd5/d9d/fde [0,4194304] 0 2024-02-16T22:56:36.579 INFO:tasks.workunit.client.0.smithi064.stdout:8/708: getdents df/d31/d2f/d3e/d5f/d93 0 2024-02-16T22:56:36.579 INFO:tasks.workunit.client.0.smithi064.stdout:9/795: chown d3/d11/d9e/da5/ff9 7219 1 2024-02-16T22:56:36.582 INFO:tasks.workunit.client.0.smithi064.stdout:9/796: write d3/d89/d9a/fce [730285,58852] 0 2024-02-16T22:56:36.584 INFO:tasks.workunit.client.0.smithi064.stdout:9/797: fdatasync d3/d11/d15/d23/f6e 0 2024-02-16T22:56:36.586 INFO:tasks.workunit.client.0.smithi064.stdout:9/798: chown d3/d9/d6a/db1/d38/fca 1431 1 2024-02-16T22:56:36.586 INFO:tasks.workunit.client.0.smithi064.stdout:2/928: creat d7/d14/d1e/dfd/f139 x:0 0 0 2024-02-16T22:56:36.589 INFO:tasks.workunit.client.0.smithi064.stdout:3/856: getdents d4/d13/d1b/d27/d82 0 2024-02-16T22:56:36.589 INFO:tasks.workunit.client.0.smithi064.stdout:2/929: chown d7/d14/d1e/d34/d3f/f4e 1 1 2024-02-16T22:56:36.591 INFO:tasks.workunit.client.0.smithi064.stdout:6/847: dwrite d0/d6/d10/db2/ff8 [0,4194304] 0 2024-02-16T22:56:36.599 INFO:tasks.workunit.client.0.smithi064.stdout:4/801: rename dc/dd/d32/d46/d79 to dc/de/dfb/dff 0 2024-02-16T22:56:36.601 INFO:tasks.workunit.client.0.smithi064.stdout:8/709: unlink df/c11 0 2024-02-16T22:56:36.606 INFO:tasks.workunit.client.0.smithi064.stdout:5/754: creat d6/d25/df1/d91/da5/de3/d43/db9/dfc/ffd x:0 0 0 2024-02-16T22:56:36.612 INFO:tasks.workunit.client.0.smithi064.stdout:3/857: fsync d4/fd 0 2024-02-16T22:56:36.613 INFO:tasks.workunit.client.0.smithi064.stdout:3/858: stat d4/d14 0 2024-02-16T22:56:36.614 INFO:tasks.workunit.client.0.smithi064.stdout:3/859: chown d4/d13/d4d/fb8 99008438 1 2024-02-16T22:56:36.615 INFO:tasks.workunit.client.0.smithi064.stdout:2/930: unlink d7/d14/d1e/fa4 0 2024-02-16T22:56:36.616 INFO:tasks.workunit.client.0.smithi064.stdout:1/688: dwrite d2/df/d14/d2c/f3c [0,4194304] 0 2024-02-16T22:56:36.618 INFO:tasks.workunit.client.0.smithi064.stdout:1/689: fdatasync d2/df/d14/d18/d48/d6b/d75/fcf 0 2024-02-16T22:56:36.618 INFO:tasks.workunit.client.0.smithi064.stdout:1/690: readlink d2/df/d14/d18/l8b 0 2024-02-16T22:56:36.621 INFO:tasks.workunit.client.0.smithi064.stdout:7/784: creat d5/d33/d42/d81/dc4/dcc/ffb x:0 0 0 2024-02-16T22:56:36.626 INFO:tasks.workunit.client.0.smithi064.stdout:9/799: write d3/d11/d15/fe6 [974684,120066] 0 2024-02-16T22:56:36.630 INFO:tasks.workunit.client.0.smithi064.stdout:8/710: creat df/d31/d61/d94/ff6 x:0 0 0 2024-02-16T22:56:36.632 INFO:tasks.workunit.client.0.smithi064.stdout:5/755: unlink d6/d25/df1/d91/da5/de3/d4b/l51 0 2024-02-16T22:56:36.634 INFO:tasks.workunit.client.0.smithi064.stdout:5/756: dread - d6/d18/d95/fea zero size 2024-02-16T22:56:36.641 INFO:tasks.workunit.client.0.smithi064.stdout:4/802: write dc/dd/d32/f3d [4007844,127341] 0 2024-02-16T22:56:36.641 INFO:tasks.workunit.client.0.smithi064.stdout:6/848: dwrite d0/d3c/fb7 [0,4194304] 0 2024-02-16T22:56:36.642 INFO:tasks.workunit.client.0.smithi064.stdout:6/849: write d0/d6/d10/db2/ff8 [76525,123380] 0 2024-02-16T22:56:36.650 INFO:tasks.workunit.client.0.smithi064.stdout:2/931: dread d7/d14/d1e/f81 [0,4194304] 0 2024-02-16T22:56:36.650 INFO:tasks.workunit.client.0.smithi064.stdout:2/932: chown d7/d14/d1e/d34/dad/d10c/ld7 11818 1 2024-02-16T22:56:36.673 INFO:tasks.workunit.client.0.smithi064.stdout:1/691: write d2/df/d14/f59 [516274,25228] 0 2024-02-16T22:56:36.673 INFO:tasks.workunit.client.0.smithi064.stdout:9/800: write d3/d9/d6a/db1/d2a/d2e/d31/d3f/f43 [7858503,67798] 0 2024-02-16T22:56:36.675 INFO:tasks.workunit.client.0.smithi064.stdout:9/801: write d3/d9/d6a/db1/fb9 [1954392,18034] 0 2024-02-16T22:56:36.679 INFO:tasks.workunit.client.0.smithi064.stdout:5/757: readlink d6/d18/d19/l57 0 2024-02-16T22:56:36.692 INFO:tasks.workunit.client.0.smithi064.stdout:2/933: read d7/d14/d1e/d34/d3f/d9a/dd9/d95/fac [3856081,79593] 0 2024-02-16T22:56:36.695 INFO:tasks.workunit.client.0.smithi064.stdout:9/802: rename d3/d11/lbe to d3/d11/d15/d23/ddd/l109 0 2024-02-16T22:56:36.697 INFO:tasks.workunit.client.0.smithi064.stdout:7/785: write d5/d33/d45/d64/f91 [1006250,95673] 0 2024-02-16T22:56:36.697 INFO:tasks.workunit.client.0.smithi064.stdout:4/803: dread dc/de/dfb/dff/fc7 [0,4194304] 0 2024-02-16T22:56:36.700 INFO:tasks.workunit.client.1.smithi192.stdout:4/972: unlink d1/d26/d66/l83 0 2024-02-16T22:56:36.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/904: symlink d3/de/d27/d34/d69/d85/d89/l134 0 2024-02-16T22:56:36.706 INFO:tasks.workunit.client.1.smithi192.stdout:0/951: rename d1/d3/d18/l57 to d1/d3/d29/dcd/dcf/l133 0 2024-02-16T22:56:36.712 INFO:tasks.workunit.client.1.smithi192.stdout:8/919: mkdir d5/d26/d83/d86/d5f/db2/d126 0 2024-02-16T22:56:36.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/973: creat d1/d15/d29/dd4/f147 x:0 0 0 2024-02-16T22:56:36.720 INFO:tasks.workunit.client.1.smithi192.stdout:0/952: fdatasync d1/f36 0 2024-02-16T22:56:36.722 INFO:tasks.workunit.client.1.smithi192.stdout:6/953: symlink da/df/d71/dde/l130 0 2024-02-16T22:56:36.722 INFO:tasks.workunit.client.1.smithi192.stdout:6/954: fdatasync da/d40/db6/db8/ff0 0 2024-02-16T22:56:36.723 INFO:tasks.workunit.client.1.smithi192.stdout:6/955: readlink da/df/d18/l127 0 2024-02-16T22:56:36.725 INFO:tasks.workunit.client.1.smithi192.stdout:8/920: symlink d5/d26/d8f/dc0/dc3/l127 0 2024-02-16T22:56:36.728 INFO:tasks.workunit.client.1.smithi192.stdout:4/974: symlink d1/d15/d48/d8d/l148 0 2024-02-16T22:56:36.730 INFO:tasks.workunit.client.1.smithi192.stdout:1/905: dwrite d3/de/d27/d34/d52/f54 [0,4194304] 0 2024-02-16T22:56:36.735 INFO:tasks.workunit.client.1.smithi192.stdout:6/956: rename da/df/d71/dde/l130 to da/d76/dd5/l131 0 2024-02-16T22:56:36.735 INFO:tasks.workunit.client.1.smithi192.stdout:5/949: getdents d6/d8/d75/d8c/d106 0 2024-02-16T22:56:36.736 INFO:tasks.workunit.client.1.smithi192.stdout:5/950: stat d6/fd 0 2024-02-16T22:56:36.736 INFO:tasks.workunit.client.1.smithi192.stdout:5/951: chown d6/d8/d77/d11a 10248262 1 2024-02-16T22:56:36.737 INFO:tasks.workunit.client.1.smithi192.stdout:5/952: readlink d6/d20/l91 0 2024-02-16T22:56:36.739 INFO:tasks.workunit.client.1.smithi192.stdout:6/957: dread da/df/d12/d15/d1b/f8a [0,4194304] 0 2024-02-16T22:56:36.740 INFO:tasks.workunit.client.1.smithi192.stdout:3/861: link da/d11/d63/d6b/d66/d75/c101 da/d11/d20/d39/c129 0 2024-02-16T22:56:36.741 INFO:tasks.workunit.client.1.smithi192.stdout:3/862: dread - da/d11/d2b/ddc/f125 zero size 2024-02-16T22:56:36.743 INFO:tasks.workunit.client.1.smithi192.stdout:4/975: symlink d1/d15/d48/dc3/d115/d11d/d12b/l149 0 2024-02-16T22:56:36.744 INFO:tasks.workunit.client.1.smithi192.stdout:5/953: creat d6/d12/d2d/f132 x:0 0 0 2024-02-16T22:56:36.745 INFO:tasks.workunit.client.1.smithi192.stdout:6/958: symlink da/db9/d126/l132 0 2024-02-16T22:56:36.746 INFO:tasks.workunit.client.1.smithi192.stdout:0/953: truncate d1/d3/de/d22/d37/f87 2181651 0 2024-02-16T22:56:36.747 INFO:tasks.workunit.client.1.smithi192.stdout:8/921: link d5/d26/d83/d86/l2a d5/d26/d83/d86/d5f/db2/l128 0 2024-02-16T22:56:36.747 INFO:tasks.workunit.client.1.smithi192.stdout:1/906: write d3/de/d27/d34/d69/d85/d89/daa/ff6 [790047,12940] 0 2024-02-16T22:56:36.747 INFO:tasks.workunit.client.1.smithi192.stdout:3/863: stat da/d11/f38 0 2024-02-16T22:56:36.747 INFO:tasks.workunit.client.1.smithi192.stdout:4/976: mkdir d1/d15/d23/d14a 0 2024-02-16T22:56:36.751 INFO:tasks.workunit.client.1.smithi192.stdout:4/977: dread d1/d15/d29/fdf [0,4194304] 0 2024-02-16T22:56:36.753 INFO:tasks.workunit.client.1.smithi192.stdout:3/864: dread da/f12 [0,4194304] 0 2024-02-16T22:56:36.802 INFO:tasks.workunit.client.0.smithi064.stdout:3/860: creat d4/d1f/d34/f122 x:0 0 0 2024-02-16T22:56:36.802 INFO:tasks.workunit.client.1.smithi192.stdout:5/954: creat d6/d8/d86/f133 x:0 0 0 2024-02-16T22:56:36.804 INFO:tasks.workunit.client.1.smithi192.stdout:6/959: rename da/df/d6b/d83 to da/df/d12/d57/d81/dfc/d133 0 2024-02-16T22:56:36.806 INFO:tasks.workunit.client.0.smithi064.stdout:4/804: mknod dc/de/dfb/dff/de3/c100 0 2024-02-16T22:56:36.806 INFO:tasks.workunit.client.0.smithi064.stdout:7/786: dread d5/f23 [0,4194304] 0 2024-02-16T22:56:36.807 INFO:tasks.workunit.client.0.smithi064.stdout:1/692: creat d2/df/d14/d2c/fe7 x:0 0 0 2024-02-16T22:56:36.809 INFO:tasks.workunit.client.1.smithi192.stdout:8/922: mknod d5/d26/d83/db4/c129 0 2024-02-16T22:56:36.809 INFO:tasks.workunit.client.1.smithi192.stdout:8/923: dread - d5/f124 zero size 2024-02-16T22:56:36.811 INFO:tasks.workunit.client.1.smithi192.stdout:1/907: creat d3/d5b/d77/dd2/f135 x:0 0 0 2024-02-16T22:56:36.811 INFO:tasks.workunit.client.0.smithi064.stdout:6/850: dwrite d0/d6/d10/d33/da4/ddd/f112 [0,4194304] 0 2024-02-16T22:56:36.812 INFO:tasks.workunit.client.1.smithi192.stdout:1/908: chown d3/de/d27/d34/d52/d63/l6f 19082770 1 2024-02-16T22:56:36.814 INFO:tasks.workunit.client.0.smithi064.stdout:8/711: dread df/f24 [0,4194304] 0 2024-02-16T22:56:36.821 INFO:tasks.workunit.client.1.smithi192.stdout:5/955: truncate d6/d12/d4e/f1c 250649 0 2024-02-16T22:56:36.822 INFO:tasks.workunit.client.1.smithi192.stdout:5/956: fsync d6/d12/d4e/d58/d8f/fa2 0 2024-02-16T22:56:36.823 INFO:tasks.workunit.client.1.smithi192.stdout:5/957: fdatasync d6/d8/d80/f8a 0 2024-02-16T22:56:36.824 INFO:tasks.workunit.client.1.smithi192.stdout:6/960: mkdir da/df/d58/d134 0 2024-02-16T22:56:36.824 INFO:tasks.workunit.client.1.smithi192.stdout:5/958: write d6/d12/d1d/d56/d108/f109 [871383,699] 0 2024-02-16T22:56:36.827 INFO:tasks.workunit.client.0.smithi064.stdout:4/805: rename dc/d1d to dc/de/d28/d57/d77/d101 0 2024-02-16T22:56:36.829 INFO:tasks.workunit.client.0.smithi064.stdout:1/693: fsync d2/df/d11/d1a/d30/dbd/fb7 0 2024-02-16T22:56:36.831 INFO:tasks.workunit.client.0.smithi064.stdout:6/851: unlink d0/d6/da/lc3 0 2024-02-16T22:56:36.837 INFO:tasks.workunit.client.0.smithi064.stdout:9/803: link d3/d9/d1e/d42/d7b/lb4 d3/d9/d1e/d42/l10a 0 2024-02-16T22:56:36.838 INFO:tasks.workunit.client.1.smithi192.stdout:8/924: write d5/d16/fd2 [3099632,36033] 0 2024-02-16T22:56:36.843 INFO:tasks.workunit.client.1.smithi192.stdout:8/925: dwrite d5/db/d47/f4a [0,4194304] 0 2024-02-16T22:56:36.844 INFO:tasks.workunit.client.0.smithi064.stdout:7/787: mknod d5/d33/d45/d73/dd7/cfc 0 2024-02-16T22:56:36.847 INFO:tasks.workunit.client.0.smithi064.stdout:4/806: mknod dc/ddc/de8/da1/c102 0 2024-02-16T22:56:36.851 INFO:tasks.workunit.client.1.smithi192.stdout:3/865: dwrite da/d11/d63/d6b/faa [0,4194304] 0 2024-02-16T22:56:36.852 INFO:tasks.workunit.client.0.smithi064.stdout:3/861: dwrite d4/d13/d4d/d5c/d78/fed [0,4194304] 0 2024-02-16T22:56:36.854 INFO:tasks.workunit.client.0.smithi064.stdout:3/862: chown d4/d13/d4d/d5c/d78/fed 2753674 1 2024-02-16T22:56:36.854 INFO:tasks.workunit.client.0.smithi064.stdout:8/712: chown df/d31/d47/lf3 1066 1 2024-02-16T22:56:36.857 INFO:tasks.workunit.client.1.smithi192.stdout:1/909: dread d3/de/d27/d34/d52/fe0 [0,4194304] 0 2024-02-16T22:56:36.859 INFO:tasks.workunit.client.0.smithi064.stdout:2/934: link l4 d7/d14/d1e/d34/dad/l13a 0 2024-02-16T22:56:36.860 INFO:tasks.workunit.client.1.smithi192.stdout:4/978: creat d1/d26/d66/d89/f14b x:0 0 0 2024-02-16T22:56:36.860 INFO:tasks.workunit.client.1.smithi192.stdout:4/979: chown d1/d15/d48/d8d/d128 1 1 2024-02-16T22:56:36.863 INFO:tasks.workunit.client.0.smithi064.stdout:7/788: rename d5/d33/d53/c71 to d5/d33/d53/df2/d54/dfa/cfd 0 2024-02-16T22:56:36.864 INFO:tasks.workunit.client.1.smithi192.stdout:0/954: getdents d1/d3/de/d22 0 2024-02-16T22:56:36.867 INFO:tasks.workunit.client.1.smithi192.stdout:6/961: mknod da/df/d12/d15/c135 0 2024-02-16T22:56:36.868 INFO:tasks.workunit.client.0.smithi064.stdout:1/694: fsync d2/df/f54 0 2024-02-16T22:56:36.869 INFO:tasks.workunit.client.0.smithi064.stdout:1/695: dread - d2/df/d14/d2c/fe7 zero size 2024-02-16T22:56:36.869 INFO:tasks.workunit.client.1.smithi192.stdout:5/959: symlink d6/d12/l134 0 2024-02-16T22:56:36.871 INFO:tasks.workunit.client.0.smithi064.stdout:8/713: creat df/d31/d47/ff7 x:0 0 0 2024-02-16T22:56:36.871 INFO:tasks.workunit.client.1.smithi192.stdout:3/866: dread da/d11/d35/f70 [0,4194304] 0 2024-02-16T22:56:36.871 INFO:tasks.workunit.client.1.smithi192.stdout:1/910: creat d3/de/d27/d34/d52/d63/d7a/dc9/f136 x:0 0 0 2024-02-16T22:56:36.872 INFO:tasks.workunit.client.1.smithi192.stdout:5/960: chown d6/d12/d2d/d5c/f99 46826 1 2024-02-16T22:56:36.872 INFO:tasks.workunit.client.0.smithi064.stdout:9/804: unlink d3/d9/d6a/db1/f56 0 2024-02-16T22:56:36.873 INFO:tasks.workunit.client.0.smithi064.stdout:9/805: write d3/d9/d6a/db1/f27 [5041876,90977] 0 2024-02-16T22:56:36.873 INFO:tasks.workunit.client.1.smithi192.stdout:8/926: dwrite d5/d26/d83/fdb [0,4194304] 0 2024-02-16T22:56:36.874 INFO:tasks.workunit.client.0.smithi064.stdout:9/806: readlink d3/d11/d15/d23/l35 0 2024-02-16T22:56:36.874 INFO:tasks.workunit.client.1.smithi192.stdout:4/980: rename d1/d15/d29 to d1/d6b/d14c 0 2024-02-16T22:56:36.879 INFO:tasks.workunit.client.0.smithi064.stdout:3/863: rename d4/d1f/d5b/d51/dc1/c101 to d4/d13/d1b/d27/d82/db0/c123 0 2024-02-16T22:56:36.882 INFO:tasks.workunit.client.0.smithi064.stdout:7/789: mkdir d5/d33/d53/dfe 0 2024-02-16T22:56:36.883 INFO:tasks.workunit.client.0.smithi064.stdout:9/807: dwrite d3/d11/d15/fe6 [4194304,4194304] 0 2024-02-16T22:56:36.883 INFO:tasks.workunit.client.1.smithi192.stdout:6/962: dwrite da/df/fef [0,4194304] 0 2024-02-16T22:56:36.885 INFO:tasks.workunit.client.0.smithi064.stdout:9/808: dread - d3/d9/d6a/db1/d2a/d2e/fc0 zero size 2024-02-16T22:56:36.886 INFO:tasks.workunit.client.1.smithi192.stdout:6/963: stat da/d40/db6/d12e 0 2024-02-16T22:56:36.886 INFO:tasks.workunit.client.0.smithi064.stdout:4/807: mknod dc/dd/d16/c103 0 2024-02-16T22:56:36.888 INFO:tasks.workunit.client.1.smithi192.stdout:6/964: readlink da/df/d12/d57/daf/dcf/l11b 0 2024-02-16T22:56:36.889 INFO:tasks.workunit.client.1.smithi192.stdout:3/867: dread da/d11/d20/d39/d53/f8a [0,4194304] 0 2024-02-16T22:56:36.891 INFO:tasks.workunit.client.1.smithi192.stdout:4/981: dread - d1/d26/d6f/df4/f133 zero size 2024-02-16T22:56:36.892 INFO:tasks.workunit.client.1.smithi192.stdout:4/982: chown d1/d6b/d14c/c46 26 1 2024-02-16T22:56:36.893 INFO:tasks.workunit.client.1.smithi192.stdout:0/955: symlink d1/d3/de/d22/d37/l134 0 2024-02-16T22:56:36.894 INFO:tasks.workunit.client.0.smithi064.stdout:8/714: creat df/d31/d2f/d3e/ddf/ff8 x:0 0 0 2024-02-16T22:56:36.898 INFO:tasks.workunit.client.0.smithi064.stdout:3/864: rmdir d4/d1f/d34 39 2024-02-16T22:56:36.899 INFO:tasks.workunit.client.0.smithi064.stdout:3/865: dread - d4/d13/d1b/d27/d81/f10d zero size 2024-02-16T22:56:36.903 INFO:tasks.workunit.client.0.smithi064.stdout:9/809: unlink d3/d9/d6a/db1/d2a/d2e/d31/d95/c96 0 2024-02-16T22:56:36.906 INFO:tasks.workunit.client.0.smithi064.stdout:4/808: fsync dc/de/d28/d57/d77/d101/f4c 0 2024-02-16T22:56:36.907 INFO:tasks.workunit.client.1.smithi192.stdout:1/911: write d3/de/d27/f4a [3538621,1243] 0 2024-02-16T22:56:36.907 INFO:tasks.workunit.client.0.smithi064.stdout:2/935: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/f15 [0,4194304] 0 2024-02-16T22:56:36.908 INFO:tasks.workunit.client.0.smithi064.stdout:2/936: chown d7/d14/d25/d46/d55/d7f/l9f 602 1 2024-02-16T22:56:36.909 INFO:tasks.workunit.client.1.smithi192.stdout:4/983: creat d1/d26/d66/d89/da8/dc5/d111/d12f/f14d x:0 0 0 2024-02-16T22:56:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:5/961: rename d6/d8/l31 to d6/d12/d4e/d58/l135 0 2024-02-16T22:56:36.915 INFO:tasks.workunit.client.0.smithi064.stdout:3/866: mknod d4/d1f/d5b/d51/dc1/c124 0 2024-02-16T22:56:36.916 INFO:tasks.workunit.client.1.smithi192.stdout:0/956: creat d1/d3/de/d22/da7/d112/f135 x:0 0 0 2024-02-16T22:56:36.917 INFO:tasks.workunit.client.0.smithi064.stdout:3/867: chown d4/d13/d1b/d27/d82/l119 1789602 1 2024-02-16T22:56:36.918 INFO:tasks.workunit.client.1.smithi192.stdout:3/868: dread f9 [0,4194304] 0 2024-02-16T22:56:36.918 INFO:tasks.workunit.client.0.smithi064.stdout:2/937: dwrite d7/d14/d1e/d34/dad/d10c/dea/f10f [0,4194304] 0 2024-02-16T22:56:36.918 INFO:tasks.workunit.client.0.smithi064.stdout:7/790: symlink d5/d33/d42/d81/d82/df3/lff 0 2024-02-16T22:56:36.919 INFO:tasks.workunit.client.0.smithi064.stdout:3/868: write d4/d13/d1b/d4f/f6f [5025122,866] 0 2024-02-16T22:56:36.921 INFO:tasks.workunit.client.0.smithi064.stdout:3/869: chown d4/d14/d2b/l10e 11084 1 2024-02-16T22:56:36.922 INFO:tasks.workunit.client.0.smithi064.stdout:2/938: chown d7/d14/d1e/d34/d3f/d9a/dd9/d60/dce/c134 12 1 2024-02-16T22:56:36.923 INFO:tasks.workunit.client.1.smithi192.stdout:6/965: dwrite da/df/d12/d15/fdf [0,4194304] 0 2024-02-16T22:56:36.923 INFO:tasks.workunit.client.1.smithi192.stdout:5/962: dread d6/d12/d4e/d58/fb7 [0,4194304] 0 2024-02-16T22:56:36.926 INFO:tasks.workunit.client.1.smithi192.stdout:1/912: read d3/de/d27/d34/d52/d5a/fa2 [3994524,102040] 0 2024-02-16T22:56:36.926 INFO:tasks.workunit.client.1.smithi192.stdout:0/957: rmdir d1/d3/d18 39 2024-02-16T22:56:36.932 INFO:tasks.workunit.client.0.smithi064.stdout:2/939: dread d7/d14/d1e/d34/f87 [0,4194304] 0 2024-02-16T22:56:36.932 INFO:tasks.workunit.client.0.smithi064.stdout:9/810: symlink d3/d9/d6a/db1/d2a/d2e/da3/dea/l10b 0 2024-02-16T22:56:36.936 INFO:tasks.workunit.client.1.smithi192.stdout:1/913: mkdir d3/de/d27/d34/d36/d47/dcd/d137 0 2024-02-16T22:56:36.938 INFO:tasks.workunit.client.1.smithi192.stdout:4/984: mknod d1/d26/d34/db9/c14e 0 2024-02-16T22:56:36.940 INFO:tasks.workunit.client.0.smithi064.stdout:1/696: getdents d2/df/d11/d25 0 2024-02-16T22:56:36.941 INFO:tasks.workunit.client.0.smithi064.stdout:8/715: mkdir df/d2a/df9 0 2024-02-16T22:56:36.941 INFO:tasks.workunit.client.0.smithi064.stdout:2/940: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/f15 [0,4194304] 0 2024-02-16T22:56:36.942 INFO:tasks.workunit.client.1.smithi192.stdout:3/869: truncate da/d11/d2b/ddc/ff2 737192 0 2024-02-16T22:56:36.949 INFO:tasks.workunit.client.0.smithi064.stdout:2/941: dread d7/d14/d1e/d34/d3f/f4e [4194304,4194304] 0 2024-02-16T22:56:36.952 INFO:tasks.workunit.client.1.smithi192.stdout:1/914: truncate d3/de/d27/d34/d69/d85/d89/daa/fdd 140687 0 2024-02-16T22:56:36.954 INFO:tasks.workunit.client.1.smithi192.stdout:4/985: creat d1/d26/d34/db9/d110/f14f x:0 0 0 2024-02-16T22:56:36.955 INFO:tasks.workunit.client.0.smithi064.stdout:1/697: mkdir d2/de8 0 2024-02-16T22:56:36.955 INFO:tasks.workunit.client.0.smithi064.stdout:1/698: chown d2/f43 115 1 2024-02-16T22:56:36.957 INFO:tasks.workunit.client.0.smithi064.stdout:9/811: dread d3/d9/f48 [0,4194304] 0 2024-02-16T22:56:36.957 INFO:tasks.workunit.client.0.smithi064.stdout:1/699: chown d2/df/d14/d2c/db5/dc6 3248 1 2024-02-16T22:56:36.957 INFO:tasks.workunit.client.1.smithi192.stdout:5/963: creat d6/d12/d4e/d5b/f136 x:0 0 0 2024-02-16T22:56:36.957 INFO:tasks.workunit.client.0.smithi064.stdout:4/809: write dc/de/d6d/f87 [2882163,88996] 0 2024-02-16T22:56:36.961 INFO:tasks.workunit.client.1.smithi192.stdout:0/958: write d1/d3/d29/f73 [763645,57234] 0 2024-02-16T22:56:36.962 INFO:tasks.workunit.client.1.smithi192.stdout:6/966: link da/df/d18/d52/da9/f125 da/df/d6b/f136 0 2024-02-16T22:56:36.962 INFO:tasks.workunit.client.0.smithi064.stdout:3/870: link d4/d14/c5a d4/d1f/d5b/d51/d7c/c125 0 2024-02-16T22:56:36.963 INFO:tasks.workunit.client.0.smithi064.stdout:3/871: fsync d4/d13/d1b/d27/d99/d7d/f11d 0 2024-02-16T22:56:36.964 INFO:tasks.workunit.client.1.smithi192.stdout:1/915: rename d3/de/d27/d34/d69/d85/d89/cc5 to d3/d5b/db1/db6/d132/c138 0 2024-02-16T22:56:36.964 INFO:tasks.workunit.client.0.smithi064.stdout:3/872: chown d4/d13/d1b/d27/d61/d63 41842 1 2024-02-16T22:56:36.964 INFO:tasks.workunit.client.0.smithi064.stdout:3/873: stat d4/d13/d4d/d5c/fcf 0 2024-02-16T22:56:36.971 INFO:tasks.workunit.client.0.smithi064.stdout:0/663: sync 2024-02-16T22:56:36.972 INFO:tasks.workunit.client.0.smithi064.stdout:0/664: write da/dc/d3b/d72/d7c/fad [988270,115372] 0 2024-02-16T22:56:36.972 INFO:tasks.workunit.client.0.smithi064.stdout:2/942: fsync d7/d14/d4b/f88 0 2024-02-16T22:56:36.974 INFO:tasks.workunit.client.0.smithi064.stdout:1/700: mknod d2/df/d11/d1a/d1d/ce9 0 2024-02-16T22:56:36.976 INFO:tasks.workunit.client.1.smithi192.stdout:3/870: mkdir da/d12a 0 2024-02-16T22:56:36.980 INFO:tasks.workunit.client.0.smithi064.stdout:9/812: creat d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/f10c x:0 0 0 2024-02-16T22:56:36.985 INFO:tasks.workunit.client.1.smithi192.stdout:6/967: dread da/d40/fcb [0,4194304] 0 2024-02-16T22:56:36.986 INFO:tasks.workunit.client.1.smithi192.stdout:4/986: rename d1/d6b/d14c/f12e to d1/d15/d48/dc3/d115/d11d/d12b/f150 0 2024-02-16T22:56:36.991 INFO:tasks.workunit.client.0.smithi064.stdout:8/716: truncate df/d31/d2f/d3e/f9c 710722 0 2024-02-16T22:56:36.994 INFO:tasks.workunit.client.0.smithi064.stdout:2/943: mkdir d7/d14/d25/d46/d55/d57/dbe/d13b 0 2024-02-16T22:56:36.995 INFO:tasks.workunit.client.0.smithi064.stdout:0/665: read da/dc/de/f76 [2995337,67773] 0 2024-02-16T22:56:36.996 INFO:tasks.workunit.client.0.smithi064.stdout:2/944: stat d7/d14/d1e/d34/f61 0 2024-02-16T22:56:36.999 INFO:tasks.workunit.client.0.smithi064.stdout:3/874: dwrite d4/d13/d1b/d27/d99/fdb [0,4194304] 0 2024-02-16T22:56:37.002 INFO:tasks.workunit.client.0.smithi064.stdout:2/945: dread d7/d14/d25/d46/d55/d7f/fe7 [0,4194304] 0 2024-02-16T22:56:37.002 INFO:tasks.workunit.client.0.smithi064.stdout:4/810: mknod dc/de/d28/d57/c104 0 2024-02-16T22:56:37.003 INFO:tasks.workunit.client.1.smithi192.stdout:3/871: dwrite da/d11/d63/d6b/d80/fd0 [0,4194304] 0 2024-02-16T22:56:37.004 INFO:tasks.workunit.client.0.smithi064.stdout:1/701: write d2/df/d11/f3b [4887650,57163] 0 2024-02-16T22:56:37.006 INFO:tasks.workunit.client.0.smithi064.stdout:9/813: rename d3/d9/d6a/db1/d2a/d2e/d31/f4e to d3/d11/d9e/f10d 0 2024-02-16T22:56:37.009 INFO:tasks.workunit.client.1.smithi192.stdout:4/987: unlink d1/d26/d66/d89/da8/dc5/c143 0 2024-02-16T22:56:37.010 INFO:tasks.workunit.client.0.smithi064.stdout:3/875: dwrite d4/d13/d1b/d27/d44/f4b [0,4194304] 0 2024-02-16T22:56:37.010 INFO:tasks.workunit.client.1.smithi192.stdout:1/916: mknod d3/de/d27/d34/d69/d85/d89/c139 0 2024-02-16T22:56:37.011 INFO:tasks.workunit.client.1.smithi192.stdout:1/917: chown d3/de/d27/d34/d36/l5f 8 1 2024-02-16T22:56:37.015 INFO:tasks.workunit.client.1.smithi192.stdout:5/964: link d6/d20/d93/c130 d6/d12/d4e/d10c/c137 0 2024-02-16T22:56:37.017 INFO:tasks.workunit.client.0.smithi064.stdout:8/717: creat df/d31/d8f/dc9/d73/ffa x:0 0 0 2024-02-16T22:56:37.019 INFO:tasks.workunit.client.1.smithi192.stdout:0/959: mkdir d1/d3/d39/d136 0 2024-02-16T22:56:37.022 INFO:tasks.workunit.client.1.smithi192.stdout:3/872: truncate da/d11/d20/d39/d53/f5b 707736 0 2024-02-16T22:56:37.023 INFO:tasks.workunit.client.1.smithi192.stdout:4/988: mkdir d1/d15/d48/d8d/d11b/d151 0 2024-02-16T22:56:37.031 INFO:tasks.workunit.client.0.smithi064.stdout:1/702: creat d2/df/d14/d2c/db5/de3/fea x:0 0 0 2024-02-16T22:56:37.033 INFO:tasks.workunit.client.1.smithi192.stdout:3/873: fsync da/d11/d63/d6b/d4e/fe0 0 2024-02-16T22:56:37.034 INFO:tasks.workunit.client.0.smithi064.stdout:2/946: dread d7/d14/d25/f27 [0,4194304] 0 2024-02-16T22:56:37.034 INFO:tasks.workunit.client.1.smithi192.stdout:3/874: fsync da/d11/d63/d6b/d80/fb5 0 2024-02-16T22:56:37.035 INFO:tasks.workunit.client.0.smithi064.stdout:9/814: mknod d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/c10e 0 2024-02-16T22:56:37.037 INFO:tasks.workunit.client.1.smithi192.stdout:1/918: creat d3/de/d27/d34/d69/d85/d89/daa/d126/f13a x:0 0 0 2024-02-16T22:56:37.038 INFO:tasks.workunit.client.0.smithi064.stdout:3/876: symlink d4/d13/d1b/d27/d44/l126 0 2024-02-16T22:56:37.040 INFO:tasks.workunit.client.1.smithi192.stdout:4/989: unlink d1/d26/d6f/df4/f105 0 2024-02-16T22:56:37.041 INFO:tasks.workunit.client.1.smithi192.stdout:1/919: dread d3/de/d27/d34/d52/ff8 [0,4194304] 0 2024-02-16T22:56:37.043 INFO:tasks.workunit.client.1.smithi192.stdout:5/965: rename d6/d12/d4e/d10c/c10d to d6/d8/c138 0 2024-02-16T22:56:37.044 INFO:tasks.workunit.client.1.smithi192.stdout:5/966: read d6/d12/f18 [3363784,39856] 0 2024-02-16T22:56:37.044 INFO:tasks.workunit.client.0.smithi064.stdout:8/718: rename df/d31/d2f/d3e/d5f/dee to df/d31/d8f/dc9/d73/dfb 0 2024-02-16T22:56:37.046 INFO:tasks.workunit.client.1.smithi192.stdout:6/968: getdents da/df/d6b 0 2024-02-16T22:56:37.047 INFO:tasks.workunit.client.1.smithi192.stdout:6/969: chown da/df/d6b/lf5 2033 1 2024-02-16T22:56:37.050 INFO:tasks.workunit.client.0.smithi064.stdout:2/947: mkdir d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/dd6/d13c 0 2024-02-16T22:56:37.054 INFO:tasks.workunit.client.1.smithi192.stdout:4/990: mkdir d1/d26/d34/db9/ddb/d152 0 2024-02-16T22:56:37.056 INFO:tasks.workunit.client.1.smithi192.stdout:1/920: mkdir d3/de/d27/d34/d52/d72/d11c/d13b 0 2024-02-16T22:56:37.057 INFO:tasks.workunit.client.1.smithi192.stdout:1/921: dread - d3/de/d27/d34/d69/d85/f127 zero size 2024-02-16T22:56:37.059 INFO:tasks.workunit.client.1.smithi192.stdout:6/970: write da/df/f1c [3581390,70322] 0 2024-02-16T22:56:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:4/811: write dc/dd/d32/d46/d5a/f83 [706468,74552] 0 2024-02-16T22:56:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:4/812: truncate dc/dd/d32/d46/d5a/d7e/d9a/da2/ffd 72448 0 2024-02-16T22:56:37.064 INFO:tasks.workunit.client.1.smithi192.stdout:4/991: fsync d1/d26/d34/f44 0 2024-02-16T22:56:37.064 INFO:tasks.workunit.client.1.smithi192.stdout:0/960: dwrite d1/d3/de/d22/d37/d67/d34/da1/fab [0,4194304] 0 2024-02-16T22:56:37.067 INFO:tasks.workunit.client.1.smithi192.stdout:1/922: creat d3/d5b/db1/db6/d132/f13c x:0 0 0 2024-02-16T22:56:37.070 INFO:tasks.workunit.client.0.smithi064.stdout:0/666: dwrite da/dc/de/d12/d2c/d2e/d38/d7e/fa3 [0,4194304] 0 2024-02-16T22:56:37.073 INFO:tasks.workunit.client.1.smithi192.stdout:5/967: creat d6/d12/d4e/d5b/d67/d78/f139 x:0 0 0 2024-02-16T22:56:37.074 INFO:tasks.workunit.client.0.smithi064.stdout:1/703: write d2/df/f1c [4764846,55733] 0 2024-02-16T22:56:37.075 INFO:tasks.workunit.client.1.smithi192.stdout:6/971: dread da/df/d12/d57/d81/dfc/f119 [0,4194304] 0 2024-02-16T22:56:37.075 INFO:tasks.workunit.client.1.smithi192.stdout:5/968: chown d6/d12/d1d/fb0 3008 1 2024-02-16T22:56:37.075 INFO:tasks.workunit.client.0.smithi064.stdout:1/704: fdatasync d2/df/d11/d25/fe5 0 2024-02-16T22:56:37.076 INFO:tasks.workunit.client.1.smithi192.stdout:6/972: fsync da/df/d12/d15/f31 0 2024-02-16T22:56:37.080 INFO:tasks.workunit.client.1.smithi192.stdout:4/992: dread d1/d26/f28 [0,4194304] 0 2024-02-16T22:56:37.082 INFO:tasks.workunit.client.1.smithi192.stdout:4/993: chown d1/d26/d66/d89/f9 15198916 1 2024-02-16T22:56:37.084 INFO:tasks.workunit.client.0.smithi064.stdout:3/877: dwrite d4/d1f/d5b/d51/d7c/f49 [0,4194304] 0 2024-02-16T22:56:37.085 INFO:tasks.workunit.client.1.smithi192.stdout:0/961: creat d1/d3/de/d22/d37/d47/d53/f137 x:0 0 0 2024-02-16T22:56:37.088 INFO:tasks.workunit.client.1.smithi192.stdout:3/875: getdents da/d11/d20/d39/dca 0 2024-02-16T22:56:37.093 INFO:tasks.workunit.client.1.smithi192.stdout:8/927: sync 2024-02-16T22:56:37.094 INFO:tasks.workunit.client.1.smithi192.stdout:6/973: creat da/df/d71/dde/f137 x:0 0 0 2024-02-16T22:56:37.096 INFO:tasks.workunit.client.1.smithi192.stdout:4/994: creat d1/d6b/f153 x:0 0 0 2024-02-16T22:56:37.098 INFO:tasks.workunit.client.1.smithi192.stdout:1/923: mknod d3/de/d27/d34/d52/d5a/d105/c13d 0 2024-02-16T22:56:37.098 INFO:tasks.workunit.client.1.smithi192.stdout:1/924: chown d3/d5b/f8a 3201186 1 2024-02-16T22:56:37.099 INFO:tasks.workunit.client.1.smithi192.stdout:0/962: truncate d1/d3/de/d22/f85 1042652 0 2024-02-16T22:56:37.101 INFO:tasks.workunit.client.1.smithi192.stdout:5/969: getdents d6/d12/d2d/d5c/df6/d120 0 2024-02-16T22:56:37.103 INFO:tasks.workunit.client.1.smithi192.stdout:8/928: creat d5/db/d47/d43/d90/f12a x:0 0 0 2024-02-16T22:56:37.104 INFO:tasks.workunit.client.1.smithi192.stdout:8/929: fdatasync d5/d26/d83/d86/f41 0 2024-02-16T22:56:37.104 INFO:tasks.workunit.client.1.smithi192.stdout:4/995: rename d1/d15/d48/d8d/c9d to d1/d15/d48/d8d/d11b/c154 0 2024-02-16T22:56:37.105 INFO:tasks.workunit.client.1.smithi192.stdout:0/963: dwrite d1/d3/de/d22/da7/d112/f135 [0,4194304] 0 2024-02-16T22:56:37.109 INFO:tasks.workunit.client.1.smithi192.stdout:0/964: truncate d1/d3/de/d22/d37/d67/d34/da1/fab 4270155 0 2024-02-16T22:56:37.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:36 smithi064 ceph-mon[29226]: pgmap v23: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 84 MiB/s rd, 183 MiB/s wr, 417 op/s 2024-02-16T22:56:37.122 INFO:tasks.workunit.client.1.smithi192.stdout:3/876: creat da/d11/d63/d6b/d66/f12b x:0 0 0 2024-02-16T22:56:37.126 INFO:tasks.workunit.client.1.smithi192.stdout:4/996: write d1/f5 [212726,32480] 0 2024-02-16T22:56:37.129 INFO:tasks.workunit.client.0.smithi064.stdout:9/815: symlink d3/d89/d9a/dcf/l10f 0 2024-02-16T22:56:37.129 INFO:tasks.workunit.client.1.smithi192.stdout:1/925: write d3/de/d27/d34/d52/fe0 [2871312,44066] 0 2024-02-16T22:56:37.130 INFO:tasks.workunit.client.1.smithi192.stdout:1/926: read - d3/d5b/d77/dd2/f106 zero size 2024-02-16T22:56:37.131 INFO:tasks.workunit.client.1.smithi192.stdout:0/965: rename d1/d3/de/d22/da7/fdf to d1/dd6/f138 0 2024-02-16T22:56:37.133 INFO:tasks.workunit.client.0.smithi064.stdout:9/816: dread d3/d9/d1e/d42/d7b/d86/fd7 [0,4194304] 0 2024-02-16T22:56:37.133 INFO:tasks.workunit.client.0.smithi064.stdout:8/719: mknod df/d31/d2f/d38/d48/da2/de2/cfc 0 2024-02-16T22:56:37.135 INFO:tasks.workunit.client.0.smithi064.stdout:4/813: dread - dc/de/fb9 zero size 2024-02-16T22:56:37.135 INFO:tasks.workunit.client.1.smithi192.stdout:5/970: write d6/d12/d4e/d46/f6b [1748388,78861] 0 2024-02-16T22:56:37.137 INFO:tasks.workunit.client.0.smithi064.stdout:1/705: mknod d2/df/d11/dba/ceb 0 2024-02-16T22:56:37.138 INFO:tasks.workunit.client.1.smithi192.stdout:6/974: dwrite da/df/f6e [0,4194304] 0 2024-02-16T22:56:37.139 INFO:tasks.workunit.client.1.smithi192.stdout:3/877: rename da/d11/d20/d39/l122 to da/d11/dd4/deb/da1/db9/ded/l12c 0 2024-02-16T22:56:37.139 INFO:tasks.workunit.client.1.smithi192.stdout:6/975: write da/df/d12/d15/d1b/d70/fd3 [129708,17009] 0 2024-02-16T22:56:37.166 INFO:tasks.workunit.client.0.smithi064.stdout:3/878: symlink d4/d13/d4d/d5c/d78/d117/l127 0 2024-02-16T22:56:37.168 INFO:tasks.workunit.client.1.smithi192.stdout:4/997: creat d1/d15/d48/d8d/d9b/d10a/f155 x:0 0 0 2024-02-16T22:56:37.168 INFO:tasks.workunit.client.1.smithi192.stdout:8/930: write d5/d26/d83/fb9 [64742,52401] 0 2024-02-16T22:56:37.169 INFO:tasks.workunit.client.1.smithi192.stdout:1/927: symlink d3/de/d27/d34/d36/d47/dcd/d137/l13e 0 2024-02-16T22:56:37.171 INFO:tasks.workunit.client.0.smithi064.stdout:8/720: truncate df/d31/d2f/d38/d48/f76 852479 0 2024-02-16T22:56:37.172 INFO:tasks.workunit.client.1.smithi192.stdout:5/971: readlink d6/d12/d1d/lab 0 2024-02-16T22:56:37.172 INFO:tasks.workunit.client.0.smithi064.stdout:8/721: stat df/d31/d8f/dab/fcd 0 2024-02-16T22:56:37.173 INFO:tasks.workunit.client.1.smithi192.stdout:5/972: fsync d6/fd 0 2024-02-16T22:56:37.173 INFO:tasks.workunit.client.0.smithi064.stdout:4/814: creat dc/dd/d32/d46/f105 x:0 0 0 2024-02-16T22:56:37.174 INFO:tasks.workunit.client.1.smithi192.stdout:5/973: write d6/d12/d2d/f43 [4586093,106354] 0 2024-02-16T22:56:37.174 INFO:tasks.workunit.client.1.smithi192.stdout:8/931: dwrite d5/db/faa [0,4194304] 0 2024-02-16T22:56:37.178 INFO:tasks.workunit.client.1.smithi192.stdout:6/976: mkdir da/df/d6b/d7d/d138 0 2024-02-16T22:56:37.178 INFO:tasks.workunit.client.0.smithi064.stdout:3/879: read - d4/d13/d1b/d27/d61/d63/de1/ffa zero size 2024-02-16T22:56:37.182 INFO:tasks.workunit.client.0.smithi064.stdout:2/948: creat d7/d14/d1e/d34/d73/f13d x:0 0 0 2024-02-16T22:56:37.184 INFO:tasks.workunit.client.0.smithi064.stdout:2/949: truncate d7/d14/d1e/d34/d3f/d9a/dd9/d103/f110 1024880 0 2024-02-16T22:56:37.186 INFO:tasks.workunit.client.1.smithi192.stdout:4/998: mkdir d1/d26/d66/d89/da8/da9/d134/d156 0 2024-02-16T22:56:37.190 INFO:tasks.workunit.client.1.smithi192.stdout:1/928: mknod d3/de/d27/d34/d36/d47/dcd/d137/c13f 0 2024-02-16T22:56:37.190 INFO:tasks.workunit.client.0.smithi064.stdout:4/815: mknod dc/de/dfb/c106 0 2024-02-16T22:56:37.197 INFO:tasks.workunit.client.1.smithi192.stdout:5/974: symlink d6/d12/d4e/d5b/d67/l13a 0 2024-02-16T22:56:37.197 INFO:tasks.workunit.client.1.smithi192.stdout:3/878: write da/d11/d20/d39/d53/f93 [741303,110801] 0 2024-02-16T22:56:37.198 INFO:tasks.workunit.client.0.smithi064.stdout:2/950: chown d7/d14/d25/d46/d55/d57/dbe/cc0 55699 1 2024-02-16T22:56:37.200 INFO:tasks.workunit.client.1.smithi192.stdout:0/966: rename d1/d3/d39/fa3 to d1/d3/d39/d136/f139 0 2024-02-16T22:56:37.202 INFO:tasks.workunit.client.0.smithi064.stdout:4/816: read - dc/dd/d16/d7c/fab zero size 2024-02-16T22:56:37.202 INFO:tasks.workunit.client.0.smithi064.stdout:8/722: write df/d31/d61/d79/f9b [2082857,75571] 0 2024-02-16T22:56:37.204 INFO:tasks.workunit.client.1.smithi192.stdout:1/929: creat d3/de/d27/d34/d52/d63/d7a/f140 x:0 0 0 2024-02-16T22:56:37.205 INFO:tasks.workunit.client.0.smithi064.stdout:0/667: getdents da/d26/de0/d93 0 2024-02-16T22:56:37.209 INFO:tasks.workunit.client.0.smithi064.stdout:1/706: link l1 d2/df/d14/d2c/db5/dc6/lec 0 2024-02-16T22:56:37.209 INFO:tasks.workunit.client.0.smithi064.stdout:3/880: write d4/d13/d1b/d27/d61/d63/ffe [471964,12296] 0 2024-02-16T22:56:37.214 INFO:tasks.workunit.client.0.smithi064.stdout:2/951: fsync d7/d14/d1e/d34/d3f/f4e 0 2024-02-16T22:56:37.214 INFO:tasks.workunit.client.1.smithi192.stdout:8/932: write d5/d26/d83/fe5 [492026,56446] 0 2024-02-16T22:56:37.215 INFO:tasks.workunit.client.1.smithi192.stdout:3/879: dwrite da/d11/d63/d6b/d66/d75/f76 [0,4194304] 0 2024-02-16T22:56:37.215 INFO:tasks.workunit.client.0.smithi064.stdout:4/817: dread dc/d4a/f7d [0,4194304] 0 2024-02-16T22:56:37.216 INFO:tasks.workunit.client.1.smithi192.stdout:8/933: write d5/d26/d83/fb9 [2389644,88290] 0 2024-02-16T22:56:37.217 INFO:tasks.workunit.client.1.smithi192.stdout:6/977: rename da/df/d18/d52/de6 to da/df/d6b/d139 0 2024-02-16T22:56:37.217 INFO:tasks.workunit.client.1.smithi192.stdout:8/934: readlink d5/d26/d83/d86/l70 0 2024-02-16T22:56:37.218 INFO:tasks.workunit.client.0.smithi064.stdout:4/818: write dc/de/d28/d57/d77/d101/d43/f5f [4502331,29944] 0 2024-02-16T22:56:37.219 INFO:tasks.workunit.client.1.smithi192.stdout:6/978: write da/df/d18/d52/f11d [826114,71079] 0 2024-02-16T22:56:37.223 INFO:tasks.workunit.client.1.smithi192.stdout:1/930: mkdir d3/de/d27/d34/d52/d63/d7a/d141 0 2024-02-16T22:56:37.225 INFO:tasks.workunit.client.0.smithi064.stdout:3/881: mkdir d4/d1f/d5b/d128 0 2024-02-16T22:56:37.227 INFO:tasks.workunit.client.0.smithi064.stdout:1/707: rename d2/df/d14/d18/d48/d6b/fd0 to d2/df/d14/d2c/db5/de3/de6/fed 0 2024-02-16T22:56:37.229 INFO:tasks.workunit.client.0.smithi064.stdout:0/668: write da/dc/de/d12/d1d/f34 [4459076,65923] 0 2024-02-16T22:56:37.230 INFO:tasks.workunit.client.1.smithi192.stdout:1/931: dwrite d3/de/d27/d34/d52/d63/d7a/f140 [0,4194304] 0 2024-02-16T22:56:37.233 INFO:tasks.workunit.client.1.smithi192.stdout:3/880: fsync da/d11/dd4/deb/f7b 0 2024-02-16T22:56:37.234 INFO:tasks.workunit.client.1.smithi192.stdout:3/881: chown da/d11/d63/d6b/d66/f121 0 1 2024-02-16T22:56:37.235 INFO:tasks.workunit.client.1.smithi192.stdout:0/967: creat d1/d3/d18/f13a x:0 0 0 2024-02-16T22:56:37.235 INFO:tasks.workunit.client.0.smithi064.stdout:8/723: dwrite df/d31/d2f/d38/f39 [0,4194304] 0 2024-02-16T22:56:37.237 INFO:tasks.workunit.client.0.smithi064.stdout:3/882: fdatasync d4/d1f/d5b/fe4 0 2024-02-16T22:56:37.241 INFO:tasks.workunit.client.0.smithi064.stdout:1/708: rename l1 to d2/df/lee 0 2024-02-16T22:56:37.242 INFO:tasks.workunit.client.1.smithi192.stdout:1/932: mkdir d3/d5b/d77/d142 0 2024-02-16T22:56:37.245 INFO:tasks.workunit.client.1.smithi192.stdout:5/975: write d6/fc [959193,2298] 0 2024-02-16T22:56:37.246 INFO:tasks.workunit.client.0.smithi064.stdout:2/952: mkdir d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/d121/d13e 0 2024-02-16T22:56:37.247 INFO:tasks.workunit.client.1.smithi192.stdout:6/979: mknod da/df/d12/c13a 0 2024-02-16T22:56:37.248 INFO:tasks.workunit.client.0.smithi064.stdout:0/669: rmdir da/dc/d3b/d72 39 2024-02-16T22:56:37.249 INFO:tasks.workunit.client.1.smithi192.stdout:6/980: stat da/df/d12/d15/d1b/d70/l7f 0 2024-02-16T22:56:37.251 INFO:tasks.workunit.client.1.smithi192.stdout:0/968: fdatasync d1/d3/de/f10 0 2024-02-16T22:56:37.253 INFO:tasks.workunit.client.0.smithi064.stdout:8/724: mkdir df/d31/d47/dfd 0 2024-02-16T22:56:37.253 INFO:tasks.workunit.client.1.smithi192.stdout:5/976: dread d6/d12/d2d/d5c/df6/dbc/d102/f104 [0,4194304] 0 2024-02-16T22:56:37.255 INFO:tasks.workunit.client.1.smithi192.stdout:3/882: mknod da/d11/d63/d6b/d66/c12d 0 2024-02-16T22:56:37.256 INFO:tasks.workunit.client.1.smithi192.stdout:6/981: truncate da/df/d18/d46/fa4 405598 0 2024-02-16T22:56:37.256 INFO:tasks.workunit.client.0.smithi064.stdout:2/953: read d7/d14/d1e/d34/d3f/d9a/dd9/f8d [506981,33615] 0 2024-02-16T22:56:37.257 INFO:tasks.workunit.client.1.smithi192.stdout:6/982: write da/df/d18/d46/f80 [237279,68477] 0 2024-02-16T22:56:37.263 INFO:tasks.workunit.client.0.smithi064.stdout:0/670: dwrite da/d26/fd0 [8388608,4194304] 0 2024-02-16T22:56:37.266 INFO:tasks.workunit.client.1.smithi192.stdout:5/977: creat d6/d12/d4e/d5b/d67/f13b x:0 0 0 2024-02-16T22:56:37.272 INFO:tasks.workunit.client.1.smithi192.stdout:3/883: chown da/d11/d63/dbf/f103 1 1 2024-02-16T22:56:37.272 INFO:tasks.workunit.client.1.smithi192.stdout:3/884: chown da/d11/d63/d6b/d66/c12d 24605215 1 2024-02-16T22:56:37.273 INFO:tasks.workunit.client.1.smithi192.stdout:3/885: readlink da/d11/d63/d6b/d66/d75/l8e 0 2024-02-16T22:56:37.273 INFO:tasks.workunit.client.1.smithi192.stdout:0/969: dread d1/d3/de/d22/d37/d67/d34/fbc [0,4194304] 0 2024-02-16T22:56:37.273 INFO:tasks.workunit.client.0.smithi064.stdout:8/725: fdatasync df/d31/d2f/d4f/f75 0 2024-02-16T22:56:37.275 INFO:tasks.workunit.client.1.smithi192.stdout:6/983: mkdir da/df/d12/d57/d13b 0 2024-02-16T22:56:37.276 INFO:tasks.workunit.client.0.smithi064.stdout:3/883: creat d4/d1f/f129 x:0 0 0 2024-02-16T22:56:37.277 INFO:tasks.workunit.client.1.smithi192.stdout:1/933: write d3/f50 [885879,92738] 0 2024-02-16T22:56:37.277 INFO:tasks.workunit.client.0.smithi064.stdout:1/709: link d2/df/d11/d1a/d30/dbd/d91/f64 d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf/fef 0 2024-02-16T22:56:37.279 INFO:tasks.workunit.client.0.smithi064.stdout:1/710: chown d2/df/d11/d1a/d30/dbd/d91/d56/d5b/c74 40 1 2024-02-16T22:56:37.281 INFO:tasks.workunit.client.0.smithi064.stdout:2/954: write d7/fb1 [462549,81245] 0 2024-02-16T22:56:37.283 INFO:tasks.workunit.client.1.smithi192.stdout:3/886: rename da/d11/dd4/lfc to da/df/l12e 0 2024-02-16T22:56:37.284 INFO:tasks.workunit.client.0.smithi064.stdout:8/726: mkdir df/d31/d2f/dfe 0 2024-02-16T22:56:37.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:36 smithi192 ceph-mon[36254]: pgmap v23: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 84 MiB/s rd, 183 MiB/s wr, 417 op/s 2024-02-16T22:56:37.285 INFO:tasks.workunit.client.0.smithi064.stdout:3/884: creat d4/d13/d1b/d27/d99/f12a x:0 0 0 2024-02-16T22:56:37.286 INFO:tasks.workunit.client.1.smithi192.stdout:1/934: creat d3/de/da4/f143 x:0 0 0 2024-02-16T22:56:37.288 INFO:tasks.workunit.client.0.smithi064.stdout:8/727: rmdir df/d31 39 2024-02-16T22:56:37.289 INFO:tasks.workunit.client.1.smithi192.stdout:3/887: dwrite da/df/f119 [0,4194304] 0 2024-02-16T22:56:37.292 INFO:tasks.workunit.client.0.smithi064.stdout:3/885: dread d4/d13/d1b/f1c [0,4194304] 0 2024-02-16T22:56:37.294 INFO:tasks.workunit.client.0.smithi064.stdout:2/955: unlink d7/d14/d25/l105 0 2024-02-16T22:56:37.297 INFO:tasks.workunit.client.1.smithi192.stdout:6/984: dread da/df/d12/d15/f31 [0,4194304] 0 2024-02-16T22:56:37.297 INFO:tasks.workunit.client.1.smithi192.stdout:1/935: mkdir d3/de/d27/d34/d69/d85/d89/daa/d126/d144 0 2024-02-16T22:56:37.300 INFO:tasks.workunit.client.1.smithi192.stdout:0/970: rmdir d1/dc7/d9c/d12c/dfd 0 2024-02-16T22:56:37.301 INFO:tasks.workunit.client.1.smithi192.stdout:5/978: truncate d6/d8/fa3 921435 0 2024-02-16T22:56:37.302 INFO:tasks.workunit.client.0.smithi064.stdout:1/711: write d2/df/f1e [758376,4451] 0 2024-02-16T22:56:37.303 INFO:tasks.workunit.client.0.smithi064.stdout:2/956: dwrite d7/d14/d25/d46/d55/d7f/f99 [8388608,4194304] 0 2024-02-16T22:56:37.304 INFO:tasks.workunit.client.1.smithi192.stdout:3/888: rmdir da/d11/dd4 39 2024-02-16T22:56:37.309 INFO:tasks.workunit.client.1.smithi192.stdout:4/999: sync 2024-02-16T22:56:37.310 INFO:tasks.workunit.client.1.smithi192.stdout:0/971: rename d1/d3/de/d22/d37/d67/d34 to d1/d3/de/d22/da7/d10d/d13b 0 2024-02-16T22:56:37.314 INFO:tasks.workunit.client.0.smithi064.stdout:8/728: unlink df/l14 0 2024-02-16T22:56:37.316 INFO:tasks.workunit.client.1.smithi192.stdout:3/889: read da/df/f49 [656999,44036] 0 2024-02-16T22:56:37.319 INFO:tasks.workunit.client.0.smithi064.stdout:3/886: unlink d4/d13/d1b/d27/d99/cfd 0 2024-02-16T22:56:37.321 INFO:tasks.workunit.client.1.smithi192.stdout:6/985: truncate da/d40/f109 4167815 0 2024-02-16T22:56:37.322 INFO:tasks.workunit.client.0.smithi064.stdout:8/729: write df/fe7 [1046959,37618] 0 2024-02-16T22:56:37.323 INFO:tasks.workunit.client.1.smithi192.stdout:0/972: mknod d1/d3/de/d22/d37/d47/c13c 0 2024-02-16T22:56:37.330 INFO:tasks.workunit.client.0.smithi064.stdout:1/712: dread d2/df/d14/d18/d40/f68 [0,4194304] 0 2024-02-16T22:56:37.332 INFO:tasks.workunit.client.1.smithi192.stdout:3/890: creat da/d11/d20/d39/d53/dc7/f12f x:0 0 0 2024-02-16T22:56:37.334 INFO:tasks.workunit.client.0.smithi064.stdout:3/887: write d4/d13/d4d/d5c/fcf [1036345,24226] 0 2024-02-16T22:56:37.337 INFO:tasks.workunit.client.1.smithi192.stdout:0/973: creat d1/d3/d29/dcd/dcf/f13d x:0 0 0 2024-02-16T22:56:37.337 INFO:tasks.workunit.client.0.smithi064.stdout:2/957: link d7/d14/d4b/cd4 d7/d14/d1e/d34/d3f/d9a/dd9/d60/c13f 0 2024-02-16T22:56:37.338 INFO:tasks.workunit.client.0.smithi064.stdout:2/958: readlink l3 0 2024-02-16T22:56:37.338 INFO:tasks.workunit.client.1.smithi192.stdout:5/979: link d6/d12/d4e/le5 d6/d8/d75/d8c/dc4/l13c 0 2024-02-16T22:56:37.338 INFO:tasks.workunit.client.1.smithi192.stdout:0/974: chown d1/d3/d18/f13a 2753599 1 2024-02-16T22:56:37.339 INFO:tasks.workunit.client.1.smithi192.stdout:1/936: getdents d3/de/d27/d34/d52/db7 0 2024-02-16T22:56:37.339 INFO:tasks.workunit.client.0.smithi064.stdout:1/713: symlink d2/df/d14/d2c/de1/lf0 0 2024-02-16T22:56:37.339 INFO:tasks.workunit.client.1.smithi192.stdout:1/937: rename d3/d40 to d3/d40/d9f/d10a/d145 22 2024-02-16T22:56:37.342 INFO:tasks.workunit.client.1.smithi192.stdout:6/986: symlink da/df/d18/d52/l13c 0 2024-02-16T22:56:37.342 INFO:tasks.workunit.client.1.smithi192.stdout:1/938: dread d3/de/d27/d34/d36/d47/dcd/f11b [0,4194304] 0 2024-02-16T22:56:37.342 INFO:tasks.workunit.client.0.smithi064.stdout:2/959: mkdir d7/d14/d25/d46/d140 0 2024-02-16T22:56:37.343 INFO:tasks.workunit.client.1.smithi192.stdout:1/939: readlink d3/de/d27/d34/d52/d63/l92 0 2024-02-16T22:56:37.344 INFO:tasks.workunit.client.1.smithi192.stdout:0/975: unlink d1/d3/d29/ccc 0 2024-02-16T22:56:37.345 INFO:tasks.workunit.client.1.smithi192.stdout:0/976: chown d1/d3/d29/f27 9306328 1 2024-02-16T22:56:37.346 INFO:tasks.workunit.client.1.smithi192.stdout:5/980: creat d6/d12/d4e/d10c/f13d x:0 0 0 2024-02-16T22:56:37.346 INFO:tasks.workunit.client.1.smithi192.stdout:0/977: write d1/d3/de/d22/d37/d47/fda [1813535,61610] 0 2024-02-16T22:56:37.348 INFO:tasks.workunit.client.0.smithi064.stdout:8/730: getdents df/d31/d2f/d68 0 2024-02-16T22:56:37.349 INFO:tasks.workunit.client.1.smithi192.stdout:3/891: symlink da/d11/d63/d6b/d4e/d11a/l130 0 2024-02-16T22:56:37.349 INFO:tasks.workunit.client.0.smithi064.stdout:8/731: write df/d31/d2f/d7f/fd5 [4997546,54077] 0 2024-02-16T22:56:37.350 INFO:tasks.workunit.client.0.smithi064.stdout:8/732: fdatasync df/d31/d2f/f59 0 2024-02-16T22:56:37.352 INFO:tasks.workunit.client.0.smithi064.stdout:8/733: truncate df/d31/d61/d79/f9b 4321209 0 2024-02-16T22:56:37.353 INFO:tasks.workunit.client.0.smithi064.stdout:3/888: rename d4/d13/d1b/d27/d61/d63/l64 to d4/d1f/l12b 0 2024-02-16T22:56:37.358 INFO:tasks.workunit.client.0.smithi064.stdout:2/960: write d7/d14/d4b/f64 [7424783,16051] 0 2024-02-16T22:56:37.360 INFO:tasks.workunit.client.1.smithi192.stdout:5/981: read d6/d12/f18 [7653139,73756] 0 2024-02-16T22:56:37.361 INFO:tasks.workunit.client.0.smithi064.stdout:8/734: fdatasync df/d31/d61/d79/f8c 0 2024-02-16T22:56:37.362 INFO:tasks.workunit.client.1.smithi192.stdout:5/982: chown d6/d12/d4e/d10c/c137 3670322 1 2024-02-16T22:56:37.362 INFO:tasks.workunit.client.1.smithi192.stdout:0/978: truncate d1/d3/de/d22/fd2 322903 0 2024-02-16T22:56:37.363 INFO:tasks.workunit.client.1.smithi192.stdout:5/983: write d6/d12/d4e/d58/ffe [954565,116909] 0 2024-02-16T22:56:37.364 INFO:tasks.workunit.client.1.smithi192.stdout:8/935: sync 2024-02-16T22:56:37.365 INFO:tasks.workunit.client.1.smithi192.stdout:3/892: chown da/d6a/ldb 1005201 1 2024-02-16T22:56:37.366 INFO:tasks.workunit.client.0.smithi064.stdout:1/714: dwrite d2/df/d14/d2c/db5/de3/de6/fed [0,4194304] 0 2024-02-16T22:56:37.367 INFO:tasks.workunit.client.1.smithi192.stdout:6/987: dread da/df/d12/d15/d1b/fa3 [0,4194304] 0 2024-02-16T22:56:37.368 INFO:tasks.workunit.client.1.smithi192.stdout:1/940: symlink d3/d40/l146 0 2024-02-16T22:56:37.368 INFO:tasks.workunit.client.1.smithi192.stdout:3/893: write da/d11/d63/d6b/ffd [3252685,67634] 0 2024-02-16T22:56:37.370 INFO:tasks.workunit.client.0.smithi064.stdout:3/889: dread d4/d13/d1b/d27/d44/f5f [0,4194304] 0 2024-02-16T22:56:37.372 INFO:tasks.workunit.client.1.smithi192.stdout:5/984: dwrite d6/d12/d1d/d56/ff1 [0,4194304] 0 2024-02-16T22:56:37.372 INFO:tasks.workunit.client.1.smithi192.stdout:8/936: rename d5/db/d47/l76 to d5/d16/dd5/d9d/l12b 0 2024-02-16T22:56:37.373 INFO:tasks.workunit.client.0.smithi064.stdout:2/961: mkdir d7/d14/d25/d46/d140/d141 0 2024-02-16T22:56:37.375 INFO:tasks.workunit.client.1.smithi192.stdout:5/985: truncate d6/d12/d2d/f132 211858 0 2024-02-16T22:56:37.379 INFO:tasks.workunit.client.1.smithi192.stdout:6/988: creat da/df/d12/d15/d1b/d70/f13d x:0 0 0 2024-02-16T22:56:37.382 INFO:tasks.workunit.client.1.smithi192.stdout:1/941: creat d3/de/d27/d34/d52/d63/d7a/db0/f147 x:0 0 0 2024-02-16T22:56:37.383 INFO:tasks.workunit.client.1.smithi192.stdout:3/894: truncate da/df/fe5 19281 0 2024-02-16T22:56:37.384 INFO:tasks.workunit.client.1.smithi192.stdout:8/937: symlink d5/d16/dcf/l12c 0 2024-02-16T22:56:37.387 INFO:tasks.workunit.client.1.smithi192.stdout:0/979: rename d1/d3/d29/f1a to d1/d3/d29/f13e 0 2024-02-16T22:56:37.388 INFO:tasks.workunit.client.1.smithi192.stdout:6/989: chown da/d76/dc8/ff4 825 1 2024-02-16T22:56:37.389 INFO:tasks.workunit.client.1.smithi192.stdout:6/990: chown da/df/d12/d15/d1b/d70/d120 5 1 2024-02-16T22:56:37.393 INFO:tasks.workunit.client.1.smithi192.stdout:5/986: mknod d6/d12/d4e/d58/c13e 0 2024-02-16T22:56:37.394 INFO:tasks.workunit.client.1.smithi192.stdout:0/980: creat d1/d3/de2/dff/f13f x:0 0 0 2024-02-16T22:56:37.395 INFO:tasks.workunit.client.1.smithi192.stdout:6/991: dread - da/df/d12/d15/feb zero size 2024-02-16T22:56:37.396 INFO:tasks.workunit.client.1.smithi192.stdout:3/895: mknod da/d11/dd4/deb/da1/c131 0 2024-02-16T22:56:37.397 INFO:tasks.workunit.client.1.smithi192.stdout:3/896: fdatasync da/d11/d2b/ddc/d102/f111 0 2024-02-16T22:56:37.399 INFO:tasks.workunit.client.1.smithi192.stdout:6/992: dread da/d76/dd5/de2/f100 [4194304,4194304] 0 2024-02-16T22:56:37.399 INFO:tasks.workunit.client.1.smithi192.stdout:3/897: dread da/d11/d20/d39/d53/fce [0,4194304] 0 2024-02-16T22:56:37.399 INFO:tasks.workunit.client.1.smithi192.stdout:0/981: mknod d1/d3/de2/c140 0 2024-02-16T22:56:37.400 INFO:tasks.workunit.client.1.smithi192.stdout:6/993: write da/d40/d85/f115 [258878,119688] 0 2024-02-16T22:56:37.403 INFO:tasks.workunit.client.1.smithi192.stdout:3/898: dread da/d11/d63/fa6 [0,4194304] 0 2024-02-16T22:56:37.404 INFO:tasks.workunit.client.1.smithi192.stdout:1/942: getdents d3/de 0 2024-02-16T22:56:37.405 INFO:tasks.workunit.client.1.smithi192.stdout:0/982: dread - d1/d3/d39/f104 zero size 2024-02-16T22:56:37.407 INFO:tasks.workunit.client.1.smithi192.stdout:8/938: getdents d5/db/d47/d43 0 2024-02-16T22:56:37.408 INFO:tasks.workunit.client.1.smithi192.stdout:3/899: fsync da/d11/f2f 0 2024-02-16T22:56:37.410 INFO:tasks.workunit.client.1.smithi192.stdout:1/943: mknod d3/de/d27/d34/d52/d63/d81/c148 0 2024-02-16T22:56:37.410 INFO:tasks.workunit.client.1.smithi192.stdout:3/900: write da/d11/d63/d6b/d66/d75/f9f [1152856,100460] 0 2024-02-16T22:56:37.410 INFO:tasks.workunit.client.1.smithi192.stdout:8/939: creat d5/d26/d83/d86/f12d x:0 0 0 2024-02-16T22:56:37.413 INFO:tasks.workunit.client.1.smithi192.stdout:3/901: mknod da/d11/d63/d6b/d4e/c132 0 2024-02-16T22:56:37.413 INFO:tasks.workunit.client.1.smithi192.stdout:6/994: mknod da/df/d6b/d139/c13e 0 2024-02-16T22:56:37.414 INFO:tasks.workunit.client.1.smithi192.stdout:8/940: truncate d5/d26/d83/db4/fac 2377801 0 2024-02-16T22:56:37.417 INFO:tasks.workunit.client.1.smithi192.stdout:8/941: mknod d5/d26/d83/db4/c12e 0 2024-02-16T22:56:37.417 INFO:tasks.workunit.client.1.smithi192.stdout:6/995: dread da/df/d12/f9c [0,4194304] 0 2024-02-16T22:56:37.417 INFO:tasks.workunit.client.1.smithi192.stdout:1/944: getdents d3/de/d27/d34/d52/d72/d11c 0 2024-02-16T22:56:37.418 INFO:tasks.workunit.client.1.smithi192.stdout:3/902: getdents da/df 0 2024-02-16T22:56:37.419 INFO:tasks.workunit.client.1.smithi192.stdout:1/945: truncate d3/d40/d9f/df0/f10c 26529 0 2024-02-16T22:56:37.421 INFO:tasks.workunit.client.1.smithi192.stdout:6/996: getdents da/df/d12/d15/d1b 0 2024-02-16T22:56:37.422 INFO:tasks.workunit.client.1.smithi192.stdout:1/946: mkdir d3/de/d27/d34/d36/d149 0 2024-02-16T22:56:37.422 INFO:tasks.workunit.client.1.smithi192.stdout:6/997: chown da/df/f84 84724530 1 2024-02-16T22:56:37.423 INFO:tasks.workunit.client.1.smithi192.stdout:8/942: getdents d5/db/de/df 0 2024-02-16T22:56:37.424 INFO:tasks.workunit.client.1.smithi192.stdout:1/947: rename d3/de/f35 to d3/de/d27/d34/d52/db7/f14a 0 2024-02-16T22:56:37.426 INFO:tasks.workunit.client.0.smithi064.stdout:5/758: sync 2024-02-16T22:56:37.426 INFO:tasks.workunit.client.0.smithi064.stdout:1/715: rename d2/df/d14/d18/d40/cde to d2/df/d11/cf1 0 2024-02-16T22:56:37.427 INFO:tasks.workunit.client.0.smithi064.stdout:8/735: fdatasync f8 0 2024-02-16T22:56:37.428 INFO:tasks.workunit.client.1.smithi192.stdout:8/943: creat d5/d26/d83/d86/f12f x:0 0 0 2024-02-16T22:56:37.428 INFO:tasks.workunit.client.0.smithi064.stdout:1/716: chown d2/df/d11/d1a/d30/dbd/l78 5753 1 2024-02-16T22:56:37.429 INFO:tasks.workunit.client.0.smithi064.stdout:3/890: creat d4/d13/d1b/d27/d61/d63/de1/f12c x:0 0 0 2024-02-16T22:56:37.430 INFO:tasks.workunit.client.1.smithi192.stdout:3/903: read da/d11/d20/d39/dca/ffa [168975,55563] 0 2024-02-16T22:56:37.436 INFO:tasks.workunit.client.0.smithi064.stdout:2/962: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/d60/f129 [0,4194304] 0 2024-02-16T22:56:37.436 INFO:tasks.workunit.client.1.smithi192.stdout:8/944: dwrite d5/db/de/f80 [4194304,4194304] 0 2024-02-16T22:56:37.438 INFO:tasks.workunit.client.1.smithi192.stdout:8/945: readlink d5/d26/d83/d86/d5f/l89 0 2024-02-16T22:56:37.446 INFO:tasks.workunit.client.1.smithi192.stdout:8/946: dwrite d5/db/de/f80 [0,4194304] 0 2024-02-16T22:56:37.507 INFO:tasks.workunit.client.1.smithi192.stdout:3/904: sync 2024-02-16T22:56:37.507 INFO:tasks.workunit.client.1.smithi192.stdout:3/905: chown da/d11/la4 647324396 1 2024-02-16T22:56:37.555 INFO:tasks.workunit.client.0.smithi064.stdout:3/891: symlink d4/d13/d1b/d27/d82/l12d 0 2024-02-16T22:56:37.562 INFO:tasks.workunit.client.0.smithi064.stdout:8/736: write df/d31/d8f/dc9/fd4 [2842646,105701] 0 2024-02-16T22:56:37.565 INFO:tasks.workunit.client.1.smithi192.stdout:8/947: write d5/d26/fa0 [2915340,79007] 0 2024-02-16T22:56:37.565 INFO:tasks.workunit.client.1.smithi192.stdout:3/906: creat da/d106/f133 x:0 0 0 2024-02-16T22:56:37.574 INFO:tasks.workunit.client.0.smithi064.stdout:3/892: creat d4/d13/d1b/d27/d82/db0/f12e x:0 0 0 2024-02-16T22:56:37.576 INFO:tasks.workunit.client.0.smithi064.stdout:8/737: chown df/d31/c4b 35147 1 2024-02-16T22:56:37.580 INFO:tasks.workunit.client.0.smithi064.stdout:1/717: link d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce d2/df/d14/d2c/db5/de3/de6/ff2 0 2024-02-16T22:56:37.582 INFO:tasks.workunit.client.0.smithi064.stdout:8/738: dwrite df/d31/d61/d79/f9b [0,4194304] 0 2024-02-16T22:56:37.583 INFO:tasks.workunit.client.0.smithi064.stdout:8/739: chown df/d2a/f2d 2719359 1 2024-02-16T22:56:37.584 INFO:tasks.workunit.client.0.smithi064.stdout:8/740: chown df/d31/d47/laa 145970 1 2024-02-16T22:56:37.588 INFO:tasks.workunit.client.1.smithi192.stdout:5/987: truncate d6/d12/d1d/d56/fde 429652 0 2024-02-16T22:56:37.592 INFO:tasks.workunit.client.0.smithi064.stdout:1/718: symlink d2/df/d11/d71/lf3 0 2024-02-16T22:56:37.595 INFO:tasks.workunit.client.0.smithi064.stdout:8/741: symlink df/d31/d2f/d4f/d6a/lff 0 2024-02-16T22:56:37.599 INFO:tasks.workunit.client.1.smithi192.stdout:0/983: truncate d1/dc7/f8b 353540 0 2024-02-16T22:56:37.601 INFO:tasks.workunit.client.0.smithi064.stdout:6/852: sync 2024-02-16T22:56:37.605 INFO:tasks.workunit.client.0.smithi064.stdout:8/742: read df/d2a/f70 [296733,34216] 0 2024-02-16T22:56:37.615 INFO:tasks.workunit.client.0.smithi064.stdout:6/853: read d0/d12/d7b/f99 [620282,40856] 0 2024-02-16T22:56:37.626 INFO:tasks.workunit.client.0.smithi064.stdout:6/854: mkdir d0/d6/d10/d105/db9/d118 0 2024-02-16T22:56:37.627 INFO:tasks.workunit.client.0.smithi064.stdout:8/743: dread df/f3d [0,4194304] 0 2024-02-16T22:56:37.669 INFO:tasks.workunit.client.0.smithi064.stdout:0/671: sync 2024-02-16T22:56:37.669 INFO:tasks.workunit.client.0.smithi064.stdout:4/819: sync 2024-02-16T22:56:37.670 INFO:tasks.workunit.client.0.smithi064.stdout:9/817: sync 2024-02-16T22:56:37.670 INFO:tasks.workunit.client.0.smithi064.stdout:7/791: sync 2024-02-16T22:56:37.691 INFO:tasks.workunit.client.0.smithi064.stdout:4/820: unlink dc/de/d28/d57/cd5 0 2024-02-16T22:56:37.692 INFO:tasks.workunit.client.0.smithi064.stdout:4/821: stat dc/dd/d32/d46/d5a/c60 0 2024-02-16T22:56:37.693 INFO:tasks.workunit.client.1.smithi192.stdout:3/907: write da/d11/d63/d6b/ff7 [632239,36646] 0 2024-02-16T22:56:37.695 INFO:tasks.workunit.client.1.smithi192.stdout:5/988: unlink d6/d8/d80/cfb 0 2024-02-16T22:56:37.697 INFO:tasks.workunit.client.1.smithi192.stdout:5/989: write d6/d12/d1d/fb8 [1771384,8649] 0 2024-02-16T22:56:37.698 INFO:tasks.workunit.client.0.smithi064.stdout:4/822: rename dc/dd/d32/d46/d5a/f84 to dc/de/d28/d57/d77/f107 0 2024-02-16T22:56:37.700 INFO:tasks.workunit.client.1.smithi192.stdout:1/948: chown d3/de/d27/d34/f4e 0 1 2024-02-16T22:56:37.709 INFO:tasks.workunit.client.1.smithi192.stdout:0/984: getdents d1/d3/d18/d11d 0 2024-02-16T22:56:37.711 INFO:tasks.workunit.client.0.smithi064.stdout:5/759: write d6/d18/d19/fd9 [651659,20792] 0 2024-02-16T22:56:37.712 INFO:tasks.workunit.client.1.smithi192.stdout:0/985: fdatasync d1/d3/de/d22/da7/d10d/d13b/f92 0 2024-02-16T22:56:37.713 INFO:tasks.workunit.client.1.smithi192.stdout:0/986: fdatasync d1/d3/de/d22/da7/d10d/d13b/da1/fb9 0 2024-02-16T22:56:37.716 INFO:tasks.workunit.client.1.smithi192.stdout:1/949: dread d3/de/d27/d34/d52/d63/d7a/db0/fd5 [0,4194304] 0 2024-02-16T22:56:37.717 INFO:tasks.workunit.client.1.smithi192.stdout:1/950: fdatasync d3/d5b/fc0 0 2024-02-16T22:56:37.718 INFO:tasks.workunit.client.1.smithi192.stdout:1/951: write d3/d5b/d77/dd2/f106 [57487,118974] 0 2024-02-16T22:56:37.719 INFO:tasks.workunit.client.1.smithi192.stdout:1/952: stat d3/de/d27/d34/d52/d5a/f90 0 2024-02-16T22:56:37.720 INFO:tasks.workunit.client.1.smithi192.stdout:0/987: read d1/fa [1955165,53026] 0 2024-02-16T22:56:37.721 INFO:tasks.workunit.client.0.smithi064.stdout:4/823: dread dc/dd/d16/f3a [0,4194304] 0 2024-02-16T22:56:37.721 INFO:tasks.workunit.client.0.smithi064.stdout:5/760: creat d6/d18/df8/ffe x:0 0 0 2024-02-16T22:56:37.728 INFO:tasks.workunit.client.0.smithi064.stdout:5/761: dwrite d6/d25/df1/d80/fe5 [0,4194304] 0 2024-02-16T22:56:37.736 INFO:tasks.workunit.client.0.smithi064.stdout:4/824: dread - dc/de/d28/d57/d77/d101/d43/ff6 zero size 2024-02-16T22:56:37.753 INFO:tasks.workunit.client.1.smithi192.stdout:0/988: dread d1/f24 [0,4194304] 0 2024-02-16T22:56:37.758 INFO:tasks.workunit.client.1.smithi192.stdout:0/989: creat d1/dc7/d9c/d12c/d121/d12f/f141 x:0 0 0 2024-02-16T22:56:37.758 INFO:tasks.workunit.client.1.smithi192.stdout:0/990: chown d1/dc7/d9c/c102 12 1 2024-02-16T22:56:37.765 INFO:tasks.workunit.client.1.smithi192.stdout:0/991: dwrite d1/dc7/d9c/fe4 [0,4194304] 0 2024-02-16T22:56:37.766 INFO:tasks.workunit.client.0.smithi064.stdout:9/818: unlink d3/d9/d6a/db1/d2a/fbf 0 2024-02-16T22:56:37.769 INFO:tasks.workunit.client.0.smithi064.stdout:9/819: mkdir d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/d110 0 2024-02-16T22:56:37.774 INFO:tasks.workunit.client.0.smithi064.stdout:9/820: chown d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/de4/cfd 133452812 1 2024-02-16T22:56:37.776 INFO:tasks.workunit.client.0.smithi064.stdout:9/821: truncate d3/d9/d6a/db1/fb0 4581705 0 2024-02-16T22:56:37.784 INFO:tasks.workunit.client.0.smithi064.stdout:9/822: dwrite d3/d9/d6a/db1/d2a/d2e/d31/d62/fdb [0,4194304] 0 2024-02-16T22:56:37.790 INFO:tasks.workunit.client.0.smithi064.stdout:9/823: mknod d3/d11/d15/d23/dee/c111 0 2024-02-16T22:56:37.790 INFO:tasks.workunit.client.0.smithi064.stdout:9/824: chown d3/ff4 274 1 2024-02-16T22:56:37.794 INFO:tasks.workunit.client.0.smithi064.stdout:2/963: write d7/f1a [5218692,41323] 0 2024-02-16T22:56:37.797 INFO:tasks.workunit.client.0.smithi064.stdout:9/825: getdents d3/d11/d9e/da5 0 2024-02-16T22:56:37.804 INFO:tasks.workunit.client.0.smithi064.stdout:9/826: dwrite d3/d11/f1f [0,4194304] 0 2024-02-16T22:56:37.806 INFO:tasks.workunit.client.0.smithi064.stdout:2/964: link d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/dd6/c128 d7/d14/d1e/d34/d3f/d9a/dd9/df7/c142 0 2024-02-16T22:56:37.808 INFO:tasks.workunit.client.0.smithi064.stdout:1/719: getdents d2/df/d14/d2c/db5/de3/de6 0 2024-02-16T22:56:37.811 INFO:tasks.workunit.client.0.smithi064.stdout:9/827: symlink d3/d9/d6a/db1/d2a/d2e/d31/d62/l112 0 2024-02-16T22:56:37.812 INFO:tasks.workunit.client.0.smithi064.stdout:2/965: truncate d7/f11f 250651 0 2024-02-16T22:56:37.813 INFO:tasks.workunit.client.0.smithi064.stdout:1/720: rename d2/df/d11/d1a/d30/dbd/ca5 to d2/df/d11/d1a/d30/dbd/cf4 0 2024-02-16T22:56:37.817 INFO:tasks.workunit.client.0.smithi064.stdout:2/966: fdatasync d7/d14/d1e/d34/f62 0 2024-02-16T22:56:37.819 INFO:tasks.workunit.client.0.smithi064.stdout:1/721: truncate d2/df/d14/d18/d40/fa0 393128 0 2024-02-16T22:56:37.823 INFO:tasks.workunit.client.0.smithi064.stdout:2/967: creat d7/d14/d25/daf/f143 x:0 0 0 2024-02-16T22:56:37.827 INFO:tasks.workunit.client.0.smithi064.stdout:2/968: fdatasync d7/d14/d1e/d34/dad/d10c/fbd 0 2024-02-16T22:56:37.835 INFO:tasks.workunit.client.0.smithi064.stdout:9/828: dread d3/d9/d6a/db1/fb9 [0,4194304] 0 2024-02-16T22:56:37.836 INFO:tasks.workunit.client.0.smithi064.stdout:2/969: dwrite d7/d14/d1e/d34/d3f/d9a/dd9/d95/f3a [0,4194304] 0 2024-02-16T22:56:37.839 INFO:tasks.workunit.client.0.smithi064.stdout:9/829: read d3/f63 [6972277,38603] 0 2024-02-16T22:56:37.844 INFO:tasks.workunit.client.0.smithi064.stdout:9/830: write d3/d89/fd8 [824028,60002] 0 2024-02-16T22:56:37.845 INFO:tasks.workunit.client.0.smithi064.stdout:3/893: dwrite d4/d1f/d34/fba [0,4194304] 0 2024-02-16T22:56:37.851 INFO:tasks.workunit.client.0.smithi064.stdout:2/970: creat d7/d14/d25/d46/d55/d7f/f144 x:0 0 0 2024-02-16T22:56:37.851 INFO:tasks.workunit.client.0.smithi064.stdout:9/831: dread - d3/d9/d6a/db1/d2a/d2e/da3/fc7 zero size 2024-02-16T22:56:37.855 INFO:tasks.workunit.client.0.smithi064.stdout:3/894: dwrite d4/d1f/f11b [0,4194304] 0 2024-02-16T22:56:37.857 INFO:tasks.workunit.client.0.smithi064.stdout:3/895: stat d4/d13/d1b/d27/d99/d7d/cd0 0 2024-02-16T22:56:37.861 INFO:tasks.workunit.client.0.smithi064.stdout:2/971: creat d7/d14/f145 x:0 0 0 2024-02-16T22:56:37.863 INFO:tasks.workunit.client.0.smithi064.stdout:2/972: write d7/d14/d25/d46/d55/d7f/f144 [466840,119205] 0 2024-02-16T22:56:37.874 INFO:tasks.workunit.client.0.smithi064.stdout:8/744: dwrite df/d31/d2f/d38/f62 [0,4194304] 0 2024-02-16T22:56:37.879 INFO:tasks.workunit.client.0.smithi064.stdout:8/745: mknod df/d31/d8f/dc9/c100 0 2024-02-16T22:56:37.883 INFO:tasks.workunit.client.0.smithi064.stdout:8/746: mknod df/d31/d8f/dc9/c101 0 2024-02-16T22:56:37.884 INFO:tasks.workunit.client.0.smithi064.stdout:9/832: dread d3/d11/d15/f94 [0,4194304] 0 2024-02-16T22:56:37.884 INFO:tasks.workunit.client.0.smithi064.stdout:8/747: chown df/d31/d2f/f7d 108 1 2024-02-16T22:56:37.886 INFO:tasks.workunit.client.0.smithi064.stdout:3/896: dread d4/d13/d1b/d27/d99/d7d/f9c [0,4194304] 0 2024-02-16T22:56:37.887 INFO:tasks.workunit.client.0.smithi064.stdout:3/897: chown d4/f11 1 1 2024-02-16T22:56:37.888 INFO:tasks.workunit.client.0.smithi064.stdout:3/898: dread d4/d13/d1b/f1c [0,4194304] 0 2024-02-16T22:56:37.896 INFO:tasks.workunit.client.0.smithi064.stdout:0/672: write da/f10 [3318171,48484] 0 2024-02-16T22:56:37.897 INFO:tasks.workunit.client.0.smithi064.stdout:0/673: write da/d26/f28 [4197390,51257] 0 2024-02-16T22:56:37.899 INFO:tasks.workunit.client.0.smithi064.stdout:0/674: chown da/dc/de/d12/d1d/l4c 15 1 2024-02-16T22:56:37.904 INFO:tasks.workunit.client.1.smithi192.stdout:8/948: dwrite d5/f61 [0,4194304] 0 2024-02-16T22:56:37.905 INFO:tasks.workunit.client.1.smithi192.stdout:8/949: read d5/d16/dd5/d9d/fde [3188825,6337] 0 2024-02-16T22:56:37.909 INFO:tasks.workunit.client.1.smithi192.stdout:5/990: write d6/d12/d4e/d46/d4f/f62 [763358,59034] 0 2024-02-16T22:56:37.911 INFO:tasks.workunit.client.1.smithi192.stdout:8/950: symlink d5/db/de/d10b/l130 0 2024-02-16T22:56:37.915 INFO:tasks.workunit.client.1.smithi192.stdout:8/951: mkdir d5/d16/dd5/d9d/d131 0 2024-02-16T22:56:37.916 INFO:tasks.workunit.client.0.smithi064.stdout:9/833: dread d3/d9/d1e/d42/d7b/fa8 [0,4194304] 0 2024-02-16T22:56:37.916 INFO:tasks.workunit.client.0.smithi064.stdout:8/748: dread df/f4c [0,4194304] 0 2024-02-16T22:56:37.916 INFO:tasks.workunit.client.1.smithi192.stdout:8/952: chown d5/db/d47/d43/ca6 6 1 2024-02-16T22:56:37.917 INFO:tasks.workunit.client.0.smithi064.stdout:8/749: dread - df/d31/d61/d79/dc4/fe6 zero size 2024-02-16T22:56:37.919 INFO:tasks.workunit.client.0.smithi064.stdout:9/834: symlink d3/d11/d9e/da9/l113 0 2024-02-16T22:56:37.920 INFO:tasks.workunit.client.1.smithi192.stdout:8/953: mknod d5/d9/d11d/c132 0 2024-02-16T22:56:37.921 INFO:tasks.workunit.client.0.smithi064.stdout:8/750: truncate f8 1234039 0 2024-02-16T22:56:37.921 INFO:tasks.workunit.client.0.smithi064.stdout:8/751: fsync df/d31/d47/ff7 0 2024-02-16T22:56:37.922 INFO:tasks.workunit.client.0.smithi064.stdout:9/835: creat d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/f114 x:0 0 0 2024-02-16T22:56:37.924 INFO:tasks.workunit.client.0.smithi064.stdout:8/752: chown df/d2a/l3c 372529720 1 2024-02-16T22:56:37.926 INFO:tasks.workunit.client.1.smithi192.stdout:8/954: unlink d5/d26/d8f/l7e 0 2024-02-16T22:56:37.926 INFO:tasks.workunit.client.0.smithi064.stdout:9/836: creat d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/f115 x:0 0 0 2024-02-16T22:56:37.927 INFO:tasks.workunit.client.0.smithi064.stdout:8/753: creat df/d31/d47/f102 x:0 0 0 2024-02-16T22:56:37.929 INFO:tasks.workunit.client.1.smithi192.stdout:1/953: dwrite d3/d5b/f8a [0,4194304] 0 2024-02-16T22:56:37.930 INFO:tasks.workunit.client.0.smithi064.stdout:9/837: symlink d3/d11/l116 0 2024-02-16T22:56:37.931 INFO:tasks.workunit.client.0.smithi064.stdout:8/754: symlink df/d31/d2f/d38/d48/da2/l103 0 2024-02-16T22:56:37.932 INFO:tasks.workunit.client.0.smithi064.stdout:8/755: stat df/d31/d2f/d4f/fa7 0 2024-02-16T22:56:37.934 INFO:tasks.workunit.client.0.smithi064.stdout:5/762: write d6/f13 [236131,68626] 0 2024-02-16T22:56:37.936 INFO:tasks.workunit.client.1.smithi192.stdout:1/954: truncate d3/de/d27/d34/d69/d85/d89/daa/fb5 1422059 0 2024-02-16T22:56:37.937 INFO:tasks.workunit.client.0.smithi064.stdout:4/825: write dc/de/d28/d57/d77/d101/d43/fb5 [525354,106460] 0 2024-02-16T22:56:37.940 INFO:tasks.workunit.client.0.smithi064.stdout:5/763: creat d6/fff x:0 0 0 2024-02-16T22:56:37.941 INFO:tasks.workunit.client.1.smithi192.stdout:1/955: creat d3/de/d27/d34/d52/d72/d11c/d13b/f14b x:0 0 0 2024-02-16T22:56:37.943 INFO:tasks.workunit.client.0.smithi064.stdout:5/764: creat d6/d18/d95/dbe/f100 x:0 0 0 2024-02-16T22:56:37.949 INFO:tasks.workunit.client.0.smithi064.stdout:7/792: symlink d5/d33/d45/l100 0 2024-02-16T22:56:37.950 INFO:tasks.workunit.client.0.smithi064.stdout:6/855: rmdir d0/d1d/da8 39 2024-02-16T22:56:37.954 INFO:tasks.workunit.client.0.smithi064.stdout:6/856: mknod d0/d6/d10/d33/da5/c119 0 2024-02-16T22:56:37.955 INFO:tasks.workunit.client.0.smithi064.stdout:6/857: truncate d0/d1d/db1/ff7 466976 0 2024-02-16T22:56:37.956 INFO:tasks.workunit.client.0.smithi064.stdout:7/793: getdents d5/d33/d53 0 2024-02-16T22:56:37.959 INFO:tasks.workunit.client.0.smithi064.stdout:6/858: symlink d0/d12/d7b/d110/l11a 0 2024-02-16T22:56:37.961 INFO:tasks.workunit.client.1.smithi192.stdout:0/992: dwrite d1/d3/de/d22/da7/d10d/d13b/f8a [0,4194304] 0 2024-02-16T22:56:37.962 INFO:tasks.workunit.client.1.smithi192.stdout:0/993: fdatasync d1/dc7/d9c/f72 0 2024-02-16T22:56:37.963 INFO:tasks.workunit.client.0.smithi064.stdout:7/794: dread d5/d33/d53/df2/d54/d41/da2/fef [0,4194304] 0 2024-02-16T22:56:37.965 INFO:tasks.workunit.client.1.smithi192.stdout:0/994: symlink d1/d3/de/d22/da7/d10d/d13b/da1/l142 0 2024-02-16T22:56:37.969 INFO:tasks.workunit.client.1.smithi192.stdout:0/995: creat d1/d3/de/d22/d37/d67/d75/db0/f143 x:0 0 0 2024-02-16T22:56:37.970 INFO:tasks.workunit.client.0.smithi064.stdout:7/795: getdents d5/d33/d53/df2/d54/d41/da2 0 2024-02-16T22:56:37.973 INFO:tasks.workunit.client.1.smithi192.stdout:0/996: truncate d1/d3/de/d22/da7/d10d/d13b/f88 72938 0 2024-02-16T22:56:37.985 INFO:tasks.workunit.client.0.smithi064.stdout:1/722: rename d2/df/d11/d1a/d30/dbd/cf4 to d2/df/d11/d1a/d30/dbd/d91/d56/cf5 0 2024-02-16T22:56:37.988 INFO:tasks.workunit.client.0.smithi064.stdout:8/756: rename df/c1d to df/d31/d2f/d3e/c104 0 2024-02-16T22:56:37.990 INFO:tasks.workunit.client.1.smithi192.stdout:6/998: rename da/df/d12/d57/fbd to da/df/d12/d57/daf/f13f 0 2024-02-16T22:56:37.992 INFO:tasks.workunit.client.1.smithi192.stdout:3/908: rmdir da/d11/d20 39 2024-02-16T22:56:37.992 INFO:tasks.workunit.client.1.smithi192.stdout:6/999: read da/df/d12/d15/d1b/d70/fd3 [76281,86435] 0 2024-02-16T22:56:37.996 INFO:tasks.workunit.client.0.smithi064.stdout:1/723: creat d2/df/d14/d2c/ff6 x:0 0 0 2024-02-16T22:56:37.997 INFO:tasks.workunit.client.1.smithi192.stdout:3/909: mknod da/d107/c134 0 2024-02-16T22:56:38.003 INFO:tasks.workunit.client.1.smithi192.stdout:1/956: rename d3/c53 to d3/de/d27/d34/de7/c14c 0 2024-02-16T22:56:38.006 INFO:tasks.workunit.client.0.smithi064.stdout:6/859: rename d0/d3c/da9/f67 to d0/d6/d10/f11b 0 2024-02-16T22:56:38.012 INFO:tasks.workunit.client.0.smithi064.stdout:8/757: getdents df/d31/d8f/dab/dae 0 2024-02-16T22:56:38.013 INFO:tasks.workunit.client.0.smithi064.stdout:8/758: fsync df/d31/d8f/d80/dec/fef 0 2024-02-16T22:56:38.014 INFO:tasks.workunit.client.1.smithi192.stdout:1/957: dwrite d3/de/f65 [0,4194304] 0 2024-02-16T22:56:38.015 INFO:tasks.workunit.client.0.smithi064.stdout:1/724: dread d2/df/d11/d1a/f34 [0,4194304] 0 2024-02-16T22:56:38.015 INFO:tasks.workunit.client.0.smithi064.stdout:2/973: write d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f5c [1258622,51480] 0 2024-02-16T22:56:38.018 INFO:tasks.workunit.client.0.smithi064.stdout:1/725: chown d2/df/d14/d2c/db5/de3/de6 18587268 1 2024-02-16T22:56:38.019 INFO:tasks.workunit.client.1.smithi192.stdout:1/958: dwrite d3/d5b/d77/dd2/f106 [0,4194304] 0 2024-02-16T22:56:38.020 INFO:tasks.workunit.client.0.smithi064.stdout:6/860: link d0/d5/lda d0/d1d/l11c 0 2024-02-16T22:56:38.023 INFO:tasks.workunit.client.0.smithi064.stdout:1/726: rmdir d2/df/d14/d18/d40 39 2024-02-16T22:56:38.029 INFO:tasks.workunit.client.0.smithi064.stdout:6/861: creat d0/d5/df/d17/f11d x:0 0 0 2024-02-16T22:56:38.035 INFO:tasks.workunit.client.0.smithi064.stdout:6/862: dread - d0/d5/df/d17/d18/fb3 zero size 2024-02-16T22:56:38.039 INFO:tasks.workunit.client.1.smithi192.stdout:5/991: write d6/d12/d4e/d46/fca [4133242,96745] 0 2024-02-16T22:56:38.039 INFO:tasks.workunit.client.0.smithi064.stdout:6/863: truncate d0/d1d/dbc/fc9 169323 0 2024-02-16T22:56:38.041 INFO:tasks.workunit.client.0.smithi064.stdout:6/864: chown d0/d1d/dbc/dec/f10b 443288678 1 2024-02-16T22:56:38.047 INFO:tasks.workunit.client.0.smithi064.stdout:9/838: write d3/d9/d1e/d3e/f65 [1748850,75517] 0 2024-02-16T22:56:38.049 INFO:tasks.workunit.client.1.smithi192.stdout:8/955: dwrite d5/db/ff8 [4194304,4194304] 0 2024-02-16T22:56:38.050 INFO:tasks.workunit.client.0.smithi064.stdout:9/839: fdatasync d3/d11/d15/d23/f3d 0 2024-02-16T22:56:38.050 INFO:tasks.workunit.client.1.smithi192.stdout:8/956: chown d5/d26/d83/d86/c10e 1949463114 1 2024-02-16T22:56:38.054 INFO:tasks.workunit.client.0.smithi064.stdout:9/840: mkdir d3/d11/d15/d23/d117 0 2024-02-16T22:56:38.064 INFO:tasks.workunit.client.1.smithi192.stdout:8/957: getdents d5/d26/d83/db4 0 2024-02-16T22:56:38.065 INFO:tasks.workunit.client.0.smithi064.stdout:4/826: dwrite f3 [0,4194304] 0 2024-02-16T22:56:38.065 INFO:tasks.workunit.client.1.smithi192.stdout:0/997: truncate d1/dc7/d9c/f72 522397 0 2024-02-16T22:56:38.069 INFO:tasks.workunit.client.0.smithi064.stdout:5/765: dwrite f4 [0,4194304] 0 2024-02-16T22:56:38.069 INFO:tasks.workunit.client.0.smithi064.stdout:4/827: write dc/de/dfb/dff/de3/fcd [2309553,66814] 0 2024-02-16T22:56:38.071 INFO:tasks.workunit.client.0.smithi064.stdout:6/865: dread d0/d6/d10/f9f [0,4194304] 0 2024-02-16T22:56:38.071 INFO:tasks.workunit.client.1.smithi192.stdout:8/958: creat d5/d9/d11d/f133 x:0 0 0 2024-02-16T22:56:38.072 INFO:tasks.workunit.client.1.smithi192.stdout:8/959: write d5/db/de/f119 [769181,112215] 0 2024-02-16T22:56:38.075 INFO:tasks.workunit.client.0.smithi064.stdout:7/796: write d5/d33/d53/df2/ff1 [4641502,85227] 0 2024-02-16T22:56:38.077 INFO:tasks.workunit.client.0.smithi064.stdout:7/797: write d5/d33/d45/d64/d56/fe1 [842629,12641] 0 2024-02-16T22:56:38.082 INFO:tasks.workunit.client.1.smithi192.stdout:8/960: creat d5/d26/d83/d86/d5f/db2/d126/f134 x:0 0 0 2024-02-16T22:56:38.091 INFO:tasks.workunit.client.1.smithi192.stdout:3/910: write da/fa8 [747924,13518] 0 2024-02-16T22:56:38.093 INFO:tasks.workunit.client.1.smithi192.stdout:1/959: write d3/de/d27/d34/d69/f8c [863329,11759] 0 2024-02-16T22:56:38.094 INFO:tasks.workunit.client.1.smithi192.stdout:3/911: dread da/d11/d63/d6b/d66/d75/f9f [0,4194304] 0 2024-02-16T22:56:38.094 INFO:tasks.workunit.client.0.smithi064.stdout:2/974: truncate d7/d14/d25/d46/f97 226252 0 2024-02-16T22:56:38.095 INFO:tasks.workunit.client.1.smithi192.stdout:3/912: chown da/d11/d63/d6b/d66/c12d 7460 1 2024-02-16T22:56:38.096 INFO:tasks.workunit.client.1.smithi192.stdout:0/998: dread d1/f3e [0,4194304] 0 2024-02-16T22:56:38.097 INFO:tasks.workunit.client.1.smithi192.stdout:1/960: rmdir d3/de/d27/d34/d69/d85/d11f 39 2024-02-16T22:56:38.097 INFO:tasks.workunit.client.1.smithi192.stdout:1/961: dread - d3/d5b/db1/f10b zero size 2024-02-16T22:56:38.099 INFO:tasks.workunit.client.1.smithi192.stdout:0/999: creat d1/d3/dac/f144 x:0 0 0 2024-02-16T22:56:38.100 INFO:tasks.workunit.client.0.smithi064.stdout:1/727: write d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf/fef [832237,27160] 0 2024-02-16T22:56:38.109 INFO:tasks.workunit.client.1.smithi192.stdout:1/962: dread d3/d40/f59 [0,4194304] 0 2024-02-16T22:56:38.110 INFO:tasks.workunit.client.0.smithi064.stdout:9/841: symlink d3/d9/d1e/d42/l118 0 2024-02-16T22:56:38.111 INFO:tasks.workunit.client.0.smithi064.stdout:9/842: chown d3/d11/d15/d23/f3d 105 1 2024-02-16T22:56:38.112 INFO:tasks.workunit.client.1.smithi192.stdout:1/963: symlink d3/de/d27/d34/d52/d5a/d105/l14d 0 2024-02-16T22:56:38.113 INFO:tasks.workunit.client.0.smithi064.stdout:6/866: unlink d0/d5/df/l97 0 2024-02-16T22:56:38.116 INFO:tasks.workunit.client.0.smithi064.stdout:4/828: creat dc/dd/d32/d22/f108 x:0 0 0 2024-02-16T22:56:38.117 INFO:tasks.workunit.client.1.smithi192.stdout:1/964: mkdir d3/de/d27/d34/d69/d85/d11f/d14e 0 2024-02-16T22:56:38.119 INFO:tasks.workunit.client.0.smithi064.stdout:2/975: rename d7/d14/d25/d46/d55/d57/dbe/d122 to d7/d14/d25/d46/d55/d57/dbe/d13b/d146 0 2024-02-16T22:56:38.121 INFO:tasks.workunit.client.0.smithi064.stdout:6/867: unlink d0/d5/df/d17/c73 0 2024-02-16T22:56:38.122 INFO:tasks.workunit.client.0.smithi064.stdout:7/798: mkdir d5/d33/d101 0 2024-02-16T22:56:38.123 INFO:tasks.workunit.client.0.smithi064.stdout:4/829: chown dc/dd/d16/d9c/cdd 96304 1 2024-02-16T22:56:38.124 INFO:tasks.workunit.client.0.smithi064.stdout:4/830: chown dc/dd/db8/fd2 38 1 2024-02-16T22:56:38.124 INFO:tasks.workunit.client.0.smithi064.stdout:7/799: truncate d5/d33/d53/df2/d65/da7/fce 1988841 0 2024-02-16T22:56:38.125 INFO:tasks.workunit.client.0.smithi064.stdout:2/976: unlink l4 0 2024-02-16T22:56:38.127 INFO:tasks.workunit.client.0.smithi064.stdout:7/800: stat d5/d33/d53/df2/l16 0 2024-02-16T22:56:38.127 INFO:tasks.workunit.client.0.smithi064.stdout:2/977: truncate d7/d14/d1e/d34/d3f/d9a/ffe 657604 0 2024-02-16T22:56:38.128 INFO:tasks.workunit.client.0.smithi064.stdout:4/831: mkdir dc/ddc/de8/df2/d109 0 2024-02-16T22:56:38.129 INFO:tasks.workunit.client.0.smithi064.stdout:2/978: chown d7/d14/d25/d46/d55/d57/dbe/d10e 887 1 2024-02-16T22:56:38.130 INFO:tasks.workunit.client.0.smithi064.stdout:2/979: readlink d7/d14/d1e/d34/d3f/d9a/dd9/l51 0 2024-02-16T22:56:38.131 INFO:tasks.workunit.client.0.smithi064.stdout:7/801: creat d5/d33/d53/df2/d54/d7b/dc6/f102 x:0 0 0 2024-02-16T22:56:38.134 INFO:tasks.workunit.client.0.smithi064.stdout:2/980: chown d7/d14/d1e/d34/d3f/d9a/dd9/d60/dce 1 1 2024-02-16T22:56:38.134 INFO:tasks.workunit.client.1.smithi192.stdout:3/913: read da/d11/d63/d6b/d4e/f56 [1993888,6943] 0 2024-02-16T22:56:38.136 INFO:tasks.workunit.client.1.smithi192.stdout:3/914: symlink da/d11/d63/d6b/d4e/d11a/l135 0 2024-02-16T22:56:38.139 INFO:tasks.workunit.client.0.smithi064.stdout:2/981: creat d7/d14/d25/d46/d55/d57/dbe/d13b/f147 x:0 0 0 2024-02-16T22:56:38.139 INFO:tasks.workunit.client.1.smithi192.stdout:3/915: stat da/d11/d20/d39/d53/f62 0 2024-02-16T22:56:38.140 INFO:tasks.workunit.client.0.smithi064.stdout:7/802: dwrite d5/d33/d42/f8f [0,4194304] 0 2024-02-16T22:56:38.141 INFO:tasks.workunit.client.0.smithi064.stdout:7/803: fdatasync d5/df9/fdf 0 2024-02-16T22:56:38.143 INFO:tasks.workunit.client.0.smithi064.stdout:7/804: fsync d5/d33/d53/da3/fda 0 2024-02-16T22:56:38.143 INFO:tasks.workunit.client.0.smithi064.stdout:2/982: chown d7/d14/d25/d46/d55/d57/dbe/d13b/f147 15029 1 2024-02-16T22:56:38.145 INFO:tasks.workunit.client.0.smithi064.stdout:7/805: write d5/df9/fdf [55716,129887] 0 2024-02-16T22:56:38.146 INFO:tasks.workunit.client.1.smithi192.stdout:3/916: rename da/d6a/fc6 to da/d11/dd4/deb/da1/db9/ded/f136 0 2024-02-16T22:56:38.150 INFO:tasks.workunit.client.0.smithi064.stdout:7/806: write d5/d33/fd9 [1001922,68651] 0 2024-02-16T22:56:38.153 INFO:tasks.workunit.client.1.smithi192.stdout:3/917: creat da/d11/dd4/f137 x:0 0 0 2024-02-16T22:56:38.155 INFO:tasks.workunit.client.0.smithi064.stdout:4/832: dread dc/de/d28/d57/d77/d101/f4c [0,4194304] 0 2024-02-16T22:56:38.155 INFO:tasks.workunit.client.0.smithi064.stdout:2/983: rename d7/d14/d25/d46/d55/d7f/f144 to d7/d14/d25/daf/de9/f148 0 2024-02-16T22:56:38.157 INFO:tasks.workunit.client.0.smithi064.stdout:7/807: creat d5/d33/d42/d81/dc4/f103 x:0 0 0 2024-02-16T22:56:38.157 INFO:tasks.workunit.client.1.smithi192.stdout:5/992: sync 2024-02-16T22:56:38.158 INFO:tasks.workunit.client.1.smithi192.stdout:3/918: creat da/d11/d20/d39/dca/f138 x:0 0 0 2024-02-16T22:56:38.163 INFO:tasks.workunit.client.1.smithi192.stdout:5/993: creat d6/d12/d4e/de6/f13f x:0 0 0 2024-02-16T22:56:38.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/919: getdents da/df 0 2024-02-16T22:56:38.167 INFO:tasks.workunit.client.0.smithi064.stdout:2/984: mkdir d7/d14/d1e/d34/d73/d149 0 2024-02-16T22:56:38.173 INFO:tasks.workunit.client.0.smithi064.stdout:4/833: dread dc/ddc/de8/fcf [0,4194304] 0 2024-02-16T22:56:38.177 INFO:tasks.workunit.client.1.smithi192.stdout:3/920: dread da/d11/d35/ff9 [0,4194304] 0 2024-02-16T22:56:38.179 INFO:tasks.workunit.client.1.smithi192.stdout:8/961: dwrite d5/d16/dd5/d9d/fde [0,4194304] 0 2024-02-16T22:56:38.179 INFO:tasks.workunit.client.1.smithi192.stdout:3/921: write da/df/f119 [2963907,69240] 0 2024-02-16T22:56:38.181 INFO:tasks.workunit.client.1.smithi192.stdout:3/922: truncate da/d11/dd4/deb/f77 4480030 0 2024-02-16T22:56:38.184 INFO:tasks.workunit.client.1.smithi192.stdout:8/962: dwrite d5/d26/d83/d86/f41 [0,4194304] 0 2024-02-16T22:56:38.191 INFO:tasks.workunit.client.1.smithi192.stdout:8/963: fsync d5/db/de/f37 0 2024-02-16T22:56:38.193 INFO:tasks.workunit.client.1.smithi192.stdout:3/923: link da/d11/dd4/deb/c114 da/d11/d63/d6b/d4e/d11a/c139 0 2024-02-16T22:56:38.195 INFO:tasks.workunit.client.1.smithi192.stdout:8/964: chown d5/c11 1643331 1 2024-02-16T22:56:38.197 INFO:tasks.workunit.client.1.smithi192.stdout:3/924: dread da/d11/d63/d6b/d66/d75/f76 [0,4194304] 0 2024-02-16T22:56:38.204 INFO:tasks.workunit.client.1.smithi192.stdout:1/965: fsync d3/de/d27/d34/d69/d85/d89/daa/dfb/f102 0 2024-02-16T22:56:38.210 INFO:tasks.workunit.client.1.smithi192.stdout:1/966: creat d3/de/d27/d34/d52/f14f x:0 0 0 2024-02-16T22:56:38.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/967: dread d3/d5b/f8a [0,4194304] 0 2024-02-16T22:56:38.217 INFO:tasks.workunit.client.1.smithi192.stdout:5/994: write d6/d12/d4e/d58/d8f/fe1 [4660532,70706] 0 2024-02-16T22:56:38.218 INFO:tasks.workunit.client.1.smithi192.stdout:5/995: chown d6/d8/d80/f40 6 1 2024-02-16T22:56:38.221 INFO:tasks.workunit.client.0.smithi064.stdout:2/985: mknod d7/d14/d1e/d34/d3f/d9a/dd9/d95/c14a 0 2024-02-16T22:56:38.221 INFO:tasks.workunit.client.1.smithi192.stdout:8/965: write d5/d9/f54 [7472709,102074] 0 2024-02-16T22:56:38.221 INFO:tasks.workunit.client.1.smithi192.stdout:3/925: truncate da/d11/dd4/deb/da1/db9/ded/f136 3111470 0 2024-02-16T22:56:38.223 INFO:tasks.workunit.client.0.smithi064.stdout:9/843: truncate d3/d9/d6a/db1/d2a/d2e/d31/d3f/f43 4580287 0 2024-02-16T22:56:38.224 INFO:tasks.workunit.client.0.smithi064.stdout:7/808: rmdir d5/d33/d42/d81/dc4 39 2024-02-16T22:56:38.226 INFO:tasks.workunit.client.1.smithi192.stdout:8/966: creat d5/d9/d115/f135 x:0 0 0 2024-02-16T22:56:38.228 INFO:tasks.workunit.client.0.smithi064.stdout:2/986: fsync d7/d14/d4b/fc7 0 2024-02-16T22:56:38.231 INFO:tasks.workunit.client.0.smithi064.stdout:5/766: dwrite d6/d25/df1/d91/da5/de3/ff3 [0,4194304] 0 2024-02-16T22:56:38.233 INFO:tasks.workunit.client.1.smithi192.stdout:1/968: dread d3/de/d27/d34/d52/f70 [0,4194304] 0 2024-02-16T22:56:38.238 INFO:tasks.workunit.client.1.smithi192.stdout:5/996: dread d6/d12/d4e/d58/ff8 [0,4194304] 0 2024-02-16T22:56:38.385 INFO:tasks.workunit.client.0.smithi064.stdout:9/844: mknod d3/d11/d9e/c119 0 2024-02-16T22:56:38.389 INFO:tasks.workunit.client.0.smithi064.stdout:4/834: write dc/de/f74 [954662,68254] 0 2024-02-16T22:56:38.391 INFO:tasks.workunit.client.0.smithi064.stdout:7/809: unlink d5/d33/d53/f6e 0 2024-02-16T22:56:38.393 INFO:tasks.workunit.client.1.smithi192.stdout:1/969: rename d3/de/d27/d34/d36/d47 to d3/de/d27/d34/d36/d149/d150 0 2024-02-16T22:56:38.394 INFO:tasks.workunit.client.1.smithi192.stdout:5/997: mknod d6/d8/d86/d10f/c140 0 2024-02-16T22:56:38.395 INFO:tasks.workunit.client.1.smithi192.stdout:5/998: chown d6/f2b 15460 1 2024-02-16T22:56:38.398 INFO:tasks.workunit.client.1.smithi192.stdout:8/967: write d5/db/d47/d43/d67/fd7 [921423,40015] 0 2024-02-16T22:56:38.399 INFO:tasks.workunit.client.1.smithi192.stdout:3/926: rename da/d11/d20/fd3 to da/d11/d20/f13a 0 2024-02-16T22:56:38.402 INFO:tasks.workunit.client.0.smithi064.stdout:5/767: creat d6/d25/df1/d91/f101 x:0 0 0 2024-02-16T22:56:38.406 INFO:tasks.workunit.client.0.smithi064.stdout:9/845: mknod d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/c11a 0 2024-02-16T22:56:38.409 INFO:tasks.workunit.client.0.smithi064.stdout:4/835: dread - dc/ddc/de8/da1/fbb zero size 2024-02-16T22:56:38.409 INFO:tasks.workunit.client.1.smithi192.stdout:8/968: dread d5/db/d47/d43/d67/f85 [0,4194304] 0 2024-02-16T22:56:38.412 INFO:tasks.workunit.client.0.smithi064.stdout:7/810: mknod d5/d33/d45/d73/dd7/c104 0 2024-02-16T22:56:38.413 INFO:tasks.workunit.client.1.smithi192.stdout:5/999: write d6/d12/d4e/d5b/d67/d78/fef [701812,88668] 0 2024-02-16T22:56:38.414 INFO:tasks.workunit.client.0.smithi064.stdout:3/899: sync 2024-02-16T22:56:38.414 INFO:tasks.workunit.client.0.smithi064.stdout:0/675: sync 2024-02-16T22:56:38.415 INFO:tasks.workunit.client.1.smithi192.stdout:1/970: dwrite d3/f2d [0,4194304] 0 2024-02-16T22:56:38.415 INFO:tasks.workunit.client.0.smithi064.stdout:2/987: symlink d7/d14/d1e/d34/d3f/d12a/l14b 0 2024-02-16T22:56:38.416 INFO:tasks.workunit.client.1.smithi192.stdout:8/969: creat d5/d26/d8f/dc0/f136 x:0 0 0 2024-02-16T22:56:38.417 INFO:tasks.workunit.client.0.smithi064.stdout:5/768: rename d6/d25/df1/d80/fec to d6/d25/da4/dc1/f102 0 2024-02-16T22:56:38.418 INFO:tasks.workunit.client.0.smithi064.stdout:5/769: stat d6/d25/da4/dc1/f60 0 2024-02-16T22:56:38.420 INFO:tasks.workunit.client.1.smithi192.stdout:8/970: creat d5/d16/de1/f137 x:0 0 0 2024-02-16T22:56:38.421 INFO:tasks.workunit.client.0.smithi064.stdout:9/846: chown d3/d89/d9a/lf5 21 1 2024-02-16T22:56:38.423 INFO:tasks.workunit.client.0.smithi064.stdout:7/811: mknod d5/d33/d53/df2/d54/d41/c105 0 2024-02-16T22:56:38.424 INFO:tasks.workunit.client.1.smithi192.stdout:8/971: creat d5/d26/d8f/dc0/dc3/f138 x:0 0 0 2024-02-16T22:56:38.424 INFO:tasks.workunit.client.1.smithi192.stdout:8/972: fsync d5/db/de/df/f11b 0 2024-02-16T22:56:38.426 INFO:tasks.workunit.client.0.smithi064.stdout:3/900: creat d4/d13/d1b/d27/d82/f12f x:0 0 0 2024-02-16T22:56:38.427 INFO:tasks.workunit.client.0.smithi064.stdout:2/988: symlink d7/d14/d25/d46/d55/d57/l14c 0 2024-02-16T22:56:38.429 INFO:tasks.workunit.client.1.smithi192.stdout:1/971: dread d3/de/d27/d34/f4e [0,4194304] 0 2024-02-16T22:56:38.431 INFO:tasks.workunit.client.1.smithi192.stdout:8/973: dwrite d5/d26/d83/db4/f105 [0,4194304] 0 2024-02-16T22:56:38.433 INFO:tasks.workunit.client.0.smithi064.stdout:3/901: dwrite d4/d1f/f20 [0,4194304] 0 2024-02-16T22:56:38.435 INFO:tasks.workunit.client.0.smithi064.stdout:0/676: truncate da/dc/d3b/d72/d7c/fcb 450386 0 2024-02-16T22:56:38.438 INFO:tasks.workunit.client.1.smithi192.stdout:1/972: unlink d3/de/d27/d34/de7/f117 0 2024-02-16T22:56:38.440 INFO:tasks.workunit.client.0.smithi064.stdout:3/902: readlink d4/d13/d1b/d27/d44/l11a 0 2024-02-16T22:56:38.440 INFO:tasks.workunit.client.0.smithi064.stdout:0/677: truncate da/dc/fdd 517604 0 2024-02-16T22:56:38.440 INFO:tasks.workunit.client.0.smithi064.stdout:0/678: read da/dc/fdd [381436,83932] 0 2024-02-16T22:56:38.442 INFO:tasks.workunit.client.1.smithi192.stdout:8/974: symlink d5/d26/l139 0 2024-02-16T22:56:38.443 INFO:tasks.workunit.client.1.smithi192.stdout:8/975: dread - d5/d26/d83/d86/f117 zero size 2024-02-16T22:56:38.445 INFO:tasks.workunit.client.1.smithi192.stdout:1/973: truncate d3/de/d27/d34/d52/d5a/fa2 132656 0 2024-02-16T22:56:38.446 INFO:tasks.workunit.client.0.smithi064.stdout:4/836: truncate dc/de/d28/d57/d77/d101/d43/fb5 371561 0 2024-02-16T22:56:38.447 INFO:tasks.workunit.client.0.smithi064.stdout:4/837: dread - dc/de/fb9 zero size 2024-02-16T22:56:38.449 INFO:tasks.workunit.client.0.smithi064.stdout:0/679: rmdir da/dc 39 2024-02-16T22:56:38.449 INFO:tasks.workunit.client.0.smithi064.stdout:5/770: dread d6/d25/df1/d91/da5/de3/f59 [0,4194304] 0 2024-02-16T22:56:38.451 INFO:tasks.workunit.client.1.smithi192.stdout:1/974: mknod d3/de/d27/d34/d36/d149/d150/dcd/c151 0 2024-02-16T22:56:38.452 INFO:tasks.workunit.client.0.smithi064.stdout:7/812: dwrite d5/d33/d53/df2/d54/f3f [4194304,4194304] 0 2024-02-16T22:56:38.455 INFO:tasks.workunit.client.1.smithi192.stdout:1/975: mkdir d3/d5b/db1/d152 0 2024-02-16T22:56:38.461 INFO:tasks.workunit.client.0.smithi064.stdout:5/771: rmdir d6 39 2024-02-16T22:56:38.462 INFO:tasks.workunit.client.0.smithi064.stdout:7/813: mknod d5/df9/c106 0 2024-02-16T22:56:38.463 INFO:tasks.workunit.client.1.smithi192.stdout:1/976: symlink d3/d5b/d77/d142/l153 0 2024-02-16T22:56:38.465 INFO:tasks.workunit.client.0.smithi064.stdout:4/838: dread dc/de/d6d/f9f [0,4194304] 0 2024-02-16T22:56:38.466 INFO:tasks.workunit.client.0.smithi064.stdout:0/680: mknod da/dc/de/ce2 0 2024-02-16T22:56:38.467 INFO:tasks.workunit.client.1.smithi192.stdout:1/977: symlink d3/de/d27/d34/d69/l154 0 2024-02-16T22:56:38.467 INFO:tasks.workunit.client.0.smithi064.stdout:0/681: dread - da/dc/de/d12/d2c/d2e/d38/d87/fcc zero size 2024-02-16T22:56:38.468 INFO:tasks.workunit.client.0.smithi064.stdout:7/814: mknod d5/d33/d53/df2/d65/d7f/db2/c107 0 2024-02-16T22:56:38.469 INFO:tasks.workunit.client.1.smithi192.stdout:1/978: stat d3/l20 0 2024-02-16T22:56:38.469 INFO:tasks.workunit.client.0.smithi064.stdout:7/815: dread - d5/d33/d53/df2/d54/d41/f68 zero size 2024-02-16T22:56:38.470 INFO:tasks.workunit.client.0.smithi064.stdout:4/839: write dc/dd/d16/f3a [6608935,3945] 0 2024-02-16T22:56:38.471 INFO:tasks.workunit.client.0.smithi064.stdout:0/682: rename da/d71/da4/fca to da/d26/de0/d9e/dc0/fe3 0 2024-02-16T22:56:38.472 INFO:tasks.workunit.client.1.smithi192.stdout:1/979: creat d3/de/d27/d34/d52/d63/f155 x:0 0 0 2024-02-16T22:56:38.475 INFO:tasks.workunit.client.0.smithi064.stdout:4/840: rename dc/dd/d32/d4b/fa4 to dc/dd/d16/d6e/f10a 0 2024-02-16T22:56:38.477 INFO:tasks.workunit.client.0.smithi064.stdout:4/841: readlink dc/de/dfb/dff/de3/l97 0 2024-02-16T22:56:38.478 INFO:tasks.workunit.client.0.smithi064.stdout:0/683: dwrite da/dc/de/d12/d1d/f1f [0,4194304] 0 2024-02-16T22:56:38.479 INFO:tasks.workunit.client.0.smithi064.stdout:7/816: symlink d5/d33/d53/dfe/l108 0 2024-02-16T22:56:38.488 INFO:tasks.workunit.client.0.smithi064.stdout:0/684: dwrite da/dc/d3b/f5f [0,4194304] 0 2024-02-16T22:56:38.489 INFO:tasks.workunit.client.1.smithi192.stdout:8/976: truncate d5/d9/d115/f11f 2125771 0 2024-02-16T22:56:38.490 INFO:tasks.workunit.client.0.smithi064.stdout:7/817: write d5/d33/d42/d81/dc4/dcc/ffb [94050,75857] 0 2024-02-16T22:56:38.490 INFO:tasks.workunit.client.0.smithi064.stdout:3/903: truncate d4/d1f/d34/fda 3247938 0 2024-02-16T22:56:38.492 INFO:tasks.workunit.client.1.smithi192.stdout:3/927: sync 2024-02-16T22:56:38.492 INFO:tasks.workunit.client.1.smithi192.stdout:1/980: dread d3/de/f1b [0,4194304] 0 2024-02-16T22:56:38.493 INFO:tasks.workunit.client.0.smithi064.stdout:0/685: symlink da/d71/da4/le4 0 2024-02-16T22:56:38.493 INFO:tasks.workunit.client.1.smithi192.stdout:1/981: dread - d3/f7e zero size 2024-02-16T22:56:38.494 INFO:tasks.workunit.client.1.smithi192.stdout:1/982: truncate d3/de/d27/d34/d52/db7/ff2 771791 0 2024-02-16T22:56:38.499 INFO:tasks.workunit.client.0.smithi064.stdout:5/772: write d6/d25/df1/d91/da5/de3/f54 [3736491,80994] 0 2024-02-16T22:56:38.504 INFO:tasks.workunit.client.0.smithi064.stdout:7/818: rename d5/d33/d53/df2/f1d to d5/d33/d42/d81/d82/f109 0 2024-02-16T22:56:38.504 INFO:tasks.workunit.client.0.smithi064.stdout:5/773: stat d6/d25/da4/dc1/d85/fa3 0 2024-02-16T22:56:38.506 INFO:tasks.workunit.client.0.smithi064.stdout:3/904: rename d4/d13/d1b/d27/d44/c79 to d4/d13/d1b/d27/d44/d97/c130 0 2024-02-16T22:56:38.508 INFO:tasks.workunit.client.1.smithi192.stdout:3/928: getdents da/d11/dd4/deb/da1 0 2024-02-16T22:56:38.509 INFO:tasks.workunit.client.0.smithi064.stdout:5/774: mkdir d6/d25/df1/d91/da5/de3/d43/db9/d103 0 2024-02-16T22:56:38.513 INFO:tasks.workunit.client.0.smithi064.stdout:0/686: rename da/dc/de/d37/daa/db4/faf to da/dc/de/d12/d2c/d2e/d62/db6/fe5 0 2024-02-16T22:56:38.513 INFO:tasks.workunit.client.0.smithi064.stdout:4/842: write dc/dd/fc8 [608651,111655] 0 2024-02-16T22:56:38.514 INFO:tasks.workunit.client.1.smithi192.stdout:8/977: dread d5/d16/f27 [0,4194304] 0 2024-02-16T22:56:38.514 INFO:tasks.workunit.client.1.smithi192.stdout:3/929: fsync da/f12 0 2024-02-16T22:56:38.514 INFO:tasks.workunit.client.0.smithi064.stdout:4/843: chown dc/de/f99 179907 1 2024-02-16T22:56:38.515 INFO:tasks.workunit.client.1.smithi192.stdout:1/983: dread d3/de/d27/d34/d36/d149/d150/d5d/fce [0,4194304] 0 2024-02-16T22:56:38.516 INFO:tasks.workunit.client.0.smithi064.stdout:3/905: creat d4/d13/d1b/d27/d82/db0/f131 x:0 0 0 2024-02-16T22:56:38.517 INFO:tasks.workunit.client.0.smithi064.stdout:3/906: stat d4/d13/d1b/d27/d99/fe2 0 2024-02-16T22:56:38.521 INFO:tasks.workunit.client.1.smithi192.stdout:3/930: dwrite da/f1a [0,4194304] 0 2024-02-16T22:56:38.523 INFO:tasks.workunit.client.1.smithi192.stdout:8/978: rmdir d5/d26 39 2024-02-16T22:56:38.530 INFO:tasks.workunit.client.0.smithi064.stdout:4/844: symlink dc/dd/d32/d46/l10b 0 2024-02-16T22:56:38.530 INFO:tasks.workunit.client.0.smithi064.stdout:4/845: chown dc/d4a/l96 3662157 1 2024-02-16T22:56:38.532 INFO:tasks.workunit.client.1.smithi192.stdout:3/931: creat da/d106/f13b x:0 0 0 2024-02-16T22:56:38.532 INFO:tasks.workunit.client.1.smithi192.stdout:3/932: fsync da/d11/dd4/f137 0 2024-02-16T22:56:38.537 INFO:tasks.workunit.client.0.smithi064.stdout:7/819: rename d5/d33/d42/d81/c95 to d5/d7d/df0/c10a 0 2024-02-16T22:56:38.541 INFO:tasks.workunit.client.1.smithi192.stdout:8/979: mknod d5/d26/dad/c13a 0 2024-02-16T22:56:38.544 INFO:tasks.workunit.client.1.smithi192.stdout:8/980: rename d5/d26/d8f/dc0/ff4 to d5/d26/d8f/dc0/dc3/f13b 0 2024-02-16T22:56:38.545 INFO:tasks.workunit.client.1.smithi192.stdout:1/984: dwrite d3/de/d27/d34/d69/d85/fdc [0,4194304] 0 2024-02-16T22:56:38.545 INFO:tasks.workunit.client.0.smithi064.stdout:5/775: write d6/d18/d90/fc2 [366360,31025] 0 2024-02-16T22:56:38.546 INFO:tasks.workunit.client.1.smithi192.stdout:3/933: creat da/d11/d20/d39/f13c x:0 0 0 2024-02-16T22:56:38.548 INFO:tasks.workunit.client.1.smithi192.stdout:3/934: write da/d11/d2b/ddc/d102/f111 [864494,28841] 0 2024-02-16T22:56:38.549 INFO:tasks.workunit.client.0.smithi064.stdout:0/687: truncate da/d26/fd0 848917 0 2024-02-16T22:56:38.549 INFO:tasks.workunit.client.0.smithi064.stdout:3/907: dwrite d4/d14/d2b/fe9 [0,4194304] 0 2024-02-16T22:56:38.550 INFO:tasks.workunit.client.0.smithi064.stdout:7/820: dread - d5/d33/d45/f52 zero size 2024-02-16T22:56:38.551 INFO:tasks.workunit.client.1.smithi192.stdout:3/935: dread - da/d11/d63/d6b/d66/f12b zero size 2024-02-16T22:56:38.552 INFO:tasks.workunit.client.0.smithi064.stdout:7/821: stat d5/d33/d42/d81/dc4 0 2024-02-16T22:56:38.552 INFO:tasks.workunit.client.0.smithi064.stdout:7/822: dread - d5/d33/d53/f99 zero size 2024-02-16T22:56:38.553 INFO:tasks.workunit.client.0.smithi064.stdout:4/846: mknod dc/c10c 0 2024-02-16T22:56:38.553 INFO:tasks.workunit.client.0.smithi064.stdout:4/847: chown dc/de/d28/d57/d77/d101/ld6 2045993 1 2024-02-16T22:56:38.554 INFO:tasks.workunit.client.1.smithi192.stdout:3/936: creat da/d11/d63/d6b/d80/f13d x:0 0 0 2024-02-16T22:56:38.558 INFO:tasks.workunit.client.0.smithi064.stdout:3/908: mkdir d4/d13/d1b/d27/d81/d102/d132 0 2024-02-16T22:56:38.559 INFO:tasks.workunit.client.1.smithi192.stdout:3/937: fsync da/d11/d63/dbf/f103 0 2024-02-16T22:56:38.560 INFO:tasks.workunit.client.0.smithi064.stdout:5/776: read d6/f24 [2990763,23176] 0 2024-02-16T22:56:38.562 INFO:tasks.workunit.client.1.smithi192.stdout:3/938: symlink da/d11/d35/l13e 0 2024-02-16T22:56:38.564 INFO:tasks.workunit.client.0.smithi064.stdout:5/777: fdatasync d6/d25/df1/d91/da5/de3/f53 0 2024-02-16T22:56:38.564 INFO:tasks.workunit.client.0.smithi064.stdout:7/823: link d5/d33/d45/d64/d56/d66/ldc d5/d33/d45/d73/l10b 0 2024-02-16T22:56:38.566 INFO:tasks.workunit.client.0.smithi064.stdout:7/824: write d5/d7d/fa0 [3505058,103056] 0 2024-02-16T22:56:38.568 INFO:tasks.workunit.client.0.smithi064.stdout:5/778: symlink d6/d25/df1/d91/da5/de3/d43/d7d/de2/l104 0 2024-02-16T22:56:38.569 INFO:tasks.workunit.client.1.smithi192.stdout:3/939: dread da/d11/d35/ff9 [0,4194304] 0 2024-02-16T22:56:38.569 INFO:tasks.workunit.client.0.smithi064.stdout:5/779: chown d6/d25/df1/d91/da5/fcb 3743 1 2024-02-16T22:56:38.579 INFO:tasks.workunit.client.0.smithi064.stdout:7/825: dwrite d5/d33/d53/df2/d54/f3f [8388608,4194304] 0 2024-02-16T22:56:38.583 INFO:tasks.workunit.client.0.smithi064.stdout:7/826: truncate d5/f8b 1997539 0 2024-02-16T22:56:38.583 INFO:tasks.workunit.client.1.smithi192.stdout:1/985: dwrite d3/de/f32 [0,4194304] 0 2024-02-16T22:56:38.586 INFO:tasks.workunit.client.0.smithi064.stdout:4/848: dwrite dc/dd/d32/fa5 [0,4194304] 0 2024-02-16T22:56:38.587 INFO:tasks.workunit.client.0.smithi064.stdout:7/827: fdatasync d5/f8b 0 2024-02-16T22:56:38.592 INFO:tasks.workunit.client.0.smithi064.stdout:4/849: mkdir dc/dd/d32/d46/d5a/d7e/d9a/df8/d10d 0 2024-02-16T22:56:38.599 INFO:tasks.workunit.client.1.smithi192.stdout:3/940: fdatasync da/d11/d63/fa6 0 2024-02-16T22:56:38.600 INFO:tasks.workunit.client.1.smithi192.stdout:1/986: symlink d3/de/d27/d34/d69/d85/d89/daa/d126/l156 0 2024-02-16T22:56:38.601 INFO:tasks.workunit.client.0.smithi064.stdout:1/728: sync 2024-02-16T22:56:38.601 INFO:tasks.workunit.client.0.smithi064.stdout:8/759: sync 2024-02-16T22:56:38.602 INFO:tasks.workunit.client.0.smithi064.stdout:4/850: write dc/dd/d32/d46/d5a/f66 [1858624,69271] 0 2024-02-16T22:56:38.606 INFO:tasks.workunit.client.1.smithi192.stdout:3/941: mkdir da/d11/d20/d39/d13f 0 2024-02-16T22:56:38.609 INFO:tasks.workunit.client.1.smithi192.stdout:1/987: mknod d3/de/d27/c157 0 2024-02-16T22:56:38.610 INFO:tasks.workunit.client.0.smithi064.stdout:1/729: mkdir d2/df/d11/df7 0 2024-02-16T22:56:38.611 INFO:tasks.workunit.client.0.smithi064.stdout:7/828: getdents d5/d33/d42/d81 0 2024-02-16T22:56:38.613 INFO:tasks.workunit.client.0.smithi064.stdout:3/909: dwrite d4/d1f/d5b/fe4 [0,4194304] 0 2024-02-16T22:56:38.614 INFO:tasks.workunit.client.0.smithi064.stdout:4/851: fsync dc/de/d28/d57/d77/d101/dbd/fc2 0 2024-02-16T22:56:38.615 INFO:tasks.workunit.client.0.smithi064.stdout:3/910: chown d4/d13/d1b/d27/d99/d7d/fbf 83547191 1 2024-02-16T22:56:38.617 INFO:tasks.workunit.client.1.smithi192.stdout:1/988: dwrite d3/de/fc6 [4194304,4194304] 0 2024-02-16T22:56:38.623 INFO:tasks.workunit.client.1.smithi192.stdout:3/942: dread f4 [0,4194304] 0 2024-02-16T22:56:38.624 INFO:tasks.workunit.client.1.smithi192.stdout:8/981: sync 2024-02-16T22:56:38.627 INFO:tasks.workunit.client.1.smithi192.stdout:1/989: dwrite d3/de/d27/d34/d69/d85/d89/daa/d126/f13a [0,4194304] 0 2024-02-16T22:56:38.627 INFO:tasks.workunit.client.0.smithi064.stdout:4/852: unlink dc/dd/d32/d46/d5a/l89 0 2024-02-16T22:56:38.630 INFO:tasks.workunit.client.1.smithi192.stdout:3/943: mkdir da/df/d140 0 2024-02-16T22:56:38.635 INFO:tasks.workunit.client.1.smithi192.stdout:8/982: symlink d5/d26/d83/d86/d5f/l13c 0 2024-02-16T22:56:38.639 INFO:tasks.workunit.client.0.smithi064.stdout:3/911: mkdir d4/d14/d133 0 2024-02-16T22:56:38.641 INFO:tasks.workunit.client.0.smithi064.stdout:4/853: dread dc/de/d28/d57/d77/d101/f36 [0,4194304] 0 2024-02-16T22:56:38.643 INFO:tasks.workunit.client.0.smithi064.stdout:4/854: chown dc/de/d28/d57/d77/d101/c73 1849018 1 2024-02-16T22:56:38.643 INFO:tasks.workunit.client.1.smithi192.stdout:8/983: creat d5/d9/d11d/f13d x:0 0 0 2024-02-16T22:56:38.643 INFO:tasks.workunit.client.1.smithi192.stdout:8/984: fsync d5/db/de/f80 0 2024-02-16T22:56:38.645 INFO:tasks.workunit.client.0.smithi064.stdout:3/912: truncate d4/d14/fb7 334574 0 2024-02-16T22:56:38.647 INFO:tasks.workunit.client.0.smithi064.stdout:3/913: chown d4/d13/d1b/le3 152541476 1 2024-02-16T22:56:38.648 INFO:tasks.workunit.client.0.smithi064.stdout:4/855: dread dc/ff [0,4194304] 0 2024-02-16T22:56:38.649 INFO:tasks.workunit.client.1.smithi192.stdout:1/990: getdents d3/de/d27/d34/d69/d85/d11f 0 2024-02-16T22:56:38.651 INFO:tasks.workunit.client.1.smithi192.stdout:3/944: dread da/d11/d20/f69 [0,4194304] 0 2024-02-16T22:56:38.651 INFO:tasks.workunit.client.1.smithi192.stdout:8/985: rename d5/d26/d8f/dc0/dc3/cf3 to d5/d26/d83/d86/d5f/dbf/dfb/c13e 0 2024-02-16T22:56:38.653 INFO:tasks.workunit.client.1.smithi192.stdout:1/991: mkdir d3/d40/d9f/d158 0 2024-02-16T22:56:38.655 INFO:tasks.workunit.client.1.smithi192.stdout:8/986: dread - d5/d9/f79 zero size 2024-02-16T22:56:38.656 INFO:tasks.workunit.client.1.smithi192.stdout:8/987: write d5/db/d47/f92 [2850927,15824] 0 2024-02-16T22:56:38.656 INFO:tasks.workunit.client.1.smithi192.stdout:3/945: truncate da/d11/f96 3456935 0 2024-02-16T22:56:38.657 INFO:tasks.workunit.client.1.smithi192.stdout:8/988: dread - d5/d16/de1/f108 zero size 2024-02-16T22:56:38.657 INFO:tasks.workunit.client.0.smithi064.stdout:4/856: creat dc/de/d6d/dee/f10e x:0 0 0 2024-02-16T22:56:38.658 INFO:tasks.workunit.client.1.smithi192.stdout:3/946: write da/d11/dd4/deb/f77 [3186652,70789] 0 2024-02-16T22:56:38.658 INFO:tasks.workunit.client.1.smithi192.stdout:8/989: write d5/db/de/f80 [410206,121130] 0 2024-02-16T22:56:38.660 INFO:tasks.workunit.client.1.smithi192.stdout:1/992: fsync d3/d40/f59 0 2024-02-16T22:56:38.660 INFO:tasks.workunit.client.1.smithi192.stdout:8/990: readlink d5/db/de/d10b/l130 0 2024-02-16T22:56:38.663 INFO:tasks.workunit.client.1.smithi192.stdout:1/993: dread d3/de/d27/d34/d69/fef [0,4194304] 0 2024-02-16T22:56:38.665 INFO:tasks.workunit.client.1.smithi192.stdout:3/947: dwrite da/d11/d63/d6b/faa [0,4194304] 0 2024-02-16T22:56:38.667 INFO:tasks.workunit.client.1.smithi192.stdout:3/948: write da/d11/d2b/ddc/d102/f111 [247555,88412] 0 2024-02-16T22:56:38.673 INFO:tasks.workunit.client.1.smithi192.stdout:3/949: symlink da/d11/dd4/deb/da1/db9/ded/l141 0 2024-02-16T22:56:38.673 INFO:tasks.workunit.client.0.smithi064.stdout:1/730: write d2/df/d11/d1a/d30/dbd/fb7 [844558,35951] 0 2024-02-16T22:56:38.675 INFO:tasks.workunit.client.1.smithi192.stdout:3/950: dread da/d11/d20/d39/dca/ffa [0,4194304] 0 2024-02-16T22:56:38.675 INFO:tasks.workunit.client.0.smithi064.stdout:1/731: chown d2/c79 335354949 1 2024-02-16T22:56:38.680 INFO:tasks.workunit.client.1.smithi192.stdout:3/951: creat da/d11/d20/d39/d53/dc7/d11b/f142 x:0 0 0 2024-02-16T22:56:38.680 INFO:tasks.workunit.client.1.smithi192.stdout:8/991: dread d5/db/d47/f92 [0,4194304] 0 2024-02-16T22:56:38.681 INFO:tasks.workunit.client.0.smithi064.stdout:8/760: dwrite df/d31/d2f/d38/d48/f76 [0,4194304] 0 2024-02-16T22:56:38.685 INFO:tasks.workunit.client.1.smithi192.stdout:3/952: unlink da/d11/d63/d6b/d66/f12b 0 2024-02-16T22:56:38.686 INFO:tasks.workunit.client.0.smithi064.stdout:7/829: dwrite d5/d33/d42/d81/d82/f109 [0,4194304] 0 2024-02-16T22:56:38.686 INFO:tasks.workunit.client.0.smithi064.stdout:1/732: dwrite d2/df/d14/d2c/f3c [4194304,4194304] 0 2024-02-16T22:56:38.686 INFO:tasks.workunit.client.1.smithi192.stdout:8/992: creat d5/d16/dd5/d9d/f13f x:0 0 0 2024-02-16T22:56:38.695 INFO:tasks.workunit.client.1.smithi192.stdout:3/953: rename da/df/f23 to da/d11/d20/d39/d53/dc7/f143 0 2024-02-16T22:56:38.696 INFO:tasks.workunit.client.0.smithi064.stdout:3/914: write d4/d13/d1b/d4f/fa2 [3340277,128926] 0 2024-02-16T22:56:38.697 INFO:tasks.workunit.client.1.smithi192.stdout:8/993: write d5/d26/d83/db4/fba [1213040,48432] 0 2024-02-16T22:56:38.699 INFO:tasks.workunit.client.1.smithi192.stdout:3/954: mknod da/d11/d20/d39/dca/c144 0 2024-02-16T22:56:38.700 INFO:tasks.workunit.client.1.smithi192.stdout:3/955: fdatasync da/d106/f13b 0 2024-02-16T22:56:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:1/733: symlink d2/df/d11/d1a/d30/d3d/lf8 0 2024-02-16T22:56:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:1/734: fsync d2/f43 0 2024-02-16T22:56:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:7/830: creat d5/d33/d45/d64/d56/f10c x:0 0 0 2024-02-16T22:56:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:3/915: mknod d4/d13/d1b/d27/c134 0 2024-02-16T22:56:38.776 INFO:tasks.workunit.client.1.smithi192.stdout:1/994: sync 2024-02-16T22:56:38.791 INFO:tasks.workunit.client.0.smithi064.stdout:6/868: sync 2024-02-16T22:56:38.792 INFO:tasks.workunit.client.0.smithi064.stdout:8/761: write df/d31/f3f [3914673,26302] 0 2024-02-16T22:56:38.792 INFO:tasks.workunit.client.1.smithi192.stdout:8/994: link d5/d16/c6d d5/c140 0 2024-02-16T22:56:38.796 INFO:tasks.workunit.client.1.smithi192.stdout:1/995: unlink d3/de/f82 0 2024-02-16T22:56:38.798 INFO:tasks.workunit.client.0.smithi064.stdout:1/735: unlink d2/df/d14/d2a/f87 0 2024-02-16T22:56:38.798 INFO:tasks.workunit.client.1.smithi192.stdout:8/995: stat d5/l6c 0 2024-02-16T22:56:38.803 INFO:tasks.workunit.client.0.smithi064.stdout:8/762: rename fa to df/d31/d8f/dab/dae/f105 0 2024-02-16T22:56:38.806 INFO:tasks.workunit.client.0.smithi064.stdout:7/831: dread d5/d33/d45/d64/d56/d5d/f80 [0,4194304] 0 2024-02-16T22:56:38.807 INFO:tasks.workunit.client.1.smithi192.stdout:3/956: dwrite da/d11/f3f [4194304,4194304] 0 2024-02-16T22:56:38.807 INFO:tasks.workunit.client.1.smithi192.stdout:1/996: stat d3/de/d27/d34/d52/d63/d7a/dc9/d113 0 2024-02-16T22:56:38.811 INFO:tasks.workunit.client.1.smithi192.stdout:8/996: creat d5/d9/f141 x:0 0 0 2024-02-16T22:56:38.812 INFO:tasks.workunit.client.0.smithi064.stdout:3/916: dread d4/d13/d1b/d27/f58 [0,4194304] 0 2024-02-16T22:56:38.818 INFO:tasks.workunit.client.1.smithi192.stdout:1/997: dread d3/de/d27/d34/d36/feb [0,4194304] 0 2024-02-16T22:56:38.818 INFO:tasks.workunit.client.0.smithi064.stdout:1/736: readlink d2/df/d14/d18/d48/d6b/ldc 0 2024-02-16T22:56:38.819 INFO:tasks.workunit.client.1.smithi192.stdout:1/998: chown d3/de/d27/d34/d52/lca 943898 1 2024-02-16T22:56:38.821 INFO:tasks.workunit.client.0.smithi064.stdout:3/917: dwrite d4/d13/d1b/d27/d81/f10d [0,4194304] 0 2024-02-16T22:56:38.825 INFO:tasks.workunit.client.0.smithi064.stdout:6/869: dwrite d0/d12/f78 [0,4194304] 0 2024-02-16T22:56:38.835 INFO:tasks.workunit.client.1.smithi192.stdout:3/957: dwrite da/df/f79 [0,4194304] 0 2024-02-16T22:56:38.838 INFO:tasks.workunit.client.0.smithi064.stdout:1/737: getdents d2/df/d14/d2c/db5/de3 0 2024-02-16T22:56:38.839 INFO:tasks.workunit.client.1.smithi192.stdout:8/997: dwrite d5/d26/d8f/fb0 [0,4194304] 0 2024-02-16T22:56:38.839 INFO:tasks.workunit.client.0.smithi064.stdout:7/832: dwrite d5/d33/d53/df2/d65/d7f/f9c [0,4194304] 0 2024-02-16T22:56:38.839 INFO:tasks.workunit.client.0.smithi064.stdout:3/918: getdents d4/d1f/d5b/d51/dc1 0 2024-02-16T22:56:38.842 INFO:tasks.workunit.client.0.smithi064.stdout:6/870: link d0/d5/f90 d0/d6/da/f11e 0 2024-02-16T22:56:38.843 INFO:tasks.workunit.client.0.smithi064.stdout:7/833: write d5/d33/d45/d64/d56/d66/fe0 [267141,44885] 0 2024-02-16T22:56:38.851 INFO:tasks.workunit.client.0.smithi064.stdout:3/919: rename d4/d1f/f20 to d4/d13/d4d/d5c/d8b/f135 0 2024-02-16T22:56:38.851 INFO:tasks.workunit.client.0.smithi064.stdout:1/738: read d2/df/d11/d1a/d30/dbd/d91/f5a [2599805,11935] 0 2024-02-16T22:56:38.851 INFO:tasks.workunit.client.0.smithi064.stdout:7/834: mknod d5/d33/d53/df2/d54/d7b/dc6/c10d 0 2024-02-16T22:56:38.854 INFO:tasks.workunit.client.0.smithi064.stdout:6/871: dwrite d0/d3c/dc5/f102 [0,4194304] 0 2024-02-16T22:56:38.856 INFO:tasks.workunit.client.0.smithi064.stdout:7/835: write d5/d33/d53/df2/d65/da7/dad/fee [316204,18012] 0 2024-02-16T22:56:38.863 INFO:tasks.workunit.client.0.smithi064.stdout:3/920: dwrite d4/d1f/d34/f122 [0,4194304] 0 2024-02-16T22:56:38.865 INFO:tasks.workunit.client.0.smithi064.stdout:6/872: rename d0/d5/df/d17/d4a/fe2 to d0/d6/d10/d33/da4/ddd/f11f 0 2024-02-16T22:56:38.866 INFO:tasks.workunit.client.0.smithi064.stdout:3/921: dread - d4/d13/d1b/d27/d61/d63/ffb zero size 2024-02-16T22:56:38.866 INFO:tasks.workunit.client.0.smithi064.stdout:1/739: dwrite d2/df/d11/d1a/d30/fd8 [0,4194304] 0 2024-02-16T22:56:38.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:38 smithi064 ceph-mon[29226]: pgmap v24: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 126 MiB/s rd, 279 MiB/s wr, 629 op/s 2024-02-16T22:56:38.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:38 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:38.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:38 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:38.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:38 smithi064 ceph-mon[29226]: Reconfiguring alertmanager.smithi064 (dependencies changed)... 2024-02-16T22:56:38.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:38 smithi064 ceph-mon[29226]: Reconfiguring daemon alertmanager.smithi064 on smithi064 2024-02-16T22:56:38.869 INFO:tasks.workunit.client.0.smithi064.stdout:7/836: symlink d5/d33/d53/df2/d54/l10e 0 2024-02-16T22:56:38.869 INFO:tasks.workunit.client.0.smithi064.stdout:6/873: rmdir d0/d5 39 2024-02-16T22:56:38.869 INFO:tasks.workunit.client.0.smithi064.stdout:1/740: symlink d2/lf9 0 2024-02-16T22:56:38.870 INFO:tasks.workunit.client.0.smithi064.stdout:3/922: creat d4/d1f/d5b/f136 x:0 0 0 2024-02-16T22:56:38.873 INFO:tasks.workunit.client.0.smithi064.stdout:3/923: mknod d4/d13/d1b/d27/d99/c137 0 2024-02-16T22:56:38.875 INFO:tasks.workunit.client.0.smithi064.stdout:7/837: read d5/d33/d53/df2/d54/d41/f4b [2637235,69699] 0 2024-02-16T22:56:38.880 INFO:tasks.workunit.client.0.smithi064.stdout:7/838: fdatasync d5/d33/d45/d64/f85 0 2024-02-16T22:56:38.881 INFO:tasks.workunit.client.0.smithi064.stdout:7/839: stat d5/d33/c77 0 2024-02-16T22:56:38.882 INFO:tasks.workunit.client.0.smithi064.stdout:3/924: link d4/d1f/ld4 d4/d13/d1b/d4f/l138 0 2024-02-16T22:56:38.883 INFO:tasks.workunit.client.0.smithi064.stdout:3/925: chown d4/d13/d1b/d27/d61/la9 5 1 2024-02-16T22:56:38.884 INFO:tasks.workunit.client.0.smithi064.stdout:7/840: truncate d5/d33/f8a 134650 0 2024-02-16T22:56:38.885 INFO:tasks.workunit.client.0.smithi064.stdout:6/874: dread d0/d5/f27 [0,4194304] 0 2024-02-16T22:56:38.885 INFO:tasks.workunit.client.0.smithi064.stdout:3/926: dread - d4/d13/d1b/d27/d82/db0/fcd zero size 2024-02-16T22:56:38.886 INFO:tasks.workunit.client.1.smithi192.stdout:8/998: dread d5/d26/d83/fb9 [0,4194304] 0 2024-02-16T22:56:38.886 INFO:tasks.workunit.client.0.smithi064.stdout:3/927: stat d4/cb 0 2024-02-16T22:56:38.886 INFO:tasks.workunit.client.0.smithi064.stdout:6/875: chown d0/d3c/dc5/f102 587926 1 2024-02-16T22:56:38.887 INFO:tasks.workunit.client.0.smithi064.stdout:3/928: readlink d4/d13/d1b/d27/d81/l90 0 2024-02-16T22:56:38.888 INFO:tasks.workunit.client.0.smithi064.stdout:7/841: creat d5/d33/d42/f10f x:0 0 0 2024-02-16T22:56:38.888 INFO:tasks.workunit.client.0.smithi064.stdout:7/842: fdatasync d5/d33/d42/fed 0 2024-02-16T22:56:38.889 INFO:tasks.workunit.client.1.smithi192.stdout:3/958: dwrite da/fcc [4194304,4194304] 0 2024-02-16T22:56:38.894 INFO:tasks.workunit.client.1.smithi192.stdout:8/999: dwrite d5/db/ff8 [8388608,4194304] 0 2024-02-16T22:56:38.901 INFO:tasks.workunit.client.1.smithi192.stdout:3/959: creat da/d11/d63/d6b/f145 x:0 0 0 2024-02-16T22:56:38.902 INFO:tasks.workunit.client.0.smithi064.stdout:7/843: getdents d5/d33/d53/df2/d65/d7f/db2 0 2024-02-16T22:56:38.908 INFO:tasks.workunit.client.0.smithi064.stdout:7/844: mkdir d5/d33/d42/d110 0 2024-02-16T22:56:38.908 INFO:tasks.workunit.client.0.smithi064.stdout:7/845: creat d5/d7d/df0/f111 x:0 0 0 2024-02-16T22:56:38.909 INFO:tasks.workunit.client.0.smithi064.stdout:6/876: write d0/d6/d10/d33/da5/faa [794057,76796] 0 2024-02-16T22:56:38.911 INFO:tasks.workunit.client.1.smithi192.stdout:1/999: sync 2024-02-16T22:56:38.912 INFO:tasks.workunit.client.0.smithi064.stdout:7/846: rename d5/d33/d45/d64/f9e to d5/f112 0 2024-02-16T22:56:38.913 INFO:tasks.workunit.client.0.smithi064.stdout:6/877: creat d0/d5/df/d17/f120 x:0 0 0 2024-02-16T22:56:38.913 INFO:tasks.workunit.client.0.smithi064.stdout:3/929: dwrite d4/f8 [4194304,4194304] 0 2024-02-16T22:56:38.914 INFO:tasks.workunit.client.0.smithi064.stdout:7/847: stat d5/d33/d45/d64/d56/d66/fe0 0 2024-02-16T22:56:38.914 INFO:tasks.workunit.client.0.smithi064.stdout:6/878: chown d0/d6/da/da2 16134028 1 2024-02-16T22:56:38.915 INFO:tasks.workunit.client.0.smithi064.stdout:7/848: chown d5/d7d/df0/f111 17876542 1 2024-02-16T22:56:38.916 INFO:tasks.workunit.client.0.smithi064.stdout:6/879: mknod d0/d1d/dbc/c121 0 2024-02-16T22:56:38.916 INFO:tasks.workunit.client.0.smithi064.stdout:6/880: chown d0/d1b/fc8 1902216 1 2024-02-16T22:56:38.918 INFO:tasks.workunit.client.0.smithi064.stdout:7/849: mkdir d5/d33/d53/da3/d113 0 2024-02-16T22:56:38.919 INFO:tasks.workunit.client.0.smithi064.stdout:6/881: creat d0/d5/df/d17/da1/f122 x:0 0 0 2024-02-16T22:56:38.920 INFO:tasks.workunit.client.0.smithi064.stdout:6/882: dread - d0/d1d/dbc/dec/f10b zero size 2024-02-16T22:56:38.920 INFO:tasks.workunit.client.0.smithi064.stdout:6/883: stat d0/d5/f27 0 2024-02-16T22:56:38.921 INFO:tasks.workunit.client.0.smithi064.stdout:7/850: symlink d5/d33/d53/dfe/l114 0 2024-02-16T22:56:38.925 INFO:tasks.workunit.client.0.smithi064.stdout:7/851: creat d5/d33/d53/f115 x:0 0 0 2024-02-16T22:56:38.926 INFO:tasks.workunit.client.0.smithi064.stdout:3/930: write d4/d13/d4d/d5c/ff4 [544156,46209] 0 2024-02-16T22:56:38.927 INFO:tasks.workunit.client.0.smithi064.stdout:7/852: chown d5/d33/d53/df2/l37 127427037 1 2024-02-16T22:56:38.929 INFO:tasks.workunit.client.0.smithi064.stdout:3/931: mkdir d4/d13/d4d/d139 0 2024-02-16T22:56:38.931 INFO:tasks.workunit.client.0.smithi064.stdout:7/853: read d5/d33/d53/df2/f30 [6111760,71692] 0 2024-02-16T22:56:38.932 INFO:tasks.workunit.client.0.smithi064.stdout:6/884: write d0/d12/f8b [177998,88249] 0 2024-02-16T22:56:38.933 INFO:tasks.workunit.client.0.smithi064.stdout:6/885: fdatasync d0/d6/d10/d33/da4/f115 0 2024-02-16T22:56:38.936 INFO:tasks.workunit.client.0.smithi064.stdout:3/932: getdents d4/d13/d1b/d27/d99/d7d 0 2024-02-16T22:56:38.937 INFO:tasks.workunit.client.0.smithi064.stdout:3/933: dread - d4/d13/d1b/d27/d61/d63/ffb zero size 2024-02-16T22:56:38.937 INFO:tasks.workunit.client.0.smithi064.stdout:6/886: mknod d0/d5/c123 0 2024-02-16T22:56:38.938 INFO:tasks.workunit.client.0.smithi064.stdout:3/934: chown d4/d13/d1b/f1c 5066624 1 2024-02-16T22:56:38.941 INFO:tasks.workunit.client.0.smithi064.stdout:3/935: symlink d4/d1f/d5b/d51/d7c/l13a 0 2024-02-16T22:56:38.942 INFO:tasks.workunit.client.0.smithi064.stdout:6/887: write d0/d1d/dbc/fe4 [557886,122935] 0 2024-02-16T22:56:38.943 INFO:tasks.workunit.client.0.smithi064.stdout:7/854: dwrite d5/d33/d53/da3/fb7 [0,4194304] 0 2024-02-16T22:56:38.945 INFO:tasks.workunit.client.0.smithi064.stdout:7/855: write d5/d33/d53/f115 [996695,79766] 0 2024-02-16T22:56:38.950 INFO:tasks.workunit.client.0.smithi064.stdout:7/856: read d5/d33/d53/df2/d54/f78 [563706,129728] 0 2024-02-16T22:56:38.950 INFO:tasks.workunit.client.0.smithi064.stdout:6/888: read - d0/d1d/da8/d60/f82 zero size 2024-02-16T22:56:38.953 INFO:tasks.workunit.client.0.smithi064.stdout:3/936: dwrite d4/d13/d1b/d27/d82/db0/f131 [0,4194304] 0 2024-02-16T22:56:38.956 INFO:tasks.workunit.client.0.smithi064.stdout:7/857: dread d5/d33/d53/df2/d65/d7f/f8d [0,4194304] 0 2024-02-16T22:56:38.960 INFO:tasks.workunit.client.0.smithi064.stdout:7/858: creat d5/df9/de4/f116 x:0 0 0 2024-02-16T22:56:38.963 INFO:tasks.workunit.client.0.smithi064.stdout:3/937: link d4/d1f/ld4 d4/d14/d2b/d10b/l13b 0 2024-02-16T22:56:38.964 INFO:tasks.workunit.client.0.smithi064.stdout:6/889: dwrite d0/d3c/fef [0,4194304] 0 2024-02-16T22:56:38.967 INFO:tasks.workunit.client.0.smithi064.stdout:3/938: mkdir d4/d13/d1b/d27/d82/d13c 0 2024-02-16T22:56:38.971 INFO:tasks.workunit.client.0.smithi064.stdout:6/890: getdents d0/d1d/d21/d98 0 2024-02-16T22:56:38.973 INFO:tasks.workunit.client.0.smithi064.stdout:6/891: creat d0/d5/df/d17/d18/f124 x:0 0 0 2024-02-16T22:56:38.974 INFO:tasks.workunit.client.0.smithi064.stdout:6/892: dread - d0/d85/ff4 zero size 2024-02-16T22:56:38.974 INFO:tasks.workunit.client.1.smithi192.stdout:3/960: sync 2024-02-16T22:56:38.975 INFO:tasks.workunit.client.1.smithi192.stdout:3/961: dread - da/d11/d20/d39/d53/dc7/d11b/f142 zero size 2024-02-16T22:56:38.976 INFO:tasks.workunit.client.0.smithi064.stdout:6/893: write d0/d6/d10/f9f [951381,55107] 0 2024-02-16T22:56:38.986 INFO:tasks.workunit.client.1.smithi192.stdout:3/962: dwrite da/d11/fd9 [0,4194304] 0 2024-02-16T22:56:38.986 INFO:tasks.workunit.client.0.smithi064.stdout:7/859: dwrite d5/d33/f8a [0,4194304] 0 2024-02-16T22:56:38.990 INFO:tasks.workunit.client.0.smithi064.stdout:3/939: dwrite d4/d13/fad [0,4194304] 0 2024-02-16T22:56:38.995 INFO:tasks.workunit.client.0.smithi064.stdout:6/894: dwrite d0/d89/f8e [0,4194304] 0 2024-02-16T22:56:38.998 INFO:tasks.workunit.client.0.smithi064.stdout:7/860: creat d5/d33/d45/d64/f117 x:0 0 0 2024-02-16T22:56:39.000 INFO:tasks.workunit.client.1.smithi192.stdout:3/963: dread da/d11/d63/d6b/d4e/fa5 [0,4194304] 0 2024-02-16T22:56:39.000 INFO:tasks.workunit.client.0.smithi064.stdout:2/989: sync 2024-02-16T22:56:39.000 INFO:tasks.workunit.client.0.smithi064.stdout:9/847: sync 2024-02-16T22:56:39.003 INFO:tasks.workunit.client.0.smithi064.stdout:3/940: symlink d4/d13/d1b/d27/d82/l13d 0 2024-02-16T22:56:39.004 INFO:tasks.workunit.client.0.smithi064.stdout:6/895: rename d0/d6/d10/d33/da5/c119 to d0/d12/c125 0 2024-02-16T22:56:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:9/848: readlink d3/d9/d1e/d42/l99 0 2024-02-16T22:56:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:2/990: symlink d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/d121/l14d 0 2024-02-16T22:56:39.009 INFO:tasks.workunit.client.1.smithi192.stdout:3/964: dwrite da/d11/d63/d6b/ff7 [0,4194304] 0 2024-02-16T22:56:39.033 INFO:tasks.workunit.client.0.smithi064.stdout:6/896: mknod d0/d6/d10/d33/c126 0 2024-02-16T22:56:39.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:38 smithi192 ceph-mon[36254]: pgmap v24: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 126 MiB/s rd, 279 MiB/s wr, 629 op/s 2024-02-16T22:56:39.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:38 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:39.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:38 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:39.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:38 smithi192 ceph-mon[36254]: Reconfiguring alertmanager.smithi064 (dependencies changed)... 2024-02-16T22:56:39.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:38 smithi192 ceph-mon[36254]: Reconfiguring daemon alertmanager.smithi064 on smithi064 2024-02-16T22:56:39.037 INFO:tasks.workunit.client.0.smithi064.stdout:6/897: dread - d0/d6/d10/d33/da5/fc1 zero size 2024-02-16T22:56:39.039 INFO:tasks.workunit.client.0.smithi064.stdout:9/849: dwrite d3/d9/d6a/db1/d2a/fed [0,4194304] 0 2024-02-16T22:56:39.039 INFO:tasks.workunit.client.0.smithi064.stdout:6/898: stat d0/d5/df/d17/d18/l52 0 2024-02-16T22:56:39.115 INFO:tasks.workunit.client.0.smithi064.stdout:2/991: rename d7/d14/d1e/d34/d73/l9b to d7/d14/d25/d46/d55/d57/dbe/d10e/d126/l14e 0 2024-02-16T22:56:39.119 INFO:tasks.workunit.client.0.smithi064.stdout:3/941: write d4/d13/d1b/fcc [1035464,6899] 0 2024-02-16T22:56:39.121 INFO:tasks.workunit.client.1.smithi192.stdout:3/965: write da/f12 [1660963,14062] 0 2024-02-16T22:56:39.122 INFO:tasks.workunit.client.1.smithi192.stdout:3/966: chown da/df/c117 5 1 2024-02-16T22:56:39.125 INFO:tasks.workunit.client.1.smithi192.stdout:3/967: rmdir da/d11/d20/d39/d53/dc7 39 2024-02-16T22:56:39.128 INFO:tasks.workunit.client.1.smithi192.stdout:3/968: symlink da/d11/d63/d6b/l146 0 2024-02-16T22:56:39.129 INFO:tasks.workunit.client.1.smithi192.stdout:3/969: chown da/d11/d35/c4f 4054 1 2024-02-16T22:56:39.137 INFO:tasks.workunit.client.1.smithi192.stdout:3/970: creat da/d11/d20/f147 x:0 0 0 2024-02-16T22:56:39.148 INFO:tasks.workunit.client.0.smithi064.stdout:2/992: symlink d7/d14/d25/d46/d55/d7f/dde/l14f 0 2024-02-16T22:56:39.149 INFO:tasks.workunit.client.0.smithi064.stdout:2/993: fsync d7/d14/d1e/f81 0 2024-02-16T22:56:39.152 INFO:tasks.workunit.client.0.smithi064.stdout:3/942: mknod d4/d1f/d5b/d128/c13e 0 2024-02-16T22:56:39.153 INFO:tasks.workunit.client.0.smithi064.stdout:9/850: link d3/d9/d6a/db1/d2a/d2e/d31/d95/lf2 d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/d110/l11b 0 2024-02-16T22:56:39.154 INFO:tasks.workunit.client.0.smithi064.stdout:2/994: mkdir d7/d14/d25/d11c/d150 0 2024-02-16T22:56:39.155 INFO:tasks.workunit.client.0.smithi064.stdout:2/995: fsync d7/d14/d1e/dfd/f139 0 2024-02-16T22:56:39.162 INFO:tasks.workunit.client.0.smithi064.stdout:2/996: dread d7/d14/d1e/d34/d3f/d9a/dd9/d95/d44/f49 [0,4194304] 0 2024-02-16T22:56:39.164 INFO:tasks.workunit.client.0.smithi064.stdout:2/997: dread - d7/d14/d1e/fe3 zero size 2024-02-16T22:56:39.166 INFO:tasks.workunit.client.0.smithi064.stdout:3/943: dread d4/d13/d1b/d27/d99/d7d/fc8 [4194304,4194304] 0 2024-02-16T22:56:39.167 INFO:tasks.workunit.client.0.smithi064.stdout:2/998: dread - d7/d14/d1e/d34/d3f/d9a/dd9/d60/fca zero size 2024-02-16T22:56:39.168 INFO:tasks.workunit.client.0.smithi064.stdout:3/944: readlink d4/d1f/d5b/d51/d7c/l13a 0 2024-02-16T22:56:39.169 INFO:tasks.workunit.client.0.smithi064.stdout:6/899: write d0/d6/d10/f11b [1833782,59845] 0 2024-02-16T22:56:39.170 INFO:tasks.workunit.client.0.smithi064.stdout:9/851: dread d3/d9/f12 [0,4194304] 0 2024-02-16T22:56:39.173 INFO:tasks.workunit.client.0.smithi064.stdout:2/999: symlink d7/d14/d25/d46/d55/d57/dbe/d10e/d126/l151 0 2024-02-16T22:56:39.174 INFO:tasks.workunit.client.0.smithi064.stdout:9/852: rmdir d3/d11/d15/d23 39 2024-02-16T22:56:39.181 INFO:tasks.workunit.client.0.smithi064.stdout:6/900: creat d0/d6/f127 x:0 0 0 2024-02-16T22:56:39.182 INFO:tasks.workunit.client.0.smithi064.stdout:9/853: write d3/d9/fa2 [784269,102014] 0 2024-02-16T22:56:39.182 INFO:tasks.workunit.client.0.smithi064.stdout:3/945: link d4/d13/d1b/d27/d44/l11a d4/d13/d1b/d27/d61/l13f 0 2024-02-16T22:56:39.184 INFO:tasks.workunit.client.0.smithi064.stdout:9/854: mkdir d3/d89/d9a/dcf/d11c 0 2024-02-16T22:56:39.188 INFO:tasks.workunit.client.0.smithi064.stdout:6/901: link d0/d1d/da8/fdb d0/d1d/d21/f128 0 2024-02-16T22:56:39.191 INFO:tasks.workunit.client.0.smithi064.stdout:6/902: fsync d0/d5/f27 0 2024-02-16T22:56:39.193 INFO:tasks.workunit.client.0.smithi064.stdout:3/946: rename d4/d13/d1b/d27/d61/l13f to d4/d14/d2b/d10b/l140 0 2024-02-16T22:56:39.194 INFO:tasks.workunit.client.0.smithi064.stdout:3/947: stat d4/d13/d1b/d27/d81/l90 0 2024-02-16T22:56:39.195 INFO:tasks.workunit.client.0.smithi064.stdout:6/903: rmdir d0/d12/d7b/d110 39 2024-02-16T22:56:39.197 INFO:tasks.workunit.client.0.smithi064.stdout:3/948: chown d4/d1f/l12b 25364180 1 2024-02-16T22:56:39.197 INFO:tasks.workunit.client.0.smithi064.stdout:3/949: stat d4/d13/d4d/cbd 0 2024-02-16T22:56:39.198 INFO:tasks.workunit.client.0.smithi064.stdout:6/904: fdatasync d0/d6/d10/f117 0 2024-02-16T22:56:39.199 INFO:tasks.workunit.client.0.smithi064.stdout:9/855: dread d3/d9/d6a/db1/f27 [0,4194304] 0 2024-02-16T22:56:39.201 INFO:tasks.workunit.client.0.smithi064.stdout:3/950: read - d4/d13/d1b/d27/d44/fec zero size 2024-02-16T22:56:39.201 INFO:tasks.workunit.client.0.smithi064.stdout:3/951: dread - d4/f42 zero size 2024-02-16T22:56:39.203 INFO:tasks.workunit.client.0.smithi064.stdout:6/905: mknod d0/d1d/db1/c129 0 2024-02-16T22:56:39.203 INFO:tasks.workunit.client.0.smithi064.stdout:6/906: chown d0/d5/df/d17/da1 3428 1 2024-02-16T22:56:39.204 INFO:tasks.workunit.client.0.smithi064.stdout:6/907: chown d0/d1d/dbc/dec/ffe 2525600 1 2024-02-16T22:56:39.205 INFO:tasks.workunit.client.0.smithi064.stdout:9/856: fdatasync d3/d9/d6a/db1/d2a/d2e/f59 0 2024-02-16T22:56:39.206 INFO:tasks.workunit.client.0.smithi064.stdout:6/908: mknod d0/d5/df/d17/c12a 0 2024-02-16T22:56:39.207 INFO:tasks.workunit.client.0.smithi064.stdout:9/857: truncate d3/d9/d1e/fe1 188774 0 2024-02-16T22:56:39.211 INFO:tasks.workunit.client.0.smithi064.stdout:6/909: write d0/d6/d10/f117 [343456,2217] 0 2024-02-16T22:56:39.213 INFO:tasks.workunit.client.0.smithi064.stdout:6/910: rename d0/d12/le1 to d0/d3c/l12b 0 2024-02-16T22:56:39.215 INFO:tasks.workunit.client.0.smithi064.stdout:6/911: symlink d0/d6/d10/d33/da4/l12c 0 2024-02-16T22:56:39.217 INFO:tasks.workunit.client.0.smithi064.stdout:6/912: rmdir d0/d6/d10/d105/db9/d118 0 2024-02-16T22:56:39.218 INFO:tasks.workunit.client.1.smithi192.stdout:3/971: sync 2024-02-16T22:56:39.219 INFO:tasks.workunit.client.1.smithi192.stdout:3/972: mknod da/df/d140/c148 0 2024-02-16T22:56:39.222 INFO:tasks.workunit.client.1.smithi192.stdout:3/973: link da/d11/d20/f4a da/d11/d20/d39/d53/f149 0 2024-02-16T22:56:39.223 INFO:tasks.workunit.client.1.smithi192.stdout:3/974: chown da/d6a/ldb 4978383 1 2024-02-16T22:56:39.232 INFO:tasks.workunit.client.1.smithi192.stdout:3/975: write da/d11/d63/d6b/f47 [842472,128942] 0 2024-02-16T22:56:39.234 INFO:tasks.workunit.client.1.smithi192.stdout:3/976: mknod da/d11/c14a 0 2024-02-16T22:56:39.238 INFO:tasks.workunit.client.1.smithi192.stdout:3/977: rename da/d11/d35/ld5 to da/d11/d20/d39/d53/l14b 0 2024-02-16T22:56:39.239 INFO:tasks.workunit.client.1.smithi192.stdout:3/978: mknod da/d11/d2b/ddc/c14c 0 2024-02-16T22:56:39.239 INFO:tasks.workunit.client.1.smithi192.stdout:3/979: chown da/d11/d20/d39/d13f 178 1 2024-02-16T22:56:39.255 INFO:tasks.workunit.client.1.smithi192.stdout:3/980: dread da/d11/d20/d39/d53/dc7/f143 [0,4194304] 0 2024-02-16T22:56:39.257 INFO:tasks.workunit.client.1.smithi192.stdout:3/981: mkdir da/d11/d20/d14d 0 2024-02-16T22:56:39.266 INFO:tasks.workunit.client.0.smithi064.stdout:0/688: sync 2024-02-16T22:56:39.270 INFO:tasks.workunit.client.0.smithi064.stdout:0/689: link da/dc/de/d12/d2c/d2e/d38/c74 da/dc/d3b/d5a/d8f/da8/db7/ce6 0 2024-02-16T22:56:39.271 INFO:tasks.workunit.client.0.smithi064.stdout:0/690: creat da/dc/de/d12/d2c/d2e/d38/fe7 x:0 0 0 2024-02-16T22:56:39.273 INFO:tasks.workunit.client.0.smithi064.stdout:0/691: mknod da/dc/de/d12/d2c/d2e/d38/dcf/ce8 0 2024-02-16T22:56:39.322 INFO:tasks.workunit.client.0.smithi064.stdout:0/692: dwrite da/dc/de/d12/f29 [4194304,4194304] 0 2024-02-16T22:56:39.326 INFO:tasks.workunit.client.1.smithi192.stdout:3/982: sync 2024-02-16T22:56:39.330 INFO:tasks.workunit.client.1.smithi192.stdout:3/983: dread da/d11/d20/d39/d53/f62 [0,4194304] 0 2024-02-16T22:56:39.330 INFO:tasks.workunit.client.0.smithi064.stdout:0/693: dread da/dc/de/d12/f49 [0,4194304] 0 2024-02-16T22:56:39.332 INFO:tasks.workunit.client.1.smithi192.stdout:3/984: mkdir da/d11/d20/d39/dca/d14e 0 2024-02-16T22:56:39.333 INFO:tasks.workunit.client.0.smithi064.stdout:0/694: fdatasync da/dc/de/d12/d2c/d2e/d38/d87/f9f 0 2024-02-16T22:56:39.337 INFO:tasks.workunit.client.0.smithi064.stdout:0/695: write da/d26/d3f/f75 [1763260,103569] 0 2024-02-16T22:56:39.345 INFO:tasks.workunit.client.0.smithi064.stdout:0/696: dwrite da/d26/f32 [0,4194304] 0 2024-02-16T22:56:39.346 INFO:tasks.workunit.client.0.smithi064.stdout:0/697: read da/d71/f7d [1064232,94191] 0 2024-02-16T22:56:39.347 INFO:tasks.workunit.client.1.smithi192.stdout:3/985: dread da/d11/d63/d6b/d80/fd0 [0,4194304] 0 2024-02-16T22:56:39.350 INFO:tasks.workunit.client.1.smithi192.stdout:3/986: creat da/d11/d63/d6b/d4e/f14f x:0 0 0 2024-02-16T22:56:39.352 INFO:tasks.workunit.client.1.smithi192.stdout:3/987: symlink da/d11/d63/d6b/d4e/d11a/l150 0 2024-02-16T22:56:39.353 INFO:tasks.workunit.client.0.smithi064.stdout:0/698: unlink da/dc/de/d12/d2c/d2e/d38/f67 0 2024-02-16T22:56:39.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/988: stat da/d11/d35/l120 0 2024-02-16T22:56:39.357 INFO:tasks.workunit.client.1.smithi192.stdout:3/989: getdents da/d11/dd4/deb/da1/db9 0 2024-02-16T22:56:39.357 INFO:tasks.workunit.client.0.smithi064.stdout:0/699: dread da/dc/d3b/f5f [0,4194304] 0 2024-02-16T22:56:39.358 INFO:tasks.workunit.client.1.smithi192.stdout:3/990: creat da/d107/f151 x:0 0 0 2024-02-16T22:56:39.403 INFO:tasks.workunit.client.0.smithi064.stdout:0/700: fsync da/dc/d3b/d72/d7c/fcb 0 2024-02-16T22:56:39.404 INFO:tasks.workunit.client.1.smithi192.stdout:3/991: readlink da/d11/dd4/deb/la9 0 2024-02-16T22:56:39.405 INFO:tasks.workunit.client.0.smithi064.stdout:5/780: sync 2024-02-16T22:56:39.408 INFO:tasks.workunit.client.0.smithi064.stdout:5/781: creat d6/d25/df1/d91/da5/de3/d43/d7d/de2/f105 x:0 0 0 2024-02-16T22:56:39.415 INFO:tasks.workunit.client.0.smithi064.stdout:5/782: dread d6/d25/df1/f7f [0,4194304] 0 2024-02-16T22:56:39.416 INFO:tasks.workunit.client.0.smithi064.stdout:5/783: write d6/fb1 [1145366,112207] 0 2024-02-16T22:56:39.417 INFO:tasks.workunit.client.0.smithi064.stdout:0/701: dread da/dc/de/d12/d2c/d2e/d38/d7e/fa3 [0,4194304] 0 2024-02-16T22:56:39.420 INFO:tasks.workunit.client.0.smithi064.stdout:5/784: mknod d6/d25/df1/d91/da5/de3/d43/db9/d103/c106 0 2024-02-16T22:56:39.421 INFO:tasks.workunit.client.0.smithi064.stdout:0/702: truncate da/dc/d3b/d72/d7c/fa5 228811 0 2024-02-16T22:56:39.421 INFO:tasks.workunit.client.0.smithi064.stdout:5/785: write d6/d25/df1/d91/da5/de3/fd7 [8999602,15679] 0 2024-02-16T22:56:39.422 INFO:tasks.workunit.client.1.smithi192.stdout:3/992: dread da/d11/d20/fab [0,4194304] 0 2024-02-16T22:56:39.423 INFO:tasks.workunit.client.0.smithi064.stdout:0/703: mkdir da/dc/de/d37/daa/de9 0 2024-02-16T22:56:39.424 INFO:tasks.workunit.client.0.smithi064.stdout:5/786: dread d6/d25/df1/d91/fc4 [0,4194304] 0 2024-02-16T22:56:39.426 INFO:tasks.workunit.client.1.smithi192.stdout:3/993: getdents da/d11/d35 0 2024-02-16T22:56:39.429 INFO:tasks.workunit.client.0.smithi064.stdout:0/704: rename da/f5c to da/dc/d3b/fea 0 2024-02-16T22:56:39.430 INFO:tasks.workunit.client.0.smithi064.stdout:5/787: rmdir d6/d25/df1/d91/da5/de3/d43/d7d/de2 39 2024-02-16T22:56:39.434 INFO:tasks.workunit.client.0.smithi064.stdout:0/705: link da/dc/de/d12/d2c/d2e/f36 da/dc/d3b/d5a/d8f/da8/db7/feb 0 2024-02-16T22:56:39.435 INFO:tasks.workunit.client.0.smithi064.stdout:5/788: link d6/d25/d48/f98 d6/d25/da4/dc1/f107 0 2024-02-16T22:56:39.436 INFO:tasks.workunit.client.0.smithi064.stdout:0/706: symlink da/d71/da4/lec 0 2024-02-16T22:56:39.448 INFO:tasks.workunit.client.0.smithi064.stdout:5/789: dwrite d6/d25/df1/d91/da5/de3/d43/d7d/faa [0,4194304] 0 2024-02-16T22:56:39.452 INFO:tasks.workunit.client.0.smithi064.stdout:0/707: dread da/dc/de/d12/fc7 [0,4194304] 0 2024-02-16T22:56:39.453 INFO:tasks.workunit.client.0.smithi064.stdout:5/790: rmdir d6/d25/df2 0 2024-02-16T22:56:39.455 INFO:tasks.workunit.client.0.smithi064.stdout:0/708: read da/dc/de/d12/d2c/f92 [8265960,7392] 0 2024-02-16T22:56:39.465 INFO:tasks.workunit.client.0.smithi064.stdout:5/791: write d6/d25/df1/d80/fe0 [188395,78980] 0 2024-02-16T22:56:39.468 INFO:tasks.workunit.client.0.smithi064.stdout:5/792: symlink d6/d18/dc5/l108 0 2024-02-16T22:56:39.469 INFO:tasks.workunit.client.0.smithi064.stdout:0/709: dwrite da/d26/f66 [0,4194304] 0 2024-02-16T22:56:39.471 INFO:tasks.workunit.client.0.smithi064.stdout:0/710: write da/dc/d3b/d72/db3/fda [1201999,112317] 0 2024-02-16T22:56:39.474 INFO:tasks.workunit.client.0.smithi064.stdout:0/711: creat da/dc/d3b/d72/d7c/fed x:0 0 0 2024-02-16T22:56:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:5/793: dwrite d6/d25/df1/d91/da5/fbd [0,4194304] 0 2024-02-16T22:56:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:5/794: write d6/d18/ff7 [255027,101208] 0 2024-02-16T22:56:39.482 INFO:tasks.workunit.client.0.smithi064.stdout:5/795: getdents d6/d25/da4/dc1/d8e 0 2024-02-16T22:56:39.485 INFO:tasks.workunit.client.0.smithi064.stdout:5/796: creat d6/d25/da4/dc1/dd4/f109 x:0 0 0 2024-02-16T22:56:39.486 INFO:tasks.workunit.client.1.smithi192.stdout:3/994: sync 2024-02-16T22:56:39.488 INFO:tasks.workunit.client.1.smithi192.stdout:3/995: mkdir da/d12a/d152 0 2024-02-16T22:56:39.488 INFO:tasks.workunit.client.1.smithi192.stdout:3/996: fdatasync da/d11/d2b/ddc/f109 0 2024-02-16T22:56:39.491 INFO:tasks.workunit.client.0.smithi064.stdout:5/797: getdents d6/d25/df1/d91 0 2024-02-16T22:56:39.491 INFO:tasks.workunit.client.0.smithi064.stdout:0/712: dwrite da/dc/de/d12/d1d/f3d [0,4194304] 0 2024-02-16T22:56:39.497 INFO:tasks.workunit.client.0.smithi064.stdout:5/798: rename d6/d18/d95/fea to d6/d25/df1/d91/da5/de3/d4b/f10a 0 2024-02-16T22:56:39.506 INFO:tasks.workunit.client.0.smithi064.stdout:5/799: dwrite d6/d18/fee [0,4194304] 0 2024-02-16T22:56:39.506 INFO:tasks.workunit.client.0.smithi064.stdout:0/713: dwrite da/d26/fb9 [0,4194304] 0 2024-02-16T22:56:39.512 INFO:tasks.workunit.client.0.smithi064.stdout:0/714: creat da/dc/d3b/d72/db3/fee x:0 0 0 2024-02-16T22:56:39.515 INFO:tasks.workunit.client.0.smithi064.stdout:0/715: mknod da/dc/de/d37/cef 0 2024-02-16T22:56:39.517 INFO:tasks.workunit.client.1.smithi192.stdout:3/997: dread da/d11/f38 [0,4194304] 0 2024-02-16T22:56:39.518 INFO:tasks.workunit.client.1.smithi192.stdout:3/998: dread da/df/f49 [0,4194304] 0 2024-02-16T22:56:39.519 INFO:tasks.workunit.client.0.smithi064.stdout:5/800: truncate d6/d18/d90/fc2 197892 0 2024-02-16T22:56:39.520 INFO:tasks.workunit.client.0.smithi064.stdout:0/716: unlink da/dc/d3b/fea 0 2024-02-16T22:56:39.521 INFO:tasks.workunit.client.1.smithi192.stdout:3/999: rmdir da/d6a 39 2024-02-16T22:56:39.522 INFO:tasks.workunit.client.0.smithi064.stdout:5/801: rename d6/d25/df1/d91/da5/de3/f59 to d6/d25/d48/f10b 0 2024-02-16T22:56:39.523 INFO:tasks.workunit.client.0.smithi064.stdout:5/802: write d6/d18/d19/f2e [1721304,88760] 0 2024-02-16T22:56:39.524 INFO:tasks.workunit.client.1.smithi192.stderr:+ rm -rf -- ./tmp.elFSIvtpwP 2024-02-16T22:56:39.525 INFO:tasks.workunit.client.0.smithi064.stdout:5/803: write d6/d25/df1/d80/fda [423584,23553] 0 2024-02-16T22:56:39.526 INFO:tasks.workunit.client.0.smithi064.stdout:5/804: write d6/d18/d19/f1f [2495193,83630] 0 2024-02-16T22:56:39.533 INFO:tasks.workunit.client.0.smithi064.stdout:4/857: sync 2024-02-16T22:56:39.540 INFO:tasks.workunit.client.0.smithi064.stdout:4/858: symlink dc/dd/d32/d46/d5a/d7e/d9a/df8/d10d/l10f 0 2024-02-16T22:56:39.544 INFO:tasks.workunit.client.0.smithi064.stdout:4/859: mkdir dc/ddc/de8/d110 0 2024-02-16T22:56:39.545 INFO:tasks.workunit.client.0.smithi064.stdout:0/717: dwrite da/dc/de/d37/fa6 [0,4194304] 0 2024-02-16T22:56:39.548 INFO:tasks.workunit.client.0.smithi064.stdout:5/805: creat d6/d25/df1/d91/da5/de3/f10c x:0 0 0 2024-02-16T22:56:39.552 INFO:tasks.workunit.client.0.smithi064.stdout:5/806: chown d6/d25/df1/d91/da5/de3/l40 9315362 1 2024-02-16T22:56:39.554 INFO:tasks.workunit.client.0.smithi064.stdout:0/718: creat da/dc/d3b/ff0 x:0 0 0 2024-02-16T22:56:39.555 INFO:tasks.workunit.client.0.smithi064.stdout:0/719: stat da/d26/f66 0 2024-02-16T22:56:39.555 INFO:tasks.workunit.client.0.smithi064.stdout:5/807: read - d6/d25/da4/dc1/fc0 zero size 2024-02-16T22:56:39.555 INFO:tasks.workunit.client.0.smithi064.stdout:4/860: dread dc/de/d28/d57/d77/f107 [0,4194304] 0 2024-02-16T22:56:39.556 INFO:tasks.workunit.client.0.smithi064.stdout:5/808: chown d6/d25/d48/c70 493 1 2024-02-16T22:56:39.557 INFO:tasks.workunit.client.0.smithi064.stdout:0/720: write da/dc/de/d37/fa6 [5074708,19626] 0 2024-02-16T22:56:39.560 INFO:tasks.workunit.client.0.smithi064.stdout:4/861: rename dc/de/dfb/dff/de3/faf to dc/ddc/de8/d110/f111 0 2024-02-16T22:56:39.562 INFO:tasks.workunit.client.0.smithi064.stdout:5/809: truncate d6/d25/df1/f8f 95450 0 2024-02-16T22:56:39.569 INFO:tasks.workunit.client.0.smithi064.stdout:0/721: creat da/ff1 x:0 0 0 2024-02-16T22:56:39.569 INFO:tasks.workunit.client.0.smithi064.stdout:0/722: chown da/dc/d3b/d72 1081 1 2024-02-16T22:56:39.570 INFO:tasks.workunit.client.0.smithi064.stdout:4/862: rename dc/dd/d32/d58/f8e to dc/dd/d32/d46/d5a/d7e/d9a/da2/f112 0 2024-02-16T22:56:39.571 INFO:tasks.workunit.client.0.smithi064.stdout:5/810: truncate d6/d25/df1/d91/da5/de3/f53 8007176 0 2024-02-16T22:56:39.578 INFO:tasks.workunit.client.0.smithi064.stdout:4/863: rename dc/de/d6d/f87 to dc/ddc/de8/d110/f113 0 2024-02-16T22:56:39.579 INFO:tasks.workunit.client.0.smithi064.stdout:0/723: creat da/ff2 x:0 0 0 2024-02-16T22:56:39.583 INFO:tasks.workunit.client.0.smithi064.stdout:0/724: truncate da/dc/de/d12/f6c 276396 0 2024-02-16T22:56:39.585 INFO:tasks.workunit.client.0.smithi064.stdout:3/952: sync 2024-02-16T22:56:39.585 INFO:tasks.workunit.client.0.smithi064.stdout:1/741: sync 2024-02-16T22:56:39.585 INFO:tasks.workunit.client.0.smithi064.stdout:9/858: sync 2024-02-16T22:56:39.585 INFO:tasks.workunit.client.0.smithi064.stdout:7/861: sync 2024-02-16T22:56:39.585 INFO:tasks.workunit.client.0.smithi064.stdout:6/913: sync 2024-02-16T22:56:39.585 INFO:tasks.workunit.client.0.smithi064.stdout:8/763: sync 2024-02-16T22:56:39.586 INFO:tasks.workunit.client.0.smithi064.stdout:1/742: stat d2/df/d11/d25/c31 0 2024-02-16T22:56:39.588 INFO:tasks.workunit.client.0.smithi064.stdout:7/862: readlink d5/d33/d53/df2/d54/d41/l7a 0 2024-02-16T22:56:39.589 INFO:tasks.workunit.client.0.smithi064.stdout:0/725: symlink da/dc/de/d12/d2c/d2e/d38/dcf/lf3 0 2024-02-16T22:56:39.591 INFO:tasks.workunit.client.0.smithi064.stdout:3/953: chown d4/d14/d2b/d10b/l140 3114 1 2024-02-16T22:56:39.591 INFO:tasks.workunit.client.0.smithi064.stdout:5/811: dread d6/d25/d48/f4a [0,4194304] 0 2024-02-16T22:56:39.591 INFO:tasks.workunit.client.0.smithi064.stdout:9/859: write d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/f114 [751223,102254] 0 2024-02-16T22:56:39.598 INFO:tasks.workunit.client.0.smithi064.stdout:6/914: mkdir d0/d5/df/d17/ded/d12d 0 2024-02-16T22:56:39.599 INFO:tasks.workunit.client.0.smithi064.stdout:6/915: dread - d0/d6/d10/f9d zero size 2024-02-16T22:56:39.599 INFO:tasks.workunit.client.0.smithi064.stdout:7/863: dwrite d5/d33/d42/f8f [0,4194304] 0 2024-02-16T22:56:39.602 INFO:tasks.workunit.client.0.smithi064.stdout:7/864: stat d5/d33/d53/df2/d54/d7b/fb1 0 2024-02-16T22:56:39.604 INFO:tasks.workunit.client.0.smithi064.stdout:8/764: dread df/d31/d2f/fc3 [0,4194304] 0 2024-02-16T22:56:39.605 INFO:tasks.workunit.client.0.smithi064.stdout:7/865: readlink d5/d33/d53/df2/d54/d7b/ld3 0 2024-02-16T22:56:39.616 INFO:tasks.workunit.client.0.smithi064.stdout:5/812: dread - d6/d25/df1/d91/da5/de3/d43/db9/f83 zero size 2024-02-16T22:56:39.622 INFO:tasks.workunit.client.0.smithi064.stdout:1/743: truncate d2/df/d14/d18/fd1 938356 0 2024-02-16T22:56:39.622 INFO:tasks.workunit.client.0.smithi064.stdout:4/864: dwrite dc/d4a/f4d [0,4194304] 0 2024-02-16T22:56:39.624 INFO:tasks.workunit.client.0.smithi064.stdout:1/744: write d2/df/d14/d18/d48/fb8 [3088907,109738] 0 2024-02-16T22:56:39.627 INFO:tasks.workunit.client.0.smithi064.stdout:8/765: read df/d31/d2f/d3e/f9c [299564,426] 0 2024-02-16T22:56:39.636 INFO:tasks.workunit.client.0.smithi064.stdout:3/954: write d4/d13/d1b/d27/d44/d97/fdc [1026876,9350] 0 2024-02-16T22:56:39.640 INFO:tasks.workunit.client.0.smithi064.stdout:6/916: dread d0/d6/da/f6a [0,4194304] 0 2024-02-16T22:56:39.643 INFO:tasks.workunit.client.0.smithi064.stdout:9/860: dwrite d3/d9/d6a/db1/d2a/d2e/d31/fc3 [0,4194304] 0 2024-02-16T22:56:39.651 INFO:tasks.workunit.client.0.smithi064.stdout:1/745: truncate d2/df/d14/d18/d48/d6b/d75/f77 872580 0 2024-02-16T22:56:39.651 INFO:tasks.workunit.client.0.smithi064.stdout:9/861: dwrite d3/d89/d9a/fce [0,4194304] 0 2024-02-16T22:56:39.657 INFO:tasks.workunit.client.0.smithi064.stdout:0/726: dwrite da/d26/de0/d9e/dc0/fe3 [0,4194304] 0 2024-02-16T22:56:39.659 INFO:tasks.workunit.client.0.smithi064.stdout:7/866: symlink d5/d33/d45/d64/d56/l118 0 2024-02-16T22:56:39.662 INFO:tasks.workunit.client.0.smithi064.stdout:3/955: mkdir d4/d14/d2b/d141 0 2024-02-16T22:56:39.667 INFO:tasks.workunit.client.0.smithi064.stdout:6/917: symlink d0/d1d/d21/l12e 0 2024-02-16T22:56:39.784 INFO:tasks.workunit.client.0.smithi064.stdout:8/766: creat df/d31/d2f/d38/f106 x:0 0 0 2024-02-16T22:56:39.790 INFO:tasks.workunit.client.0.smithi064.stdout:5/813: link d6/d25/da4/dc1/f102 d6/d18/d90/f10d 0 2024-02-16T22:56:39.794 INFO:tasks.workunit.client.0.smithi064.stdout:3/956: link d4/d13/d1b/d27/d81/f10d d4/d13/d1b/d27/d81/f142 0 2024-02-16T22:56:39.795 INFO:tasks.workunit.client.0.smithi064.stdout:4/865: write dc/dd/d32/d46/d5a/d7e/d9a/da2/f112 [565915,41924] 0 2024-02-16T22:56:39.796 INFO:tasks.workunit.client.0.smithi064.stdout:8/767: mkdir df/d31/d2f/d38/d48/d107 0 2024-02-16T22:56:39.797 INFO:tasks.workunit.client.0.smithi064.stdout:4/866: chown dc/dd/d32/d46/d5a/dcb/cdb 3508 1 2024-02-16T22:56:39.797 INFO:tasks.workunit.client.0.smithi064.stdout:5/814: rmdir d6/d25/df1/d91/da5 39 2024-02-16T22:56:39.799 INFO:tasks.workunit.client.0.smithi064.stdout:3/957: unlink d4/d1f/d5b/d51/fa4 0 2024-02-16T22:56:39.801 INFO:tasks.workunit.client.0.smithi064.stdout:3/958: write d4/d13/d1b/f68 [4382961,73929] 0 2024-02-16T22:56:39.803 INFO:tasks.workunit.client.0.smithi064.stdout:8/768: rename df/d2a/d5c/l85 to df/d31/d8f/dc9/d73/dfb/l108 0 2024-02-16T22:56:39.805 INFO:tasks.workunit.client.0.smithi064.stdout:4/867: read - dc/dd/d32/d46/d5a/d7e/fe0 zero size 2024-02-16T22:56:39.808 INFO:tasks.workunit.client.0.smithi064.stdout:0/727: write da/dc/de/d12/f42 [359861,77222] 0 2024-02-16T22:56:39.808 INFO:tasks.workunit.client.0.smithi064.stdout:1/746: dwrite d2/df/d11/d71/f99 [0,4194304] 0 2024-02-16T22:56:39.815 INFO:tasks.workunit.client.0.smithi064.stdout:5/815: dread d6/f24 [0,4194304] 0 2024-02-16T22:56:39.815 INFO:tasks.workunit.client.0.smithi064.stdout:9/862: dwrite d3/d9/d6a/db1/d2a/d2e/f39 [0,4194304] 0 2024-02-16T22:56:39.817 INFO:tasks.workunit.client.0.smithi064.stdout:8/769: mkdir df/d31/d2f/d3e/ddf/d109 0 2024-02-16T22:56:39.818 INFO:tasks.workunit.client.0.smithi064.stdout:7/867: write d5/d33/d45/d64/f85 [3427617,76906] 0 2024-02-16T22:56:39.819 INFO:tasks.workunit.client.0.smithi064.stdout:8/770: dread - df/d31/d61/d94/ff6 zero size 2024-02-16T22:56:39.819 INFO:tasks.workunit.client.0.smithi064.stdout:3/959: chown d4/d13/d1b/d27/d61/f66 3 1 2024-02-16T22:56:39.822 INFO:tasks.workunit.client.0.smithi064.stdout:3/960: chown d4/d13/d1b/d27/d44/l11a 3926050 1 2024-02-16T22:56:39.823 INFO:tasks.workunit.client.0.smithi064.stdout:0/728: fsync da/dc/de/d12/d2c/f31 0 2024-02-16T22:56:39.829 INFO:tasks.workunit.client.0.smithi064.stdout:7/868: truncate d5/d33/d53/df2/d65/d7f/f8d 330078 0 2024-02-16T22:56:39.831 INFO:tasks.workunit.client.0.smithi064.stdout:3/961: creat d4/d13/d4d/d5c/d78/d117/f143 x:0 0 0 2024-02-16T22:56:39.842 INFO:tasks.workunit.client.0.smithi064.stdout:7/869: creat d5/d33/d53/df2/d54/d7b/dc6/f119 x:0 0 0 2024-02-16T22:56:39.845 INFO:tasks.workunit.client.0.smithi064.stdout:3/962: fdatasync d4/d13/d4d/d5c/f10c 0 2024-02-16T22:56:39.845 INFO:tasks.workunit.client.0.smithi064.stdout:8/771: dread df/d31/d61/d79/f9b [0,4194304] 0 2024-02-16T22:56:39.846 INFO:tasks.workunit.client.0.smithi064.stdout:4/868: write dc/de/d28/d57/d77/d101/faa [75557,92489] 0 2024-02-16T22:56:39.847 INFO:tasks.workunit.client.0.smithi064.stdout:8/772: chown df/d31/d2f/d38/d48/da2/de2/cfc 1 1 2024-02-16T22:56:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:7/870: creat d5/d33/d53/df2/d54/d41/f11a x:0 0 0 2024-02-16T22:56:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:9/863: write d3/d11/d9e/da5/fd6 [293539,28689] 0 2024-02-16T22:56:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:3/963: mkdir d4/d14/d2b/d10b/d144 0 2024-02-16T22:56:39.856 INFO:tasks.workunit.client.0.smithi064.stdout:4/869: dwrite f3 [0,4194304] 0 2024-02-16T22:56:39.856 INFO:tasks.workunit.client.0.smithi064.stdout:8/773: rmdir df/d31/d2f/d38/d48/db5 39 2024-02-16T22:56:39.858 INFO:tasks.workunit.client.0.smithi064.stdout:5/816: dwrite d6/f24 [0,4194304] 0 2024-02-16T22:56:39.862 INFO:tasks.workunit.client.0.smithi064.stdout:1/747: write d2/df/d14/d18/d40/f68 [408529,50810] 0 2024-02-16T22:56:39.863 INFO:tasks.workunit.client.0.smithi064.stdout:0/729: write da/dc/de/d12/d2c/d2e/d38/f7a [928890,98943] 0 2024-02-16T22:56:39.864 INFO:tasks.workunit.client.0.smithi064.stdout:0/730: readlink da/dc/de/l14 0 2024-02-16T22:56:39.865 INFO:tasks.workunit.client.0.smithi064.stdout:3/964: mkdir d4/d13/d4d/d145 0 2024-02-16T22:56:39.866 INFO:tasks.workunit.client.0.smithi064.stdout:4/870: rmdir dc/de/d28/d57 39 2024-02-16T22:56:39.872 INFO:tasks.workunit.client.0.smithi064.stdout:9/864: fsync d3/d11/d15/d23/f73 0 2024-02-16T22:56:39.880 INFO:tasks.workunit.client.0.smithi064.stdout:9/865: dwrite d3/d9/d6a/db1/d2a/fc9 [0,4194304] 0 2024-02-16T22:56:39.880 INFO:tasks.workunit.client.0.smithi064.stdout:3/965: truncate d4/d13/fa7 694729 0 2024-02-16T22:56:39.885 INFO:tasks.workunit.client.0.smithi064.stdout:9/866: write d3/f8a [4647492,100678] 0 2024-02-16T22:56:39.885 INFO:tasks.workunit.client.0.smithi064.stdout:7/871: dwrite d5/d33/d53/df2/d54/f40 [0,4194304] 0 2024-02-16T22:56:39.889 INFO:tasks.workunit.client.0.smithi064.stdout:9/867: stat d3/d9/d6a/db1/d2a/d2e/da3/dea/l10b 0 2024-02-16T22:56:39.899 INFO:tasks.workunit.client.0.smithi064.stdout:4/871: dwrite dc/de/dfb/dff/fc7 [0,4194304] 0 2024-02-16T22:56:39.899 INFO:tasks.workunit.client.0.smithi064.stdout:3/966: chown d4/d13/l25 28548 1 2024-02-16T22:56:39.900 INFO:tasks.workunit.client.0.smithi064.stdout:8/774: link df/d31/d61/d79/d81/cac df/d31/d8f/d80/dec/c10a 0 2024-02-16T22:56:39.904 INFO:tasks.workunit.client.0.smithi064.stdout:8/775: symlink df/d31/d2f/d3e/d5f/l10b 0 2024-02-16T22:56:39.906 INFO:tasks.workunit.client.0.smithi064.stdout:3/967: creat d4/d13/d4d/d5c/d78/f146 x:0 0 0 2024-02-16T22:56:39.907 INFO:tasks.workunit.client.0.smithi064.stdout:5/817: truncate d6/f37 1853186 0 2024-02-16T22:56:39.908 INFO:tasks.workunit.client.0.smithi064.stdout:5/818: chown d6/d25/da4/dc1/d8e/cd5 0 1 2024-02-16T22:56:39.910 INFO:tasks.workunit.client.0.smithi064.stdout:4/872: symlink dc/de/dfb/dff/de3/l114 0 2024-02-16T22:56:39.912 INFO:tasks.workunit.client.0.smithi064.stdout:9/868: mkdir d3/d11/d15/d11d 0 2024-02-16T22:56:39.912 INFO:tasks.workunit.client.0.smithi064.stdout:1/748: write d2/df/d14/d2c/f49 [405292,55874] 0 2024-02-16T22:56:39.915 INFO:tasks.workunit.client.0.smithi064.stdout:8/776: fsync df/d31/d61/fa8 0 2024-02-16T22:56:39.917 INFO:tasks.workunit.client.0.smithi064.stdout:3/968: rename d4/d1f/d5b/d51 to d4/d13/d1b/d27/d82/d147 0 2024-02-16T22:56:39.918 INFO:tasks.workunit.client.0.smithi064.stdout:0/731: dread da/d26/f82 [0,4194304] 0 2024-02-16T22:56:39.920 INFO:tasks.workunit.client.0.smithi064.stdout:9/869: fdatasync d3/d9/d6a/db1/f29 0 2024-02-16T22:56:39.921 INFO:tasks.workunit.client.0.smithi064.stdout:9/870: chown d3/d9/d6a/db1 16776 1 2024-02-16T22:56:39.924 INFO:tasks.workunit.client.0.smithi064.stdout:8/777: rmdir df/d31/d2f/d38/d48/da2/de2 39 2024-02-16T22:56:39.927 INFO:tasks.workunit.client.0.smithi064.stdout:8/778: chown df/d31/d2f/d4f/f8d 13034 1 2024-02-16T22:56:39.928 INFO:tasks.workunit.client.0.smithi064.stdout:9/871: dwrite d3/d9/d1e/d3e/f65 [0,4194304] 0 2024-02-16T22:56:39.932 INFO:tasks.workunit.client.0.smithi064.stdout:0/732: mkdir da/d26/de0/df4 0 2024-02-16T22:56:39.935 INFO:tasks.workunit.client.0.smithi064.stdout:5/819: mknod d6/d18/d19/c10e 0 2024-02-16T22:56:39.941 INFO:tasks.workunit.client.0.smithi064.stdout:1/749: mkdir d2/de8/dfa 0 2024-02-16T22:56:39.946 INFO:tasks.workunit.client.0.smithi064.stdout:9/872: fdatasync d3/d11/f28 0 2024-02-16T22:56:39.953 INFO:tasks.workunit.client.0.smithi064.stdout:3/969: dwrite d4/d1f/f32 [0,4194304] 0 2024-02-16T22:56:39.961 INFO:tasks.workunit.client.0.smithi064.stdout:9/873: unlink d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/c92 0 2024-02-16T22:56:39.966 INFO:tasks.workunit.client.0.smithi064.stdout:4/873: getdents dc/dd/d32/d58 0 2024-02-16T22:56:39.967 INFO:tasks.workunit.client.0.smithi064.stdout:5/820: symlink d6/d18/d95/de6/l10f 0 2024-02-16T22:56:39.968 INFO:tasks.workunit.client.0.smithi064.stdout:0/733: dwrite da/dc/de/d12/d2c/f31 [0,4194304] 0 2024-02-16T22:56:39.972 INFO:tasks.workunit.client.0.smithi064.stdout:1/750: mknod d2/df/d11/d1a/cfb 0 2024-02-16T22:56:39.976 INFO:tasks.workunit.client.0.smithi064.stdout:1/751: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce [0,4194304] 0 2024-02-16T22:56:39.979 INFO:tasks.workunit.client.0.smithi064.stdout:8/779: rename df/d2a/f34 to df/d31/d2f/d38/d48/da2/f10c 0 2024-02-16T22:56:39.980 INFO:tasks.workunit.client.0.smithi064.stdout:8/780: write df/d31/d8f/d80/dec/fef [665377,12729] 0 2024-02-16T22:56:39.981 INFO:tasks.workunit.client.0.smithi064.stdout:9/874: mkdir d3/d11/d11e 0 2024-02-16T22:56:39.982 INFO:tasks.workunit.client.0.smithi064.stdout:9/875: chown d3/d9/d1e/le7 8291982 1 2024-02-16T22:56:39.982 INFO:tasks.workunit.client.0.smithi064.stdout:4/874: dread - dc/de/d28/d57/d77/d101/d43/ffa zero size 2024-02-16T22:56:39.983 INFO:tasks.workunit.client.0.smithi064.stdout:5/821: creat d6/d18/d95/f110 x:0 0 0 2024-02-16T22:56:39.984 INFO:tasks.workunit.client.0.smithi064.stdout:0/734: chown da/d26/de0/d93/cd1 10552 1 2024-02-16T22:56:39.986 INFO:tasks.workunit.client.0.smithi064.stdout:0/735: chown da/dc/de/d12/d2c/f31 63969 1 2024-02-16T22:56:39.989 INFO:tasks.workunit.client.0.smithi064.stdout:8/781: dread df/d31/d2f/d38/fba [0,4194304] 0 2024-02-16T22:56:39.990 INFO:tasks.workunit.client.0.smithi064.stdout:4/875: mknod dc/dd/d32/d46/d5a/d7e/c115 0 2024-02-16T22:56:39.991 INFO:tasks.workunit.client.0.smithi064.stdout:5/822: truncate d6/d25/df1/fce 102269 0 2024-02-16T22:56:39.992 INFO:tasks.workunit.client.0.smithi064.stdout:4/876: write dc/de/dfb/dff/fc7 [1968779,128981] 0 2024-02-16T22:56:39.993 INFO:tasks.workunit.client.0.smithi064.stdout:5/823: chown d6/d25/da4/dc1/dd4/f109 997741733 1 2024-02-16T22:56:39.995 INFO:tasks.workunit.client.0.smithi064.stdout:0/736: creat da/d26/ff5 x:0 0 0 2024-02-16T22:56:39.997 INFO:tasks.workunit.client.0.smithi064.stdout:4/877: mknod dc/dd/d32/d46/d5a/d7e/d9a/df8/c116 0 2024-02-16T22:56:39.998 INFO:tasks.workunit.client.0.smithi064.stdout:1/752: creat d2/df/d11/d1a/d30/ffc x:0 0 0 2024-02-16T22:56:40.001 INFO:tasks.workunit.client.0.smithi064.stdout:9/876: dread d3/d11/d15/d23/f3d [0,4194304] 0 2024-02-16T22:56:40.007 INFO:tasks.workunit.client.0.smithi064.stdout:9/877: creat d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/f11f x:0 0 0 2024-02-16T22:56:40.012 INFO:tasks.workunit.client.0.smithi064.stdout:4/878: creat dc/de/d28/d57/f117 x:0 0 0 2024-02-16T22:56:40.014 INFO:tasks.workunit.client.0.smithi064.stdout:1/753: write d2/df/d14/d18/f28 [1333918,85831] 0 2024-02-16T22:56:40.020 INFO:tasks.workunit.client.0.smithi064.stdout:9/878: rmdir d3/d11/d15/dab/dcb 0 2024-02-16T22:56:40.021 INFO:tasks.workunit.client.0.smithi064.stdout:1/754: dwrite d2/df/d11/d1a/d1d/fdb [0,4194304] 0 2024-02-16T22:56:40.028 INFO:tasks.workunit.client.0.smithi064.stdout:1/755: symlink d2/df/d14/d18/d48/lfd 0 2024-02-16T22:56:40.031 INFO:tasks.workunit.client.0.smithi064.stdout:9/879: symlink d3/l120 0 2024-02-16T22:56:40.032 INFO:tasks.workunit.client.0.smithi064.stdout:4/879: truncate dc/de/d28/d57/d77/d101/d43/d7a/f9b 3433848 0 2024-02-16T22:56:40.033 INFO:tasks.workunit.client.0.smithi064.stdout:1/756: mknod d2/df/d11/d25/cfe 0 2024-02-16T22:56:40.034 INFO:tasks.workunit.client.0.smithi064.stdout:9/880: truncate d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/fe9 522816 0 2024-02-16T22:56:40.038 INFO:tasks.workunit.client.0.smithi064.stdout:9/881: creat d3/d9/d1e/d3e/f121 x:0 0 0 2024-02-16T22:56:40.042 INFO:tasks.workunit.client.0.smithi064.stdout:9/882: rmdir d3/d9/d6a/db1/d2a/d2e/d31/d62 39 2024-02-16T22:56:40.043 INFO:tasks.workunit.client.0.smithi064.stdout:9/883: write d3/d11/d9e/f108 [262012,22868] 0 2024-02-16T22:56:40.044 INFO:tasks.workunit.client.0.smithi064.stdout:9/884: chown d3/d11/d15/d23/l2d 209091 1 2024-02-16T22:56:40.049 INFO:tasks.workunit.client.0.smithi064.stdout:9/885: creat d3/d9/d6a/db1/f122 x:0 0 0 2024-02-16T22:56:40.049 INFO:tasks.workunit.client.0.smithi064.stdout:1/757: write d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce [647598,16888] 0 2024-02-16T22:56:40.055 INFO:tasks.workunit.client.0.smithi064.stdout:1/758: stat d2/df/d11/d25/fc5 0 2024-02-16T22:56:40.057 INFO:tasks.workunit.client.0.smithi064.stdout:1/759: creat d2/df/d14/d2c/db5/dc6/fff x:0 0 0 2024-02-16T22:56:40.060 INFO:tasks.workunit.client.0.smithi064.stdout:1/760: mkdir d2/df/d14/d2c/db5/de3/d100 0 2024-02-16T22:56:40.064 INFO:tasks.workunit.client.0.smithi064.stdout:9/886: dread d3/f69 [0,4194304] 0 2024-02-16T22:56:40.074 INFO:tasks.workunit.client.0.smithi064.stdout:6/918: sync 2024-02-16T22:56:40.080 INFO:tasks.workunit.client.0.smithi064.stdout:1/761: write d2/df/d14/f8d [221445,45040] 0 2024-02-16T22:56:40.144 INFO:tasks.workunit.client.0.smithi064.stdout:9/887: truncate d3/d11/d15/d23/f6e 1747646 0 2024-02-16T22:56:40.144 INFO:tasks.workunit.client.0.smithi064.stdout:6/919: symlink d0/d12/d7b/l12f 0 2024-02-16T22:56:40.148 INFO:tasks.workunit.client.0.smithi064.stdout:6/920: symlink d0/d6/d10/d105/db9/l130 0 2024-02-16T22:56:40.151 INFO:tasks.workunit.client.0.smithi064.stdout:6/921: mknod d0/d3c/da9/c131 0 2024-02-16T22:56:40.151 INFO:tasks.workunit.client.0.smithi064.stdout:9/888: link d3/d9/d6a/db1/d2a/d2e/f59 d3/d11/d15/d23/dee/f123 0 2024-02-16T22:56:40.155 INFO:tasks.workunit.client.0.smithi064.stdout:6/922: rename d0/d89/cf3 to d0/d1d/dfa/c132 0 2024-02-16T22:56:40.160 INFO:tasks.workunit.client.0.smithi064.stdout:9/889: dwrite d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/f114 [0,4194304] 0 2024-02-16T22:56:40.160 INFO:tasks.workunit.client.0.smithi064.stdout:9/890: readlink d3/d9/le2 0 2024-02-16T22:56:40.166 INFO:tasks.workunit.client.0.smithi064.stdout:6/923: link d0/d1d/da8/fdb d0/d1b/f133 0 2024-02-16T22:56:40.166 INFO:tasks.workunit.client.0.smithi064.stdout:1/762: write d2/df/d11/d25/fb9 [5230051,61843] 0 2024-02-16T22:56:40.168 INFO:tasks.workunit.client.0.smithi064.stdout:1/763: readlink d2/df/d11/d1a/d30/dbd/d91/lda 0 2024-02-16T22:56:40.169 INFO:tasks.workunit.client.0.smithi064.stdout:1/764: stat d2/df/d11/dba/ceb 0 2024-02-16T22:56:40.172 INFO:tasks.workunit.client.0.smithi064.stdout:1/765: creat d2/df/d11/d1a/d30/dbd/f101 x:0 0 0 2024-02-16T22:56:40.175 INFO:tasks.workunit.client.0.smithi064.stdout:1/766: mknod d2/df/d11/d1a/c102 0 2024-02-16T22:56:40.176 INFO:tasks.workunit.client.0.smithi064.stdout:9/891: getdents d3/d11/d15/d23/dee 0 2024-02-16T22:56:40.177 INFO:tasks.workunit.client.0.smithi064.stdout:1/767: symlink d2/df/d14/d2c/db5/l103 0 2024-02-16T22:56:40.180 INFO:tasks.workunit.client.0.smithi064.stdout:1/768: mknod d2/df/d14/d18/d48/d84/c104 0 2024-02-16T22:56:40.182 INFO:tasks.workunit.client.0.smithi064.stdout:9/892: dwrite d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/f115 [0,4194304] 0 2024-02-16T22:56:40.184 INFO:tasks.workunit.client.0.smithi064.stdout:1/769: fdatasync d2/df/d11/d1a/d1d/f58 0 2024-02-16T22:56:40.191 INFO:tasks.workunit.client.0.smithi064.stdout:1/770: mkdir d2/df/d14/d18/d48/d105 0 2024-02-16T22:56:40.193 INFO:tasks.workunit.client.0.smithi064.stdout:6/924: dwrite d0/d6/da/f11e [0,4194304] 0 2024-02-16T22:56:40.195 INFO:tasks.workunit.client.0.smithi064.stdout:9/893: write d3/d9/d1e/d3e/d76/f79 [1245481,34047] 0 2024-02-16T22:56:40.197 INFO:tasks.workunit.client.0.smithi064.stdout:1/771: mkdir d2/df/d14/d18/d48/d6b/d75/d106 0 2024-02-16T22:56:40.199 INFO:tasks.workunit.client.0.smithi064.stdout:6/925: read d0/d5/df/f86 [3571481,79895] 0 2024-02-16T22:56:40.200 INFO:tasks.workunit.client.0.smithi064.stdout:9/894: mknod d3/d11/d11e/c124 0 2024-02-16T22:56:40.202 INFO:tasks.workunit.client.0.smithi064.stdout:1/772: dread d2/df/d14/d18/d40/fa0 [0,4194304] 0 2024-02-16T22:56:40.204 INFO:tasks.workunit.client.0.smithi064.stdout:1/773: write d2/df/d14/d18/d40/f68 [2705242,7345] 0 2024-02-16T22:56:40.209 INFO:tasks.workunit.client.0.smithi064.stdout:9/895: rename d3/d9/d6a/db1/d2a/d2e/d31/d3f/lf7 to d3/d9/d1e/d42/d7b/l125 0 2024-02-16T22:56:40.213 INFO:tasks.workunit.client.0.smithi064.stdout:9/896: mkdir d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/de4/d126 0 2024-02-16T22:56:40.216 INFO:tasks.workunit.client.0.smithi064.stdout:9/897: mknod d3/d11/d15/c127 0 2024-02-16T22:56:40.220 INFO:tasks.workunit.client.0.smithi064.stdout:9/898: symlink d3/d11/d15/d23/d117/l128 0 2024-02-16T22:56:40.223 INFO:tasks.workunit.client.0.smithi064.stdout:1/774: dread d2/df/d14/d2c/f5f [0,4194304] 0 2024-02-16T22:56:40.224 INFO:tasks.workunit.client.0.smithi064.stdout:1/775: stat d2/df/d14/d2a/fbf 0 2024-02-16T22:56:40.224 INFO:tasks.workunit.client.0.smithi064.stdout:6/926: dread d0/d1d/d21/f128 [0,4194304] 0 2024-02-16T22:56:40.225 INFO:tasks.workunit.client.0.smithi064.stdout:1/776: dread - d2/df/d11/d1a/d30/dbd/f101 zero size 2024-02-16T22:56:40.229 INFO:tasks.workunit.client.0.smithi064.stdout:9/899: symlink d3/d9/d6a/db1/d2a/d2e/d31/d62/df6/l129 0 2024-02-16T22:56:40.230 INFO:tasks.workunit.client.0.smithi064.stdout:1/777: readlink d2/df/d11/dba/lc8 0 2024-02-16T22:56:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:6/927: symlink d0/d6/d10/l134 0 2024-02-16T22:56:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:9/900: read - d3/ff4 zero size 2024-02-16T22:56:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:1/778: creat d2/df/d14/d18/d48/d6b/d75/f107 x:0 0 0 2024-02-16T22:56:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:6/928: truncate d0/d5/df/d17/d4a/f6b 881955 0 2024-02-16T22:56:40.240 INFO:tasks.workunit.client.0.smithi064.stdout:6/929: truncate d0/d1d/da8/fdb 2003006 0 2024-02-16T22:56:40.242 INFO:tasks.workunit.client.0.smithi064.stdout:1/779: dwrite d2/df/d14/d2c/db5/de3/fea [0,4194304] 0 2024-02-16T22:56:40.243 INFO:tasks.workunit.client.0.smithi064.stdout:9/901: mkdir d3/d12a 0 2024-02-16T22:56:40.253 INFO:tasks.workunit.client.0.smithi064.stdout:6/930: symlink d0/d1b/l135 0 2024-02-16T22:56:40.253 INFO:tasks.workunit.client.0.smithi064.stdout:1/780: truncate d2/df/d14/f52 928648 0 2024-02-16T22:56:40.256 INFO:tasks.workunit.client.0.smithi064.stdout:9/902: mkdir d3/d11/d15/dab/db6/dd5/d12b 0 2024-02-16T22:56:40.256 INFO:tasks.workunit.client.0.smithi064.stdout:6/931: readlink d0/d3c/l12b 0 2024-02-16T22:56:40.259 INFO:tasks.workunit.client.0.smithi064.stdout:1/781: mkdir d2/df/d14/d2a/d108 0 2024-02-16T22:56:40.267 INFO:tasks.workunit.client.0.smithi064.stdout:1/782: truncate d2/df/d11/d71/fd3 1269013 0 2024-02-16T22:56:40.270 INFO:tasks.workunit.client.0.smithi064.stdout:9/903: dread d3/d9/d6a/db1/d38/fa6 [0,4194304] 0 2024-02-16T22:56:40.280 INFO:tasks.workunit.client.0.smithi064.stdout:9/904: getdents d3/d11/d15/d23 0 2024-02-16T22:56:40.365 INFO:tasks.workunit.client.0.smithi064.stdout:9/905: creat d3/d9/d6a/db1/d38/f12c x:0 0 0 2024-02-16T22:56:40.365 INFO:tasks.workunit.client.0.smithi064.stdout:6/932: write d0/d6/d10/d33/f2a [4908843,129936] 0 2024-02-16T22:56:40.369 INFO:tasks.workunit.client.0.smithi064.stdout:9/906: creat d3/d9/dec/f12d x:0 0 0 2024-02-16T22:56:40.369 INFO:tasks.workunit.client.0.smithi064.stdout:1/783: write d2/df/d14/f2d [4233582,50982] 0 2024-02-16T22:56:40.370 INFO:tasks.workunit.client.0.smithi064.stdout:6/933: mkdir d0/d85/d136 0 2024-02-16T22:56:40.375 INFO:tasks.workunit.client.0.smithi064.stdout:1/784: truncate d2/df/d11/d25/fe5 426291 0 2024-02-16T22:56:40.384 INFO:tasks.workunit.client.0.smithi064.stdout:6/934: dread d0/d6/da/fba [0,4194304] 0 2024-02-16T22:56:40.385 INFO:tasks.workunit.client.0.smithi064.stdout:9/907: write d3/d9/d1e/fe1 [764099,61735] 0 2024-02-16T22:56:40.387 INFO:tasks.workunit.client.0.smithi064.stdout:6/935: creat d0/d5/df/f137 x:0 0 0 2024-02-16T22:56:40.392 INFO:tasks.workunit.client.0.smithi064.stdout:1/785: dwrite d2/df/d11/f8f [0,4194304] 0 2024-02-16T22:56:40.393 INFO:tasks.workunit.client.0.smithi064.stdout:1/786: stat d2/df/d14/d18/d48 0 2024-02-16T22:56:40.395 INFO:tasks.workunit.client.0.smithi064.stdout:1/787: readlink d2/df/d11/d1a/d30/dbd/d91/d56/d5b/lb3 0 2024-02-16T22:56:40.396 INFO:tasks.workunit.client.0.smithi064.stdout:1/788: fdatasync d2/df/d14/d18/d48/d6b/d75/f107 0 2024-02-16T22:56:40.400 INFO:tasks.workunit.client.0.smithi064.stdout:1/789: mkdir d2/de8/d109 0 2024-02-16T22:56:40.403 INFO:tasks.workunit.client.0.smithi064.stdout:9/908: dwrite d3/d9/d6a/db1/d2a/d2e/f9d [4194304,4194304] 0 2024-02-16T22:56:40.403 INFO:tasks.workunit.client.0.smithi064.stdout:6/936: dwrite d0/d1d/da8/d60/f7c [0,4194304] 0 2024-02-16T22:56:40.418 INFO:tasks.workunit.client.0.smithi064.stdout:6/937: creat d0/d5/df/d17/ded/d12d/f138 x:0 0 0 2024-02-16T22:56:40.422 INFO:tasks.workunit.client.0.smithi064.stdout:9/909: getdents d3/d9/d1e/d3e 0 2024-02-16T22:56:40.423 INFO:tasks.workunit.client.0.smithi064.stdout:9/910: truncate d3/d9/d6a/db1/d2a/fc9 5128389 0 2024-02-16T22:56:40.426 INFO:tasks.workunit.client.0.smithi064.stdout:1/790: write d2/df/d14/f52 [104509,100810] 0 2024-02-16T22:56:40.426 INFO:tasks.workunit.client.0.smithi064.stdout:9/911: truncate d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/f11f 705779 0 2024-02-16T22:56:40.431 INFO:tasks.workunit.client.0.smithi064.stdout:6/938: write d0/d5/fcd [2878525,57986] 0 2024-02-16T22:56:40.436 INFO:tasks.workunit.client.0.smithi064.stdout:9/912: write d3/f8f [1937515,4647] 0 2024-02-16T22:56:40.437 INFO:tasks.workunit.client.0.smithi064.stdout:6/939: symlink d0/d5/df/d17/d18/l139 0 2024-02-16T22:56:40.440 INFO:tasks.workunit.client.0.smithi064.stdout:9/913: dread d3/d9/d6a/db1/d2a/fc9 [0,4194304] 0 2024-02-16T22:56:40.445 INFO:tasks.workunit.client.0.smithi064.stdout:7/872: sync 2024-02-16T22:56:40.446 INFO:tasks.workunit.client.0.smithi064.stdout:9/914: dwrite d3/f8f [0,4194304] 0 2024-02-16T22:56:40.452 INFO:tasks.workunit.client.0.smithi064.stdout:1/791: dread d2/df/d11/d1a/d30/dbd/f72 [0,4194304] 0 2024-02-16T22:56:40.453 INFO:tasks.workunit.client.0.smithi064.stdout:7/873: creat d5/d33/d101/f11b x:0 0 0 2024-02-16T22:56:40.455 INFO:tasks.workunit.client.0.smithi064.stdout:6/940: dwrite d0/d6/d10/d33/f55 [0,4194304] 0 2024-02-16T22:56:40.458 INFO:tasks.workunit.client.0.smithi064.stdout:1/792: creat d2/f10a x:0 0 0 2024-02-16T22:56:40.460 INFO:tasks.workunit.client.0.smithi064.stdout:9/915: dwrite d3/d9/f48 [0,4194304] 0 2024-02-16T22:56:40.460 INFO:tasks.workunit.client.0.smithi064.stdout:7/874: rename d5/d33/d45/f46 to d5/df9/de4/df5/df7/f11c 0 2024-02-16T22:56:40.462 INFO:tasks.workunit.client.0.smithi064.stdout:1/793: dread - d2/df/d14/d18/d48/d6b/d75/f107 zero size 2024-02-16T22:56:40.462 INFO:tasks.workunit.client.0.smithi064.stdout:9/916: write d3/d88/f106 [596525,47547] 0 2024-02-16T22:56:40.466 INFO:tasks.workunit.client.0.smithi064.stdout:6/941: truncate d0/d89/fb6 5113212 0 2024-02-16T22:56:40.469 INFO:tasks.workunit.client.0.smithi064.stdout:1/794: rename d2/df/d14/f8d to d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf/f10b 0 2024-02-16T22:56:40.469 INFO:tasks.workunit.client.0.smithi064.stdout:6/942: truncate d0/d6/d10/d33/da5/f114 139201 0 2024-02-16T22:56:40.470 INFO:tasks.workunit.client.0.smithi064.stdout:1/795: dread - d2/fa6 zero size 2024-02-16T22:56:40.471 INFO:tasks.workunit.client.0.smithi064.stdout:9/917: mkdir d3/d9/d6a/db1/d2a/d2e/da3/dea/d12e 0 2024-02-16T22:56:40.478 INFO:tasks.workunit.client.0.smithi064.stdout:1/796: write d2/df/d11/d1a/d1d/f24 [590615,86092] 0 2024-02-16T22:56:40.479 INFO:tasks.workunit.client.0.smithi064.stdout:3/970: sync 2024-02-16T22:56:40.479 INFO:tasks.workunit.client.0.smithi064.stdout:9/918: chown d3/d9/d6a/db1/c50 1905244725 1 2024-02-16T22:56:40.485 INFO:tasks.workunit.client.0.smithi064.stdout:7/875: dwrite d5/d33/d53/df2/d54/f78 [0,4194304] 0 2024-02-16T22:56:40.487 INFO:tasks.workunit.client.0.smithi064.stdout:6/943: write d0/d5/f42 [5598488,68552] 0 2024-02-16T22:56:40.489 INFO:tasks.workunit.client.0.smithi064.stdout:9/919: truncate d3/d11/d15/d23/f73 4787009 0 2024-02-16T22:56:40.492 INFO:tasks.workunit.client.0.smithi064.stdout:7/876: dread - d5/d33/d53/df2/d54/d41/fea zero size 2024-02-16T22:56:40.501 INFO:tasks.workunit.client.0.smithi064.stdout:1/797: dread d2/df/d14/d18/d48/d84/f9a [0,4194304] 0 2024-02-16T22:56:40.501 INFO:tasks.workunit.client.0.smithi064.stdout:9/920: rename d3/d89/d9a/lf5 to d3/d11/d15/d23/dee/l12f 0 2024-02-16T22:56:40.505 INFO:tasks.workunit.client.0.smithi064.stdout:3/971: creat d4/d13/d1b/d27/f148 x:0 0 0 2024-02-16T22:56:40.505 INFO:tasks.workunit.client.0.smithi064.stdout:5/824: sync 2024-02-16T22:56:40.505 INFO:tasks.workunit.client.0.smithi064.stdout:8/782: sync 2024-02-16T22:56:40.505 INFO:tasks.workunit.client.0.smithi064.stdout:4/880: sync 2024-02-16T22:56:40.505 INFO:tasks.workunit.client.0.smithi064.stdout:0/737: sync 2024-02-16T22:56:40.507 INFO:tasks.workunit.client.0.smithi064.stdout:0/738: readlink da/dc/de/d12/d2c/d2e/d38/l8a 0 2024-02-16T22:56:40.511 INFO:tasks.workunit.client.0.smithi064.stdout:9/921: mknod d3/d11/d15/dab/db6/dd5/c130 0 2024-02-16T22:56:40.513 INFO:tasks.workunit.client.0.smithi064.stdout:6/944: dwrite d0/d5/df/d17/d18/fb3 [0,4194304] 0 2024-02-16T22:56:40.515 INFO:tasks.workunit.client.0.smithi064.stdout:8/783: rmdir df/d31/d61 39 2024-02-16T22:56:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:7/877: creat d5/d33/ddd/f11d x:0 0 0 2024-02-16T22:56:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:3/972: rename d4/d13/d1b/d27/d82/d147/c8c to d4/d13/d1b/d27/d82/d147/c149 0 2024-02-16T22:56:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:6/945: mkdir d0/d3c/da9/d13a 0 2024-02-16T22:56:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:4/881: symlink dc/dd/l118 0 2024-02-16T22:56:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:1/798: truncate d2/df/d11/f8f 3148592 0 2024-02-16T22:56:40.587 INFO:tasks.workunit.client.0.smithi064.stdout:8/784: dread - df/d31/d61/d94/ff6 zero size 2024-02-16T22:56:40.588 INFO:tasks.workunit.client.0.smithi064.stdout:9/922: symlink d3/d9/d6a/db1/d2a/d2e/l131 0 2024-02-16T22:56:40.595 INFO:tasks.workunit.client.0.smithi064.stdout:9/923: truncate d3/d9/d6a/db1/d2a/d2e/d31/d3f/faa 5159841 0 2024-02-16T22:56:40.595 INFO:tasks.workunit.client.0.smithi064.stdout:4/882: dwrite dc/dd/d32/f3d [0,4194304] 0 2024-02-16T22:56:40.597 INFO:tasks.workunit.client.0.smithi064.stdout:3/973: symlink d4/d13/d1b/d27/d82/d147/dc1/l14a 0 2024-02-16T22:56:40.601 INFO:tasks.workunit.client.0.smithi064.stdout:5/825: creat d6/d25/df1/d91/da5/de3/d43/d7d/de2/f111 x:0 0 0 2024-02-16T22:56:40.606 INFO:tasks.workunit.client.0.smithi064.stdout:8/785: creat df/d31/d8f/d80/f10d x:0 0 0 2024-02-16T22:56:40.607 INFO:tasks.workunit.client.0.smithi064.stdout:6/946: dread d0/d3c/fb7 [0,4194304] 0 2024-02-16T22:56:40.612 INFO:tasks.workunit.client.0.smithi064.stdout:8/786: dwrite df/d31/d2f/d38/f39 [0,4194304] 0 2024-02-16T22:56:40.616 INFO:tasks.workunit.client.0.smithi064.stdout:4/883: fsync dc/dd/d16/fa6 0 2024-02-16T22:56:40.617 INFO:tasks.workunit.client.0.smithi064.stdout:3/974: fsync d4/d13/d41/f8d 0 2024-02-16T22:56:40.621 INFO:tasks.workunit.client.0.smithi064.stdout:1/799: getdents d2/df/d14/d2a/dc2 0 2024-02-16T22:56:40.637 INFO:tasks.workunit.client.0.smithi064.stdout:3/975: mkdir d4/d13/d1b/d27/d82/d147/d7c/d14b 0 2024-02-16T22:56:40.637 INFO:tasks.workunit.client.0.smithi064.stdout:3/976: readlink d4/d13/d1b/d27/d82/l119 0 2024-02-16T22:56:40.638 INFO:tasks.workunit.client.0.smithi064.stdout:3/977: fdatasync d4/d13/d1b/d27/d44/d97/fdc 0 2024-02-16T22:56:40.639 INFO:tasks.workunit.client.0.smithi064.stdout:5/826: mkdir d6/d18/d19/d112 0 2024-02-16T22:56:40.640 INFO:tasks.workunit.client.0.smithi064.stdout:3/978: truncate d4/d13/d4d/d5c/d78/d117/f11e 666656 0 2024-02-16T22:56:40.644 INFO:tasks.workunit.client.0.smithi064.stdout:6/947: link d0/d6/d10/f9f d0/d1d/da8/d60/f13b 0 2024-02-16T22:56:40.647 INFO:tasks.workunit.client.0.smithi064.stdout:8/787: rename df/d31/d61/d79/d81/f88 to df/d31/d2f/d38/d48/db5/f10e 0 2024-02-16T22:56:40.648 INFO:tasks.workunit.client.0.smithi064.stdout:5/827: dwrite d6/d25/df1/d80/fda [0,4194304] 0 2024-02-16T22:56:40.649 INFO:tasks.workunit.client.0.smithi064.stdout:9/924: dwrite d3/d11/d15/d23/f6e [0,4194304] 0 2024-02-16T22:56:40.651 INFO:tasks.workunit.client.0.smithi064.stdout:1/800: dread d2/df/d14/d18/f26 [0,4194304] 0 2024-02-16T22:56:40.652 INFO:tasks.workunit.client.0.smithi064.stdout:9/925: dread d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/f11f [0,4194304] 0 2024-02-16T22:56:40.656 INFO:tasks.workunit.client.0.smithi064.stdout:7/878: truncate d5/d33/d53/df2/f22 4074385 0 2024-02-16T22:56:40.662 INFO:tasks.workunit.client.0.smithi064.stdout:6/948: creat d0/d5/df/d17/da1/f13c x:0 0 0 2024-02-16T22:56:40.667 INFO:tasks.workunit.client.0.smithi064.stdout:5/828: mknod d6/d25/df1/d91/da5/c113 0 2024-02-16T22:56:40.670 INFO:tasks.workunit.client.0.smithi064.stdout:4/884: dwrite dc/d4a/f7d [4194304,4194304] 0 2024-02-16T22:56:40.674 INFO:tasks.workunit.client.0.smithi064.stdout:7/879: stat d5/l14 0 2024-02-16T22:56:40.675 INFO:tasks.workunit.client.0.smithi064.stdout:3/979: creat d4/d14/d2b/d141/f14c x:0 0 0 2024-02-16T22:56:40.681 INFO:tasks.workunit.client.0.smithi064.stdout:7/880: dwrite d5/d33/d53/df2/d65/ff6 [0,4194304] 0 2024-02-16T22:56:40.701 INFO:tasks.workunit.client.0.smithi064.stdout:7/881: dread d5/d33/d42/d81/d82/f109 [0,4194304] 0 2024-02-16T22:56:40.702 INFO:tasks.workunit.client.0.smithi064.stdout:1/801: getdents d2/df/d14/d2c/db5/de3 0 2024-02-16T22:56:40.702 INFO:tasks.workunit.client.0.smithi064.stdout:7/882: fdatasync d5/d33/d45/d64/f91 0 2024-02-16T22:56:40.703 INFO:tasks.workunit.client.0.smithi064.stdout:8/788: write df/d31/d2f/d4f/f8d [1854442,62368] 0 2024-02-16T22:56:40.704 INFO:tasks.workunit.client.0.smithi064.stdout:4/885: link dc/de/d28/d57/d77/cf9 dc/de/d28/d57/d77/dc6/c119 0 2024-02-16T22:56:40.706 INFO:tasks.workunit.client.0.smithi064.stdout:6/949: link d0/d89/fae d0/d6/da/f13d 0 2024-02-16T22:56:40.707 INFO:tasks.workunit.client.0.smithi064.stdout:5/829: getdents d6/d18/d95/de6 0 2024-02-16T22:56:40.709 INFO:tasks.workunit.client.0.smithi064.stdout:9/926: write d3/d9/d6a/db1/d2a/d2e/da3/fc7 [298705,117985] 0 2024-02-16T22:56:40.713 INFO:tasks.workunit.client.0.smithi064.stdout:0/739: sync 2024-02-16T22:56:40.717 INFO:tasks.workunit.client.0.smithi064.stdout:6/950: rmdir d0/d6/d10 39 2024-02-16T22:56:40.719 INFO:tasks.workunit.client.0.smithi064.stdout:5/830: creat d6/d25/df1/d91/da5/de3/d43/db9/d103/f114 x:0 0 0 2024-02-16T22:56:40.721 INFO:tasks.workunit.client.0.smithi064.stdout:9/927: unlink d3/d11/d15/f94 0 2024-02-16T22:56:40.724 INFO:tasks.workunit.client.0.smithi064.stdout:3/980: truncate d4/d1f/d5b/fe4 3103202 0 2024-02-16T22:56:40.728 INFO:tasks.workunit.client.0.smithi064.stdout:8/789: write df/d31/d2f/d4f/f66 [5459988,21700] 0 2024-02-16T22:56:40.732 INFO:tasks.workunit.client.0.smithi064.stdout:1/802: dwrite d2/df/d11/d25/fc5 [0,4194304] 0 2024-02-16T22:56:40.741 INFO:tasks.workunit.client.0.smithi064.stdout:9/928: mkdir d3/d11/d15/dab/db6/dd5/d12b/d132 0 2024-02-16T22:56:40.743 INFO:tasks.workunit.client.0.smithi064.stdout:4/886: dwrite dc/dd/d16/f5e [0,4194304] 0 2024-02-16T22:56:40.748 INFO:tasks.workunit.client.0.smithi064.stdout:1/803: symlink d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf/l10c 0 2024-02-16T22:56:40.748 INFO:tasks.workunit.client.0.smithi064.stdout:7/883: rename d5/d33/d53/df2/c6f to d5/d33/d42/d81/d82/c11e 0 2024-02-16T22:56:40.748 INFO:tasks.workunit.client.0.smithi064.stdout:6/951: truncate d0/d85/f92 63963 0 2024-02-16T22:56:40.748 INFO:tasks.workunit.client.0.smithi064.stdout:7/884: dread - d5/d33/d45/d64/d56/f10c zero size 2024-02-16T22:56:40.750 INFO:tasks.workunit.client.0.smithi064.stdout:8/790: write df/d31/d2f/d4f/d6a/fb0 [1037479,30256] 0 2024-02-16T22:56:40.751 INFO:tasks.workunit.client.0.smithi064.stdout:8/791: stat df/d31/d2f/f7d 0 2024-02-16T22:56:40.752 INFO:tasks.workunit.client.0.smithi064.stdout:8/792: write df/d31/d2f/d7f/fd5 [345528,114986] 0 2024-02-16T22:56:40.754 INFO:tasks.workunit.client.0.smithi064.stdout:9/929: rmdir d3/d9/d6a/db1 39 2024-02-16T22:56:40.755 INFO:tasks.workunit.client.0.smithi064.stdout:9/930: write d3/d89/d9a/fce [1262169,52740] 0 2024-02-16T22:56:40.759 INFO:tasks.workunit.client.0.smithi064.stdout:1/804: fsync d2/df/d14/d18/d48/d6b/d75/fac 0 2024-02-16T22:56:40.762 INFO:tasks.workunit.client.0.smithi064.stdout:5/831: rename d6/d18/d90 to d6/d25/df1/d91/da5/de3/d43/db9/d115 0 2024-02-16T22:56:40.764 INFO:tasks.workunit.client.0.smithi064.stdout:0/740: getdents da/dc/de/d12/d2c/d2e 0 2024-02-16T22:56:40.766 INFO:tasks.workunit.client.0.smithi064.stdout:7/885: dread d5/d33/d53/f9f [0,4194304] 0 2024-02-16T22:56:40.769 INFO:tasks.workunit.client.0.smithi064.stdout:6/952: write d0/d6/d10/db2/ff1 [4693558,23797] 0 2024-02-16T22:56:40.773 INFO:tasks.workunit.client.0.smithi064.stdout:8/793: creat df/d31/d8f/dc9/d73/dfb/f10f x:0 0 0 2024-02-16T22:56:40.778 INFO:tasks.workunit.client.0.smithi064.stdout:4/887: symlink dc/dd/d16/l11a 0 2024-02-16T22:56:40.785 INFO:tasks.workunit.client.0.smithi064.stdout:9/931: write d3/d11/d9e/da9/fd9 [1939589,40147] 0 2024-02-16T22:56:40.787 INFO:tasks.workunit.client.0.smithi064.stdout:6/953: symlink d0/d6/d10/d105/l13e 0 2024-02-16T22:56:40.791 INFO:tasks.workunit.client.0.smithi064.stdout:8/794: dread df/d2a/f57 [0,4194304] 0 2024-02-16T22:56:40.797 INFO:tasks.workunit.client.0.smithi064.stdout:9/932: read - d3/d11/d15/dab/fcc zero size 2024-02-16T22:56:40.798 INFO:tasks.workunit.client.0.smithi064.stdout:4/888: write dc/f19 [3766820,65828] 0 2024-02-16T22:56:40.798 INFO:tasks.workunit.client.0.smithi064.stdout:5/832: dread d6/d25/df1/d91/da5/de3/f93 [0,4194304] 0 2024-02-16T22:56:40.803 INFO:tasks.workunit.client.0.smithi064.stdout:8/795: rename df/d31/d8f/dab/fd1 to df/d31/dc1/f110 0 2024-02-16T22:56:40.805 INFO:tasks.workunit.client.0.smithi064.stdout:7/886: write d5/f8b [2000950,39267] 0 2024-02-16T22:56:40.806 INFO:tasks.workunit.client.0.smithi064.stdout:1/805: creat d2/df/d11/d1a/d30/dbd/d91/d56/f10d x:0 0 0 2024-02-16T22:56:40.806 INFO:tasks.workunit.client.0.smithi064.stdout:6/954: write d0/d5/df/f63 [166541,63528] 0 2024-02-16T22:56:40.807 INFO:tasks.workunit.client.0.smithi064.stdout:0/741: link da/dc/de/d12/d2c/d2e/c56 da/dc/de/d37/cf6 0 2024-02-16T22:56:40.809 INFO:tasks.workunit.client.0.smithi064.stdout:5/833: stat d6/d25/da4/dc1/f107 0 2024-02-16T22:56:40.811 INFO:tasks.workunit.client.0.smithi064.stdout:1/806: fsync d2/df/d14/f88 0 2024-02-16T22:56:40.812 INFO:tasks.workunit.client.0.smithi064.stdout:1/807: write d2/df/d11/d1a/d30/dbd/f101 [722265,24673] 0 2024-02-16T22:56:40.815 INFO:tasks.workunit.client.0.smithi064.stdout:7/887: rename d5/d33/d45/d64/d56 to d5/d33/d53/df2/d54/d7b/d11f 0 2024-02-16T22:56:40.817 INFO:tasks.workunit.client.0.smithi064.stdout:6/955: mknod d0/d3c/da9/c13f 0 2024-02-16T22:56:40.820 INFO:tasks.workunit.client.0.smithi064.stdout:7/888: dwrite d5/d7d/fa0 [0,4194304] 0 2024-02-16T22:56:40.825 INFO:tasks.workunit.client.0.smithi064.stdout:5/834: creat d6/d25/da4/dc1/d8e/f116 x:0 0 0 2024-02-16T22:56:40.827 INFO:tasks.workunit.client.0.smithi064.stdout:1/808: rmdir d2/df/d11/d1a/d30/dbd/d91/d56/d5b 39 2024-02-16T22:56:40.828 INFO:tasks.workunit.client.0.smithi064.stdout:5/835: read d6/d25/da4/dc1/f69 [1658669,107131] 0 2024-02-16T22:56:40.833 INFO:tasks.workunit.client.0.smithi064.stdout:0/742: creat da/da0/ff7 x:0 0 0 2024-02-16T22:56:40.837 INFO:tasks.workunit.client.0.smithi064.stdout:7/889: mkdir d5/d33/d53/df2/d65/da7/d120 0 2024-02-16T22:56:40.840 INFO:tasks.workunit.client.0.smithi064.stdout:4/889: link dc/ddc/de8/da1/cc9 dc/d4a/c11b 0 2024-02-16T22:56:40.840 INFO:tasks.workunit.client.0.smithi064.stdout:8/796: link df/d31/d2f/d4f/f89 df/d31/d2f/dfe/f111 0 2024-02-16T22:56:40.852 INFO:tasks.workunit.client.0.smithi064.stdout:1/809: write d2/df/d14/d18/d48/fd4 [393353,30800] 0 2024-02-16T22:56:40.855 INFO:tasks.workunit.client.0.smithi064.stdout:6/956: write d0/d1d/d21/ffd [416383,65667] 0 2024-02-16T22:56:40.856 INFO:tasks.workunit.client.0.smithi064.stdout:4/890: mkdir dc/de/d28/d57/d77/dc6/d11c 0 2024-02-16T22:56:40.856 INFO:tasks.workunit.client.0.smithi064.stdout:6/957: write d0/d1d/d21/ffd [919696,6870] 0 2024-02-16T22:56:40.858 INFO:tasks.workunit.client.0.smithi064.stdout:4/891: write dc/dd/fc8 [413198,59173] 0 2024-02-16T22:56:40.859 INFO:tasks.workunit.client.0.smithi064.stdout:6/958: read d0/d85/f92 [733337,63959] 0 2024-02-16T22:56:40.862 INFO:tasks.workunit.client.0.smithi064.stdout:8/797: dread df/d31/d2f/f7d [0,4194304] 0 2024-02-16T22:56:40.862 INFO:tasks.workunit.client.0.smithi064.stdout:6/959: chown d0/d6/d10/d33/d72/de8/f101 0 1 2024-02-16T22:56:40.864 INFO:tasks.workunit.client.0.smithi064.stdout:9/933: getdents d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/d110 0 2024-02-16T22:56:40.866 INFO:tasks.workunit.client.0.smithi064.stdout:7/890: dwrite d5/d33/d42/d81/d82/df3/fbe [0,4194304] 0 2024-02-16T22:56:40.875 INFO:tasks.workunit.client.0.smithi064.stdout:0/743: dwrite da/dc/d3b/d72/f99 [0,4194304] 0 2024-02-16T22:56:40.879 INFO:tasks.workunit.client.0.smithi064.stdout:6/960: symlink d0/d1d/db1/l140 0 2024-02-16T22:56:40.879 INFO:tasks.workunit.client.0.smithi064.stdout:6/961: chown d0/d12 248 1 2024-02-16T22:56:40.880 INFO:tasks.workunit.client.0.smithi064.stdout:6/962: write d0/d1d/dbc/fe4 [888907,3650] 0 2024-02-16T22:56:40.881 INFO:tasks.workunit.client.0.smithi064.stdout:9/934: rename d3/d11/d9e/da9/fd9 to d3/d11/d9e/f133 0 2024-02-16T22:56:40.883 INFO:tasks.workunit.client.0.smithi064.stdout:1/810: symlink d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf/l10e 0 2024-02-16T22:56:40.884 INFO:tasks.workunit.client.0.smithi064.stdout:1/811: write d2/df/d11/d71/fe4 [765491,5429] 0 2024-02-16T22:56:40.888 INFO:tasks.workunit.client.0.smithi064.stdout:5/836: creat d6/d25/da4/dc1/d8e/f117 x:0 0 0 2024-02-16T22:56:40.889 INFO:tasks.workunit.client.0.smithi064.stdout:5/837: dread - d6/d25/da4/dc1/fc0 zero size 2024-02-16T22:56:40.890 INFO:tasks.workunit.client.0.smithi064.stdout:4/892: write dc/de/fb9 [74740,50204] 0 2024-02-16T22:56:40.890 INFO:tasks.workunit.client.0.smithi064.stdout:8/798: truncate df/f19 85456 0 2024-02-16T22:56:40.890 INFO:tasks.workunit.client.0.smithi064.stdout:4/893: chown dc/de/d28/d57/d77 22283327 1 2024-02-16T22:56:40.892 INFO:tasks.workunit.client.0.smithi064.stdout:4/894: read dc/dd/d16/f5e [1035649,1815] 0 2024-02-16T22:56:40.892 INFO:tasks.workunit.client.0.smithi064.stdout:4/895: stat dc/ddc/de8/da1/cc9 0 2024-02-16T22:56:40.893 INFO:tasks.workunit.client.0.smithi064.stdout:6/963: mkdir d0/d6/da/da2/d141 0 2024-02-16T22:56:40.894 INFO:tasks.workunit.client.0.smithi064.stdout:6/964: fsync d0/d5/df/d17/ded/d12d/f138 0 2024-02-16T22:56:40.895 INFO:tasks.workunit.client.0.smithi064.stdout:7/891: link d5/f23 d5/d33/d53/df2/d54/d7b/d11f/d66/f121 0 2024-02-16T22:56:40.898 INFO:tasks.workunit.client.0.smithi064.stdout:0/744: creat da/dc/de/d12/d2c/d2e/d96/ff8 x:0 0 0 2024-02-16T22:56:40.902 INFO:tasks.workunit.client.0.smithi064.stdout:5/838: truncate d6/d25/df1/f7f 253765 0 2024-02-16T22:56:40.904 INFO:tasks.workunit.client.0.smithi064.stdout:8/799: dread df/d31/d2f/d4f/fa7 [4194304,4194304] 0 2024-02-16T22:56:40.910 INFO:tasks.workunit.client.0.smithi064.stdout:1/812: creat d2/df/d11/df7/f10f x:0 0 0 2024-02-16T22:56:40.914 INFO:tasks.workunit.client.0.smithi064.stdout:9/935: write d3/d9/d6a/db1/d38/f66 [508850,7560] 0 2024-02-16T22:56:40.915 INFO:tasks.workunit.client.0.smithi064.stdout:8/800: mknod df/d31/d2f/d3e/c112 0 2024-02-16T22:56:40.916 INFO:tasks.workunit.client.0.smithi064.stdout:8/801: chown df/d2a/d5c/fed 17121732 1 2024-02-16T22:56:40.918 INFO:tasks.workunit.client.0.smithi064.stdout:1/813: creat d2/df/d11/d1a/d30/d3d/f110 x:0 0 0 2024-02-16T22:56:40.920 INFO:tasks.workunit.client.0.smithi064.stdout:1/814: fsync d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce 0 2024-02-16T22:56:40.921 INFO:tasks.workunit.client.0.smithi064.stdout:4/896: rename dc/de/c13 to dc/dd/d16/c11d 0 2024-02-16T22:56:40.921 INFO:tasks.workunit.client.0.smithi064.stdout:8/802: dread df/d31/d61/d79/f9b [0,4194304] 0 2024-02-16T22:56:40.922 INFO:tasks.workunit.client.0.smithi064.stdout:9/936: fdatasync d3/d9/d1e/d3e/d76/fda 0 2024-02-16T22:56:40.924 INFO:tasks.workunit.client.0.smithi064.stdout:9/937: dread - d3/d11/d15/dab/fcc zero size 2024-02-16T22:56:40.924 INFO:tasks.workunit.client.0.smithi064.stdout:5/839: dread d6/d25/df1/d91/da5/de3/f54 [0,4194304] 0 2024-02-16T22:56:40.931 INFO:tasks.workunit.client.0.smithi064.stdout:8/803: mkdir df/d31/d2f/d113 0 2024-02-16T22:56:40.933 INFO:tasks.workunit.client.0.smithi064.stdout:6/965: getdents d0/d1d/da8/d60 0 2024-02-16T22:56:40.933 INFO:tasks.workunit.client.0.smithi064.stdout:3/981: sync 2024-02-16T22:56:40.933 INFO:tasks.workunit.client.0.smithi064.stdout:6/966: chown d0/d3c/da9/d76/fd9 1840931610 1 2024-02-16T22:56:40.934 INFO:tasks.workunit.client.0.smithi064.stdout:6/967: write d0/d5/fcd [2153264,14622] 0 2024-02-16T22:56:40.935 INFO:tasks.workunit.client.0.smithi064.stdout:9/938: creat d3/d11/d9e/da9/f134 x:0 0 0 2024-02-16T22:56:40.938 INFO:tasks.workunit.client.0.smithi064.stdout:5/840: symlink d6/d25/df1/d91/da5/de3/d43/d7d/de2/l118 0 2024-02-16T22:56:40.940 INFO:tasks.workunit.client.0.smithi064.stdout:7/892: rename d5/d33/d53/df2/f22 to d5/d33/d53/df2/d54/df4/f122 0 2024-02-16T22:56:40.942 INFO:tasks.workunit.client.0.smithi064.stdout:4/897: creat dc/ddc/df0/f11e x:0 0 0 2024-02-16T22:56:40.942 INFO:tasks.workunit.client.0.smithi064.stdout:8/804: truncate df/d31/d61/fa8 2314390 0 2024-02-16T22:56:40.945 INFO:tasks.workunit.client.0.smithi064.stdout:9/939: creat d3/d88/f135 x:0 0 0 2024-02-16T22:56:40.949 INFO:tasks.workunit.client.0.smithi064.stdout:0/745: rename da/dc/de/d12/d1d/c55 to da/dc/de/d37/daa/de9/cf9 0 2024-02-16T22:56:40.950 INFO:tasks.workunit.client.0.smithi064.stdout:1/815: link d2/l10 d2/df/d14/d2a/d108/l111 0 2024-02-16T22:56:40.951 INFO:tasks.workunit.client.0.smithi064.stdout:1/816: write d2/df/d11/d1a/d30/ffc [287155,126698] 0 2024-02-16T22:56:40.952 INFO:tasks.workunit.client.0.smithi064.stdout:7/893: truncate d5/d33/d53/df2/d54/f63 2268290 0 2024-02-16T22:56:40.956 INFO:tasks.workunit.client.0.smithi064.stdout:8/805: read df/f32 [3237074,106843] 0 2024-02-16T22:56:40.957 INFO:tasks.workunit.client.0.smithi064.stdout:4/898: dread dc/dd/d32/fa5 [0,4194304] 0 2024-02-16T22:56:40.961 INFO:tasks.workunit.client.0.smithi064.stdout:3/982: mkdir d4/d1f/d14d 0 2024-02-16T22:56:40.962 INFO:tasks.workunit.client.0.smithi064.stdout:3/983: readlink d4/d14/d2b/d10b/l140 0 2024-02-16T22:56:40.962 INFO:tasks.workunit.client.0.smithi064.stdout:9/940: write d3/d9/d6a/db1/d2a/d2e/d31/d3f/faa [1436612,67572] 0 2024-02-16T22:56:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:0/746: mknod da/dc/d3b/d5a/cfa 0 2024-02-16T22:56:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:0/747: chown da/dc/d3b/d72/db3 0 1 2024-02-16T22:56:40.973 INFO:tasks.workunit.client.0.smithi064.stdout:7/894: creat d5/d33/d53/dfe/f123 x:0 0 0 2024-02-16T22:56:40.977 INFO:tasks.workunit.client.0.smithi064.stdout:0/748: dwrite da/dc/de/d12/f42 [0,4194304] 0 2024-02-16T22:56:40.979 INFO:tasks.workunit.client.0.smithi064.stdout:6/968: dwrite d0/d6/d10/d33/da5/fc1 [0,4194304] 0 2024-02-16T22:56:40.981 INFO:tasks.workunit.client.0.smithi064.stdout:6/969: fsync d0/d1d/d21/f2d 0 2024-02-16T22:56:40.992 INFO:tasks.workunit.client.0.smithi064.stdout:5/841: creat d6/d25/df1/d91/da5/de3/d43/d7d/f119 x:0 0 0 2024-02-16T22:56:40.993 INFO:tasks.workunit.client.0.smithi064.stdout:5/842: dread - d6/ff0 zero size 2024-02-16T22:56:40.997 INFO:tasks.workunit.client.0.smithi064.stdout:7/895: rename d5/d33/d53/df2/d54/d7b/fb1 to d5/d33/d53/df2/d65/da7/d120/f124 0 2024-02-16T22:56:40.998 INFO:tasks.workunit.client.0.smithi064.stdout:8/806: fdatasync df/f1b 0 2024-02-16T22:56:40.999 INFO:tasks.workunit.client.0.smithi064.stdout:7/896: truncate d5/df9/fdf 408280 0 2024-02-16T22:56:41.001 INFO:tasks.workunit.client.0.smithi064.stdout:3/984: symlink d4/d1f/l14e 0 2024-02-16T22:56:41.004 INFO:tasks.workunit.client.0.smithi064.stdout:5/843: fdatasync d6/d25/d48/f4a 0 2024-02-16T22:56:41.005 INFO:tasks.workunit.client.0.smithi064.stdout:4/899: write dc/dd/d32/d22/f93 [1569129,21280] 0 2024-02-16T22:56:41.007 INFO:tasks.workunit.client.0.smithi064.stdout:1/817: creat d2/df/d14/d2c/f112 x:0 0 0 2024-02-16T22:56:41.009 INFO:tasks.workunit.client.0.smithi064.stdout:1/818: readlink d2/df/d14/d18/d48/d6b/lbe 0 2024-02-16T22:56:41.011 INFO:tasks.workunit.client.0.smithi064.stdout:0/749: rename da/dc/de/d12/d2c/l58 to da/dc/de/d12/d2c/d2e/d38/dbe/lfb 0 2024-02-16T22:56:41.013 INFO:tasks.workunit.client.0.smithi064.stdout:9/941: dwrite d3/d11/d15/dab/fcc [0,4194304] 0 2024-02-16T22:56:41.013 INFO:tasks.workunit.client.0.smithi064.stdout:4/900: dread dc/dd/d32/f3d [0,4194304] 0 2024-02-16T22:56:41.018 INFO:tasks.workunit.client.0.smithi064.stdout:1/819: dwrite d2/df/d11/d25/fc5 [0,4194304] 0 2024-02-16T22:56:41.026 INFO:tasks.workunit.client.0.smithi064.stdout:5/844: mkdir d6/d25/df1/d80/d11a 0 2024-02-16T22:56:41.031 INFO:tasks.workunit.client.0.smithi064.stdout:8/807: symlink df/d31/d2f/dfe/l114 0 2024-02-16T22:56:41.033 INFO:tasks.workunit.client.0.smithi064.stdout:6/970: write d0/d89/fae [2276121,46104] 0 2024-02-16T22:56:41.034 INFO:tasks.workunit.client.0.smithi064.stdout:4/901: symlink dc/de/dfb/l11f 0 2024-02-16T22:56:41.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:40 smithi192 ceph-mon[36254]: pgmap v25: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 83 MiB/s rd, 182 MiB/s wr, 419 op/s 2024-02-16T22:56:41.035 INFO:tasks.workunit.client.0.smithi064.stdout:3/985: write d4/d13/d1b/fcb [1085550,15963] 0 2024-02-16T22:56:41.036 INFO:tasks.workunit.client.0.smithi064.stdout:9/942: read d3/d9/d1e/d42/f54 [907518,113819] 0 2024-02-16T22:56:41.036 INFO:tasks.workunit.client.0.smithi064.stdout:3/986: dread - d4/d14/d2b/d141/f14c zero size 2024-02-16T22:56:41.038 INFO:tasks.workunit.client.0.smithi064.stdout:9/943: write d3/d11/d9e/da9/f134 [395155,46343] 0 2024-02-16T22:56:41.039 INFO:tasks.workunit.client.0.smithi064.stdout:7/897: mkdir d5/d33/d125 0 2024-02-16T22:56:41.042 INFO:tasks.workunit.client.0.smithi064.stdout:1/820: creat d2/df/d14/d18/d48/d84/f113 x:0 0 0 2024-02-16T22:56:41.050 INFO:tasks.workunit.client.0.smithi064.stdout:3/987: unlink d4/d13/d1b/d27/fe7 0 2024-02-16T22:56:41.052 INFO:tasks.workunit.client.0.smithi064.stdout:9/944: creat d3/d11/d9e/da9/f136 x:0 0 0 2024-02-16T22:56:41.053 INFO:tasks.workunit.client.0.smithi064.stdout:9/945: chown d3/d11/d15/d23/l2d 1281576 1 2024-02-16T22:56:41.055 INFO:tasks.workunit.client.0.smithi064.stdout:1/821: unlink d2/df/c2e 0 2024-02-16T22:56:41.059 INFO:tasks.workunit.client.0.smithi064.stdout:8/808: creat df/d31/d2f/d38/d48/da2/de2/f115 x:0 0 0 2024-02-16T22:56:41.060 INFO:tasks.workunit.client.0.smithi064.stdout:6/971: mknod d0/d5/df/d17/ded/d106/c142 0 2024-02-16T22:56:41.061 INFO:tasks.workunit.client.0.smithi064.stdout:6/972: stat d0/d12/d7b/l12f 0 2024-02-16T22:56:41.062 INFO:tasks.workunit.client.0.smithi064.stdout:0/750: truncate da/dc/de/d12/f29 5333494 0 2024-02-16T22:56:41.065 INFO:tasks.workunit.client.0.smithi064.stdout:7/898: rename d5/d33/d53/df2/c4d to d5/d33/d53/df2/d65/da7/c126 0 2024-02-16T22:56:41.067 INFO:tasks.workunit.client.0.smithi064.stdout:9/946: mkdir d3/d9/d6a/db1/d2a/d2e/da3/dea/d137 0 2024-02-16T22:56:41.068 INFO:tasks.workunit.client.0.smithi064.stdout:5/845: link d6/d25/df1/d91/da5/de3/d43/d7d/f119 d6/d25/da4/dc1/d8e/f11b 0 2024-02-16T22:56:41.076 INFO:tasks.workunit.client.0.smithi064.stdout:8/809: unlink df/d31/d8f/d80/dec/fef 0 2024-02-16T22:56:41.076 INFO:tasks.workunit.client.0.smithi064.stdout:1/822: dread d2/df/d11/d1a/d30/dbd/fe2 [0,4194304] 0 2024-02-16T22:56:41.079 INFO:tasks.workunit.client.0.smithi064.stdout:4/902: truncate dc/de/dfb/dff/de3/fc1 30967 0 2024-02-16T22:56:41.083 INFO:tasks.workunit.client.0.smithi064.stdout:0/751: mkdir da/dc/d3b/d72/db3/dfc 0 2024-02-16T22:56:41.091 INFO:tasks.workunit.client.0.smithi064.stdout:4/903: dread dc/dd/d16/fa6 [0,4194304] 0 2024-02-16T22:56:41.093 INFO:tasks.workunit.client.0.smithi064.stdout:5/846: creat d6/d18/dc5/f11c x:0 0 0 2024-02-16T22:56:41.109 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:40 smithi064 ceph-mon[29226]: pgmap v25: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 83 MiB/s rd, 182 MiB/s wr, 419 op/s 2024-02-16T22:56:41.139 INFO:tasks.workunit.client.0.smithi064.stdout:8/810: symlink df/d31/d2f/d3e/d5f/d93/dd2/l116 0 2024-02-16T22:56:41.140 INFO:tasks.workunit.client.0.smithi064.stdout:1/823: truncate d2/df/d11/fc1 788822 0 2024-02-16T22:56:41.141 INFO:tasks.workunit.client.0.smithi064.stdout:6/973: getdents d0/d6/da/da2/d141 0 2024-02-16T22:56:41.141 INFO:tasks.workunit.client.0.smithi064.stdout:6/974: chown d0/d6/f127 35648 1 2024-02-16T22:56:41.142 INFO:tasks.workunit.client.0.smithi064.stdout:6/975: fsync d0/d1d/da8/d60/f7c 0 2024-02-16T22:56:41.143 INFO:tasks.workunit.client.0.smithi064.stdout:0/752: mkdir da/dc/de/d37/daa/dfd 0 2024-02-16T22:56:41.143 INFO:tasks.workunit.client.0.smithi064.stdout:3/988: link d4/d13/d4d/d5c/lca d4/d13/d4d/l14f 0 2024-02-16T22:56:41.225 INFO:tasks.workunit.client.0.smithi064.stdout:8/811: symlink df/d31/d2f/d3e/ddf/l117 0 2024-02-16T22:56:41.226 INFO:tasks.workunit.client.0.smithi064.stdout:0/753: mkdir da/dc/de/d12/d2c/d2e/d38/dbe/dfe 0 2024-02-16T22:56:41.227 INFO:tasks.workunit.client.0.smithi064.stdout:7/899: write d5/d33/d53/df2/fb [2190193,93983] 0 2024-02-16T22:56:41.228 INFO:tasks.workunit.client.0.smithi064.stdout:3/989: dwrite d4/d13/d1b/d27/d99/d7d/f11d [0,4194304] 0 2024-02-16T22:56:41.229 INFO:tasks.workunit.client.0.smithi064.stdout:0/754: chown da/dc/d3b/d72/db3 2116 1 2024-02-16T22:56:41.229 INFO:tasks.workunit.client.0.smithi064.stdout:5/847: mkdir d6/d25/df1/d91/d11d 0 2024-02-16T22:56:41.283 INFO:tasks.workunit.client.0.smithi064.stdout:6/976: fsync d0/d1d/dbc/fc9 0 2024-02-16T22:56:41.284 INFO:tasks.workunit.client.0.smithi064.stdout:9/947: getdents d3/d9/d6a/db1/d2a/d2e/d31/d95 0 2024-02-16T22:56:41.285 INFO:tasks.workunit.client.0.smithi064.stdout:3/990: fsync d4/d1f/f67 0 2024-02-16T22:56:41.286 INFO:tasks.workunit.client.0.smithi064.stdout:8/812: write df/d31/d61/d79/f8c [134014,124464] 0 2024-02-16T22:56:41.287 INFO:tasks.workunit.client.0.smithi064.stdout:7/900: mkdir d5/d33/d53/df2/d54/d7b/d11f/d5d/d127 0 2024-02-16T22:56:41.290 INFO:tasks.workunit.client.0.smithi064.stdout:9/948: creat d3/d11/d9e/f138 x:0 0 0 2024-02-16T22:56:41.291 INFO:tasks.workunit.client.0.smithi064.stdout:9/949: chown d3/d9/d1e/d3e/d76 332012 1 2024-02-16T22:56:41.293 INFO:tasks.workunit.client.0.smithi064.stdout:4/904: dread dc/f6c [0,4194304] 0 2024-02-16T22:56:41.298 INFO:tasks.workunit.client.0.smithi064.stdout:6/977: truncate d0/d5/df/d17/ff2 414454 0 2024-02-16T22:56:41.299 INFO:tasks.workunit.client.0.smithi064.stdout:5/848: write d6/d25/da4/dc1/f76 [543830,110473] 0 2024-02-16T22:56:41.300 INFO:tasks.workunit.client.0.smithi064.stdout:7/901: symlink d5/df9/de4/df5/l128 0 2024-02-16T22:56:41.301 INFO:tasks.workunit.client.0.smithi064.stdout:9/950: rmdir d3/d9/d6a/db1/d2a/d2e/da3 39 2024-02-16T22:56:41.301 INFO:tasks.workunit.client.0.smithi064.stdout:0/755: dread da/dc/de/d12/f2a [4194304,4194304] 0 2024-02-16T22:56:41.305 INFO:tasks.workunit.client.0.smithi064.stdout:4/905: mknod dc/dd/d32/d46/d5a/d7e/c120 0 2024-02-16T22:56:41.305 INFO:tasks.workunit.client.0.smithi064.stdout:3/991: mkdir d4/d13/d1b/d150 0 2024-02-16T22:56:41.307 INFO:tasks.workunit.client.0.smithi064.stdout:8/813: dwrite df/d31/d61/f65 [0,4194304] 0 2024-02-16T22:56:41.308 INFO:tasks.workunit.client.0.smithi064.stdout:6/978: symlink d0/d6/d10/d33/da4/ddd/l143 0 2024-02-16T22:56:41.308 INFO:tasks.workunit.client.0.smithi064.stdout:4/906: read dc/dd/d32/d58/f38 [2001,82036] 0 2024-02-16T22:56:41.312 INFO:tasks.workunit.client.0.smithi064.stdout:9/951: rmdir d3/d9/d1e/d3e 39 2024-02-16T22:56:41.314 INFO:tasks.workunit.client.0.smithi064.stdout:3/992: dread - d4/d13/d1b/d27/d44/fe0 zero size 2024-02-16T22:56:41.314 INFO:tasks.workunit.client.0.smithi064.stdout:5/849: mkdir d6/d18/d19/d112/d11e 0 2024-02-16T22:56:41.319 INFO:tasks.workunit.client.0.smithi064.stdout:8/814: read df/d31/d2f/d3e/d5f/fc7 [4061949,89642] 0 2024-02-16T22:56:41.320 INFO:tasks.workunit.client.0.smithi064.stdout:7/902: fsync d5/d33/d53/df2/f19 0 2024-02-16T22:56:41.324 INFO:tasks.workunit.client.0.smithi064.stdout:0/756: creat da/dc/d3b/d72/db3/dfc/fff x:0 0 0 2024-02-16T22:56:41.325 INFO:tasks.workunit.client.0.smithi064.stdout:3/993: dwrite d4/d14/d2b/d141/f14c [0,4194304] 0 2024-02-16T22:56:41.326 INFO:tasks.workunit.client.0.smithi064.stdout:5/850: truncate d6/d18/d95/dbe/fe1 142542 0 2024-02-16T22:56:41.331 INFO:tasks.workunit.client.0.smithi064.stdout:7/903: read - d5/d33/d53/df2/d54/d7b/d11f/f10c zero size 2024-02-16T22:56:41.332 INFO:tasks.workunit.client.0.smithi064.stdout:4/907: dread dc/de/d28/d57/d77/d101/f4c [0,4194304] 0 2024-02-16T22:56:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:0/757: mkdir da/dc/de/d12/d1d/d100 0 2024-02-16T22:56:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:8/815: dwrite df/d31/d47/ff7 [0,4194304] 0 2024-02-16T22:56:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:9/952: dwrite d3/d9/d1e/d42/d7b/fa8 [0,4194304] 0 2024-02-16T22:56:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:0/758: chown da/d71/cde 184 1 2024-02-16T22:56:41.343 INFO:tasks.workunit.client.0.smithi064.stdout:6/979: write d0/d5/df/d17/da1/f10f [502153,123563] 0 2024-02-16T22:56:41.346 INFO:tasks.workunit.client.0.smithi064.stdout:0/759: dwrite da/dc/d3b/d72/db3/dfc/fff [0,4194304] 0 2024-02-16T22:56:41.351 INFO:tasks.workunit.client.0.smithi064.stdout:7/904: creat d5/d33/d53/df2/d65/d7f/f129 x:0 0 0 2024-02-16T22:56:41.356 INFO:tasks.workunit.client.0.smithi064.stdout:9/953: rmdir d3/d89/d9a 39 2024-02-16T22:56:41.357 INFO:tasks.workunit.client.0.smithi064.stdout:8/816: symlink df/d31/d2f/d3e/ddf/l118 0 2024-02-16T22:56:41.359 INFO:tasks.workunit.client.0.smithi064.stdout:5/851: creat d6/d25/df1/d91/da5/de3/d4b/f11f x:0 0 0 2024-02-16T22:56:41.359 INFO:tasks.workunit.client.0.smithi064.stdout:3/994: creat d4/d13/f151 x:0 0 0 2024-02-16T22:56:41.360 INFO:tasks.workunit.client.0.smithi064.stdout:3/995: stat d4/d13/f62 0 2024-02-16T22:56:41.366 INFO:tasks.workunit.client.0.smithi064.stdout:6/980: mkdir d0/d1d/dfa/d144 0 2024-02-16T22:56:41.367 INFO:tasks.workunit.client.0.smithi064.stdout:6/981: chown d0/d6/d10/d33/l20 2062490063 1 2024-02-16T22:56:41.367 INFO:tasks.workunit.client.0.smithi064.stdout:0/760: creat da/dc/de/d12/d1d/f101 x:0 0 0 2024-02-16T22:56:41.367 INFO:tasks.workunit.client.0.smithi064.stdout:5/852: dwrite d6/d18/d19/f1f [0,4194304] 0 2024-02-16T22:56:41.368 INFO:tasks.workunit.client.0.smithi064.stdout:0/761: stat da/dc/de/d12/d2c/fc5 0 2024-02-16T22:56:41.368 INFO:tasks.workunit.client.0.smithi064.stdout:7/905: symlink d5/d33/d53/df2/d54/df4/l12a 0 2024-02-16T22:56:41.371 INFO:tasks.workunit.client.0.smithi064.stdout:0/762: truncate da/dc/de/d12/d2c/d2e/d38/dbe/dc6/fd8 470218 0 2024-02-16T22:56:41.371 INFO:tasks.workunit.client.0.smithi064.stdout:9/954: readlink d3/d9/d1e/d42/d7b/d86/l90 0 2024-02-16T22:56:41.376 INFO:tasks.workunit.client.0.smithi064.stdout:3/996: creat d4/d13/d1b/d27/d99/f152 x:0 0 0 2024-02-16T22:56:41.379 INFO:tasks.workunit.client.0.smithi064.stdout:6/982: symlink d0/d89/l145 0 2024-02-16T22:56:41.383 INFO:tasks.workunit.client.0.smithi064.stdout:4/908: dwrite dc/dd/d16/f5d [0,4194304] 0 2024-02-16T22:56:41.388 INFO:tasks.workunit.client.0.smithi064.stdout:9/955: truncate d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/f11f 651961 0 2024-02-16T22:56:41.388 INFO:tasks.workunit.client.0.smithi064.stdout:8/817: dwrite df/f4c [0,4194304] 0 2024-02-16T22:56:41.389 INFO:tasks.workunit.client.0.smithi064.stdout:0/763: write da/dc/d3b/f7b [2131086,55191] 0 2024-02-16T22:56:41.389 INFO:tasks.workunit.client.0.smithi064.stdout:8/818: write df/d31/d8f/dc9/d73/ffa [508158,17278] 0 2024-02-16T22:56:41.395 INFO:tasks.workunit.client.0.smithi064.stdout:4/909: truncate dc/dd/d4f/fa0 180942 0 2024-02-16T22:56:41.399 INFO:tasks.workunit.client.0.smithi064.stdout:0/764: readlink da/d26/d3f/l9a 0 2024-02-16T22:56:41.400 INFO:tasks.workunit.client.0.smithi064.stdout:7/906: fsync d5/d33/d53/df2/d65/da7/fce 0 2024-02-16T22:56:41.401 INFO:tasks.workunit.client.0.smithi064.stdout:7/907: chown d5/d7d/fa0 3726 1 2024-02-16T22:56:41.401 INFO:tasks.workunit.client.0.smithi064.stdout:8/819: unlink df/d31/d47/la9 0 2024-02-16T22:56:41.402 INFO:tasks.workunit.client.0.smithi064.stdout:6/983: write d0/d1b/f23 [1360013,76303] 0 2024-02-16T22:56:41.403 INFO:tasks.workunit.client.0.smithi064.stdout:9/956: dread d3/d9/d1e/d42/f9f [0,4194304] 0 2024-02-16T22:56:41.407 INFO:tasks.workunit.client.0.smithi064.stdout:0/765: mknod da/d26/de0/d93/c102 0 2024-02-16T22:56:41.408 INFO:tasks.workunit.client.0.smithi064.stdout:8/820: dread df/d31/d2f/d4f/d6a/fb0 [0,4194304] 0 2024-02-16T22:56:41.409 INFO:tasks.workunit.client.0.smithi064.stdout:5/853: truncate d6/d25/df1/d91/da5/de3/d43/db9/d115/f10d 351089 0 2024-02-16T22:56:41.410 INFO:tasks.workunit.client.0.smithi064.stdout:7/908: symlink d5/d33/d53/df2/d54/df4/l12b 0 2024-02-16T22:56:41.412 INFO:tasks.workunit.client.0.smithi064.stdout:4/910: write dc/de/d28/fbf [1757677,105088] 0 2024-02-16T22:56:41.418 INFO:tasks.workunit.client.0.smithi064.stdout:3/997: getdents d4/d13/d1b/d27/d44 0 2024-02-16T22:56:41.419 INFO:tasks.workunit.client.0.smithi064.stdout:0/766: readlink da/l70 0 2024-02-16T22:56:41.421 INFO:tasks.workunit.client.0.smithi064.stdout:8/821: rename df/d31/d2f/lbc to df/d31/d8f/dab/dae/l119 0 2024-02-16T22:56:41.425 INFO:tasks.workunit.client.0.smithi064.stdout:7/909: fdatasync d5/d33/d53/f99 0 2024-02-16T22:56:41.426 INFO:tasks.workunit.client.0.smithi064.stdout:7/910: chown d5/d33/d53/df2/d54 63651 1 2024-02-16T22:56:41.431 INFO:tasks.workunit.client.0.smithi064.stdout:3/998: dread d4/d13/d1b/d27/d44/f5f [0,4194304] 0 2024-02-16T22:56:41.436 INFO:tasks.workunit.client.0.smithi064.stdout:5/854: dread d6/d18/f56 [0,4194304] 0 2024-02-16T22:56:41.440 INFO:tasks.workunit.client.0.smithi064.stdout:9/957: truncate d3/f8f 810408 0 2024-02-16T22:56:41.443 INFO:tasks.workunit.client.0.smithi064.stdout:7/911: creat d5/d33/d42/d81/f12c x:0 0 0 2024-02-16T22:56:41.444 INFO:tasks.workunit.client.0.smithi064.stdout:4/911: dwrite dc/de/d28/d57/d77/f107 [0,4194304] 0 2024-02-16T22:56:41.446 INFO:tasks.workunit.client.0.smithi064.stdout:6/984: getdents d0/d1d/da8/d60 0 2024-02-16T22:56:41.447 INFO:tasks.workunit.client.0.smithi064.stdout:6/985: dread - d0/d6/da/f95 zero size 2024-02-16T22:56:41.449 INFO:tasks.workunit.client.0.smithi064.stdout:5/855: fsync d6/d25/d48/f4a 0 2024-02-16T22:56:41.450 INFO:tasks.workunit.client.0.smithi064.stdout:6/986: stat d0/d3c/fef 0 2024-02-16T22:56:41.451 INFO:tasks.workunit.client.0.smithi064.stdout:0/767: unlink da/dc/de/d12/f6c 0 2024-02-16T22:56:41.457 INFO:tasks.workunit.client.0.smithi064.stdout:8/822: dwrite df/d31/d8f/d80/fe5 [0,4194304] 0 2024-02-16T22:56:41.457 INFO:tasks.workunit.client.0.smithi064.stdout:7/912: dwrite d5/f23 [0,4194304] 0 2024-02-16T22:56:41.457 INFO:tasks.workunit.client.0.smithi064.stdout:9/958: rename d3/d9/d1e/d42/l99 to d3/d11/d15/dab/db6/dd5/d12b/d132/l139 0 2024-02-16T22:56:41.458 INFO:tasks.workunit.client.0.smithi064.stdout:0/768: readlink da/dc/de/d12/d2c/d2e/d38/l52 0 2024-02-16T22:56:41.464 INFO:tasks.workunit.client.0.smithi064.stdout:3/999: mkdir d4/d1f/d34/d153 0 2024-02-16T22:56:41.465 INFO:tasks.workunit.client.0.smithi064.stdout:4/912: dread dc/dd/d4f/fa0 [0,4194304] 0 2024-02-16T22:56:41.470 INFO:tasks.workunit.client.0.smithi064.stdout:6/987: dwrite d0/d5/df/d17/d18/fb3 [0,4194304] 0 2024-02-16T22:56:41.472 INFO:tasks.workunit.client.0.smithi064.stdout:0/769: dwrite da/dc/de/d12/d2c/f31 [0,4194304] 0 2024-02-16T22:56:41.472 INFO:tasks.workunit.client.0.smithi064.stdout:9/959: truncate d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/f10c 792898 0 2024-02-16T22:56:41.481 INFO:tasks.workunit.client.0.smithi064.stdout:7/913: rename d5/d33/d53/df2/d54/d7b/d11f/d5d to d5/d7d/d12d 0 2024-02-16T22:56:41.485 INFO:tasks.workunit.client.0.smithi064.stdout:4/913: mkdir dc/de/d28/d57/d77/d101/d121 0 2024-02-16T22:56:41.486 INFO:tasks.workunit.client.0.smithi064.stdout:4/914: stat dc/dd/d32/d46/d5a/d7e/d9a/da2/ff7 0 2024-02-16T22:56:41.488 INFO:tasks.workunit.client.0.smithi064.stdout:0/770: mkdir da/d26/d103 0 2024-02-16T22:56:41.491 INFO:tasks.workunit.client.0.smithi064.stdout:6/988: rmdir d0/d5/df/d17/ded 39 2024-02-16T22:56:41.493 INFO:tasks.workunit.client.0.smithi064.stdout:8/823: dread f9 [0,4194304] 0 2024-02-16T22:56:41.495 INFO:tasks.workunit.client.0.smithi064.stdout:4/915: rename dc/dd/d32/d58/c37 to dc/ddc/de8/df2/c122 0 2024-02-16T22:56:41.496 INFO:tasks.workunit.client.0.smithi064.stdout:4/916: chown dc/de/d28/d57/d77/d101/d43/d7a 35495 1 2024-02-16T22:56:41.497 INFO:tasks.workunit.client.0.smithi064.stdout:7/914: read d5/d33/d53/f70 [1157241,41170] 0 2024-02-16T22:56:41.499 INFO:tasks.workunit.client.0.smithi064.stdout:4/917: chown dc/dd/d32/d46/d5a/d7e/d9a 2412551 1 2024-02-16T22:56:41.499 INFO:tasks.workunit.client.0.smithi064.stdout:7/915: fdatasync d5/d33/d53/df2/d65/ff6 0 2024-02-16T22:56:41.499 INFO:tasks.workunit.client.0.smithi064.stdout:8/824: mknod df/d31/d8f/d80/dec/c11a 0 2024-02-16T22:56:41.501 INFO:tasks.workunit.client.0.smithi064.stdout:0/771: symlink da/d26/d103/l104 0 2024-02-16T22:56:41.505 INFO:tasks.workunit.client.0.smithi064.stdout:5/856: link d6/d25/df1/d91/da5/de3/d43/f4d d6/d25/da4/dc1/d8e/f120 0 2024-02-16T22:56:41.508 INFO:tasks.workunit.client.0.smithi064.stdout:7/916: unlink d5/d33/d42/c6c 0 2024-02-16T22:56:41.511 INFO:tasks.workunit.client.0.smithi064.stdout:4/918: creat dc/dd/d32/d46/d5a/dcb/f123 x:0 0 0 2024-02-16T22:56:41.513 INFO:tasks.workunit.client.0.smithi064.stdout:9/960: dwrite d3/d9/d1e/f8d [0,4194304] 0 2024-02-16T22:56:41.513 INFO:tasks.workunit.client.0.smithi064.stdout:4/919: readlink dc/de/d28/d57/d77/d101/l49 0 2024-02-16T22:56:41.517 INFO:tasks.workunit.client.0.smithi064.stdout:0/772: dread - da/dc/d3b/f8e zero size 2024-02-16T22:56:41.517 INFO:tasks.workunit.client.0.smithi064.stdout:0/773: chown f9 3459 1 2024-02-16T22:56:41.518 INFO:tasks.workunit.client.0.smithi064.stdout:6/989: write d0/d1d/dbc/fde [8445,54424] 0 2024-02-16T22:56:41.521 INFO:tasks.workunit.client.0.smithi064.stdout:6/990: dread d0/d6/d10/d33/da5/f114 [0,4194304] 0 2024-02-16T22:56:41.521 INFO:tasks.workunit.client.0.smithi064.stdout:5/857: truncate d6/d25/da4/dc1/fa1 1002686 0 2024-02-16T22:56:41.522 INFO:tasks.workunit.client.0.smithi064.stdout:8/825: mkdir df/d31/d2f/d38/d48/da2/d11b 0 2024-02-16T22:56:41.523 INFO:tasks.workunit.client.0.smithi064.stdout:4/920: fdatasync dc/de/dfb/dff/de3/fb7 0 2024-02-16T22:56:41.524 INFO:tasks.workunit.client.0.smithi064.stdout:7/917: dread d5/d7d/d12d/f80 [0,4194304] 0 2024-02-16T22:56:41.525 INFO:tasks.workunit.client.0.smithi064.stdout:9/961: chown d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/d110/l11b 14752928 1 2024-02-16T22:56:41.526 INFO:tasks.workunit.client.0.smithi064.stdout:0/774: creat da/dc/d3b/d5a/d8f/da8/f105 x:0 0 0 2024-02-16T22:56:41.529 INFO:tasks.workunit.client.0.smithi064.stdout:6/991: symlink d0/d89/l146 0 2024-02-16T22:56:41.529 INFO:tasks.workunit.client.0.smithi064.stdout:8/826: mkdir df/d31/d2f/d3e/d5f/d93/d11c 0 2024-02-16T22:56:41.532 INFO:tasks.workunit.client.0.smithi064.stdout:4/921: dread dc/de/dfb/dff/de3/fc1 [0,4194304] 0 2024-02-16T22:56:41.534 INFO:tasks.workunit.client.0.smithi064.stdout:4/922: dread - dc/dd/d32/d46/d5a/dcb/f123 zero size 2024-02-16T22:56:41.535 INFO:tasks.workunit.client.0.smithi064.stdout:4/923: readlink dc/dd/d16/ld9 0 2024-02-16T22:56:41.537 INFO:tasks.workunit.client.0.smithi064.stdout:5/858: mkdir d6/d25/d121 0 2024-02-16T22:56:41.540 INFO:tasks.workunit.client.0.smithi064.stdout:7/918: mkdir d5/d33/d53/df2/d54/d7b/df8/d12e 0 2024-02-16T22:56:41.541 INFO:tasks.workunit.client.0.smithi064.stdout:7/919: dread - d5/d33/d53/f75 zero size 2024-02-16T22:56:41.542 INFO:tasks.workunit.client.0.smithi064.stdout:7/920: chown d5/d33/d53/da3 310563764 1 2024-02-16T22:56:41.545 INFO:tasks.workunit.client.0.smithi064.stdout:6/992: rename d0/d1b/fcb to d0/d5/df/d17/ded/f147 0 2024-02-16T22:56:41.547 INFO:tasks.workunit.client.0.smithi064.stdout:0/775: dread f9 [0,4194304] 0 2024-02-16T22:56:41.548 INFO:tasks.workunit.client.0.smithi064.stdout:4/924: dread f3 [0,4194304] 0 2024-02-16T22:56:41.548 INFO:tasks.workunit.client.0.smithi064.stdout:8/827: creat df/d2a/ddb/f11d x:0 0 0 2024-02-16T22:56:41.548 INFO:tasks.workunit.client.0.smithi064.stdout:5/859: mknod d6/d25/df1/d91/da5/de3/d43/db9/dfc/c122 0 2024-02-16T22:56:41.560 INFO:tasks.workunit.client.0.smithi064.stdout:5/860: mknod d6/d25/df1/d91/da5/de3/d43/d7d/de2/c123 0 2024-02-16T22:56:41.561 INFO:tasks.workunit.client.0.smithi064.stdout:9/962: link d3/d11/d9e/da9/cc1 d3/d9/d6a/db1/d2a/d2e/d31/c13a 0 2024-02-16T22:56:41.561 INFO:tasks.workunit.client.0.smithi064.stdout:5/861: chown d6/d18/df8 118 1 2024-02-16T22:56:41.562 INFO:tasks.workunit.client.0.smithi064.stdout:8/828: dwrite df/d31/d2f/fc3 [0,4194304] 0 2024-02-16T22:56:41.563 INFO:tasks.workunit.client.0.smithi064.stdout:5/862: write d6/f24 [1793465,56691] 0 2024-02-16T22:56:41.567 INFO:tasks.workunit.client.0.smithi064.stdout:7/921: creat d5/d33/d53/df2/d54/d7b/d11f/f12f x:0 0 0 2024-02-16T22:56:41.568 INFO:tasks.workunit.client.0.smithi064.stdout:5/863: mkdir d6/d25/d48/d124 0 2024-02-16T22:56:41.569 INFO:tasks.workunit.client.0.smithi064.stdout:6/993: rename d0/d5/df to d0/d1d/da8/d60/dbf/d148 0 2024-02-16T22:56:41.571 INFO:tasks.workunit.client.0.smithi064.stdout:7/922: creat d5/df9/de4/f130 x:0 0 0 2024-02-16T22:56:41.571 INFO:tasks.workunit.client.0.smithi064.stdout:5/864: creat d6/d25/df1/d91/da5/de3/d43/db9/d103/f125 x:0 0 0 2024-02-16T22:56:41.574 INFO:tasks.workunit.client.0.smithi064.stdout:0/776: rename da/d26/de0/d9e/dc0 to da/dc/de/d37/daa/d106 0 2024-02-16T22:56:41.574 INFO:tasks.workunit.client.0.smithi064.stdout:0/777: chown da/dc/de/fc8 24 1 2024-02-16T22:56:41.577 INFO:tasks.workunit.client.0.smithi064.stdout:7/923: symlink d5/d33/d53/df2/d65/d7f/l131 0 2024-02-16T22:56:41.578 INFO:tasks.workunit.client.0.smithi064.stdout:9/963: rename d3/d9/d1e/d42/l118 to d3/d11/d15/d11d/l13b 0 2024-02-16T22:56:41.619 INFO:tasks.workunit.client.0.smithi064.stdout:1/824: sync 2024-02-16T22:56:41.619 INFO:tasks.workunit.client.0.smithi064.stdout:0/778: unlink da/d26/f28 0 2024-02-16T22:56:41.620 INFO:tasks.workunit.client.0.smithi064.stdout:4/925: write dc/d4a/f4d [68512,29161] 0 2024-02-16T22:56:41.626 INFO:tasks.workunit.client.0.smithi064.stdout:8/829: rename df/d31/d8f/dc9/fd4 to df/d31/d8f/dc9/d73/f11e 0 2024-02-16T22:56:41.628 INFO:tasks.workunit.client.0.smithi064.stdout:9/964: truncate d3/ff4 245856 0 2024-02-16T22:56:41.741 INFO:tasks.workunit.client.0.smithi064.stdout:0/779: creat da/dc/de/d12/d2c/d2e/d38/dbe/f107 x:0 0 0 2024-02-16T22:56:41.742 INFO:tasks.workunit.client.0.smithi064.stdout:9/965: mknod d3/d11/d9e/da5/c13c 0 2024-02-16T22:56:41.743 INFO:tasks.workunit.client.0.smithi064.stdout:0/780: fsync da/d26/fb9 0 2024-02-16T22:56:41.745 INFO:tasks.workunit.client.0.smithi064.stdout:8/830: write df/f1b [5779089,67425] 0 2024-02-16T22:56:41.745 INFO:tasks.workunit.client.0.smithi064.stdout:6/994: dwrite d0/d1d/d21/ffd [0,4194304] 0 2024-02-16T22:56:41.748 INFO:tasks.workunit.client.0.smithi064.stdout:4/926: mknod dc/de/d28/d57/d77/dc6/d11c/c124 0 2024-02-16T22:56:41.752 INFO:tasks.workunit.client.0.smithi064.stdout:4/927: write dc/de/d28/d57/d77/d101/faa [299000,120453] 0 2024-02-16T22:56:41.753 INFO:tasks.workunit.client.0.smithi064.stdout:0/781: dwrite da/dc/de/d37/fa6 [0,4194304] 0 2024-02-16T22:56:41.766 INFO:tasks.workunit.client.0.smithi064.stdout:0/782: fdatasync da/dc/d3b/d72/d7c/fcb 0 2024-02-16T22:56:41.766 INFO:tasks.workunit.client.0.smithi064.stdout:1/825: link d2/df/d14/d2a/d108/l111 d2/df/d11/l114 0 2024-02-16T22:56:41.769 INFO:tasks.workunit.client.0.smithi064.stdout:6/995: mkdir d0/d6/da/d149 0 2024-02-16T22:56:41.770 INFO:tasks.workunit.client.0.smithi064.stdout:4/928: dwrite dc/ddc/de8/d110/f113 [0,4194304] 0 2024-02-16T22:56:41.771 INFO:tasks.workunit.client.0.smithi064.stdout:9/966: dwrite d3/d9/d6a/fb2 [0,4194304] 0 2024-02-16T22:56:41.772 INFO:tasks.workunit.client.0.smithi064.stdout:4/929: chown dc/de/fb9 579285 1 2024-02-16T22:56:41.773 INFO:tasks.workunit.client.0.smithi064.stdout:9/967: stat d3/d9/d6a/lf8 0 2024-02-16T22:56:41.773 INFO:tasks.workunit.client.0.smithi064.stdout:5/865: rename d6/d25/df1/d91/da5/de3/d43/db9/d115/fc2 to d6/d25/da4/dc1/d8e/f126 0 2024-02-16T22:56:41.774 INFO:tasks.workunit.client.0.smithi064.stdout:4/930: write dc/de/d28/fbf [1286856,93743] 0 2024-02-16T22:56:41.780 INFO:tasks.workunit.client.0.smithi064.stdout:1/826: chown d2/df/d14/lbc 21496 1 2024-02-16T22:56:41.781 INFO:tasks.workunit.client.0.smithi064.stdout:0/783: rmdir da/dc/de/d12/d2c 39 2024-02-16T22:56:41.784 INFO:tasks.workunit.client.0.smithi064.stdout:6/996: fsync d0/d6/d10/d33/d72/de8/f101 0 2024-02-16T22:56:41.788 INFO:tasks.workunit.client.0.smithi064.stdout:8/831: rename df/d2a/ddb to df/d31/d2f/d3e/ddf/d109/d11f 0 2024-02-16T22:56:41.788 INFO:tasks.workunit.client.0.smithi064.stdout:8/832: chown df/d31/d47/f67 311424 1 2024-02-16T22:56:41.791 INFO:tasks.workunit.client.0.smithi064.stdout:4/931: creat dc/de/d28/d57/d77/f125 x:0 0 0 2024-02-16T22:56:41.796 INFO:tasks.workunit.client.0.smithi064.stdout:8/833: dwrite df/d31/d2f/fc3 [0,4194304] 0 2024-02-16T22:56:41.796 INFO:tasks.workunit.client.0.smithi064.stdout:0/784: read - da/dc/de/d12/d2c/d2e/d38/dbe/f107 zero size 2024-02-16T22:56:41.797 INFO:tasks.workunit.client.0.smithi064.stdout:8/834: chown df/d31/d2f/d3e/d5f/d93/dd2/ce1 75400 1 2024-02-16T22:56:41.799 INFO:tasks.workunit.client.0.smithi064.stdout:6/997: truncate d0/d6/d10/db2/ffc 609494 0 2024-02-16T22:56:41.801 INFO:tasks.workunit.client.0.smithi064.stdout:6/998: fdatasync d0/d5/f42 0 2024-02-16T22:56:41.802 INFO:tasks.workunit.client.0.smithi064.stdout:0/785: dwrite da/dc/de/d12/d2c/d2e/f36 [8388608,4194304] 0 2024-02-16T22:56:41.803 INFO:tasks.workunit.client.0.smithi064.stdout:6/999: dread - d0/d6/d10/d33/d72/de8/f101 zero size 2024-02-16T22:56:41.808 INFO:tasks.workunit.client.0.smithi064.stdout:4/932: creat dc/de/d28/d57/d77/d101/dbd/f126 x:0 0 0 2024-02-16T22:56:41.819 INFO:tasks.workunit.client.0.smithi064.stdout:0/786: dread da/dc/de/d12/f49 [0,4194304] 0 2024-02-16T22:56:41.821 INFO:tasks.workunit.client.0.smithi064.stdout:0/787: unlink da/dc/f1e 0 2024-02-16T22:56:41.825 INFO:tasks.workunit.client.0.smithi064.stdout:0/788: dwrite da/dc/d3b/d5a/d8f/da8/db7/feb [0,4194304] 0 2024-02-16T22:56:41.868 INFO:tasks.workunit.client.0.smithi064.stdout:5/866: truncate d6/d25/df1/d80/fe5 2128332 0 2024-02-16T22:56:41.870 INFO:tasks.workunit.client.0.smithi064.stdout:1/827: write d2/df/f13 [1218988,17328] 0 2024-02-16T22:56:41.871 INFO:tasks.workunit.client.0.smithi064.stdout:9/968: dwrite d3/d11/d15/d23/dee/f123 [0,4194304] 0 2024-02-16T22:56:41.873 INFO:tasks.workunit.client.0.smithi064.stdout:4/933: write dc/dd/d16/d7c/fab [901125,94436] 0 2024-02-16T22:56:41.875 INFO:tasks.workunit.client.0.smithi064.stdout:9/969: write d3/d9/d6a/db1/d2a/d2e/f39 [4594070,85433] 0 2024-02-16T22:56:41.876 INFO:tasks.workunit.client.0.smithi064.stdout:1/828: stat d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce 0 2024-02-16T22:56:41.877 INFO:tasks.workunit.client.0.smithi064.stdout:5/867: dread d6/d25/df1/d91/da5/de3/d43/d7d/fb4 [0,4194304] 0 2024-02-16T22:56:41.882 INFO:tasks.workunit.client.0.smithi064.stdout:4/934: rmdir dc/de/d28/d57 39 2024-02-16T22:56:41.883 INFO:tasks.workunit.client.0.smithi064.stdout:9/970: dwrite d3/d11/d9e/f108 [0,4194304] 0 2024-02-16T22:56:41.886 INFO:tasks.workunit.client.0.smithi064.stdout:1/829: mknod d2/df/c115 0 2024-02-16T22:56:41.892 INFO:tasks.workunit.client.0.smithi064.stdout:4/935: dwrite dc/d4a/f4d [0,4194304] 0 2024-02-16T22:56:41.897 INFO:tasks.workunit.client.0.smithi064.stdout:1/830: dwrite d2/df/d14/d2c/fe7 [0,4194304] 0 2024-02-16T22:56:41.903 INFO:tasks.workunit.client.0.smithi064.stdout:9/971: rmdir d3/d89 39 2024-02-16T22:56:41.904 INFO:tasks.workunit.client.0.smithi064.stdout:9/972: readlink d3/d9/d6a/l74 0 2024-02-16T22:56:41.908 INFO:tasks.workunit.client.0.smithi064.stdout:4/936: symlink dc/dd/d32/d46/d5a/d7e/d9a/df8/d10d/l127 0 2024-02-16T22:56:41.910 INFO:tasks.workunit.client.0.smithi064.stdout:1/831: fsync d2/df/d14/d18/d48/d84/fa4 0 2024-02-16T22:56:41.911 INFO:tasks.workunit.client.0.smithi064.stdout:9/973: creat d3/d11/d9e/da9/f13d x:0 0 0 2024-02-16T22:56:41.912 INFO:tasks.workunit.client.0.smithi064.stdout:1/832: write d2/df/d11/d1a/d30/ffc [582542,6570] 0 2024-02-16T22:56:41.913 INFO:tasks.workunit.client.0.smithi064.stdout:4/937: unlink dc/dd/c35 0 2024-02-16T22:56:41.914 INFO:tasks.workunit.client.0.smithi064.stdout:1/833: write d2/df/d14/d2c/f112 [325986,64980] 0 2024-02-16T22:56:41.920 INFO:tasks.workunit.client.0.smithi064.stdout:4/938: chown dc/de/d28/d57/d77/d101/d43/f5f 54790 1 2024-02-16T22:56:41.922 INFO:tasks.workunit.client.0.smithi064.stdout:1/834: rename d2/df/d14/d2c/db5/de3/de6/ff2 to d2/df/d14/d18/d40/f116 0 2024-02-16T22:56:41.923 INFO:tasks.workunit.client.0.smithi064.stdout:1/835: chown d2/fb0 416 1 2024-02-16T22:56:41.923 INFO:tasks.workunit.client.0.smithi064.stdout:1/836: stat d2/df/d11/d1a/d30/d3d/l82 0 2024-02-16T22:56:41.931 INFO:tasks.workunit.client.0.smithi064.stdout:1/837: mkdir d2/df/d14/d2c/db5/d117 0 2024-02-16T22:56:41.936 INFO:tasks.workunit.client.0.smithi064.stdout:9/974: write d3/ff4 [95913,49709] 0 2024-02-16T22:56:41.941 INFO:tasks.workunit.client.0.smithi064.stdout:9/975: dread d3/d11/d9e/f108 [0,4194304] 0 2024-02-16T22:56:41.945 INFO:tasks.workunit.client.0.smithi064.stdout:4/939: dwrite dc/de/d28/d57/d77/d101/dbd/fc2 [0,4194304] 0 2024-02-16T22:56:41.947 INFO:tasks.workunit.client.0.smithi064.stdout:1/838: write d2/df/d11/d1a/d1d/f4d [1188277,130920] 0 2024-02-16T22:56:41.950 INFO:tasks.workunit.client.0.smithi064.stdout:4/940: creat dc/dd/d32/d22/f128 x:0 0 0 2024-02-16T22:56:41.954 INFO:tasks.workunit.client.0.smithi064.stdout:1/839: symlink d2/df/d11/d1a/d30/d3d/l118 0 2024-02-16T22:56:41.964 INFO:tasks.workunit.client.0.smithi064.stdout:9/976: dread d3/d89/d9a/fce [0,4194304] 0 2024-02-16T22:56:41.969 INFO:tasks.workunit.client.0.smithi064.stdout:9/977: dwrite d3/d9/d6a/fb2 [0,4194304] 0 2024-02-16T22:56:41.972 INFO:tasks.workunit.client.0.smithi064.stdout:9/978: mknod d3/d11/d9e/da5/df3/c13e 0 2024-02-16T22:56:41.973 INFO:tasks.workunit.client.0.smithi064.stdout:9/979: mknod d3/d9/d6a/db1/c13f 0 2024-02-16T22:56:41.975 INFO:tasks.workunit.client.0.smithi064.stdout:9/980: creat d3/f140 x:0 0 0 2024-02-16T22:56:41.986 INFO:tasks.workunit.client.0.smithi064.stdout:1/840: write d2/df/d14/d18/d40/fa0 [347936,84522] 0 2024-02-16T22:56:41.989 INFO:tasks.workunit.client.0.smithi064.stdout:1/841: fsync d2/df/d14/d2c/fe7 0 2024-02-16T22:56:41.990 INFO:tasks.workunit.client.0.smithi064.stdout:4/941: dwrite dc/de/dfb/dff/de3/fb7 [0,4194304] 0 2024-02-16T22:56:41.998 INFO:tasks.workunit.client.0.smithi064.stdout:4/942: creat dc/ddc/de8/da1/f129 x:0 0 0 2024-02-16T22:56:42.000 INFO:tasks.workunit.client.0.smithi064.stdout:9/981: dwrite d3/d9/d6a/db1/d2a/d2e/d31/d3f/f5b [0,4194304] 0 2024-02-16T22:56:42.003 INFO:tasks.workunit.client.0.smithi064.stdout:1/842: dread d2/df/d14/d2a/fbf [0,4194304] 0 2024-02-16T22:56:42.007 INFO:tasks.workunit.client.0.smithi064.stdout:4/943: dwrite dc/dd/d32/d46/f105 [0,4194304] 0 2024-02-16T22:56:42.012 INFO:tasks.workunit.client.0.smithi064.stdout:4/944: mknod dc/ddc/de8/da1/c12a 0 2024-02-16T22:56:42.013 INFO:tasks.workunit.client.0.smithi064.stdout:1/843: dwrite d2/df/d11/d25/fb9 [0,4194304] 0 2024-02-16T22:56:42.018 INFO:tasks.workunit.client.0.smithi064.stdout:1/844: mknod d2/df/d14/d2a/c119 0 2024-02-16T22:56:42.018 INFO:tasks.workunit.client.0.smithi064.stdout:9/982: write d3/d9/d6a/db1/d2a/d2e/d31/d3f/d80/fe9 [721200,55240] 0 2024-02-16T22:56:42.019 INFO:tasks.workunit.client.0.smithi064.stdout:4/945: rmdir dc/de/d28/d57/d77/dc6/d11c 39 2024-02-16T22:56:42.019 INFO:tasks.workunit.client.0.smithi064.stdout:1/845: fdatasync d2/df/d11/d1a/d30/d3d/f110 0 2024-02-16T22:56:42.020 INFO:tasks.workunit.client.0.smithi064.stdout:9/983: write d3/f140 [1032431,126110] 0 2024-02-16T22:56:42.021 INFO:tasks.workunit.client.0.smithi064.stdout:4/946: chown c2 1499636 1 2024-02-16T22:56:42.021 INFO:tasks.workunit.client.0.smithi064.stdout:9/984: readlink d3/d9/d1e/l67 0 2024-02-16T22:56:42.027 INFO:tasks.workunit.client.0.smithi064.stdout:9/985: mkdir d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/de4/d141 0 2024-02-16T22:56:42.029 INFO:tasks.workunit.client.0.smithi064.stdout:4/947: mkdir dc/d12b 0 2024-02-16T22:56:42.034 INFO:tasks.workunit.client.0.smithi064.stdout:1/846: getdents d2/df/d11/d1a/d30/d3d 0 2024-02-16T22:56:42.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:41 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:56:42.035 INFO:tasks.workunit.client.0.smithi064.stdout:1/847: dread - d2/df/d14/d2c/ff6 zero size 2024-02-16T22:56:42.036 INFO:tasks.workunit.client.0.smithi064.stdout:4/948: unlink dc/dd/d16/d6e/le6 0 2024-02-16T22:56:42.045 INFO:tasks.workunit.client.0.smithi064.stdout:1/848: rename d2/df/d11/d1a/c81 to d2/df/d14/d18/d48/d84/c11a 0 2024-02-16T22:56:42.046 INFO:tasks.workunit.client.0.smithi064.stdout:1/849: dread - d2/f10a zero size 2024-02-16T22:56:42.047 INFO:tasks.workunit.client.0.smithi064.stdout:9/986: link d3/d11/d15/d23/dee/l12f d3/d9/d6a/db1/d2a/d2e/d31/d95/dc6/de4/l142 0 2024-02-16T22:56:42.050 INFO:tasks.workunit.client.0.smithi064.stdout:1/850: symlink d2/df/d14/d18/l11b 0 2024-02-16T22:56:42.052 INFO:tasks.workunit.client.0.smithi064.stdout:9/987: rename d3/d11/d9e to d3/d9/d6a/db1/d2a/d2e/d31/d3f/d143 0 2024-02-16T22:56:42.054 INFO:tasks.workunit.client.0.smithi064.stdout:9/988: write d3/d9/d6a/db1/d2a/d2e/d31/d3f/d143/da9/f134 [1424377,31380] 0 2024-02-16T22:56:42.054 INFO:tasks.workunit.client.0.smithi064.stdout:1/851: symlink d2/df/d14/d2c/db5/l11c 0 2024-02-16T22:56:42.057 INFO:tasks.workunit.client.0.smithi064.stdout:4/949: write dc/dd/d32/d58/f12 [1144984,64533] 0 2024-02-16T22:56:42.057 INFO:tasks.workunit.client.0.smithi064.stdout:9/989: mknod d3/d9/d1e/d42/c144 0 2024-02-16T22:56:42.058 INFO:tasks.workunit.client.0.smithi064.stdout:9/990: stat d3/d9/dec 0 2024-02-16T22:56:42.063 INFO:tasks.workunit.client.0.smithi064.stdout:9/991: readlink d3/d9/d6a/db1/d2a/d2e/da3/dea/l10b 0 2024-02-16T22:56:42.064 INFO:tasks.workunit.client.0.smithi064.stdout:1/852: creat d2/df/d14/d2c/db5/de3/d100/f11d x:0 0 0 2024-02-16T22:56:42.066 INFO:tasks.workunit.client.0.smithi064.stdout:9/992: fsync d3/d11/d15/f7f 0 2024-02-16T22:56:42.070 INFO:tasks.workunit.client.0.smithi064.stdout:4/950: creat dc/de/f12c x:0 0 0 2024-02-16T22:56:42.072 INFO:tasks.workunit.client.0.smithi064.stdout:1/853: creat d2/df/d14/d2a/d108/f11e x:0 0 0 2024-02-16T22:56:42.073 INFO:tasks.workunit.client.0.smithi064.stdout:9/993: write d3/d9/d6a/db1/d2a/d2e/d31/d3f/d143/f108 [3149544,4177] 0 2024-02-16T22:56:42.075 INFO:tasks.workunit.client.0.smithi064.stdout:4/951: unlink dc/ddc/de8/da1/fc3 0 2024-02-16T22:56:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:1/854: mkdir d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d11f 0 2024-02-16T22:56:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:9/994: symlink d3/d9/d6a/db1/d2a/d2e/d31/d3f/d143/da5/df3/l145 0 2024-02-16T22:56:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:1/855: truncate d2/df/d14/d2c/f49 1691180 0 2024-02-16T22:56:42.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:41 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:56:42.123 INFO:tasks.workunit.client.0.smithi064.stdout:4/952: creat dc/d12b/f12d x:0 0 0 2024-02-16T22:56:42.125 INFO:tasks.workunit.client.0.smithi064.stdout:9/995: creat d3/d11/d15/f146 x:0 0 0 2024-02-16T22:56:42.126 INFO:tasks.workunit.client.0.smithi064.stdout:9/996: write d3/f63 [4917820,75737] 0 2024-02-16T22:56:42.129 INFO:tasks.workunit.client.0.smithi064.stdout:4/953: unlink dc/dd/d32/d22/f93 0 2024-02-16T22:56:42.130 INFO:tasks.workunit.client.0.smithi064.stdout:1/856: rmdir d2/df/d14/d2a/dc2 0 2024-02-16T22:56:42.131 INFO:tasks.workunit.client.0.smithi064.stdout:1/857: write d2/df/d11/d1a/d30/dbd/d91/d56/f10d [251303,111205] 0 2024-02-16T22:56:42.133 INFO:tasks.workunit.client.0.smithi064.stdout:9/997: getdents d3/d9/d6a/db1/d2a/d2e/d31/d3f/d143/da5/d102 0 2024-02-16T22:56:42.143 INFO:tasks.workunit.client.0.smithi064.stdout:7/924: sync 2024-02-16T22:56:42.144 INFO:tasks.workunit.client.0.smithi064.stdout:4/954: rename dc/dd/d16/d6e/f10a to dc/dd/f12e 0 2024-02-16T22:56:42.153 INFO:tasks.workunit.client.0.smithi064.stdout:4/955: rename dc/dd/d4f/f5c to dc/de/d28/d57/d77/dc6/d11c/f12f 0 2024-02-16T22:56:42.155 INFO:tasks.workunit.client.0.smithi064.stdout:9/998: getdents d3/d11/d15/d23/d117 0 2024-02-16T22:56:42.156 INFO:tasks.workunit.client.0.smithi064.stdout:9/999: write d3/d9/d1e/f8d [12085,122760] 0 2024-02-16T22:56:42.158 INFO:tasks.workunit.client.0.smithi064.stdout:4/956: write dc/de/f99 [687473,106465] 0 2024-02-16T22:56:42.160 INFO:tasks.workunit.client.0.smithi064.stdout:4/957: read dc/dd/d16/d7c/fab [883643,15297] 0 2024-02-16T22:56:42.169 INFO:tasks.workunit.client.0.smithi064.stdout:4/958: getdents dc/dd/db8 0 2024-02-16T22:56:42.171 INFO:tasks.workunit.client.0.smithi064.stdout:4/959: symlink dc/dd/d32/d46/d5a/l130 0 2024-02-16T22:56:42.172 INFO:tasks.workunit.client.0.smithi064.stdout:4/960: readlink dc/dd/d16/lb6 0 2024-02-16T22:56:42.189 INFO:tasks.workunit.client.0.smithi064.stdout:1/858: dwrite d2/df/d14/d18/d48/d6b/fd5 [0,4194304] 0 2024-02-16T22:56:42.195 INFO:tasks.workunit.client.0.smithi064.stdout:7/925: dwrite d5/f112 [0,4194304] 0 2024-02-16T22:56:42.210 INFO:tasks.workunit.client.0.smithi064.stdout:4/961: dwrite dc/dd/d32/d4b/fce [0,4194304] 0 2024-02-16T22:56:42.212 INFO:tasks.workunit.client.0.smithi064.stdout:1/859: dwrite d2/df/d14/d2c/fe0 [0,4194304] 0 2024-02-16T22:56:42.216 INFO:tasks.workunit.client.0.smithi064.stdout:7/926: dwrite d5/f27 [0,4194304] 0 2024-02-16T22:56:42.218 INFO:tasks.workunit.client.0.smithi064.stdout:4/962: rmdir dc/dd/d32/d46/d5a 39 2024-02-16T22:56:42.223 INFO:tasks.workunit.client.0.smithi064.stdout:7/927: mkdir d5/d7d/df0/d132 0 2024-02-16T22:56:42.229 INFO:tasks.workunit.client.0.smithi064.stdout:7/928: fsync d5/d33/d53/f70 0 2024-02-16T22:56:42.231 INFO:tasks.workunit.client.0.smithi064.stdout:4/963: link dc/dd/d32/d22/c39 dc/dd/db8/c131 0 2024-02-16T22:56:42.237 INFO:tasks.workunit.client.0.smithi064.stdout:4/964: dwrite dc/de/d28/d57/d77/f107 [0,4194304] 0 2024-02-16T22:56:42.240 INFO:tasks.workunit.client.0.smithi064.stdout:1/860: write d2/df/d14/d2a/fbf [809538,59846] 0 2024-02-16T22:56:42.245 INFO:tasks.workunit.client.0.smithi064.stdout:4/965: dwrite dc/dd/fc8 [0,4194304] 0 2024-02-16T22:56:42.254 INFO:tasks.workunit.client.0.smithi064.stdout:1/861: rename d2/df/d14/l1b to d2/df/d14/d18/l120 0 2024-02-16T22:56:42.255 INFO:tasks.workunit.client.0.smithi064.stdout:1/862: dread - d2/df/d14/d2c/ff6 zero size 2024-02-16T22:56:42.256 INFO:tasks.workunit.client.0.smithi064.stdout:0/789: sync 2024-02-16T22:56:42.256 INFO:tasks.workunit.client.0.smithi064.stdout:8/835: sync 2024-02-16T22:56:42.256 INFO:tasks.workunit.client.0.smithi064.stdout:5/868: sync 2024-02-16T22:56:42.258 INFO:tasks.workunit.client.0.smithi064.stdout:7/929: rename d5/d33/d42/d81/d82/df3/fd5 to d5/d33/d53/df2/d54/d7b/de7/f133 0 2024-02-16T22:56:42.258 INFO:tasks.workunit.client.0.smithi064.stdout:1/863: write d2/df/d14/d2c/db5/de3/de6/fed [1520494,84268] 0 2024-02-16T22:56:42.260 INFO:tasks.workunit.client.0.smithi064.stdout:5/869: mkdir d6/d25/df1/d80/d127 0 2024-02-16T22:56:42.263 INFO:tasks.workunit.client.0.smithi064.stdout:1/864: read - d2/df/d14/d18/d48/d6b/d75/f8c zero size 2024-02-16T22:56:42.264 INFO:tasks.workunit.client.0.smithi064.stdout:5/870: symlink d6/d25/da4/dc1/dd4/l128 0 2024-02-16T22:56:42.266 INFO:tasks.workunit.client.0.smithi064.stdout:7/930: truncate d5/d33/d42/d81/d82/fd4 476429 0 2024-02-16T22:56:42.267 INFO:tasks.workunit.client.0.smithi064.stdout:7/931: write d5/d33/fd9 [435066,46400] 0 2024-02-16T22:56:42.272 INFO:tasks.workunit.client.0.smithi064.stdout:4/966: dwrite dc/de/d28/d57/d77/d101/f8b [0,4194304] 0 2024-02-16T22:56:42.272 INFO:tasks.workunit.client.0.smithi064.stdout:0/790: creat da/dc/de/d12/d1d/d100/f108 x:0 0 0 2024-02-16T22:56:42.275 INFO:tasks.workunit.client.0.smithi064.stdout:8/836: write df/d31/d8f/d80/f84 [3247163,34646] 0 2024-02-16T22:56:42.275 INFO:tasks.workunit.client.0.smithi064.stdout:4/967: readlink dc/dd/d32/d58/l1b 0 2024-02-16T22:56:42.276 INFO:tasks.workunit.client.0.smithi064.stdout:1/865: creat d2/de8/d109/f121 x:0 0 0 2024-02-16T22:56:42.279 INFO:tasks.workunit.client.0.smithi064.stdout:7/932: dwrite d5/d33/d42/fed [0,4194304] 0 2024-02-16T22:56:42.279 INFO:tasks.workunit.client.0.smithi064.stdout:8/837: write df/f24 [4448524,94591] 0 2024-02-16T22:56:42.282 INFO:tasks.workunit.client.0.smithi064.stdout:5/871: creat d6/d25/df1/d91/da5/de3/f129 x:0 0 0 2024-02-16T22:56:42.286 INFO:tasks.workunit.client.0.smithi064.stdout:7/933: chown d5/d33/d45/fb5 3913 1 2024-02-16T22:56:42.289 INFO:tasks.workunit.client.0.smithi064.stdout:0/791: dwrite da/d26/d3f/f75 [0,4194304] 0 2024-02-16T22:56:42.290 INFO:tasks.workunit.client.0.smithi064.stdout:8/838: fsync df/d31/d61/d79/f9b 0 2024-02-16T22:56:42.292 INFO:tasks.workunit.client.0.smithi064.stdout:5/872: mkdir d6/d18/d19/d112/d12a 0 2024-02-16T22:56:42.301 INFO:tasks.workunit.client.0.smithi064.stdout:1/866: creat d2/df/d11/d1a/f122 x:0 0 0 2024-02-16T22:56:42.302 INFO:tasks.workunit.client.0.smithi064.stdout:1/867: chown d2/df/d14/d18/d48/d84/fa4 140 1 2024-02-16T22:56:42.303 INFO:tasks.workunit.client.0.smithi064.stdout:4/968: creat dc/dd/d32/d46/f132 x:0 0 0 2024-02-16T22:56:42.304 INFO:tasks.workunit.client.0.smithi064.stdout:0/792: mknod da/dc/d3b/c109 0 2024-02-16T22:56:42.306 INFO:tasks.workunit.client.0.smithi064.stdout:7/934: rename d5/d33/d53/df2/d54/d41/da2/fef to d5/d33/d42/d81/d82/f134 0 2024-02-16T22:56:42.307 INFO:tasks.workunit.client.0.smithi064.stdout:4/969: mknod dc/dd/d32/d4b/c133 0 2024-02-16T22:56:42.308 INFO:tasks.workunit.client.0.smithi064.stdout:1/868: truncate d2/df/d11/d1a/d30/f63 348476 0 2024-02-16T22:56:42.310 INFO:tasks.workunit.client.0.smithi064.stdout:1/869: chown d2/df/d11/d1a/d1d/ce9 114501813 1 2024-02-16T22:56:42.319 INFO:tasks.workunit.client.0.smithi064.stdout:8/839: dread df/d31/d2f/d38/d48/f49 [0,4194304] 0 2024-02-16T22:56:42.327 INFO:tasks.workunit.client.0.smithi064.stdout:0/793: dread da/dc/de/f76 [0,4194304] 0 2024-02-16T22:56:42.328 INFO:tasks.workunit.client.0.smithi064.stdout:5/873: dwrite d6/d25/df1/d91/da5/de3/d43/d7d/f119 [0,4194304] 0 2024-02-16T22:56:42.333 INFO:tasks.workunit.client.0.smithi064.stdout:1/870: rmdir d2 39 2024-02-16T22:56:42.335 INFO:tasks.workunit.client.0.smithi064.stdout:5/874: fdatasync d6/d25/da4/dc1/d85/f9f 0 2024-02-16T22:56:42.336 INFO:tasks.workunit.client.0.smithi064.stdout:7/935: write d5/d33/d53/f70 [953998,70236] 0 2024-02-16T22:56:42.336 INFO:tasks.workunit.client.0.smithi064.stdout:4/970: write dc/dd/d32/d46/d5a/f83 [159651,36651] 0 2024-02-16T22:56:42.343 INFO:tasks.workunit.client.0.smithi064.stdout:8/840: dwrite df/d31/d2f/d3e/f83 [4194304,4194304] 0 2024-02-16T22:56:42.345 INFO:tasks.workunit.client.0.smithi064.stdout:5/875: symlink d6/d25/df1/d91/da5/de3/d43/d7d/de2/l12b 0 2024-02-16T22:56:42.348 INFO:tasks.workunit.client.0.smithi064.stdout:7/936: rename d5/d33/d53/da3/fb7 to d5/d33/d53/df2/d65/da7/d120/f135 0 2024-02-16T22:56:42.350 INFO:tasks.workunit.client.0.smithi064.stdout:4/971: unlink dc/dd/d32/d22/f9d 0 2024-02-16T22:56:42.357 INFO:tasks.workunit.client.0.smithi064.stdout:4/972: dread dc/de/d6d/dee/fed [0,4194304] 0 2024-02-16T22:56:42.358 INFO:tasks.workunit.client.0.smithi064.stdout:0/794: dwrite da/dc/de/d12/d2c/fc5 [0,4194304] 0 2024-02-16T22:56:42.358 INFO:tasks.workunit.client.0.smithi064.stdout:7/937: truncate d5/d33/d53/df2/f30 376921 0 2024-02-16T22:56:42.366 INFO:tasks.workunit.client.0.smithi064.stdout:4/973: symlink dc/dd/db8/l134 0 2024-02-16T22:56:42.368 INFO:tasks.workunit.client.0.smithi064.stdout:7/938: creat d5/df9/de4/f136 x:0 0 0 2024-02-16T22:56:42.368 INFO:tasks.workunit.client.0.smithi064.stdout:5/876: dread d6/d25/df1/d80/fda [0,4194304] 0 2024-02-16T22:56:42.369 INFO:tasks.workunit.client.0.smithi064.stdout:7/939: chown d5/d33/d53/f115 1 1 2024-02-16T22:56:42.372 INFO:tasks.workunit.client.0.smithi064.stdout:4/974: symlink dc/dd/d32/d58/l135 0 2024-02-16T22:56:42.373 INFO:tasks.workunit.client.0.smithi064.stdout:0/795: creat da/dc/de/d12/d2c/d2e/d38/dbe/dfe/f10a x:0 0 0 2024-02-16T22:56:42.375 INFO:tasks.workunit.client.0.smithi064.stdout:7/940: mknod d5/df9/c137 0 2024-02-16T22:56:42.377 INFO:tasks.workunit.client.0.smithi064.stdout:0/796: dread da/dc/de/f76 [0,4194304] 0 2024-02-16T22:56:42.378 INFO:tasks.workunit.client.0.smithi064.stdout:0/797: chown da/dc/d3b/d72/d7c/fcb 0 1 2024-02-16T22:56:42.380 INFO:tasks.workunit.client.0.smithi064.stdout:5/877: link d6/f13 d6/d18/d19/d112/d12a/f12c 0 2024-02-16T22:56:42.380 INFO:tasks.workunit.client.0.smithi064.stdout:5/878: dread - d6/d18/df8/ffe zero size 2024-02-16T22:56:42.382 INFO:tasks.workunit.client.0.smithi064.stdout:8/841: link df/d2a/d5c/cde df/d31/d2f/d38/c120 0 2024-02-16T22:56:42.383 INFO:tasks.workunit.client.0.smithi064.stdout:0/798: creat da/dc/de/d12/d2c/d2e/d62/f10b x:0 0 0 2024-02-16T22:56:42.385 INFO:tasks.workunit.client.0.smithi064.stdout:0/799: write da/dc/de/d37/fa6 [3537605,73988] 0 2024-02-16T22:56:42.386 INFO:tasks.workunit.client.0.smithi064.stdout:5/879: rmdir d6/d25/da4 39 2024-02-16T22:56:42.386 INFO:tasks.workunit.client.0.smithi064.stdout:4/975: dread dc/dd/d32/d58/f12 [0,4194304] 0 2024-02-16T22:56:42.387 INFO:tasks.workunit.client.0.smithi064.stdout:0/800: dread - da/dc/de/d12/d2c/d2e/d96/ff8 zero size 2024-02-16T22:56:42.387 INFO:tasks.workunit.client.0.smithi064.stdout:8/842: mkdir df/d31/d8f/dc9/d121 0 2024-02-16T22:56:42.389 INFO:tasks.workunit.client.0.smithi064.stdout:8/843: write df/f4c [2764235,65366] 0 2024-02-16T22:56:42.391 INFO:tasks.workunit.client.0.smithi064.stdout:0/801: creat da/dc/de/f10c x:0 0 0 2024-02-16T22:56:42.392 INFO:tasks.workunit.client.0.smithi064.stdout:8/844: fsync df/d31/f41 0 2024-02-16T22:56:42.394 INFO:tasks.workunit.client.0.smithi064.stdout:8/845: chown df/d31/dc1/lcc 1499409 1 2024-02-16T22:56:42.395 INFO:tasks.workunit.client.0.smithi064.stdout:0/802: mkdir da/dc/d3b/d72/db3/d10d 0 2024-02-16T22:56:42.395 INFO:tasks.workunit.client.0.smithi064.stdout:5/880: dwrite d6/d25/df1/d91/da5/de3/f10c [0,4194304] 0 2024-02-16T22:56:42.396 INFO:tasks.workunit.client.0.smithi064.stdout:0/803: fsync da/dc/de/d12/d1d/f101 0 2024-02-16T22:56:42.399 INFO:tasks.workunit.client.0.smithi064.stdout:5/881: creat d6/d25/d48/d124/f12d x:0 0 0 2024-02-16T22:56:42.399 INFO:tasks.workunit.client.0.smithi064.stdout:0/804: creat da/d26/de0/d9e/f10e x:0 0 0 2024-02-16T22:56:42.401 INFO:tasks.workunit.client.0.smithi064.stdout:8/846: link df/d31/d2f/f44 df/d31/d2f/d7f/f122 0 2024-02-16T22:56:42.401 INFO:tasks.workunit.client.0.smithi064.stdout:5/882: creat d6/d18/d95/f12e x:0 0 0 2024-02-16T22:56:42.402 INFO:tasks.workunit.client.0.smithi064.stdout:0/805: creat da/dc/d3b/dc2/f10f x:0 0 0 2024-02-16T22:56:42.405 INFO:tasks.workunit.client.0.smithi064.stdout:5/883: symlink d6/d25/df1/d91/da5/de3/d43/db9/dfc/l12f 0 2024-02-16T22:56:42.488 INFO:tasks.workunit.client.0.smithi064.stdout:4/976: read dc/dd/d32/f8d [2003024,80797] 0 2024-02-16T22:56:42.562 INFO:tasks.workunit.client.0.smithi064.stdout:4/977: symlink dc/ddc/de8/da1/l136 0 2024-02-16T22:56:42.564 INFO:tasks.workunit.client.0.smithi064.stdout:4/978: rmdir dc/ddc/de8/df2 39 2024-02-16T22:56:42.570 INFO:tasks.workunit.client.0.smithi064.stdout:4/979: creat dc/ddc/de8/f137 x:0 0 0 2024-02-16T22:56:42.573 INFO:tasks.workunit.client.0.smithi064.stdout:4/980: creat dc/d12b/f138 x:0 0 0 2024-02-16T22:56:42.577 INFO:tasks.workunit.client.0.smithi064.stdout:4/981: rename dc/ddc/de8/cbe to dc/dd/d32/d22/c139 0 2024-02-16T22:56:42.580 INFO:tasks.workunit.client.0.smithi064.stdout:4/982: symlink dc/ddc/de8/da1/l13a 0 2024-02-16T22:56:42.582 INFO:tasks.workunit.client.0.smithi064.stdout:4/983: mkdir dc/dd/d4f/d13b 0 2024-02-16T22:56:42.592 INFO:tasks.workunit.client.0.smithi064.stdout:7/941: dwrite d5/d33/d42/d81/f94 [0,4194304] 0 2024-02-16T22:56:42.596 INFO:tasks.workunit.client.0.smithi064.stdout:7/942: chown d5/d33/d45/d73/l10b 89111181 1 2024-02-16T22:56:42.597 INFO:tasks.workunit.client.0.smithi064.stdout:7/943: fsync f0 0 2024-02-16T22:56:42.600 INFO:tasks.workunit.client.0.smithi064.stdout:4/984: dread dc/de/d28/d57/d77/d101/d43/d7a/f9b [0,4194304] 0 2024-02-16T22:56:42.601 INFO:tasks.workunit.client.0.smithi064.stdout:4/985: write dc/de/dfb/dff/de3/fb7 [290220,114465] 0 2024-02-16T22:56:42.603 INFO:tasks.workunit.client.0.smithi064.stdout:7/944: rename d5/l3e to d5/d33/d53/df2/d54/d7b/df8/d12e/l138 0 2024-02-16T22:56:42.606 INFO:tasks.workunit.client.0.smithi064.stdout:8/847: write df/d31/d8f/dc9/d73/fc8 [383394,69702] 0 2024-02-16T22:56:42.610 INFO:tasks.workunit.client.0.smithi064.stdout:5/884: write d6/d18/d95/dbe/ffb [618271,105382] 0 2024-02-16T22:56:42.612 INFO:tasks.workunit.client.0.smithi064.stdout:0/806: dwrite da/d26/f33 [0,4194304] 0 2024-02-16T22:56:42.613 INFO:tasks.workunit.client.0.smithi064.stdout:0/807: read - da/dc/d3b/f8e zero size 2024-02-16T22:56:42.614 INFO:tasks.workunit.client.0.smithi064.stdout:7/945: creat d5/d7d/df0/f139 x:0 0 0 2024-02-16T22:56:42.618 INFO:tasks.workunit.client.0.smithi064.stdout:8/848: dread - df/d31/d61/d94/fa6 zero size 2024-02-16T22:56:42.624 INFO:tasks.workunit.client.0.smithi064.stdout:0/808: unlink da/d71/cde 0 2024-02-16T22:56:42.628 INFO:tasks.workunit.client.0.smithi064.stdout:1/871: sync 2024-02-16T22:56:42.628 INFO:tasks.workunit.client.0.smithi064.stdout:7/946: creat d5/d7d/d12d/f13a x:0 0 0 2024-02-16T22:56:42.631 INFO:tasks.workunit.client.0.smithi064.stdout:8/849: mkdir df/d31/d61/d79/d81/d123 0 2024-02-16T22:56:42.641 INFO:tasks.workunit.client.0.smithi064.stdout:1/872: read d2/df/d11/d1a/f34 [196792,87028] 0 2024-02-16T22:56:42.642 INFO:tasks.workunit.client.0.smithi064.stdout:0/809: dread da/dc/d3b/d5a/d8f/fbb [0,4194304] 0 2024-02-16T22:56:42.643 INFO:tasks.workunit.client.0.smithi064.stdout:8/850: creat df/d31/d2f/d3e/f124 x:0 0 0 2024-02-16T22:56:42.644 INFO:tasks.workunit.client.0.smithi064.stdout:8/851: fsync df/f4c 0 2024-02-16T22:56:42.652 INFO:tasks.workunit.client.0.smithi064.stdout:1/873: symlink d2/df/d14/d2a/l123 0 2024-02-16T22:56:42.657 INFO:tasks.workunit.client.0.smithi064.stdout:0/810: symlink da/dc/de/d12/d1d/d100/l110 0 2024-02-16T22:56:42.663 INFO:tasks.workunit.client.0.smithi064.stdout:8/852: symlink df/d31/d2f/d3e/ddf/d109/l125 0 2024-02-16T22:56:42.671 INFO:tasks.workunit.client.0.smithi064.stdout:0/811: creat da/dc/de/d37/daa/de9/f111 x:0 0 0 2024-02-16T22:56:42.674 INFO:tasks.workunit.client.0.smithi064.stdout:8/853: creat df/d31/d61/d79/dc4/f126 x:0 0 0 2024-02-16T22:56:42.676 INFO:tasks.workunit.client.0.smithi064.stdout:7/947: getdents d5/d7d 0 2024-02-16T22:56:42.676 INFO:tasks.workunit.client.0.smithi064.stdout:7/948: readlink d5/l21 0 2024-02-16T22:56:42.682 INFO:tasks.workunit.client.0.smithi064.stdout:8/854: fdatasync df/d31/d2f/d38/d48/db5/fb8 0 2024-02-16T22:56:42.684 INFO:tasks.workunit.client.0.smithi064.stdout:7/949: symlink d5/d33/d53/df2/d54/df4/l13b 0 2024-02-16T22:56:42.686 INFO:tasks.workunit.client.0.smithi064.stdout:0/812: dread da/dc/d3b/f5f [0,4194304] 0 2024-02-16T22:56:42.687 INFO:tasks.workunit.client.0.smithi064.stdout:0/813: stat da/d26/de0/d93/fc3 0 2024-02-16T22:56:42.688 INFO:tasks.workunit.client.0.smithi064.stdout:0/814: chown da/d71/da4/lec 0 1 2024-02-16T22:56:42.693 INFO:tasks.workunit.client.0.smithi064.stdout:8/855: rename df/d31/d8f/dab/dae/fbe to df/d31/d8f/f127 0 2024-02-16T22:56:42.694 INFO:tasks.workunit.client.0.smithi064.stdout:0/815: symlink da/dc/de/d37/daa/d106/l112 0 2024-02-16T22:56:42.695 INFO:tasks.workunit.client.0.smithi064.stdout:0/816: chown da/d26/ff5 1216727 1 2024-02-16T22:56:42.695 INFO:tasks.workunit.client.0.smithi064.stdout:0/817: chown da/dc/d3b/dc2/f10f 13207300 1 2024-02-16T22:56:42.697 INFO:tasks.workunit.client.0.smithi064.stdout:8/856: read df/f1a [4669952,29229] 0 2024-02-16T22:56:42.767 INFO:tasks.workunit.client.0.smithi064.stdout:5/885: dwrite d6/d25/df1/f7f [0,4194304] 0 2024-02-16T22:56:42.770 INFO:tasks.workunit.client.0.smithi064.stdout:5/886: mknod d6/d18/d95/c130 0 2024-02-16T22:56:42.774 INFO:tasks.workunit.client.0.smithi064.stdout:5/887: getdents d6/d18/d19/d112/d12a 0 2024-02-16T22:56:42.779 INFO:tasks.workunit.client.0.smithi064.stdout:5/888: truncate d6/d25/da4/dc1/f76 685441 0 2024-02-16T22:56:42.784 INFO:tasks.workunit.client.0.smithi064.stdout:1/874: write d2/df/d11/fd2 [824181,99728] 0 2024-02-16T22:56:42.787 INFO:tasks.workunit.client.0.smithi064.stdout:1/875: creat d2/df/d14/d2c/db5/de3/d100/f124 x:0 0 0 2024-02-16T22:56:42.790 INFO:tasks.workunit.client.0.smithi064.stdout:1/876: mkdir d2/df/d14/d2c/db5/d117/d125 0 2024-02-16T22:56:42.791 INFO:tasks.workunit.client.0.smithi064.stdout:1/877: write d2/df/d11/d1a/d1d/fdb [698088,86716] 0 2024-02-16T22:56:42.810 INFO:tasks.workunit.client.0.smithi064.stdout:4/986: unlink dc/dd/ccc 0 2024-02-16T22:56:42.811 INFO:tasks.workunit.client.0.smithi064.stdout:4/987: chown dc/dd/db8/l134 386644 1 2024-02-16T22:56:42.811 INFO:tasks.workunit.client.0.smithi064.stdout:0/818: symlink da/dc/de/d37/l113 0 2024-02-16T22:56:42.812 INFO:tasks.workunit.client.0.smithi064.stdout:0/819: readlink da/d26/d3f/l41 0 2024-02-16T22:56:42.813 INFO:tasks.workunit.client.0.smithi064.stdout:4/988: creat dc/dd/d4f/f13c x:0 0 0 2024-02-16T22:56:42.814 INFO:tasks.workunit.client.0.smithi064.stdout:0/820: read da/d26/f32 [2236483,2912] 0 2024-02-16T22:56:42.818 INFO:tasks.workunit.client.0.smithi064.stdout:4/989: creat dc/ddc/de8/df2/f13d x:0 0 0 2024-02-16T22:56:42.819 INFO:tasks.workunit.client.0.smithi064.stdout:0/821: dread da/dc/de/d12/d2c/d2e/d38/f7a [0,4194304] 0 2024-02-16T22:56:42.828 INFO:tasks.workunit.client.0.smithi064.stdout:4/990: link dc/de/dfb/dff/de3/fb2 dc/de/d28/d57/d77/f13e 0 2024-02-16T22:56:42.828 INFO:tasks.workunit.client.0.smithi064.stdout:5/889: write d6/d25/df1/d80/f81 [1008898,127239] 0 2024-02-16T22:56:42.831 INFO:tasks.workunit.client.0.smithi064.stdout:4/991: truncate dc/dd/d32/d46/d5a/d7e/d9a/da2/ff7 816530 0 2024-02-16T22:56:42.833 INFO:tasks.workunit.client.0.smithi064.stdout:4/992: symlink dc/d12b/l13f 0 2024-02-16T22:56:42.834 INFO:tasks.workunit.client.0.smithi064.stdout:4/993: chown dc/de/d6d/dee 40 1 2024-02-16T22:56:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:5/890: creat d6/d25/df1/d91/da5/de3/d43/d7d/f131 x:0 0 0 2024-02-16T22:56:42.845 INFO:tasks.workunit.client.0.smithi064.stdout:4/994: rename dc/de/dfb/dff/de3 to dc/d12b/d140 0 2024-02-16T22:56:42.846 INFO:tasks.workunit.client.0.smithi064.stdout:0/822: dwrite da/dc/d3b/d5a/d8f/fbb [0,4194304] 0 2024-02-16T22:56:42.850 INFO:tasks.workunit.client.0.smithi064.stdout:0/823: mkdir da/dc/de/d37/daa/de9/d114 0 2024-02-16T22:56:42.852 INFO:tasks.workunit.client.0.smithi064.stdout:4/995: write dc/de/d6d/dee/ff5 [675521,7398] 0 2024-02-16T22:56:42.855 INFO:tasks.workunit.client.0.smithi064.stdout:5/891: dwrite d6/d25/da4/dc1/d85/f9f [0,4194304] 0 2024-02-16T22:56:42.856 INFO:tasks.workunit.client.0.smithi064.stdout:0/824: truncate da/dc/de/d37/d88/f9c 1052149 0 2024-02-16T22:56:42.859 INFO:tasks.workunit.client.0.smithi064.stdout:5/892: stat d6/d25/df1/d91/da5/de3/d43/db9/dfc/c122 0 2024-02-16T22:56:42.864 INFO:tasks.workunit.client.0.smithi064.stdout:4/996: dwrite dc/de/d28/d57/f117 [0,4194304] 0 2024-02-16T22:56:42.871 INFO:tasks.workunit.client.0.smithi064.stdout:0/825: write f9 [4204693,35362] 0 2024-02-16T22:56:42.873 INFO:tasks.workunit.client.0.smithi064.stdout:0/826: chown da/dc/de/d12/d2c/d2e/d38/dbe/dfe/f10a 3 1 2024-02-16T22:56:42.885 INFO:tasks.workunit.client.0.smithi064.stdout:5/893: creat d6/d25/df1/d91/da5/de3/d43/d7d/f132 x:0 0 0 2024-02-16T22:56:42.888 INFO:tasks.workunit.client.0.smithi064.stdout:0/827: read - da/dc/de/d12/d2c/d2e/d38/d7e/fc4 zero size 2024-02-16T22:56:42.889 INFO:tasks.workunit.client.0.smithi064.stdout:5/894: mknod d6/d25/df1/d91/da5/de3/d4b/c133 0 2024-02-16T22:56:42.893 INFO:tasks.workunit.client.0.smithi064.stdout:5/895: mknod d6/d18/d19/d112/d12a/c134 0 2024-02-16T22:56:42.894 INFO:tasks.workunit.client.0.smithi064.stdout:4/997: dwrite dc/dd/d32/f3d [0,4194304] 0 2024-02-16T22:56:42.894 INFO:tasks.workunit.client.0.smithi064.stdout:5/896: fsync d6/d18/d95/f12e 0 2024-02-16T22:56:42.897 INFO:tasks.workunit.client.0.smithi064.stdout:5/897: mknod d6/d25/df1/d91/c135 0 2024-02-16T22:56:42.900 INFO:tasks.workunit.client.0.smithi064.stdout:5/898: unlink d6/d18/d19/fc8 0 2024-02-16T22:56:42.901 INFO:tasks.workunit.client.0.smithi064.stdout:4/998: link dc/dd/d32/d58/l135 dc/de/d28/d57/l141 0 2024-02-16T22:56:42.901 INFO:tasks.workunit.client.0.smithi064.stdout:5/899: chown d6/d25/da4/dc1/d8e/le4 37 1 2024-02-16T22:56:42.902 INFO:tasks.workunit.client.0.smithi064.stdout:5/900: stat d6/d25 0 2024-02-16T22:56:42.903 INFO:tasks.workunit.client.0.smithi064.stdout:4/999: rmdir dc/d4a 39 2024-02-16T22:56:42.925 INFO:tasks.workunit.client.0.smithi064.stdout:5/901: dwrite d6/f34 [0,4194304] 0 2024-02-16T22:56:42.933 INFO:tasks.workunit.client.0.smithi064.stdout:5/902: dwrite d6/d25/da4/dc1/dd4/f109 [0,4194304] 0 2024-02-16T22:56:42.935 INFO:tasks.workunit.client.0.smithi064.stdout:5/903: mknod d6/d25/d48/c136 0 2024-02-16T22:56:42.942 INFO:tasks.workunit.client.0.smithi064.stdout:5/904: link d6/d25/df1/d91/da5/de3/d43/d7d/f131 d6/d18/f137 0 2024-02-16T22:56:42.943 INFO:tasks.workunit.client.0.smithi064.stdout:5/905: chown d6/d25/d48/f4a 10149 1 2024-02-16T22:56:42.946 INFO:tasks.workunit.client.0.smithi064.stdout:5/906: dread - d6/d25/df1/d91/da5/de3/d43/db9/dfc/ffd zero size 2024-02-16T22:56:42.954 INFO:tasks.workunit.client.0.smithi064.stdout:5/907: write d6/d25/d48/f4a [8305879,114955] 0 2024-02-16T22:56:42.963 INFO:tasks.workunit.client.0.smithi064.stdout:5/908: dwrite d6/f37 [0,4194304] 0 2024-02-16T22:56:42.969 INFO:tasks.workunit.client.0.smithi064.stdout:5/909: dread d6/d25/df1/d80/f81 [0,4194304] 0 2024-02-16T22:56:42.974 INFO:tasks.workunit.client.0.smithi064.stdout:5/910: write d6/d25/df1/d80/fb8 [676974,90856] 0 2024-02-16T22:56:42.982 INFO:tasks.workunit.client.0.smithi064.stdout:5/911: dread d6/d25/df1/d91/da5/de3/d43/f58 [0,4194304] 0 2024-02-16T22:56:42.988 INFO:tasks.workunit.client.0.smithi064.stdout:5/912: write d6/d25/df1/d91/da5/de3/f62 [852151,100842] 0 2024-02-16T22:56:42.991 INFO:tasks.workunit.client.0.smithi064.stdout:5/913: write d6/d25/df1/d91/da5/de3/f93 [725917,38935] 0 2024-02-16T22:56:42.992 INFO:tasks.workunit.client.0.smithi064.stdout:5/914: chown d6/d25/df1/d91/da5/de3/d43/db9/l66 659 1 2024-02-16T22:56:43.000 INFO:tasks.workunit.client.0.smithi064.stdout:5/915: dwrite d6/d25/df1/d91/da5/de3/d43/db9/fd8 [0,4194304] 0 2024-02-16T22:56:43.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:42 smithi192 ceph-mon[36254]: pgmap v26: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 86 MiB/s rd, 189 MiB/s wr, 432 op/s 2024-02-16T22:56:43.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:42 smithi064 ceph-mon[29226]: pgmap v26: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 86 MiB/s rd, 189 MiB/s wr, 432 op/s 2024-02-16T22:56:43.262 INFO:tasks.workunit.client.0.smithi064.stdout:7/950: sync 2024-02-16T22:56:43.268 INFO:tasks.workunit.client.0.smithi064.stdout:7/951: truncate d5/df9/fdf 230132 0 2024-02-16T22:56:43.272 INFO:tasks.workunit.client.0.smithi064.stdout:7/952: mknod d5/d33/d53/df2/d54/c13c 0 2024-02-16T22:56:43.273 INFO:tasks.workunit.client.0.smithi064.stdout:7/953: readlink d5/d33/d53/df2/d54/d7b/df8/d12e/l138 0 2024-02-16T22:56:43.274 INFO:tasks.workunit.client.0.smithi064.stdout:7/954: chown d5/d33/d53/df2/d54/df4/l13b 2425 1 2024-02-16T22:56:43.275 INFO:tasks.workunit.client.0.smithi064.stdout:7/955: chown d5/d33/d53/df2/d54/c76 81731 1 2024-02-16T22:56:43.277 INFO:tasks.workunit.client.0.smithi064.stdout:7/956: fdatasync d5/d7d/d12d/f80 0 2024-02-16T22:56:43.277 INFO:tasks.workunit.client.0.smithi064.stdout:7/957: readlink d5/df9/de4/df5/l128 0 2024-02-16T22:56:43.282 INFO:tasks.workunit.client.0.smithi064.stdout:7/958: rename d5/d7d/fa0 to d5/d33/d125/f13d 0 2024-02-16T22:56:43.283 INFO:tasks.workunit.client.0.smithi064.stdout:7/959: rename d5/d33/d53 to d5/d33/d53/df2/d54/d7b/d13e 22 2024-02-16T22:56:43.287 INFO:tasks.workunit.client.0.smithi064.stdout:7/960: fsync d5/d33/d53/df2/d54/fe3 0 2024-02-16T22:56:43.294 INFO:tasks.workunit.client.0.smithi064.stdout:7/961: dread d5/d33/d53/df2/f19 [0,4194304] 0 2024-02-16T22:56:43.295 INFO:tasks.workunit.client.0.smithi064.stdout:7/962: readlink d5/d33/d53/df2/d65/d7f/l131 0 2024-02-16T22:56:43.297 INFO:tasks.workunit.client.0.smithi064.stdout:0/828: sync 2024-02-16T22:56:43.297 INFO:tasks.workunit.client.0.smithi064.stdout:8/857: sync 2024-02-16T22:56:43.297 INFO:tasks.workunit.client.0.smithi064.stdout:1/878: sync 2024-02-16T22:56:43.298 INFO:tasks.workunit.client.0.smithi064.stdout:0/829: write da/dc/d3b/d72/d7c/fed [494107,20788] 0 2024-02-16T22:56:43.300 INFO:tasks.workunit.client.0.smithi064.stdout:1/879: symlink d2/de8/dfa/l126 0 2024-02-16T22:56:43.302 INFO:tasks.workunit.client.0.smithi064.stdout:7/963: getdents d5/d33/d101 0 2024-02-16T22:56:43.303 INFO:tasks.workunit.client.0.smithi064.stdout:0/830: truncate da/dc/de/d37/daa/db4/fba 551842 0 2024-02-16T22:56:43.305 INFO:tasks.workunit.client.0.smithi064.stdout:0/831: write da/dc/de/d12/d2c/d2e/d62/f10b [767910,107260] 0 2024-02-16T22:56:43.309 INFO:tasks.workunit.client.0.smithi064.stdout:1/880: mkdir d2/df/d11/d25/d127 0 2024-02-16T22:56:43.310 INFO:tasks.workunit.client.0.smithi064.stdout:7/964: dwrite d5/d33/d53/df2/d54/d7b/d11f/d66/fe0 [0,4194304] 0 2024-02-16T22:56:43.311 INFO:tasks.workunit.client.0.smithi064.stdout:7/965: write d5/f112 [4873112,58340] 0 2024-02-16T22:56:43.311 INFO:tasks.workunit.client.0.smithi064.stdout:0/832: creat da/dc/d3b/f115 x:0 0 0 2024-02-16T22:56:43.315 INFO:tasks.workunit.client.0.smithi064.stdout:7/966: unlink d5/d33/d53/df2/d54/l4c 0 2024-02-16T22:56:43.321 INFO:tasks.workunit.client.0.smithi064.stdout:1/881: getdents d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf 0 2024-02-16T22:56:43.322 INFO:tasks.workunit.client.0.smithi064.stdout:1/882: chown d2/df/d11/d25/cfe 31261726 1 2024-02-16T22:56:43.322 INFO:tasks.workunit.client.0.smithi064.stdout:1/883: write d2/f10a [480487,67362] 0 2024-02-16T22:56:43.326 INFO:tasks.workunit.client.0.smithi064.stdout:7/967: mkdir d5/d33/d42/d81/dc4/d13f 0 2024-02-16T22:56:43.326 INFO:tasks.workunit.client.0.smithi064.stdout:7/968: fdatasync d5/d33/d53/df2/d65/d7f/f9c 0 2024-02-16T22:56:43.329 INFO:tasks.workunit.client.0.smithi064.stdout:1/884: rename d2/df/d11/d25/fc5 to d2/df/d11/d1a/d30/f128 0 2024-02-16T22:56:43.332 INFO:tasks.workunit.client.0.smithi064.stdout:1/885: creat d2/df/d11/d1a/d1d/f129 x:0 0 0 2024-02-16T22:56:43.335 INFO:tasks.workunit.client.0.smithi064.stdout:1/886: mkdir d2/df/d14/d2c/de1/d12a 0 2024-02-16T22:56:43.337 INFO:tasks.workunit.client.0.smithi064.stdout:1/887: mknod d2/df/d11/d1a/d1d/c12b 0 2024-02-16T22:56:43.349 INFO:tasks.workunit.client.0.smithi064.stdout:1/888: dwrite d2/df/d11/f8f [0,4194304] 0 2024-02-16T22:56:43.351 INFO:tasks.workunit.client.0.smithi064.stdout:1/889: rmdir d2/df/d14/d2c/db5/de3/de6 39 2024-02-16T22:56:43.353 INFO:tasks.workunit.client.0.smithi064.stdout:1/890: read - d2/df/d11/d1a/d30/dbd/f70 zero size 2024-02-16T22:56:43.355 INFO:tasks.workunit.client.0.smithi064.stdout:1/891: symlink d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf/l12c 0 2024-02-16T22:56:43.358 INFO:tasks.workunit.client.0.smithi064.stdout:1/892: chown d2/df/d11/d25/l50 12738258 1 2024-02-16T22:56:43.359 INFO:tasks.workunit.client.0.smithi064.stdout:1/893: fsync d2/f5c 0 2024-02-16T22:56:43.363 INFO:tasks.workunit.client.0.smithi064.stdout:1/894: read d2/df/d11/d71/fd3 [1124463,52435] 0 2024-02-16T22:56:43.364 INFO:tasks.workunit.client.0.smithi064.stdout:1/895: chown d2/df/d11/d71/fe4 15547820 1 2024-02-16T22:56:43.368 INFO:tasks.workunit.client.0.smithi064.stdout:1/896: rename d2/df/d14 to d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d 0 2024-02-16T22:56:43.369 INFO:tasks.workunit.client.0.smithi064.stdout:1/897: creat d2/df/f12e x:0 0 0 2024-02-16T22:56:43.379 INFO:tasks.workunit.client.0.smithi064.stdout:1/898: write d2/df/d11/d1a/d30/dbd/d91/f65 [810116,109468] 0 2024-02-16T22:56:43.383 INFO:tasks.workunit.client.0.smithi064.stdout:1/899: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/fd4 [0,4194304] 0 2024-02-16T22:56:43.385 INFO:tasks.workunit.client.0.smithi064.stdout:1/900: symlink d2/de8/l12f 0 2024-02-16T22:56:43.391 INFO:tasks.workunit.client.0.smithi064.stdout:1/901: dwrite d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d6b/fd5 [0,4194304] 0 2024-02-16T22:56:43.397 INFO:tasks.workunit.client.0.smithi064.stdout:1/902: creat d2/df/d11/d25/d127/f130 x:0 0 0 2024-02-16T22:56:43.399 INFO:tasks.workunit.client.0.smithi064.stdout:1/903: write d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/de3/d100/f124 [428990,112902] 0 2024-02-16T22:56:43.402 INFO:tasks.workunit.client.0.smithi064.stdout:1/904: creat d2/de8/d109/f131 x:0 0 0 2024-02-16T22:56:43.405 INFO:tasks.workunit.client.0.smithi064.stdout:1/905: truncate d2/df/f54 2564859 0 2024-02-16T22:56:43.405 INFO:tasks.workunit.client.0.smithi064.stdout:1/906: stat d2/df/f13 0 2024-02-16T22:56:43.406 INFO:tasks.workunit.client.0.smithi064.stdout:1/907: dread d2/df/d11/d25/fe5 [0,4194304] 0 2024-02-16T22:56:43.408 INFO:tasks.workunit.client.0.smithi064.stdout:1/908: fsync d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/dc6/fff 0 2024-02-16T22:56:43.414 INFO:tasks.workunit.client.0.smithi064.stdout:1/909: dwrite d2/df/d11/d1a/d30/ffc [0,4194304] 0 2024-02-16T22:56:43.421 INFO:tasks.workunit.client.0.smithi064.stdout:1/910: mkdir d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d6b/d75/d106/d132 0 2024-02-16T22:56:43.423 INFO:tasks.workunit.client.0.smithi064.stdout:1/911: creat d2/df/f133 x:0 0 0 2024-02-16T22:56:43.425 INFO:tasks.workunit.client.0.smithi064.stdout:1/912: chown d2/df/d11/cf1 72 1 2024-02-16T22:56:43.427 INFO:tasks.workunit.client.0.smithi064.stdout:1/913: symlink d2/df/d11/l134 0 2024-02-16T22:56:43.428 INFO:tasks.workunit.client.0.smithi064.stdout:1/914: write d2/df/d11/d1a/d1d/f24 [1755100,99118] 0 2024-02-16T22:56:43.443 INFO:tasks.workunit.client.0.smithi064.stdout:1/915: dwrite d2/f36 [0,4194304] 0 2024-02-16T22:56:43.450 INFO:tasks.workunit.client.0.smithi064.stdout:1/916: dread d2/f5c [0,4194304] 0 2024-02-16T22:56:43.450 INFO:tasks.workunit.client.0.smithi064.stdout:1/917: dread - d2/df/d11/d1a/d30/dbd/f70 zero size 2024-02-16T22:56:43.455 INFO:tasks.workunit.client.0.smithi064.stdout:1/918: creat d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/de1/f135 x:0 0 0 2024-02-16T22:56:43.487 INFO:tasks.workunit.client.0.smithi064.stdout:5/916: sync 2024-02-16T22:56:43.490 INFO:tasks.workunit.client.0.smithi064.stdout:5/917: creat d6/d25/df1/d91/da5/de3/f138 x:0 0 0 2024-02-16T22:56:43.492 INFO:tasks.workunit.client.0.smithi064.stdout:5/918: chown d6/d18/c9c 19240586 1 2024-02-16T22:56:43.494 INFO:tasks.workunit.client.0.smithi064.stdout:5/919: mknod d6/d25/da4/dc1/dd4/c139 0 2024-02-16T22:56:43.495 INFO:tasks.workunit.client.0.smithi064.stdout:5/920: read d6/d25/f89 [797775,117953] 0 2024-02-16T22:56:43.497 INFO:tasks.workunit.client.0.smithi064.stdout:5/921: symlink d6/d25/l13a 0 2024-02-16T22:56:43.501 INFO:tasks.workunit.client.0.smithi064.stdout:5/922: getdents d6/d25/df1 0 2024-02-16T22:56:43.546 INFO:tasks.workunit.client.0.smithi064.stdout:5/923: rename d6/d25/da4/dc1/f45 to d6/d18/f13b 0 2024-02-16T22:56:43.546 INFO:tasks.workunit.client.0.smithi064.stdout:5/924: dread - d6/fff zero size 2024-02-16T22:56:43.549 INFO:tasks.workunit.client.0.smithi064.stdout:5/925: mknod d6/d25/df1/d91/d11d/c13c 0 2024-02-16T22:56:43.550 INFO:tasks.workunit.client.0.smithi064.stdout:5/926: dread - d6/d18/d95/dbe/f100 zero size 2024-02-16T22:56:43.551 INFO:tasks.workunit.client.0.smithi064.stdout:5/927: dread - d6/d25/df1/d91/da5/de3/d43/db9/d103/f114 zero size 2024-02-16T22:56:43.552 INFO:tasks.workunit.client.0.smithi064.stdout:5/928: chown d6/d25/d48/c136 1 1 2024-02-16T22:56:43.552 INFO:tasks.workunit.client.0.smithi064.stdout:5/929: chown d6/d18/f137 72594507 1 2024-02-16T22:56:43.557 INFO:tasks.workunit.client.0.smithi064.stdout:5/930: dread d6/d25/df1/d91/da5/de3/d43/db9/d115/f10d [0,4194304] 0 2024-02-16T22:56:43.558 INFO:tasks.workunit.client.0.smithi064.stdout:5/931: chown d6/d25/df1/d91/da5/de3/d43/f4c 3428147 1 2024-02-16T22:56:43.578 INFO:tasks.workunit.client.0.smithi064.stdout:5/932: dwrite d6/d18/d19/fb0 [0,4194304] 0 2024-02-16T22:56:43.586 INFO:tasks.workunit.client.0.smithi064.stdout:5/933: dwrite d6/d25/df1/d91/da5/de3/f138 [0,4194304] 0 2024-02-16T22:56:43.591 INFO:tasks.workunit.client.0.smithi064.stdout:5/934: dread d6/d25/df1/d91/da5/de3/f93 [0,4194304] 0 2024-02-16T22:56:43.591 INFO:tasks.workunit.client.0.smithi064.stdout:5/935: chown d6/d18/dc5 389123074 1 2024-02-16T22:56:43.595 INFO:tasks.workunit.client.0.smithi064.stdout:5/936: creat d6/d25/df1/d80/d11a/f13d x:0 0 0 2024-02-16T22:56:43.598 INFO:tasks.workunit.client.0.smithi064.stdout:5/937: symlink d6/d18/d95/l13e 0 2024-02-16T22:56:43.599 INFO:tasks.workunit.client.0.smithi064.stdout:5/938: fdatasync d6/d25/df1/d91/da5/de3/d43/d7d/de2/f105 0 2024-02-16T22:56:43.606 INFO:tasks.workunit.client.0.smithi064.stdout:5/939: dwrite d6/d25/df1/d91/da5/de3/ff3 [4194304,4194304] 0 2024-02-16T22:56:43.607 INFO:tasks.workunit.client.0.smithi064.stdout:5/940: chown d6/d25/df1/cf5 1891372255 1 2024-02-16T22:56:43.613 INFO:tasks.workunit.client.0.smithi064.stdout:5/941: rename d6/d25/df1/d91/da5/de3/d4b/c61 to d6/d18/dc5/c13f 0 2024-02-16T22:56:43.622 INFO:tasks.workunit.client.0.smithi064.stdout:5/942: rmdir d6/d18/d19/d112/d11e 0 2024-02-16T22:56:43.688 INFO:tasks.workunit.client.0.smithi064.stdout:8/858: sync 2024-02-16T22:56:43.695 INFO:tasks.workunit.client.0.smithi064.stdout:8/859: dwrite df/fe7 [0,4194304] 0 2024-02-16T22:56:43.743 INFO:tasks.workunit.client.0.smithi064.stdout:8/860: dwrite df/f4c [4194304,4194304] 0 2024-02-16T22:56:43.748 INFO:tasks.workunit.client.0.smithi064.stdout:8/861: symlink df/d31/d2f/d3e/ddf/l128 0 2024-02-16T22:56:43.753 INFO:tasks.workunit.client.0.smithi064.stdout:8/862: truncate df/d31/d2f/d3e/ddf/ff8 392890 0 2024-02-16T22:56:43.756 INFO:tasks.workunit.client.0.smithi064.stdout:8/863: creat df/d31/d2f/d38/d48/f129 x:0 0 0 2024-02-16T22:56:43.772 INFO:tasks.workunit.client.0.smithi064.stdout:8/864: dread df/d31/d61/f65 [0,4194304] 0 2024-02-16T22:56:43.773 INFO:tasks.workunit.client.0.smithi064.stdout:8/865: dread - df/d31/dc1/f110 zero size 2024-02-16T22:56:43.780 INFO:tasks.workunit.client.0.smithi064.stdout:8/866: fsync df/d31/d2f/d38/d48/da2/f10c 0 2024-02-16T22:56:43.789 INFO:tasks.workunit.client.0.smithi064.stdout:8/867: write df/f37 [2414270,42210] 0 2024-02-16T22:56:43.791 INFO:tasks.workunit.client.0.smithi064.stdout:8/868: rmdir df/d31/d61/d79/dc4 39 2024-02-16T22:56:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:8/869: rename df/d31/d2f/d4f/d6a/lff to df/d31/d2f/d7f/l12a 0 2024-02-16T22:56:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:8/870: dwrite df/d31/d47/ff7 [0,4194304] 0 2024-02-16T22:56:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:8/871: mknod df/d31/d2f/d3e/d5f/d93/d11c/c12b 0 2024-02-16T22:56:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:8/872: mkdir df/d31/d2f/d3e/d5f/d12c 0 2024-02-16T22:56:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:8/873: dread - df/d31/d2f/d7f/fd0 zero size 2024-02-16T22:56:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:8/874: read df/d31/d2f/d3e/d5f/fc7 [2723850,91730] 0 2024-02-16T22:56:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:8/875: mkdir df/d31/d2f/d4f/d12d 0 2024-02-16T22:56:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:8/876: creat df/d31/d61/d94/f12e x:0 0 0 2024-02-16T22:56:43.815 INFO:tasks.workunit.client.0.smithi064.stdout:8/877: creat df/d31/d47/dfd/f12f x:0 0 0 2024-02-16T22:56:43.819 INFO:tasks.workunit.client.0.smithi064.stdout:8/878: unlink df/d31/d8f/d80/dec/c10a 0 2024-02-16T22:56:43.824 INFO:tasks.workunit.client.0.smithi064.stdout:8/879: symlink df/d31/d8f/dc9/d73/l130 0 2024-02-16T22:56:43.824 INFO:tasks.workunit.client.0.smithi064.stdout:8/880: write f9 [12386,57813] 0 2024-02-16T22:56:43.826 INFO:tasks.workunit.client.0.smithi064.stdout:8/881: creat df/d31/d8f/dc9/d73/f131 x:0 0 0 2024-02-16T22:56:43.829 INFO:tasks.workunit.client.0.smithi064.stdout:8/882: symlink df/d31/dc1/l132 0 2024-02-16T22:56:43.829 INFO:tasks.workunit.client.0.smithi064.stdout:8/883: fdatasync df/d31/d61/d94/f12e 0 2024-02-16T22:56:43.834 INFO:tasks.workunit.client.0.smithi064.stdout:8/884: rename df/d31/d61/d79/f8c to df/d31/d2f/d38/d48/f133 0 2024-02-16T22:56:43.835 INFO:tasks.workunit.client.0.smithi064.stdout:8/885: write df/d31/d2f/d38/f39 [3524461,12672] 0 2024-02-16T22:56:43.842 INFO:tasks.workunit.client.0.smithi064.stdout:8/886: write df/d31/f41 [1188068,99595] 0 2024-02-16T22:56:43.845 INFO:tasks.workunit.client.0.smithi064.stdout:8/887: creat df/d31/d2f/d113/f134 x:0 0 0 2024-02-16T22:56:43.847 INFO:tasks.workunit.client.0.smithi064.stdout:8/888: creat df/d31/d2f/d3e/d5f/d12c/f135 x:0 0 0 2024-02-16T22:56:43.849 INFO:tasks.workunit.client.0.smithi064.stdout:8/889: mkdir df/d31/d8f/dab/dae/d136 0 2024-02-16T22:56:43.852 INFO:tasks.workunit.client.0.smithi064.stdout:8/890: getdents df/d31 0 2024-02-16T22:56:43.857 INFO:tasks.workunit.client.0.smithi064.stdout:8/891: dwrite df/f37 [4194304,4194304] 0 2024-02-16T22:56:43.864 INFO:tasks.workunit.client.0.smithi064.stdout:8/892: getdents df/d31/d2f/d4f/d6a 0 2024-02-16T22:56:43.868 INFO:tasks.workunit.client.0.smithi064.stdout:8/893: creat df/d31/d2f/d68/de4/f137 x:0 0 0 2024-02-16T22:56:43.874 INFO:tasks.workunit.client.0.smithi064.stdout:8/894: dwrite df/d31/d8f/d80/f84 [0,4194304] 0 2024-02-16T22:56:43.875 INFO:tasks.workunit.client.0.smithi064.stdout:8/895: dread - df/d31/d8f/d80/f10d zero size 2024-02-16T22:56:43.878 INFO:tasks.workunit.client.0.smithi064.stdout:8/896: readlink df/d31/d2f/l25 0 2024-02-16T22:56:43.879 INFO:tasks.workunit.client.0.smithi064.stdout:8/897: mknod df/d31/d2f/d3e/d5f/d12c/c138 0 2024-02-16T22:56:43.881 INFO:tasks.workunit.client.0.smithi064.stdout:8/898: creat df/d31/d8f/dc9/f139 x:0 0 0 2024-02-16T22:56:43.887 INFO:tasks.workunit.client.0.smithi064.stdout:8/899: dread df/d31/d2f/d3e/f9c [0,4194304] 0 2024-02-16T22:56:43.889 INFO:tasks.workunit.client.0.smithi064.stdout:8/900: mkdir df/d31/d2f/d13a 0 2024-02-16T22:56:43.891 INFO:tasks.workunit.client.0.smithi064.stdout:8/901: unlink df/d2a/f57 0 2024-02-16T22:56:43.893 INFO:tasks.workunit.client.0.smithi064.stdout:8/902: write df/d2a/fa3 [3092831,97555] 0 2024-02-16T22:56:43.895 INFO:tasks.workunit.client.0.smithi064.stdout:8/903: mknod df/d2a/df9/c13b 0 2024-02-16T22:56:43.897 INFO:tasks.workunit.client.0.smithi064.stdout:8/904: dread - df/d31/d61/d79/dc4/f126 zero size 2024-02-16T22:56:43.908 INFO:tasks.workunit.client.0.smithi064.stdout:8/905: write df/d31/d2f/dfe/f111 [20960,91289] 0 2024-02-16T22:56:43.915 INFO:tasks.workunit.client.0.smithi064.stdout:8/906: truncate df/d31/d2f/d38/f39 2395529 0 2024-02-16T22:56:43.917 INFO:tasks.workunit.client.0.smithi064.stdout:8/907: rmdir df/d31/d2f/d38 39 2024-02-16T22:56:43.922 INFO:tasks.workunit.client.0.smithi064.stdout:8/908: symlink df/d31/d2f/l13c 0 2024-02-16T22:56:43.922 INFO:tasks.workunit.client.0.smithi064.stdout:8/909: readlink df/d31/d47/l86 0 2024-02-16T22:56:43.930 INFO:tasks.workunit.client.0.smithi064.stdout:8/910: dread df/d31/d2f/d38/d48/f54 [0,4194304] 0 2024-02-16T22:56:43.930 INFO:tasks.workunit.client.0.smithi064.stdout:8/911: fsync df/f24 0 2024-02-16T22:56:43.933 INFO:tasks.workunit.client.0.smithi064.stdout:8/912: truncate df/d31/d2f/d68/f96 866833 0 2024-02-16T22:56:43.934 INFO:tasks.workunit.client.0.smithi064.stdout:8/913: chown df/d31/d8f/dc9/d73/dfb/f10f 0 1 2024-02-16T22:56:43.936 INFO:tasks.workunit.client.0.smithi064.stdout:8/914: mknod df/d31/d61/d79/dc4/c13d 0 2024-02-16T22:56:43.938 INFO:tasks.workunit.client.0.smithi064.stdout:8/915: symlink df/d31/d2f/d38/l13e 0 2024-02-16T22:56:43.939 INFO:tasks.workunit.client.0.smithi064.stdout:8/916: read - df/d31/d47/dfd/f12f zero size 2024-02-16T22:56:43.943 INFO:tasks.workunit.client.0.smithi064.stdout:8/917: link df/d31/d61/f65 df/d31/d2f/d38/d48/da2/d11b/f13f 0 2024-02-16T22:56:43.947 INFO:tasks.workunit.client.0.smithi064.stdout:8/918: creat df/d31/d8f/dc9/d121/f140 x:0 0 0 2024-02-16T22:56:43.949 INFO:tasks.workunit.client.0.smithi064.stdout:8/919: creat df/d31/d2f/d3e/d5f/f141 x:0 0 0 2024-02-16T22:56:43.954 INFO:tasks.workunit.client.0.smithi064.stdout:8/920: dwrite df/d31/d2f/d38/d48/da2/de2/f115 [0,4194304] 0 2024-02-16T22:56:43.955 INFO:tasks.workunit.client.0.smithi064.stdout:8/921: read f8 [603672,4741] 0 2024-02-16T22:56:43.959 INFO:tasks.workunit.client.0.smithi064.stdout:8/922: link df/d31/d8f/d80/ca1 df/d31/d2f/d68/de4/c142 0 2024-02-16T22:56:43.964 INFO:tasks.workunit.client.0.smithi064.stdout:8/923: dread f3 [0,4194304] 0 2024-02-16T22:56:43.968 INFO:tasks.workunit.client.0.smithi064.stdout:8/924: rename df/d31/d2f/d68/lcf to df/d31/d2f/d4f/d12d/l143 0 2024-02-16T22:56:43.974 INFO:tasks.workunit.client.0.smithi064.stdout:8/925: dwrite df/d31/f3f [0,4194304] 0 2024-02-16T22:56:43.978 INFO:tasks.workunit.client.0.smithi064.stdout:8/926: creat df/d2a/f144 x:0 0 0 2024-02-16T22:56:43.979 INFO:tasks.workunit.client.0.smithi064.stdout:8/927: stat df/d31/d8f/dab/dae/l119 0 2024-02-16T22:56:43.982 INFO:tasks.workunit.client.0.smithi064.stdout:8/928: mkdir df/d31/d2f/d3e/d5f/d93/dd2/d145 0 2024-02-16T22:56:43.988 INFO:tasks.workunit.client.0.smithi064.stdout:8/929: link c6 df/d31/d2f/d3e/d5f/d12c/c146 0 2024-02-16T22:56:43.990 INFO:tasks.workunit.client.0.smithi064.stdout:8/930: chown df/d2a/d5c/cde 0 1 2024-02-16T22:56:43.990 INFO:tasks.workunit.client.0.smithi064.stdout:8/931: fdatasync df/d31/d2f/dfe/f111 0 2024-02-16T22:56:44.003 INFO:tasks.workunit.client.0.smithi064.stdout:8/932: dread df/d31/d2f/d4f/fa7 [0,4194304] 0 2024-02-16T22:56:44.006 INFO:tasks.workunit.client.0.smithi064.stdout:7/969: sync 2024-02-16T22:56:44.007 INFO:tasks.workunit.client.0.smithi064.stdout:0/833: sync 2024-02-16T22:56:44.007 INFO:tasks.workunit.client.0.smithi064.stdout:8/933: mknod df/d31/d61/d79/d81/d123/c147 0 2024-02-16T22:56:44.008 INFO:tasks.workunit.client.0.smithi064.stdout:0/834: write da/dc/de/d12/d2c/d2e/d38/fe7 [642375,66017] 0 2024-02-16T22:56:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:0/835: dread da/fb [0,4194304] 0 2024-02-16T22:56:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:8/934: dwrite df/f37 [4194304,4194304] 0 2024-02-16T22:56:44.018 INFO:tasks.workunit.client.0.smithi064.stdout:0/836: write da/dc/d3b/f5f [3102289,65183] 0 2024-02-16T22:56:44.019 INFO:tasks.workunit.client.0.smithi064.stdout:8/935: symlink df/d31/d8f/dc9/d73/l148 0 2024-02-16T22:56:44.020 INFO:tasks.workunit.client.0.smithi064.stdout:7/970: getdents d5/d33/d53/df2/d54/d41/da2 0 2024-02-16T22:56:44.022 INFO:tasks.workunit.client.0.smithi064.stdout:8/936: mkdir df/d31/dc1/d149 0 2024-02-16T22:56:44.023 INFO:tasks.workunit.client.0.smithi064.stdout:8/937: chown df/d31/d2f/d3e/d5f/d12c/c138 3490587 1 2024-02-16T22:56:44.024 INFO:tasks.workunit.client.0.smithi064.stdout:8/938: rmdir df/d31/d2f/d38 39 2024-02-16T22:56:44.026 INFO:tasks.workunit.client.0.smithi064.stdout:8/939: mknod df/d31/d8f/d80/dec/c14a 0 2024-02-16T22:56:44.029 INFO:tasks.workunit.client.0.smithi064.stdout:7/971: write d5/d33/d53/df2/d54/d41/f68 [389155,10940] 0 2024-02-16T22:56:44.032 INFO:tasks.workunit.client.0.smithi064.stdout:0/837: dwrite da/dc/de/fc8 [0,4194304] 0 2024-02-16T22:56:44.043 INFO:tasks.workunit.client.0.smithi064.stdout:7/972: rename d5/d33/d53/df2/d54/d7b/df8/d12e/l138 to d5/d33/d101/l140 0 2024-02-16T22:56:44.044 INFO:tasks.workunit.client.0.smithi064.stdout:8/940: truncate df/d2a/fa3 1766919 0 2024-02-16T22:56:44.045 INFO:tasks.workunit.client.0.smithi064.stdout:0/838: symlink da/dc/de/d37/daa/d106/l116 0 2024-02-16T22:56:44.047 INFO:tasks.workunit.client.0.smithi064.stdout:5/943: sync 2024-02-16T22:56:44.047 INFO:tasks.workunit.client.0.smithi064.stdout:1/919: sync 2024-02-16T22:56:44.051 INFO:tasks.workunit.client.0.smithi064.stdout:7/973: dwrite d5/d33/d53/f115 [0,4194304] 0 2024-02-16T22:56:44.053 INFO:tasks.workunit.client.0.smithi064.stdout:7/974: stat d5/d33/d53/df2/d65/d7f/l131 0 2024-02-16T22:56:44.053 INFO:tasks.workunit.client.0.smithi064.stdout:0/839: rename da/dc/de/d12/d2c/d2e/d38/l8a to da/dc/de/d12/d2c/d2e/d38/dbe/l117 0 2024-02-16T22:56:44.054 INFO:tasks.workunit.client.0.smithi064.stdout:0/840: write da/dc/de/f86 [914419,112182] 0 2024-02-16T22:56:44.056 INFO:tasks.workunit.client.0.smithi064.stdout:0/841: chown da/dc/de/d12/d1d/d100/l110 1305 1 2024-02-16T22:56:44.065 INFO:tasks.workunit.client.0.smithi064.stdout:7/975: truncate d5/d33/d53/df2/d54/d41/f87 1920964 0 2024-02-16T22:56:44.067 INFO:tasks.workunit.client.0.smithi064.stdout:5/944: unlink d6/d25/df1/d91/da5/de3/c42 0 2024-02-16T22:56:44.069 INFO:tasks.workunit.client.0.smithi064.stdout:5/945: write d6/d18/d19/f2e [853501,10258] 0 2024-02-16T22:56:44.076 INFO:tasks.workunit.client.0.smithi064.stdout:1/920: rename d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d84/fc4 to d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/de3/de6/f136 0 2024-02-16T22:56:44.080 INFO:tasks.workunit.client.0.smithi064.stdout:0/842: fsync da/dc/d3b/d72/d7c/fa5 0 2024-02-16T22:56:44.082 INFO:tasks.workunit.client.0.smithi064.stdout:5/946: rename d6/d25/f89 to d6/d25/d48/d124/f140 0 2024-02-16T22:56:44.085 INFO:tasks.workunit.client.0.smithi064.stdout:7/976: dread d5/d33/d53/df2/d65/d7f/f9c [0,4194304] 0 2024-02-16T22:56:44.086 INFO:tasks.workunit.client.0.smithi064.stdout:7/977: dread - d5/d33/d53/df2/d54/d7b/dc6/f119 zero size 2024-02-16T22:56:44.095 INFO:tasks.workunit.client.0.smithi064.stdout:8/941: dwrite df/d31/d2f/d38/f39 [0,4194304] 0 2024-02-16T22:56:44.123 INFO:tasks.workunit.client.0.smithi064.stdout:0/843: write da/dc/de/fd6 [4432987,29457] 0 2024-02-16T22:56:44.154 INFO:tasks.workunit.client.0.smithi064.stdout:7/978: fsync d5/d33/fd1 0 2024-02-16T22:56:44.155 INFO:tasks.workunit.client.0.smithi064.stdout:1/921: rename d2/df/d11/d1a/d30/dbd/d91/d56/cf5 to d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/de1/d12a/c137 0 2024-02-16T22:56:44.157 INFO:tasks.workunit.client.0.smithi064.stdout:8/942: read - df/d31/d2f/d3e/d5f/d93/feb zero size 2024-02-16T22:56:44.162 INFO:tasks.workunit.client.0.smithi064.stdout:1/922: dwrite d2/df/d11/d1a/d30/d3d/f110 [0,4194304] 0 2024-02-16T22:56:44.164 INFO:tasks.workunit.client.0.smithi064.stdout:7/979: mknod d5/d33/d53/df2/d65/d7f/db2/c141 0 2024-02-16T22:56:44.165 INFO:tasks.workunit.client.0.smithi064.stdout:7/980: readlink d5/d7d/d12d/l92 0 2024-02-16T22:56:44.167 INFO:tasks.workunit.client.0.smithi064.stdout:0/844: symlink da/l118 0 2024-02-16T22:56:44.168 INFO:tasks.workunit.client.0.smithi064.stdout:5/947: getdents d6/d25/df1/d91/da5 0 2024-02-16T22:56:44.168 INFO:tasks.workunit.client.0.smithi064.stdout:7/981: chown d5/d33/d53/df2/d54/fe6 277092792 1 2024-02-16T22:56:44.174 INFO:tasks.workunit.client.0.smithi064.stdout:8/943: dwrite df/f1b [0,4194304] 0 2024-02-16T22:56:44.175 INFO:tasks.workunit.client.0.smithi064.stdout:1/923: dwrite d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/f52 [0,4194304] 0 2024-02-16T22:56:44.178 INFO:tasks.workunit.client.0.smithi064.stdout:7/982: symlink d5/d33/d45/l142 0 2024-02-16T22:56:44.180 INFO:tasks.workunit.client.0.smithi064.stdout:8/944: creat df/d31/d2f/d3e/ddf/f14b x:0 0 0 2024-02-16T22:56:44.182 INFO:tasks.workunit.client.0.smithi064.stdout:0/845: truncate da/dc/de/d12/d2c/d2e/d38/d87/f9f 534536 0 2024-02-16T22:56:44.185 INFO:tasks.workunit.client.0.smithi064.stdout:8/945: write df/d31/d2f/dfe/f111 [798180,78892] 0 2024-02-16T22:56:44.187 INFO:tasks.workunit.client.0.smithi064.stdout:1/924: dwrite d2/de8/d109/f121 [0,4194304] 0 2024-02-16T22:56:44.187 INFO:tasks.workunit.client.0.smithi064.stdout:0/846: write da/dc/de/d12/d2c/d2e/d62/f10b [819938,125659] 0 2024-02-16T22:56:44.189 INFO:tasks.workunit.client.0.smithi064.stdout:8/946: write df/d31/d2f/d113/f134 [404440,19955] 0 2024-02-16T22:56:44.209 INFO:tasks.workunit.client.0.smithi064.stdout:0/847: stat da/dc/de/d37/daa/de9/f111 0 2024-02-16T22:56:44.211 INFO:tasks.workunit.client.0.smithi064.stdout:1/925: creat d2/df/d11/d25/d127/f138 x:0 0 0 2024-02-16T22:56:44.212 INFO:tasks.workunit.client.0.smithi064.stdout:0/848: write da/dc/de/d12/d2c/d2e/d38/dbe/dc6/fd8 [63455,46940] 0 2024-02-16T22:56:44.214 INFO:tasks.workunit.client.0.smithi064.stdout:8/947: truncate df/d31/d61/d79/f9b 4260151 0 2024-02-16T22:56:44.218 INFO:tasks.workunit.client.0.smithi064.stdout:0/849: creat da/d26/de0/d93/f119 x:0 0 0 2024-02-16T22:56:44.219 INFO:tasks.workunit.client.0.smithi064.stdout:0/850: chown da/dc/de/d12/d1d/c5e 1 1 2024-02-16T22:56:44.219 INFO:tasks.workunit.client.0.smithi064.stdout:8/948: fdatasync df/d31/d2f/d7f/f122 0 2024-02-16T22:56:44.220 INFO:tasks.workunit.client.0.smithi064.stdout:1/926: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d40/f116 [0,4194304] 0 2024-02-16T22:56:44.225 INFO:tasks.workunit.client.0.smithi064.stdout:0/851: dwrite da/dc/de/d12/d2c/d2e/f3e [0,4194304] 0 2024-02-16T22:56:44.232 INFO:tasks.workunit.client.0.smithi064.stdout:1/927: link d2/df/f12e d2/df/d11/d1a/d30/dbd/d91/f139 0 2024-02-16T22:56:44.235 INFO:tasks.workunit.client.0.smithi064.stdout:0/852: rename da/dc/de/ce2 to da/d26/de0/c11a 0 2024-02-16T22:56:44.240 INFO:tasks.workunit.client.0.smithi064.stdout:5/948: dwrite d6/f3c [0,4194304] 0 2024-02-16T22:56:44.244 INFO:tasks.workunit.client.0.smithi064.stdout:0/853: read da/dc/de/d12/d1d/f1f [684528,118461] 0 2024-02-16T22:56:44.244 INFO:tasks.workunit.client.0.smithi064.stdout:1/928: creat d2/df/d11/d25/f13a x:0 0 0 2024-02-16T22:56:44.246 INFO:tasks.workunit.client.0.smithi064.stdout:5/949: symlink d6/d25/df1/d91/da5/de3/d43/l141 0 2024-02-16T22:56:44.250 INFO:tasks.workunit.client.0.smithi064.stdout:8/949: write df/d31/d2f/d3e/d5f/d93/fe0 [1474499,44873] 0 2024-02-16T22:56:44.251 INFO:tasks.workunit.client.0.smithi064.stdout:1/929: read d2/df/d11/d1a/d1d/f4d [2214193,105787] 0 2024-02-16T22:56:44.252 INFO:tasks.workunit.client.0.smithi064.stdout:8/950: fdatasync df/d31/d61/d94/ff6 0 2024-02-16T22:56:44.252 INFO:tasks.workunit.client.0.smithi064.stdout:5/950: symlink d6/d25/l142 0 2024-02-16T22:56:44.256 INFO:tasks.workunit.client.0.smithi064.stdout:8/951: mknod df/d31/d8f/dc9/c14c 0 2024-02-16T22:56:44.256 INFO:tasks.workunit.client.0.smithi064.stdout:1/930: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d40/f116 [0,4194304] 0 2024-02-16T22:56:44.259 INFO:tasks.workunit.client.0.smithi064.stdout:8/952: truncate df/d2a/f70 652917 0 2024-02-16T22:56:44.262 INFO:tasks.workunit.client.0.smithi064.stdout:1/931: fsync d2/df/d11/d1a/d30/dbd/f72 0 2024-02-16T22:56:44.263 INFO:tasks.workunit.client.0.smithi064.stdout:8/953: mkdir df/d31/d2f/d4f/d14d 0 2024-02-16T22:56:44.265 INFO:tasks.workunit.client.0.smithi064.stdout:8/954: stat df/d31/d8f/dc9/c100 0 2024-02-16T22:56:44.265 INFO:tasks.workunit.client.0.smithi064.stdout:5/951: link d6/d25/d48/c136 d6/d25/df1/d80/d11a/c143 0 2024-02-16T22:56:44.266 INFO:tasks.workunit.client.0.smithi064.stdout:0/854: dread da/dc/d3b/d5a/fa1 [0,4194304] 0 2024-02-16T22:56:44.268 INFO:tasks.workunit.client.0.smithi064.stdout:0/855: dread da/dc/d3b/d72/d7c/fa5 [0,4194304] 0 2024-02-16T22:56:44.270 INFO:tasks.workunit.client.0.smithi064.stdout:1/932: mkdir d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d6b/d75/d13b 0 2024-02-16T22:56:44.271 INFO:tasks.workunit.client.0.smithi064.stdout:1/933: readlink d2/df/d11/d1a/d30/d3d/l118 0 2024-02-16T22:56:44.274 INFO:tasks.workunit.client.0.smithi064.stdout:0/856: creat da/dc/d3b/d72/f11b x:0 0 0 2024-02-16T22:56:44.275 INFO:tasks.workunit.client.0.smithi064.stdout:5/952: creat d6/f144 x:0 0 0 2024-02-16T22:56:44.276 INFO:tasks.workunit.client.0.smithi064.stdout:8/955: creat df/d31/d2f/d38/f14e x:0 0 0 2024-02-16T22:56:44.277 INFO:tasks.workunit.client.0.smithi064.stdout:1/934: fdatasync d2/df/f12e 0 2024-02-16T22:56:44.278 INFO:tasks.workunit.client.0.smithi064.stdout:8/956: read df/f3d [3299273,23421] 0 2024-02-16T22:56:44.279 INFO:tasks.workunit.client.0.smithi064.stdout:0/857: symlink da/dc/de/d12/d1d/dd2/l11c 0 2024-02-16T22:56:44.283 INFO:tasks.workunit.client.0.smithi064.stdout:1/935: mknod d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/de3/d100/c13c 0 2024-02-16T22:56:44.286 INFO:tasks.workunit.client.0.smithi064.stdout:8/957: mkdir df/d31/d2f/d38/d14f 0 2024-02-16T22:56:44.287 INFO:tasks.workunit.client.0.smithi064.stdout:0/858: dread da/dc/de/d12/f29 [4194304,4194304] 0 2024-02-16T22:56:44.288 INFO:tasks.workunit.client.0.smithi064.stdout:0/859: write da/dc/de/f10c [123373,91182] 0 2024-02-16T22:56:44.290 INFO:tasks.workunit.client.0.smithi064.stdout:8/958: mknod df/d31/d2f/d3e/ddf/d109/d11f/c150 0 2024-02-16T22:56:44.292 INFO:tasks.workunit.client.0.smithi064.stdout:1/936: mknod d2/df/d11/d1a/c13d 0 2024-02-16T22:56:44.292 INFO:tasks.workunit.client.0.smithi064.stdout:0/860: symlink da/l11d 0 2024-02-16T22:56:44.294 INFO:tasks.workunit.client.0.smithi064.stdout:0/861: creat da/d71/f11e x:0 0 0 2024-02-16T22:56:44.298 INFO:tasks.workunit.client.0.smithi064.stdout:0/862: creat da/dc/de/d12/f11f x:0 0 0 2024-02-16T22:56:44.308 INFO:tasks.workunit.client.0.smithi064.stdout:0/863: dwrite da/dc/d3b/d72/d7c/fcb [0,4194304] 0 2024-02-16T22:56:44.314 INFO:tasks.workunit.client.0.smithi064.stdout:0/864: dwrite da/dc/d3b/d72/db3/fda [0,4194304] 0 2024-02-16T22:56:44.318 INFO:tasks.workunit.client.0.smithi064.stdout:0/865: creat da/dc/d3b/d72/db3/dfc/f120 x:0 0 0 2024-02-16T22:56:44.325 INFO:tasks.workunit.client.0.smithi064.stdout:0/866: link da/dc/de/d37/daa/de9/f111 da/dc/de/d37/daa/f121 0 2024-02-16T22:56:44.325 INFO:tasks.workunit.client.0.smithi064.stdout:0/867: stat da/dc/de/d12/d1d/f2f 0 2024-02-16T22:56:44.328 INFO:tasks.workunit.client.0.smithi064.stdout:0/868: unlink da/dc/de/d12/d2c/d2e/d38/fe7 0 2024-02-16T22:56:44.345 INFO:tasks.workunit.client.0.smithi064.stdout:0/869: dwrite da/dc/d3b/d5a/f73 [4194304,4194304] 0 2024-02-16T22:56:44.353 INFO:tasks.workunit.client.0.smithi064.stdout:0/870: write da/dc/de/d12/d1d/f1f [1027288,60171] 0 2024-02-16T22:56:44.360 INFO:tasks.workunit.client.0.smithi064.stdout:0/871: dwrite da/dc/de/d12/d2c/d2e/d96/ff8 [0,4194304] 0 2024-02-16T22:56:44.362 INFO:tasks.workunit.client.0.smithi064.stdout:0/872: chown da/dc/d3b/d72/db3/fee 2387 1 2024-02-16T22:56:44.363 INFO:tasks.workunit.client.0.smithi064.stdout:0/873: read da/dc/de/d12/d2c/d2e/d96/ff8 [3592550,18194] 0 2024-02-16T22:56:44.369 INFO:tasks.workunit.client.0.smithi064.stdout:0/874: dread da/dc/d3b/d72/d7c/fcb [0,4194304] 0 2024-02-16T22:56:44.372 INFO:tasks.workunit.client.0.smithi064.stdout:0/875: mkdir da/dc/d3b/d5a/d122 0 2024-02-16T22:56:44.378 INFO:tasks.workunit.client.0.smithi064.stdout:0/876: getdents da/d26/d3f 0 2024-02-16T22:56:44.381 INFO:tasks.workunit.client.0.smithi064.stdout:0/877: creat da/dc/de/d12/d2c/d2e/d96/f123 x:0 0 0 2024-02-16T22:56:44.383 INFO:tasks.workunit.client.0.smithi064.stdout:0/878: creat da/dc/d3b/d72/db3/f124 x:0 0 0 2024-02-16T22:56:44.384 INFO:tasks.workunit.client.0.smithi064.stdout:0/879: readlink da/dc/de/d12/d1d/l4c 0 2024-02-16T22:56:44.385 INFO:tasks.workunit.client.0.smithi064.stdout:0/880: write da/dc/de/d12/d2c/d2e/f36 [6804937,32084] 0 2024-02-16T22:56:44.386 INFO:tasks.workunit.client.0.smithi064.stdout:0/881: write da/dc/de/d12/d1d/f1f [1906423,101455] 0 2024-02-16T22:56:44.389 INFO:tasks.workunit.client.0.smithi064.stdout:0/882: mkdir da/dc/de/d12/d1d/d100/d125 0 2024-02-16T22:56:44.391 INFO:tasks.workunit.client.0.smithi064.stdout:0/883: write da/dc/d3b/d72/db3/dfc/fff [983056,26423] 0 2024-02-16T22:56:44.392 INFO:tasks.workunit.client.0.smithi064.stdout:0/884: write da/dc/de/d12/d2c/d2e/f3e [290765,90554] 0 2024-02-16T22:56:44.394 INFO:tasks.workunit.client.0.smithi064.stdout:0/885: fsync da/dc/d3b/d72/db3/fda 0 2024-02-16T22:56:44.405 INFO:tasks.workunit.client.0.smithi064.stdout:0/886: dread da/dc/d3b/d5a/f73 [0,4194304] 0 2024-02-16T22:56:44.415 INFO:tasks.workunit.client.0.smithi064.stdout:0/887: dwrite da/dc/de/d12/d1d/f2f [0,4194304] 0 2024-02-16T22:56:44.432 INFO:tasks.workunit.client.0.smithi064.stdout:0/888: dread da/dc/de/d12/d2c/f31 [0,4194304] 0 2024-02-16T22:56:44.435 INFO:tasks.workunit.client.0.smithi064.stdout:0/889: mknod da/dc/d3b/dc2/c126 0 2024-02-16T22:56:44.910 INFO:tasks.workunit.client.0.smithi064.stdout:7/983: sync 2024-02-16T22:56:44.915 INFO:tasks.workunit.client.0.smithi064.stdout:7/984: dwrite d5/df9/de4/f136 [0,4194304] 0 2024-02-16T22:56:44.929 INFO:tasks.workunit.client.0.smithi064.stdout:7/985: write d5/d33/d45/d64/f117 [534764,44398] 0 2024-02-16T22:56:44.939 INFO:tasks.workunit.client.0.smithi064.stdout:7/986: dwrite d5/d33/d45/fb5 [0,4194304] 0 2024-02-16T22:56:44.950 INFO:tasks.workunit.client.0.smithi064.stdout:7/987: truncate d5/d33/d53/f70 3827320 0 2024-02-16T22:56:44.952 INFO:tasks.workunit.client.0.smithi064.stdout:7/988: mknod d5/d33/d53/df2/d54/dfa/c143 0 2024-02-16T22:56:44.956 INFO:tasks.workunit.client.0.smithi064.stdout:7/989: read d5/d33/d45/f98 [3975620,80220] 0 2024-02-16T22:56:44.957 INFO:tasks.workunit.client.0.smithi064.stdout:7/990: dread - d5/d33/d53/df2/d54/fe6 zero size 2024-02-16T22:56:44.959 INFO:tasks.workunit.client.0.smithi064.stdout:7/991: symlink d5/d33/d53/df2/d65/l144 0 2024-02-16T22:56:44.960 INFO:tasks.workunit.client.0.smithi064.stdout:7/992: chown d5/d33/d53/df2/d65/d7f/db2/c141 253221 1 2024-02-16T22:56:44.965 INFO:tasks.workunit.client.0.smithi064.stdout:7/993: dwrite d5/d33/d45/d64/f85 [0,4194304] 0 2024-02-16T22:56:44.966 INFO:tasks.workunit.client.0.smithi064.stdout:7/994: truncate d5/df9/de4/f130 349699 0 2024-02-16T22:56:44.976 INFO:tasks.workunit.client.0.smithi064.stdout:8/959: sync 2024-02-16T22:56:44.976 INFO:tasks.workunit.client.0.smithi064.stdout:5/953: sync 2024-02-16T22:56:44.977 INFO:tasks.workunit.client.0.smithi064.stdout:8/960: chown df/d31/d2f/d3e/d5f/d93/cbf 4 1 2024-02-16T22:56:44.986 INFO:tasks.workunit.client.0.smithi064.stdout:7/995: write d5/d7d/f93 [449786,24019] 0 2024-02-16T22:56:44.989 INFO:tasks.workunit.client.0.smithi064.stdout:8/961: write df/d31/d2f/d38/f62 [946639,21541] 0 2024-02-16T22:56:44.993 INFO:tasks.workunit.client.0.smithi064.stdout:8/962: mkdir df/d2a/d151 0 2024-02-16T22:56:44.994 INFO:tasks.workunit.client.0.smithi064.stdout:8/963: dread f3 [0,4194304] 0 2024-02-16T22:56:44.995 INFO:tasks.workunit.client.0.smithi064.stdout:8/964: fsync df/d31/d8f/d80/f84 0 2024-02-16T22:56:44.999 INFO:tasks.workunit.client.0.smithi064.stdout:8/965: truncate df/d31/d61/d94/fa6 152089 0 2024-02-16T22:56:45.000 INFO:tasks.workunit.client.0.smithi064.stdout:5/954: dread d6/d25/df1/d91/da5/de3/d43/f4c [0,4194304] 0 2024-02-16T22:56:45.001 INFO:tasks.workunit.client.0.smithi064.stdout:8/966: write df/d31/d2f/dfe/f111 [1446932,9021] 0 2024-02-16T22:56:45.004 INFO:tasks.workunit.client.0.smithi064.stdout:8/967: chown cd 19291939 1 2024-02-16T22:56:45.005 INFO:tasks.workunit.client.0.smithi064.stdout:7/996: dread d5/d33/d53/da3/fda [0,4194304] 0 2024-02-16T22:56:45.006 INFO:tasks.workunit.client.0.smithi064.stdout:5/955: link d6/d18/fee d6/d25/df1/d91/da5/de3/d43/db9/f145 0 2024-02-16T22:56:45.007 INFO:tasks.workunit.client.0.smithi064.stdout:8/968: mknod df/d2a/d5c/c152 0 2024-02-16T22:56:45.009 INFO:tasks.workunit.client.0.smithi064.stdout:8/969: stat df/d31/d2f/d38/c120 0 2024-02-16T22:56:45.009 INFO:tasks.workunit.client.0.smithi064.stdout:8/970: dread - df/d31/d2f/d7f/fd0 zero size 2024-02-16T22:56:45.018 INFO:tasks.workunit.client.0.smithi064.stdout:7/997: write d5/d33/d53/df2/d54/d41/f4b [200905,94391] 0 2024-02-16T22:56:45.019 INFO:tasks.workunit.client.0.smithi064.stdout:5/956: truncate d6/d25/da4/dc1/d8e/f11b 1439629 0 2024-02-16T22:56:45.021 INFO:tasks.workunit.client.0.smithi064.stdout:8/971: dread df/f1a [0,4194304] 0 2024-02-16T22:56:45.022 INFO:tasks.workunit.client.0.smithi064.stdout:7/998: read d5/d33/d53/df2/d54/f63 [1114607,92285] 0 2024-02-16T22:56:45.023 INFO:tasks.workunit.client.0.smithi064.stdout:5/957: mkdir d6/d25/da4/dc1/d146 0 2024-02-16T22:56:45.026 INFO:tasks.workunit.client.0.smithi064.stdout:8/972: creat df/d31/d2f/d13a/f153 x:0 0 0 2024-02-16T22:56:45.026 INFO:tasks.workunit.client.0.smithi064.stdout:5/958: mknod d6/d25/da4/dc1/dd4/c147 0 2024-02-16T22:56:45.027 INFO:tasks.workunit.client.0.smithi064.stdout:7/999: mknod d5/d33/d42/d81/c145 0 2024-02-16T22:56:45.027 INFO:tasks.workunit.client.0.smithi064.stdout:5/959: write d6/d25/df1/d80/fe0 [2475367,20202] 0 2024-02-16T22:56:45.029 INFO:tasks.workunit.client.0.smithi064.stdout:8/973: mkdir df/d31/d61/d79/d81/d154 0 2024-02-16T22:56:45.031 INFO:tasks.workunit.client.0.smithi064.stdout:5/960: truncate d6/d25/f32 465258 0 2024-02-16T22:56:45.032 INFO:tasks.workunit.client.0.smithi064.stdout:8/974: mkdir df/d31/d47/d155 0 2024-02-16T22:56:45.033 INFO:tasks.workunit.client.0.smithi064.stdout:8/975: dread - df/d31/d2f/d3e/ddf/f14b zero size 2024-02-16T22:56:45.049 INFO:tasks.workunit.client.0.smithi064.stdout:0/890: sync 2024-02-16T22:56:45.049 INFO:tasks.workunit.client.0.smithi064.stdout:1/937: sync 2024-02-16T22:56:45.051 INFO:tasks.workunit.client.0.smithi064.stdout:0/891: read - da/dc/d3b/d72/db3/f124 zero size 2024-02-16T22:56:45.052 INFO:tasks.workunit.client.0.smithi064.stdout:1/938: write d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/f28 [36838,126686] 0 2024-02-16T22:56:45.061 INFO:tasks.workunit.client.0.smithi064.stdout:8/976: write df/d31/d61/f65 [984128,8852] 0 2024-02-16T22:56:45.064 INFO:tasks.workunit.client.0.smithi064.stdout:0/892: getdents da/d71/da4 0 2024-02-16T22:56:45.074 INFO:tasks.workunit.client.0.smithi064.stdout:8/977: dread df/d31/f7c [0,4194304] 0 2024-02-16T22:56:45.078 INFO:tasks.workunit.client.0.smithi064.stdout:0/893: creat da/dc/f127 x:0 0 0 2024-02-16T22:56:45.080 INFO:tasks.workunit.client.0.smithi064.stdout:8/978: mknod df/d2a/df9/c156 0 2024-02-16T22:56:45.081 INFO:tasks.workunit.client.0.smithi064.stdout:0/894: symlink da/dc/de/d37/daa/d106/l128 0 2024-02-16T22:56:45.088 INFO:tasks.workunit.client.0.smithi064.stdout:0/895: creat da/dc/de/d37/daa/dcd/f129 x:0 0 0 2024-02-16T22:56:45.089 INFO:tasks.workunit.client.0.smithi064.stdout:1/939: dwrite d2/df/d11/d1a/d30/dbd/d91/f139 [0,4194304] 0 2024-02-16T22:56:45.096 INFO:tasks.workunit.client.0.smithi064.stdout:1/940: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce [0,4194304] 0 2024-02-16T22:56:45.100 INFO:tasks.workunit.client.0.smithi064.stdout:5/961: truncate d6/d25/da4/dc1/d8e/f11b 832954 0 2024-02-16T22:56:45.101 INFO:tasks.workunit.client.0.smithi064.stdout:1/941: dread d2/df/d11/d1a/d30/ffc [0,4194304] 0 2024-02-16T22:56:45.114 INFO:tasks.workunit.client.0.smithi064.stdout:0/896: dread da/d26/f66 [0,4194304] 0 2024-02-16T22:56:45.115 INFO:tasks.workunit.client.0.smithi064.stdout:0/897: dread - da/dc/d3b/dc2/f10f zero size 2024-02-16T22:56:45.116 INFO:tasks.workunit.client.0.smithi064.stdout:0/898: chown da/dc/de/l25 2003412305 1 2024-02-16T22:56:45.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:44 smithi064 ceph-mon[29226]: pgmap v27: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 110 MiB/s rd, 226 MiB/s wr, 566 op/s 2024-02-16T22:56:45.119 INFO:tasks.workunit.client.0.smithi064.stdout:8/979: write df/d31/dc1/f110 [448788,73546] 0 2024-02-16T22:56:45.145 INFO:tasks.workunit.client.0.smithi064.stdout:1/942: creat d2/df/d11/d1a/d30/dbd/d91/d56/d5b/f13e x:0 0 0 2024-02-16T22:56:45.146 INFO:tasks.workunit.client.0.smithi064.stdout:5/962: truncate d6/d25/df1/d91/da5/de3/d43/d7d/de2/fe7 1880681 0 2024-02-16T22:56:45.149 INFO:tasks.workunit.client.0.smithi064.stdout:0/899: fdatasync da/dc/de/d12/d2c/f6e 0 2024-02-16T22:56:45.150 INFO:tasks.workunit.client.0.smithi064.stdout:8/980: mkdir df/d31/d8f/dc9/d73/d157 0 2024-02-16T22:56:45.151 INFO:tasks.workunit.client.0.smithi064.stdout:8/981: chown df/f1b 33330374 1 2024-02-16T22:56:45.159 INFO:tasks.workunit.client.0.smithi064.stdout:0/900: mknod da/dc/de/d37/daa/de9/d114/c12a 0 2024-02-16T22:56:45.163 INFO:tasks.workunit.client.0.smithi064.stdout:5/963: rename d6/ld to d6/d18/l148 0 2024-02-16T22:56:45.226 INFO:tasks.workunit.client.0.smithi064.stdout:8/982: write df/d31/d2f/d3e/f9c [2148153,61277] 0 2024-02-16T22:56:45.231 INFO:tasks.workunit.client.0.smithi064.stdout:1/943: dwrite d2/df/d11/d25/f85 [0,4194304] 0 2024-02-16T22:56:45.234 INFO:tasks.workunit.client.0.smithi064.stdout:0/901: truncate f9 1228861 0 2024-02-16T22:56:45.239 INFO:tasks.workunit.client.0.smithi064.stdout:1/944: link d2/df/d11/d1a/d30/dbd/d91/f64 d2/df/d11/d1a/d30/dbd/d91/d56/f13f 0 2024-02-16T22:56:45.240 INFO:tasks.workunit.client.0.smithi064.stdout:5/964: truncate d6/d25/df1/d80/fe0 2268659 0 2024-02-16T22:56:45.242 INFO:tasks.workunit.client.0.smithi064.stdout:1/945: dread d2/df/f12e [0,4194304] 0 2024-02-16T22:56:45.244 INFO:tasks.workunit.client.0.smithi064.stdout:5/965: unlink d6/d25/da4/dc1/dd4/f109 0 2024-02-16T22:56:45.246 INFO:tasks.workunit.client.0.smithi064.stdout:1/946: creat d2/df/d11/dba/f140 x:0 0 0 2024-02-16T22:56:45.247 INFO:tasks.workunit.client.0.smithi064.stdout:5/966: rmdir d6/d25/df1/d80 39 2024-02-16T22:56:45.250 INFO:tasks.workunit.client.0.smithi064.stdout:5/967: mknod d6/d25/df1/c149 0 2024-02-16T22:56:45.253 INFO:tasks.workunit.client.0.smithi064.stdout:1/947: link d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d6b/d75/f107 d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d84/f141 0 2024-02-16T22:56:45.259 INFO:tasks.workunit.client.0.smithi064.stdout:5/968: symlink d6/d25/df1/l14a 0 2024-02-16T22:56:45.262 INFO:tasks.workunit.client.0.smithi064.stdout:1/948: rename d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/de3/de6 to d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/dc6/d142 0 2024-02-16T22:56:45.263 INFO:tasks.workunit.client.0.smithi064.stdout:1/949: fsync d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/f28 0 2024-02-16T22:56:45.265 INFO:tasks.workunit.client.0.smithi064.stdout:5/969: creat d6/d18/d19/d112/d12a/f14b x:0 0 0 2024-02-16T22:56:45.267 INFO:tasks.workunit.client.0.smithi064.stdout:1/950: readlink d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/l7d 0 2024-02-16T22:56:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:5/970: fdatasync d6/d25/df1/d80/fdc 0 2024-02-16T22:56:45.284 INFO:tasks.workunit.client.0.smithi064.stdout:5/971: creat d6/d25/da4/dc1/f14c x:0 0 0 2024-02-16T22:56:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:44 smithi192 ceph-mon[36254]: pgmap v27: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 110 MiB/s rd, 226 MiB/s wr, 566 op/s 2024-02-16T22:56:45.285 INFO:tasks.workunit.client.0.smithi064.stdout:5/972: chown d6/d18/df8 3630 1 2024-02-16T22:56:45.288 INFO:tasks.workunit.client.0.smithi064.stdout:5/973: creat d6/d18/d95/dbe/f14d x:0 0 0 2024-02-16T22:56:45.288 INFO:tasks.workunit.client.0.smithi064.stdout:5/974: chown d6/d18/d19/l57 96208 1 2024-02-16T22:56:45.290 INFO:tasks.workunit.client.0.smithi064.stdout:1/951: link d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/f88 d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2a/f143 0 2024-02-16T22:56:45.292 INFO:tasks.workunit.client.0.smithi064.stdout:5/975: creat d6/d18/dc5/f14e x:0 0 0 2024-02-16T22:56:45.296 INFO:tasks.workunit.client.0.smithi064.stdout:5/976: link d6/d25/d48/d124/f12d d6/d25/df1/d91/da5/de3/d43/f14f 0 2024-02-16T22:56:45.299 INFO:tasks.workunit.client.0.smithi064.stdout:5/977: unlink d6/d25/df1/d91/da5/de3/d4b/f11f 0 2024-02-16T22:56:45.299 INFO:tasks.workunit.client.0.smithi064.stdout:5/978: stat d6/d25/df1/d80/f81 0 2024-02-16T22:56:45.306 INFO:tasks.workunit.client.0.smithi064.stdout:5/979: rename d6/d18/d19/d112/d12a/c134 to d6/d25/df1/c150 0 2024-02-16T22:56:45.307 INFO:tasks.workunit.client.0.smithi064.stdout:5/980: fdatasync d6/d25/df1/d91/da5/de3/f10c 0 2024-02-16T22:56:45.309 INFO:tasks.workunit.client.0.smithi064.stdout:5/981: symlink d6/d25/df1/d91/da5/de3/d43/db9/d115/l151 0 2024-02-16T22:56:45.310 INFO:tasks.workunit.client.0.smithi064.stdout:5/982: truncate d6/f13 1266753 0 2024-02-16T22:56:45.312 INFO:tasks.workunit.client.0.smithi064.stdout:5/983: rmdir d6/d25/da4/dc1/d85 39 2024-02-16T22:56:45.316 INFO:tasks.workunit.client.0.smithi064.stdout:5/984: creat d6/d25/df1/d80/d11a/f152 x:0 0 0 2024-02-16T22:56:45.318 INFO:tasks.workunit.client.0.smithi064.stdout:5/985: mknod d6/d18/d19/d112/c153 0 2024-02-16T22:56:45.318 INFO:tasks.workunit.client.0.smithi064.stdout:5/986: readlink d6/d25/df1/d91/l9a 0 2024-02-16T22:56:45.330 INFO:tasks.workunit.client.0.smithi064.stdout:5/987: write d6/d25/df1/d91/da5/fbd [433693,74757] 0 2024-02-16T22:56:45.332 INFO:tasks.workunit.client.0.smithi064.stdout:5/988: mkdir d6/d25/df1/d80/d154 0 2024-02-16T22:56:45.350 INFO:tasks.workunit.client.0.smithi064.stdout:5/989: dwrite f4 [0,4194304] 0 2024-02-16T22:56:45.351 INFO:tasks.workunit.client.0.smithi064.stdout:5/990: chown d6/d25/df1/d91/da5/de3/d43/db9 31 1 2024-02-16T22:56:45.358 INFO:tasks.workunit.client.0.smithi064.stdout:5/991: truncate d6/d18/d19/f26 4654305 0 2024-02-16T22:56:45.361 INFO:tasks.workunit.client.0.smithi064.stdout:5/992: symlink d6/d25/d48/d124/l155 0 2024-02-16T22:56:45.364 INFO:tasks.workunit.client.0.smithi064.stdout:5/993: getdents d6/d25/df1/d91/da5 0 2024-02-16T22:56:45.375 INFO:tasks.workunit.client.0.smithi064.stdout:5/994: dwrite d6/d25/df1/d91/da5/de3/d43/db9/f145 [0,4194304] 0 2024-02-16T22:56:45.382 INFO:tasks.workunit.client.0.smithi064.stdout:5/995: dwrite d6/d25/da4/fe9 [0,4194304] 0 2024-02-16T22:56:45.387 INFO:tasks.workunit.client.0.smithi064.stdout:5/996: creat d6/d18/d19/d112/f156 x:0 0 0 2024-02-16T22:56:45.390 INFO:tasks.workunit.client.0.smithi064.stdout:5/997: mknod d6/d18/d19/d112/c157 0 2024-02-16T22:56:45.394 INFO:tasks.workunit.client.0.smithi064.stdout:5/998: rename d6/d18/f137 to d6/d25/df1/f158 0 2024-02-16T22:56:45.395 INFO:tasks.workunit.client.0.smithi064.stdout:5/999: chown d6/d25/df1/d91/da5/de3/d4b/cd0 13734562 1 2024-02-16T22:56:45.829 INFO:tasks.workunit.client.0.smithi064.stdout:8/983: sync 2024-02-16T22:56:45.836 INFO:tasks.workunit.client.0.smithi064.stdout:8/984: rename df/d31/d47/f6b to df/d31/d61/d79/f158 0 2024-02-16T22:56:45.836 INFO:tasks.workunit.client.0.smithi064.stdout:8/985: fsync df/d31/d2f/d4f/fa7 0 2024-02-16T22:56:45.839 INFO:tasks.workunit.client.0.smithi064.stdout:8/986: fdatasync df/d31/d2f/d68/f96 0 2024-02-16T22:56:45.852 INFO:tasks.workunit.client.0.smithi064.stdout:8/987: dread df/d31/d8f/d80/fe5 [0,4194304] 0 2024-02-16T22:56:45.859 INFO:tasks.workunit.client.0.smithi064.stdout:8/988: link df/d31/d2f/d38/f14e df/d31/d8f/dab/f159 0 2024-02-16T22:56:45.864 INFO:tasks.workunit.client.0.smithi064.stdout:8/989: dwrite df/d31/d2f/d4f/f66 [0,4194304] 0 2024-02-16T22:56:45.869 INFO:tasks.workunit.client.0.smithi064.stdout:0/902: sync 2024-02-16T22:56:45.869 INFO:tasks.workunit.client.0.smithi064.stdout:1/952: sync 2024-02-16T22:56:45.870 INFO:tasks.workunit.client.0.smithi064.stdout:1/953: chown d2/f10a 3158009 1 2024-02-16T22:56:45.875 INFO:tasks.workunit.client.0.smithi064.stdout:1/954: truncate d2/df/d11/d1a/d30/dbd/d91/d56/d5b/fce 1680177 0 2024-02-16T22:56:45.877 INFO:tasks.workunit.client.0.smithi064.stdout:0/903: dwrite da/dc/de/d12/d1d/d100/f108 [0,4194304] 0 2024-02-16T22:56:45.883 INFO:tasks.workunit.client.0.smithi064.stdout:1/955: mkdir d2/de8/dfa/d144 0 2024-02-16T22:56:45.885 INFO:tasks.workunit.client.0.smithi064.stdout:1/956: fsync d2/df/f13 0 2024-02-16T22:56:45.885 INFO:tasks.workunit.client.0.smithi064.stdout:0/904: dwrite da/dc/f127 [0,4194304] 0 2024-02-16T22:56:45.887 INFO:tasks.workunit.client.0.smithi064.stdout:8/990: link df/d31/d2f/d3e/f77 df/d31/d2f/d4f/d12d/f15a 0 2024-02-16T22:56:45.901 INFO:tasks.workunit.client.0.smithi064.stdout:0/905: fsync da/dc/de/f76 0 2024-02-16T22:56:45.907 INFO:tasks.workunit.client.0.smithi064.stdout:0/906: creat da/dc/de/d12/d2c/d2e/d62/db6/f12b x:0 0 0 2024-02-16T22:56:45.907 INFO:tasks.workunit.client.0.smithi064.stdout:1/957: mkdir d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d105/d145 0 2024-02-16T22:56:45.909 INFO:tasks.workunit.client.0.smithi064.stdout:1/958: write d2/df/d11/fd2 [1866529,71629] 0 2024-02-16T22:56:45.909 INFO:tasks.workunit.client.0.smithi064.stdout:0/907: mkdir da/dc/d3b/d72/d12c 0 2024-02-16T22:56:45.911 INFO:tasks.workunit.client.0.smithi064.stdout:0/908: dread - da/dc/de/d12/d2c/d2e/d62/db6/f12b zero size 2024-02-16T22:56:45.914 INFO:tasks.workunit.client.0.smithi064.stdout:1/959: rename d2/df/d11/dba/ceb to d2/df/d11/d1a/d30/dbd/d91/d56/d5b/c146 0 2024-02-16T22:56:45.915 INFO:tasks.workunit.client.0.smithi064.stdout:0/909: mkdir da/dc/d3b/dd9/d12d 0 2024-02-16T22:56:45.916 INFO:tasks.workunit.client.0.smithi064.stdout:1/960: chown d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/dc6/d142 1149353 1 2024-02-16T22:56:45.923 INFO:tasks.workunit.client.0.smithi064.stdout:0/910: rmdir da/dc/d3b/d5a/d122 0 2024-02-16T22:56:45.929 INFO:tasks.workunit.client.0.smithi064.stdout:0/911: link da/dc/d3b/d5a/d8f/fbb da/dc/d3b/d72/f12e 0 2024-02-16T22:56:45.929 INFO:tasks.workunit.client.0.smithi064.stdout:0/912: chown da/d26/f32 62000029 1 2024-02-16T22:56:45.937 INFO:tasks.workunit.client.0.smithi064.stdout:1/961: dwrite d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/fd1 [0,4194304] 0 2024-02-16T22:56:45.951 INFO:tasks.workunit.client.0.smithi064.stdout:0/913: dwrite da/dc/d3b/f8e [0,4194304] 0 2024-02-16T22:56:45.953 INFO:tasks.workunit.client.0.smithi064.stdout:0/914: dread da/dc/de/f10c [0,4194304] 0 2024-02-16T22:56:45.960 INFO:tasks.workunit.client.0.smithi064.stdout:0/915: dwrite da/f1b [0,4194304] 0 2024-02-16T22:56:45.962 INFO:tasks.workunit.client.0.smithi064.stdout:1/962: dwrite d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/f16 [0,4194304] 0 2024-02-16T22:56:45.963 INFO:tasks.workunit.client.0.smithi064.stdout:0/916: write da/dc/d3b/d72/db3/dfc/fff [837993,65517] 0 2024-02-16T22:56:45.964 INFO:tasks.workunit.client.0.smithi064.stdout:0/917: stat da/d26/de0/df4 0 2024-02-16T22:56:45.970 INFO:tasks.workunit.client.0.smithi064.stdout:1/963: creat d2/df/d11/df7/f147 x:0 0 0 2024-02-16T22:56:45.972 INFO:tasks.workunit.client.0.smithi064.stdout:0/918: symlink da/dc/d3b/d72/db3/l12f 0 2024-02-16T22:56:45.974 INFO:tasks.workunit.client.0.smithi064.stdout:1/964: unlink d2/df/l94 0 2024-02-16T22:56:45.975 INFO:tasks.workunit.client.0.smithi064.stdout:0/919: symlink da/dc/de/d37/daa/dcd/l130 0 2024-02-16T22:56:45.977 INFO:tasks.workunit.client.0.smithi064.stdout:1/965: unlink d2/f36 0 2024-02-16T22:56:45.981 INFO:tasks.workunit.client.0.smithi064.stdout:1/966: creat d2/df/d11/d1a/d30/dbd/d91/f148 x:0 0 0 2024-02-16T22:56:45.983 INFO:tasks.workunit.client.0.smithi064.stdout:0/920: dwrite da/dc/d3b/d5a/f73 [0,4194304] 0 2024-02-16T22:56:45.987 INFO:tasks.workunit.client.0.smithi064.stdout:0/921: creat da/d26/de0/d93/f131 x:0 0 0 2024-02-16T22:56:45.988 INFO:tasks.workunit.client.0.smithi064.stdout:0/922: stat da/dc/de/d12/f11f 0 2024-02-16T22:56:45.993 INFO:tasks.workunit.client.0.smithi064.stdout:0/923: unlink da/dc/de/d37/fa6 0 2024-02-16T22:56:46.009 INFO:tasks.workunit.client.0.smithi064.stdout:0/924: write da/dc/d3b/d5a/d8f/fbb [1089656,5674] 0 2024-02-16T22:56:46.012 INFO:tasks.workunit.client.0.smithi064.stdout:0/925: truncate f3 969554 0 2024-02-16T22:56:46.014 INFO:tasks.workunit.client.0.smithi064.stdout:0/926: creat da/dc/d3b/d72/db3/f132 x:0 0 0 2024-02-16T22:56:46.015 INFO:tasks.workunit.client.0.smithi064.stdout:0/927: stat da/dc/de/d37/daa/d106/l128 0 2024-02-16T22:56:46.018 INFO:tasks.workunit.client.0.smithi064.stdout:0/928: symlink da/dc/de/d12/d2c/d2e/d38/dbe/dc6/l133 0 2024-02-16T22:56:46.019 INFO:tasks.workunit.client.0.smithi064.stdout:0/929: chown da/dc/de/d12/f29 26852 1 2024-02-16T22:56:46.029 INFO:tasks.workunit.client.0.smithi064.stdout:0/930: write da/dc/d3b/ff0 [753963,88171] 0 2024-02-16T22:56:46.042 INFO:tasks.workunit.client.0.smithi064.stdout:0/931: dwrite da/dc/de/d12/d1d/f34 [0,4194304] 0 2024-02-16T22:56:46.045 INFO:tasks.workunit.client.0.smithi064.stdout:0/932: creat da/dc/de/d37/daa/de9/f134 x:0 0 0 2024-02-16T22:56:46.296 INFO:tasks.workunit.client.0.smithi064.stdout:8/991: sync 2024-02-16T22:56:46.300 INFO:tasks.workunit.client.0.smithi064.stdout:8/992: link df/d31/d2f/f7d df/d31/d2f/d3e/ddf/dea/f15b 0 2024-02-16T22:56:46.302 INFO:tasks.workunit.client.0.smithi064.stdout:8/993: chown df/d31/d8f/dc9/d73/f131 0 1 2024-02-16T22:56:46.309 INFO:tasks.workunit.client.0.smithi064.stdout:8/994: write f3 [156493,110737] 0 2024-02-16T22:56:46.313 INFO:tasks.workunit.client.0.smithi064.stdout:8/995: unlink df/d31/d2f/d3e/c104 0 2024-02-16T22:56:46.325 INFO:tasks.workunit.client.0.smithi064.stdout:8/996: dwrite df/d31/d8f/dc9/d73/dfb/f10f [0,4194304] 0 2024-02-16T22:56:46.328 INFO:tasks.workunit.client.0.smithi064.stdout:8/997: mkdir df/d31/d2f/d4f/d15c 0 2024-02-16T22:56:46.330 INFO:tasks.workunit.client.0.smithi064.stdout:8/998: fsync df/d31/d2f/d3e/d5f/f141 0 2024-02-16T22:56:46.332 INFO:tasks.workunit.client.0.smithi064.stdout:8/999: rmdir df/d31/d2f/d38/d48/da2/d11b 39 2024-02-16T22:56:46.354 INFO:tasks.workunit.client.0.smithi064.stdout:0/933: sync 2024-02-16T22:56:46.354 INFO:tasks.workunit.client.0.smithi064.stdout:1/967: sync 2024-02-16T22:56:46.357 INFO:tasks.workunit.client.0.smithi064.stdout:0/934: mkdir da/d71/d135 0 2024-02-16T22:56:46.360 INFO:tasks.workunit.client.0.smithi064.stdout:0/935: mknod da/d26/d103/c136 0 2024-02-16T22:56:46.362 INFO:tasks.workunit.client.0.smithi064.stdout:0/936: chown c7 3126651 1 2024-02-16T22:56:46.366 INFO:tasks.workunit.client.0.smithi064.stdout:0/937: rename da/dc/de/d12/d2c/d2e/d38/c74 to da/dc/de/d37/d88/c137 0 2024-02-16T22:56:46.368 INFO:tasks.workunit.client.0.smithi064.stdout:0/938: creat da/d71/d135/f138 x:0 0 0 2024-02-16T22:56:46.373 INFO:tasks.workunit.client.0.smithi064.stdout:0/939: link da/dc/de/d37/f4e da/dc/de/d12/d2c/d2e/d38/d87/f139 0 2024-02-16T22:56:46.373 INFO:tasks.workunit.client.0.smithi064.stdout:0/940: dread - da/dc/d3b/d72/db3/f124 zero size 2024-02-16T22:56:46.390 INFO:tasks.workunit.client.0.smithi064.stdout:0/941: dread da/dc/de/d12/d1d/f2b [0,4194304] 0 2024-02-16T22:56:46.402 INFO:tasks.workunit.client.0.smithi064.stdout:0/942: dwrite da/d26/de0/d93/fb5 [0,4194304] 0 2024-02-16T22:56:46.404 INFO:tasks.workunit.client.0.smithi064.stdout:0/943: write da/dc/de/fd6 [4204127,82958] 0 2024-02-16T22:56:46.417 INFO:tasks.workunit.client.0.smithi064.stdout:0/944: write da/dc/de/d12/d2c/d2e/d38/d87/f139 [2977888,65887] 0 2024-02-16T22:56:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:0/945: rmdir da 39 2024-02-16T22:56:46.430 INFO:tasks.workunit.client.0.smithi064.stdout:0/946: mknod da/dc/de/d12/d1d/d100/d125/c13a 0 2024-02-16T22:56:46.431 INFO:tasks.workunit.client.0.smithi064.stdout:0/947: fsync da/dc/de/d12/d2c/f6e 0 2024-02-16T22:56:46.442 INFO:tasks.workunit.client.0.smithi064.stdout:0/948: dwrite da/dc/de/d12/f29 [0,4194304] 0 2024-02-16T22:56:46.444 INFO:tasks.workunit.client.0.smithi064.stdout:0/949: dread - da/dc/de/d12/d2c/d2e/d38/f80 zero size 2024-02-16T22:56:46.447 INFO:tasks.workunit.client.0.smithi064.stdout:0/950: creat da/dc/de/d12/d2c/f13b x:0 0 0 2024-02-16T22:56:46.448 INFO:tasks.workunit.client.0.smithi064.stdout:0/951: rmdir da/dc/de/d37 39 2024-02-16T22:56:46.451 INFO:tasks.workunit.client.0.smithi064.stdout:0/952: symlink da/dc/d3b/dd9/d12d/l13c 0 2024-02-16T22:56:46.451 INFO:tasks.workunit.client.0.smithi064.stdout:0/953: fdatasync da/d26/f33 0 2024-02-16T22:56:46.464 INFO:tasks.workunit.client.0.smithi064.stdout:0/954: dwrite da/dc/de/d12/f42 [0,4194304] 0 2024-02-16T22:56:46.465 INFO:tasks.workunit.client.0.smithi064.stdout:0/955: dread - da/dc/d3b/f115 zero size 2024-02-16T22:56:46.473 INFO:tasks.workunit.client.0.smithi064.stdout:0/956: write da/dc/de/d37/d88/f9b [493211,84700] 0 2024-02-16T22:56:46.479 INFO:tasks.workunit.client.0.smithi064.stdout:0/957: write da/dc/de/f10c [417502,62434] 0 2024-02-16T22:56:46.482 INFO:tasks.workunit.client.0.smithi064.stdout:0/958: creat da/dc/de/d12/d2c/d2e/d38/dbe/dc6/f13d x:0 0 0 2024-02-16T22:56:46.490 INFO:tasks.workunit.client.0.smithi064.stdout:0/959: write da/dc/de/d12/d2c/d2e/d38/d87/fcc [356013,62887] 0 2024-02-16T22:56:46.491 INFO:tasks.workunit.client.0.smithi064.stdout:0/960: chown da/dc/de/d12/d2c/d2e/d38/dbe 47857363 1 2024-02-16T22:56:46.495 INFO:tasks.workunit.client.0.smithi064.stdout:0/961: mkdir da/dc/de/d12/d2c/d2e/d38/dbe/dc6/d13e 0 2024-02-16T22:56:46.630 INFO:tasks.workunit.client.0.smithi064.stdout:1/968: sync 2024-02-16T22:56:46.632 INFO:tasks.workunit.client.0.smithi064.stdout:1/969: symlink d2/de8/d109/l149 0 2024-02-16T22:56:46.637 INFO:tasks.workunit.client.0.smithi064.stdout:1/970: dread d2/df/d11/d1a/d30/dbd/d91/f65 [0,4194304] 0 2024-02-16T22:56:46.648 INFO:tasks.workunit.client.0.smithi064.stdout:1/971: dwrite d2/df/d11/d1a/d30/dbd/f101 [0,4194304] 0 2024-02-16T22:56:46.653 INFO:tasks.workunit.client.0.smithi064.stdout:1/972: truncate d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d40/fa0 972900 0 2024-02-16T22:56:46.657 INFO:tasks.workunit.client.0.smithi064.stdout:1/973: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/f112 [0,4194304] 0 2024-02-16T22:56:46.660 INFO:tasks.workunit.client.0.smithi064.stdout:1/974: rename d2/df/d11/dba to d2/df/d11/d1a/d1d/d14a 0 2024-02-16T22:56:46.663 INFO:tasks.workunit.client.0.smithi064.stdout:0/962: sync 2024-02-16T22:56:46.665 INFO:tasks.workunit.client.0.smithi064.stdout:1/975: mkdir d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d6b/d75/d106/d132/d14b 0 2024-02-16T22:56:46.665 INFO:tasks.workunit.client.0.smithi064.stdout:0/963: rmdir da/d71 39 2024-02-16T22:56:46.668 INFO:tasks.workunit.client.0.smithi064.stdout:0/964: dread da/dc/de/f10c [0,4194304] 0 2024-02-16T22:56:46.673 INFO:tasks.workunit.client.0.smithi064.stdout:0/965: dread da/dc/f127 [0,4194304] 0 2024-02-16T22:56:46.680 INFO:tasks.workunit.client.0.smithi064.stdout:0/966: dwrite da/dc/d3b/d72/db3/dfc/f120 [0,4194304] 0 2024-02-16T22:56:46.685 INFO:tasks.workunit.client.0.smithi064.stdout:0/967: symlink da/d26/d3f/l13f 0 2024-02-16T22:56:46.685 INFO:tasks.workunit.client.0.smithi064.stdout:1/976: dwrite d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d84/f9c [0,4194304] 0 2024-02-16T22:56:46.690 INFO:tasks.workunit.client.0.smithi064.stdout:0/968: creat da/dc/d3b/dc2/f140 x:0 0 0 2024-02-16T22:56:46.695 INFO:tasks.workunit.client.0.smithi064.stdout:1/977: rename d2/df/d11/d1a/d1d/fdb to d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/d117/d125/f14c 0 2024-02-16T22:56:46.700 INFO:tasks.workunit.client.0.smithi064.stdout:0/969: mknod da/dc/d3b/d72/d12c/c141 0 2024-02-16T22:56:46.703 INFO:tasks.workunit.client.0.smithi064.stdout:1/978: dwrite d2/df/d11/d1a/f122 [0,4194304] 0 2024-02-16T22:56:46.704 INFO:tasks.workunit.client.0.smithi064.stdout:1/979: dread - d2/de8/d109/f131 zero size 2024-02-16T22:56:46.707 INFO:tasks.workunit.client.0.smithi064.stdout:0/970: link da/dc/d3b/dc2/c126 da/d71/da4/c142 0 2024-02-16T22:56:46.709 INFO:tasks.workunit.client.0.smithi064.stdout:0/971: write da/dc/d3b/d5a/f73 [8298330,114885] 0 2024-02-16T22:56:46.717 INFO:tasks.workunit.client.0.smithi064.stdout:1/980: getdents d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d6b/d75/d106/d132 0 2024-02-16T22:56:46.718 INFO:tasks.workunit.client.0.smithi064.stdout:1/981: stat d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d40/lad 0 2024-02-16T22:56:46.720 INFO:tasks.workunit.client.0.smithi064.stdout:0/972: getdents da 0 2024-02-16T22:56:46.721 INFO:tasks.workunit.client.0.smithi064.stdout:0/973: fdatasync da/dc/d3b/dc2/f140 0 2024-02-16T22:56:46.722 INFO:tasks.workunit.client.0.smithi064.stdout:0/974: chown da/d26/de0/d9e/f10e 353 1 2024-02-16T22:56:46.726 INFO:tasks.workunit.client.0.smithi064.stdout:0/975: symlink da/dc/de/d12/l143 0 2024-02-16T22:56:46.726 INFO:tasks.workunit.client.0.smithi064.stdout:1/982: truncate d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d6b/d75/f77 308919 0 2024-02-16T22:56:46.729 INFO:tasks.workunit.client.0.smithi064.stdout:0/976: symlink da/dc/d3b/d5a/d8f/da8/db7/l144 0 2024-02-16T22:56:46.742 INFO:tasks.workunit.client.0.smithi064.stdout:0/977: write da/dc/f4b [669949,96137] 0 2024-02-16T22:56:46.742 INFO:tasks.workunit.client.0.smithi064.stdout:1/983: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d84/f9a [0,4194304] 0 2024-02-16T22:56:46.744 INFO:tasks.workunit.client.0.smithi064.stdout:1/984: write d2/df/d11/d1a/d1d/f129 [588017,6767] 0 2024-02-16T22:56:46.748 INFO:tasks.workunit.client.0.smithi064.stdout:0/978: rename da/d26/d3f/f75 to da/dc/d3b/f145 0 2024-02-16T22:56:46.751 INFO:tasks.workunit.client.0.smithi064.stdout:1/985: link d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/fd4 d2/de8/dfa/d144/f14d 0 2024-02-16T22:56:46.756 INFO:tasks.workunit.client.0.smithi064.stdout:1/986: truncate d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d84/f113 843867 0 2024-02-16T22:56:46.769 INFO:tasks.workunit.client.0.smithi064.stdout:1/987: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/ddf/fef [0,4194304] 0 2024-02-16T22:56:46.774 INFO:tasks.workunit.client.0.smithi064.stdout:1/988: write d2/df/d11/d25/fe5 [645840,112987] 0 2024-02-16T22:56:46.783 INFO:tasks.workunit.client.0.smithi064.stdout:1/989: dwrite d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/d117/d125/f14c [4194304,4194304] 0 2024-02-16T22:56:46.790 INFO:tasks.workunit.client.0.smithi064.stdout:1/990: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/db5/d117/d125/f14c [4194304,4194304] 0 2024-02-16T22:56:46.793 INFO:tasks.workunit.client.0.smithi064.stdout:1/991: write d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/f112 [476283,38437] 0 2024-02-16T22:56:46.807 INFO:tasks.workunit.client.0.smithi064.stdout:1/992: dwrite d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d84/f9a [0,4194304] 0 2024-02-16T22:56:46.813 INFO:tasks.workunit.client.0.smithi064.stdout:1/993: mkdir d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d18/d48/d6b/d75/d106/d132/d14e 0 2024-02-16T22:56:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:1/994: stat d2/df/d11/d1a/d30 0 2024-02-16T22:56:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:1/995: chown d2/f5c 113471 1 2024-02-16T22:56:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:1/996: chown d2/df/d11/d25/fb9 3 1 2024-02-16T22:56:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:1/997: rename d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/d2c/fe0 to d2/df/d11/d1a/d1d/d14a/f14f 0 2024-02-16T22:56:46.825 INFO:tasks.workunit.client.0.smithi064.stdout:1/998: dread d2/df/d11/d1a/d30/dbd/d91/d56/d5b/d12d/f88 [0,4194304] 0 2024-02-16T22:56:47.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:46 smithi064 ceph-mon[29226]: pgmap v28: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 74 MiB/s rd, 152 MiB/s wr, 376 op/s 2024-02-16T22:56:47.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:46 smithi192 ceph-mon[36254]: pgmap v28: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 74 MiB/s rd, 152 MiB/s wr, 376 op/s 2024-02-16T22:56:47.263 INFO:tasks.workunit.client.0.smithi064.stdout:0/979: sync 2024-02-16T22:56:47.263 INFO:tasks.workunit.client.0.smithi064.stdout:1/999: sync 2024-02-16T22:56:47.266 INFO:tasks.workunit.client.0.smithi064.stdout:0/980: rmdir da/d26/de0/d9e 39 2024-02-16T22:56:47.267 INFO:tasks.workunit.client.0.smithi064.stdout:0/981: write da/dc/de/fc8 [185759,75768] 0 2024-02-16T22:56:47.270 INFO:tasks.workunit.client.0.smithi064.stdout:0/982: creat da/dc/d3b/d72/d12c/f146 x:0 0 0 2024-02-16T22:56:47.282 INFO:tasks.workunit.client.0.smithi064.stdout:0/983: dwrite da/dc/d3b/f3c [0,4194304] 0 2024-02-16T22:56:47.290 INFO:tasks.workunit.client.0.smithi064.stdout:0/984: truncate da/dc/d3b/d72/f99 9132191 0 2024-02-16T22:56:47.309 INFO:tasks.workunit.client.0.smithi064.stdout:0/985: dread da/dc/de/d12/d2c/d2e/d38/d87/f139 [0,4194304] 0 2024-02-16T22:56:47.316 INFO:tasks.workunit.client.0.smithi064.stdout:0/986: rename da/dc/de/d12/d2c/d2e/d38/d87/f139 to da/dc/de/d37/daa/dcd/f147 0 2024-02-16T22:56:47.322 INFO:tasks.workunit.client.0.smithi064.stdout:0/987: dwrite da/dc/de/d12/d2c/d2e/d96/f123 [0,4194304] 0 2024-02-16T22:56:47.328 INFO:tasks.workunit.client.0.smithi064.stdout:0/988: fsync da/dc/de/d37/daa/db4/fba 0 2024-02-16T22:56:47.336 INFO:tasks.workunit.client.0.smithi064.stdout:0/989: rmdir da/dc/de/d12/d2c/d2e/d62 39 2024-02-16T22:56:47.341 INFO:tasks.workunit.client.0.smithi064.stdout:0/990: dwrite da/dc/de/d12/d1d/f2f [0,4194304] 0 2024-02-16T22:56:47.353 INFO:tasks.workunit.client.0.smithi064.stdout:0/991: symlink da/dc/de/d12/d2c/d2e/d96/l148 0 2024-02-16T22:56:47.354 INFO:tasks.workunit.client.0.smithi064.stdout:0/992: fdatasync da/dc/d3b/d72/f11b 0 2024-02-16T22:56:47.359 INFO:tasks.workunit.client.0.smithi064.stdout:0/993: dwrite da/d26/de0/d93/fb5 [0,4194304] 0 2024-02-16T22:56:47.360 INFO:tasks.workunit.client.0.smithi064.stdout:0/994: chown da/d71/d135 0 1 2024-02-16T22:56:47.364 INFO:tasks.workunit.client.0.smithi064.stdout:0/995: rename da/dc/d3b/d5a/l98 to da/dc/d3b/d5a/d8f/da8/db7/l149 0 2024-02-16T22:56:47.371 INFO:tasks.workunit.client.0.smithi064.stdout:0/996: rename da/dc/de/d12/d2c/d2e/d62 to da/dc/d3b/d5a/d14a 0 2024-02-16T22:56:47.387 INFO:tasks.workunit.client.0.smithi064.stdout:0/997: dread da/dc/de/d12/d2c/f92 [0,4194304] 0 2024-02-16T22:56:47.391 INFO:tasks.workunit.client.0.smithi064.stdout:0/998: mknod da/dc/de/d12/d2c/d2e/d38/dcf/c14b 0 2024-02-16T22:56:47.392 INFO:tasks.workunit.client.0.smithi064.stdout:0/999: chown da/dc/de/d12/d2c/d2e/d38/l52 23187 1 2024-02-16T22:56:47.395 INFO:tasks.workunit.client.0.smithi064.stderr:+ rm -rf -- ./tmp.5LFff9q55c 2024-02-16T22:56:48.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 -- 172.21.15.64:0/744148660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fcfb0 msgr2=0x7fc05c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 --2- 172.21.15.64:0/744148660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fcfb0 0x7fc05c0fd420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc04c004940 tx=0x7fc04c009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 -- 172.21.15.64:0/744148660 shutdown_connections 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 --2- 172.21.15.64:0/744148660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fcfb0 0x7fc05c0fd420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 --2- 172.21.15.64:0/744148660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05c0fbe40 0x7fc05c0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 -- 172.21.15.64:0/744148660 >> 172.21.15.64:0/744148660 conn(0x7fc05c0f7520 msgr2=0x7fc05c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 -- 172.21.15.64:0/744148660 shutdown_connections 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 -- 172.21.15.64:0/744148660 wait complete. 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 Processor -- start 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.594+0000 7fc0636a1700 1 -- start start 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc0636a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fbe40 0x7fc05c1105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc0636a1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05c0fcfb0 0x7fc05c112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:48.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc0636a1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c066cf0 con 0x7fc05c0fbe40 2024-02-16T22:56:48.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc0636a1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c066e30 con 0x7fc05c0fcfb0 2024-02-16T22:56:48.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc06143d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fbe40 0x7fc05c1105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:48.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc06143d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fbe40 0x7fc05c1105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44800/0 (socket says 172.21.15.64:44800) 2024-02-16T22:56:48.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc06143d700 1 -- 172.21.15.64:0/1002166847 learned_addr learned my addr 172.21.15.64:0/1002166847 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:48.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc060c3c700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05c0fcfb0 0x7fc05c112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:48.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc06143d700 1 -- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05c0fcfb0 msgr2=0x7fc05c112b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:48.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc06143d700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05c0fcfb0 0x7fc05c112b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:48.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc06143d700 1 -- 172.21.15.64:0/1002166847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c005040 con 0x7fc05c0fbe40 2024-02-16T22:56:48.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc06143d700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fbe40 0x7fc05c1105c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc05800eb40 tx=0x7fc05800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:48.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc051ffb700 1 -- 172.21.15.64:0/1002166847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc058007cc0 con 0x7fc05c0fbe40 2024-02-16T22:56:48.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.595+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c113110 con 0x7fc05c0fbe40 2024-02-16T22:56:48.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.596+0000 7fc051ffb700 1 -- 172.21.15.64:0/1002166847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc05800f040 con 0x7fc05c0fbe40 2024-02-16T22:56:48.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.596+0000 7fc051ffb700 1 -- 172.21.15.64:0/1002166847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc05800ca10 con 0x7fc05c0fbe40 2024-02-16T22:56:48.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.596+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c113650 con 0x7fc05c0fbe40 2024-02-16T22:56:48.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.597+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c04bed0 con 0x7fc05c0fbe40 2024-02-16T22:56:48.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.598+0000 7fc051ffb700 1 -- 172.21.15.64:0/1002166847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7fc05800cb70 con 0x7fc05c0fbe40 2024-02-16T22:56:48.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.598+0000 7fc051ffb700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fc04806b2e0 0x7fc04806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:48.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.598+0000 7fc051ffb700 1 -- 172.21.15.64:0/1002166847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc0580898d0 con 0x7fc05c0fbe40 2024-02-16T22:56:48.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.598+0000 7fc060c3c700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fc04806b2e0 0x7fc04806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:48.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.599+0000 7fc060c3c700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fc04806b2e0 0x7fc04806d790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc04c013740 tx=0x7fc04c009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:48.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.603+0000 7fc051ffb700 1 -- 172.21.15.64:0/1002166847 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fc05804f620 con 0x7fc05c0fbe40 2024-02-16T22:56:48.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.949+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc05c101990 con 0x7fc04806b2e0 2024-02-16T22:56:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.950+0000 7fc051ffb700 1 -- 172.21.15.64:0/1002166847 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fc05c101990 con 0x7fc04806b2e0 2024-02-16T22:56:48.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fc04806b2e0 msgr2=0x7fc04806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:48.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fc04806b2e0 0x7fc04806d790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc04c013740 tx=0x7fc04c009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:48.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fbe40 msgr2=0x7fc05c1105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:48.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fbe40 0x7fc05c1105c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc05800eb40 tx=0x7fc05800b040 comp rx=0 tx=0).stop 2024-02-16T22:56:48.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 shutdown_connections 2024-02-16T22:56:48.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc05c0fbe40 0x7fc05c1105c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:48.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fc04806b2e0 0x7fc04806d790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:48.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 --2- 172.21.15.64:0/1002166847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05c0fcfb0 0x7fc05c112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:48.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 >> 172.21.15.64:0/1002166847 conn(0x7fc05c0f7520 msgr2=0x7fc05c100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:48.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 shutdown_connections 2024-02-16T22:56:48.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:48.953+0000 7fc0636a1700 1 -- 172.21.15.64:0/1002166847 wait complete. 2024-02-16T22:56:48.965 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:56:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.077+0000 7f2ec7d03700 1 -- 172.21.15.64:0/3094465166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00fbe40 msgr2=0x7f2ec00fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.077+0000 7f2ec7d03700 1 --2- 172.21.15.64:0/3094465166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00fbe40 0x7f2ec00fc250 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc004940 tx=0x7f2ebc009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.078+0000 7f2ec7d03700 1 -- 172.21.15.64:0/3094465166 shutdown_connections 2024-02-16T22:56:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.078+0000 7f2ec7d03700 1 --2- 172.21.15.64:0/3094465166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ec00fcfb0 0x7f2ec00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.078+0000 7f2ec7d03700 1 --2- 172.21.15.64:0/3094465166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00fbe40 0x7f2ec00fc250 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.078+0000 7f2ec7d03700 1 -- 172.21.15.64:0/3094465166 >> 172.21.15.64:0/3094465166 conn(0x7f2ec00f7520 msgr2=0x7f2ec00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.078+0000 7f2ec7d03700 1 -- 172.21.15.64:0/3094465166 shutdown_connections 2024-02-16T22:56:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.078+0000 7f2ec7d03700 1 -- 172.21.15.64:0/3094465166 wait complete. 2024-02-16T22:56:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec7d03700 1 Processor -- start 2024-02-16T22:56:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec7d03700 1 -- start start 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec7d03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00fbe40 0x7f2ec010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec7d03700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ec00fcfb0 0x7f2ec010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec7d03700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec010e3f0 con 0x7f2ec00fbe40 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec7d03700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec010ec90 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec529e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ec00fcfb0 0x7f2ec010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec529e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ec00fcfb0 0x7f2ec010ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:41870/0 (socket says 172.21.15.64:41870) 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec529e700 1 -- 172.21.15.64:0/1523581911 learned_addr learned my addr 172.21.15.64:0/1523581911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec5a9f700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00fbe40 0x7f2ec010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec529e700 1 -- 172.21.15.64:0/1523581911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00fbe40 msgr2=0x7f2ec010d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec529e700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00fbe40 0x7f2ec010d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.079+0000 7f2ec529e700 1 -- 172.21.15.64:0/1523581911 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb0005040 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.080+0000 7f2ec529e700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ec00fcfb0 0x7f2ec010ddf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00096d0 tx=0x7f2eb0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:49.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.080+0000 7f2eb67fc700 1 -- 172.21.15.64:0/1523581911 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb000e840 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.080+0000 7f2eb67fc700 1 -- 172.21.15.64:0/1523581911 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eb000f040 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.080+0000 7f2eb67fc700 1 -- 172.21.15.64:0/1523581911 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb0013e50 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.080+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc005040 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.080+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec010f2d0 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.081+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec004bed0 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.082+0000 7f2eb67fc700 1 -- 172.21.15.64:0/1523581911 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f2eb000e9a0 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.082+0000 7f2eb67fc700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2eac06b270 0x7f2eac06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.082+0000 7f2eb67fc700 1 -- 172.21.15.64:0/1523581911 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f2eb0089cd0 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.082+0000 7f2ec5a9f700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2eac06b270 0x7f2eac06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.083+0000 7f2ec5a9f700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2eac06b270 0x7f2eac06d720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00b0d0 tx=0x7f2ebc009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:49.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.087+0000 7f2eb67fc700 1 -- 172.21.15.64:0/1523581911 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2eb0057740 con 0x7f2ec00fcfb0 2024-02-16T22:56:49.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:48 smithi192 ceph-mon[36254]: pgmap v29: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 82 MiB/s rd, 184 MiB/s wr, 490 op/s 2024-02-16T22:56:49.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:48 smithi064 ceph-mon[29226]: pgmap v29: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 82 MiB/s rd, 184 MiB/s wr, 490 op/s 2024-02-16T22:56:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.429+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ec010f770 con 0x7f2eac06b270 2024-02-16T22:56:49.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.430+0000 7f2eb67fc700 1 -- 172.21.15.64:0/1523581911 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f2ec010f770 con 0x7f2eac06b270 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2eac06b270 msgr2=0x7f2eac06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2eac06b270 0x7f2eac06d720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00b0d0 tx=0x7f2ebc009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ec00fcfb0 msgr2=0x7f2ec010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ec00fcfb0 0x7f2ec010ddf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00096d0 tx=0x7f2eb0009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 shutdown_connections 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00fbe40 0x7f2ec010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2eac06b270 0x7f2eac06d720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 --2- 172.21.15.64:0/1523581911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ec00fcfb0 0x7f2ec010ddf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 >> 172.21.15.64:0/1523581911 conn(0x7f2ec00f7520 msgr2=0x7f2ec0065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 shutdown_connections 2024-02-16T22:56:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.433+0000 7f2ec7d03700 1 -- 172.21.15.64:0/1523581911 wait complete. 2024-02-16T22:56:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.548+0000 7f84fa6a4700 1 -- 172.21.15.64:0/872233479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f40fb220 msgr2=0x7f84f40fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.548+0000 7f84fa6a4700 1 --2- 172.21.15.64:0/872233479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f40fb220 0x7f84f40fb630 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f84dc004940 tx=0x7f84dc009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.548+0000 7f84fa6a4700 1 -- 172.21.15.64:0/872233479 shutdown_connections 2024-02-16T22:56:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.548+0000 7f84fa6a4700 1 --2- 172.21.15.64:0/872233479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84f40fe2b0 0x7f84f40fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.548+0000 7f84fa6a4700 1 --2- 172.21.15.64:0/872233479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f40fb220 0x7f84f40fb630 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.548+0000 7f84fa6a4700 1 -- 172.21.15.64:0/872233479 >> 172.21.15.64:0/872233479 conn(0x7f84f40f6ca0 msgr2=0x7f84f40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.549+0000 7f84fa6a4700 1 -- 172.21.15.64:0/872233479 shutdown_connections 2024-02-16T22:56:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.549+0000 7f84fa6a4700 1 -- 172.21.15.64:0/872233479 wait complete. 2024-02-16T22:56:49.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.549+0000 7f84fa6a4700 1 Processor -- start 2024-02-16T22:56:49.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.549+0000 7f84fa6a4700 1 -- start start 2024-02-16T22:56:49.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84fa6a4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84f40fe2b0 0x7f84f410db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84fa6a4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f410e090 0x7f84f410ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84fa6a4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f410e550 con 0x7f84f410e090 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84fa6a4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f410e690 con 0x7f84f40fe2b0 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f410e090 0x7f84f410ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f410e090 0x7f84f410ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44844/0 (socket says 172.21.15.64:44844) 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f37fe700 1 -- 172.21.15.64:0/3278120898 learned_addr learned my addr 172.21.15.64:0/3278120898 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f3fff700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84f40fe2b0 0x7f84f410db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f37fe700 1 -- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84f40fe2b0 msgr2=0x7f84f410db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f37fe700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84f40fe2b0 0x7f84f410db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f37fe700 1 -- 172.21.15.64:0/3278120898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc005040 con 0x7f84f410e090 2024-02-16T22:56:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f3fff700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84f40fe2b0 0x7f84f410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:56:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.550+0000 7f84f37fe700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f410e090 0x7f84f410ef30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f84e400eb70 tx=0x7f84e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.551+0000 7f84f0ff9700 1 -- 172.21.15.64:0/3278120898 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84e4007cc0 con 0x7f84f410e090 2024-02-16T22:56:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.551+0000 7f84f0ff9700 1 -- 172.21.15.64:0/3278120898 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84e400f040 con 0x7f84f410e090 2024-02-16T22:56:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.551+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f410f530 con 0x7f84f410e090 2024-02-16T22:56:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.551+0000 7f84f0ff9700 1 -- 172.21.15.64:0/3278120898 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84e400ca90 con 0x7f84f410e090 2024-02-16T22:56:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.551+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f410fa40 con 0x7f84f410e090 2024-02-16T22:56:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.552+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f4107aa0 con 0x7f84f410e090 2024-02-16T22:56:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.553+0000 7f84f0ff9700 1 -- 172.21.15.64:0/3278120898 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f84e400cbf0 con 0x7f84f410e090 2024-02-16T22:56:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.553+0000 7f84f0ff9700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f84e006b3a0 0x7f84e006d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.554+0000 7f84f0ff9700 1 -- 172.21.15.64:0/3278120898 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f84e4089810 con 0x7f84f410e090 2024-02-16T22:56:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.554+0000 7f84f3fff700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f84e006b3a0 0x7f84e006d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.554+0000 7f84f3fff700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f84e006b3a0 0x7f84e006d850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84dc000c00 tx=0x7f84dc009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:49.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.561+0000 7f84f0ff9700 1 -- 172.21.15.64:0/3278120898 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f84e4057440 con 0x7f84f410e090 2024-02-16T22:56:49.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.812+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84f4105ba0 con 0x7f84e006b3a0 2024-02-16T22:56:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.823+0000 7f84f0ff9700 1 -- 172.21.15.64:0/3278120898 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f84f4105ba0 con 0x7f84e006b3a0 2024-02-16T22:56:49.824 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:56:49.824 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (7m) 32s ago 8m 25.4M - 0.25.0 c8568f914cd2 829b066b8c26 2024-02-16T22:56:49.824 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (8m) 32s ago 8m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:56:49.824 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (7m) 36s ago 7m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:56:49.824 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (6m) 32s ago 8m 84.6M - 9.4.7 954c08fa6188 529226ab33c5 2024-02-16T22:56:49.824 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (2m) 32s ago 2m 15.6M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:56:49.825 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (3m) 32s ago 3m 213M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:56:49.825 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (2m) 36s ago 2m 14.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:56:49.825 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (3m) 36s ago 2m 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:56:49.825 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:9283 running (9m) 32s ago 9m 359M - 17.2.7-513-gb88fae66 870a0944ccb8 7189c74ed7fc 2024-02-16T22:56:49.825 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (65s) 36s ago 7m 487M - 19.0.0-1450-gf78a58c0 f081f847070a 05118581c81b 2024-02-16T22:56:49.825 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (9m) 32s ago 9m 48.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:56:49.825 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (7m) 36s ago 7m 36.2M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:56:49.825 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 starting - - - - 2024-02-16T22:56:49.826 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (7m) 36s ago 7m 15.4M - 1.5.0 0da6a335fe13 c32798cb2969 2024-02-16T22:56:49.826 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (6m) 32s ago 6m 94.7M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:56:49.826 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (6m) 32s ago 6m 91.5M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:56:49.826 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (5m) 32s ago 5m 84.2M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:56:49.826 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (5m) 36s ago 5m 94.9M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:56:49.826 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (4m) 36s ago 4m 90.5M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:56:49.826 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (4m) 36s ago 4m 89.6M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:56:49.827 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (6m) 32s ago 7m 56.7M - 2.43.0 a07b618ecd1d dd20eff0542c 2024-02-16T22:56:49.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.825+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f84e006b3a0 msgr2=0x7f84e006d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.825+0000 7f84fa6a4700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f84e006b3a0 0x7f84e006d850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84dc000c00 tx=0x7f84dc009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:49.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.825+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f410e090 msgr2=0x7f84f410ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.825+0000 7f84fa6a4700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f410e090 0x7f84f410ef30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f84e400eb70 tx=0x7f84e400b040 comp rx=0 tx=0).stop 2024-02-16T22:56:49.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.826+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 shutdown_connections 2024-02-16T22:56:49.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.826+0000 7f84fa6a4700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f84e006b3a0 0x7f84e006d850 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.826+0000 7f84fa6a4700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84f40fe2b0 0x7f84f410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.826+0000 7f84fa6a4700 1 --2- 172.21.15.64:0/3278120898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f410e090 0x7f84f410ef30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.826+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 >> 172.21.15.64:0/3278120898 conn(0x7f84f40f6ca0 msgr2=0x7f84f40f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:49.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.826+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 shutdown_connections 2024-02-16T22:56:49.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.826+0000 7f84fa6a4700 1 -- 172.21.15.64:0/3278120898 wait complete. 2024-02-16T22:56:49.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.960+0000 7f96d72ed700 1 -- 172.21.15.64:0/1427858310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 msgr2=0x7f96d00fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.960+0000 7f96d72ed700 1 --2- 172.21.15.64:0/1427858310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 0x7f96d00fcc40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f96c0004a50 tx=0x7f96c000af40 comp rx=0 tx=0).stop 2024-02-16T22:56:49.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.960+0000 7f96d72ed700 1 -- 172.21.15.64:0/1427858310 shutdown_connections 2024-02-16T22:56:49.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.960+0000 7f96d72ed700 1 --2- 172.21.15.64:0/1427858310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 0x7f96d00fcc40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.960+0000 7f96d72ed700 1 --2- 172.21.15.64:0/1427858310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d00fb5e0 0x7f96d00fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.960+0000 7f96d72ed700 1 -- 172.21.15.64:0/1427858310 >> 172.21.15.64:0/1427858310 conn(0x7f96d00f6ca0 msgr2=0x7f96d00f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:49.962 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:49 smithi064 ceph-mon[29226]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:49.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.961+0000 7f96d72ed700 1 -- 172.21.15.64:0/1427858310 shutdown_connections 2024-02-16T22:56:49.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.961+0000 7f96d72ed700 1 -- 172.21.15.64:0/1427858310 wait complete. 2024-02-16T22:56:49.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.961+0000 7f96d72ed700 1 Processor -- start 2024-02-16T22:56:49.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.961+0000 7f96d72ed700 1 -- start start 2024-02-16T22:56:49.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d72ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d00fb5e0 0x7f96d010b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d72ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 0x7f96d010bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d72ed700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d010c320 con 0x7f96d00fc7f0 2024-02-16T22:56:49.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d72ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d010cbc0 con 0x7f96d00fb5e0 2024-02-16T22:56:49.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d4888700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 0x7f96d010bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d4888700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 0x7f96d010bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44866/0 (socket says 172.21.15.64:44866) 2024-02-16T22:56:49.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d4888700 1 -- 172.21.15.64:0/1857992095 learned_addr learned my addr 172.21.15.64:0/1857992095 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:49.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d5089700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d00fb5e0 0x7f96d010b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d4888700 1 -- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d00fb5e0 msgr2=0x7f96d010b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:49.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d4888700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d00fb5e0 0x7f96d010b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:49.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d4888700 1 -- 172.21.15.64:0/1857992095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c0005040 con 0x7f96d00fc7f0 2024-02-16T22:56:49.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d5089700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d00fb5e0 0x7f96d010b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:56:49.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.962+0000 7f96d4888700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 0x7f96d010bd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f96c00049a0 tx=0x7f96c000af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:49.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.963+0000 7f96c5ffb700 1 -- 172.21.15.64:0/1857992095 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96c000f040 con 0x7f96d00fc7f0 2024-02-16T22:56:49.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.963+0000 7f96c5ffb700 1 -- 172.21.15.64:0/1857992095 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96c0004290 con 0x7f96d00fc7f0 2024-02-16T22:56:49.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.963+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d010ce40 con 0x7f96d00fc7f0 2024-02-16T22:56:49.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.963+0000 7f96c5ffb700 1 -- 172.21.15.64:0/1857992095 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96c001e410 con 0x7f96d00fc7f0 2024-02-16T22:56:49.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.963+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d010d380 con 0x7f96d00fc7f0 2024-02-16T22:56:49.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.964+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d0105880 con 0x7f96d00fc7f0 2024-02-16T22:56:49.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.965+0000 7f96c5ffb700 1 -- 172.21.15.64:0/1857992095 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f96c000a880 con 0x7f96d00fc7f0 2024-02-16T22:56:49.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.965+0000 7f96c5ffb700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f96bc06b2e0 0x7f96bc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:49.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.966+0000 7f96c5ffb700 1 -- 172.21.15.64:0/1857992095 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f96c008d9b0 con 0x7f96d00fc7f0 2024-02-16T22:56:49.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.966+0000 7f96d5089700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f96bc06b2e0 0x7f96bc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:49.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.966+0000 7f96d5089700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f96bc06b2e0 0x7f96bc06d790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009b80 tx=0x7f96cc00b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:49.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:49.974+0000 7f96c5ffb700 1 -- 172.21.15.64:0/1857992095 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f96c005b4c0 con 0x7f96d00fc7f0 2024-02-16T22:56:50.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:49 smithi192 ceph-mon[36254]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:50.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.412+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f96d004bed0 con 0x7f96d00fc7f0 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.413+0000 7f96c5ffb700 1 -- 172.21.15.64:0/1857992095 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f96c005b050 con 0x7f96d00fc7f0 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:56:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 13, 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:56:50.415 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:56:50.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f96bc06b2e0 msgr2=0x7f96bc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f96bc06b2e0 0x7f96bc06d790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009b80 tx=0x7f96cc00b040 comp rx=0 tx=0).stop 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 msgr2=0x7f96d010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 0x7f96d010bd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f96c00049a0 tx=0x7f96c000af40 comp rx=0 tx=0).stop 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 shutdown_connections 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f96bc06b2e0 0x7f96bc06d790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d00fb5e0 0x7f96d010b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 --2- 172.21.15.64:0/1857992095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96d00fc7f0 0x7f96d010bd20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 >> 172.21.15.64:0/1857992095 conn(0x7f96d00f6ca0 msgr2=0x7f96d00f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 shutdown_connections 2024-02-16T22:56:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.416+0000 7f96d72ed700 1 -- 172.21.15.64:0/1857992095 wait complete. 2024-02-16T22:56:50.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.535+0000 7f6d918af700 1 -- 172.21.15.64:0/872174684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 msgr2=0x7f6d8c0fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:50.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.535+0000 7f6d918af700 1 --2- 172.21.15.64:0/872174684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 0x7f6d8c0fb630 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c004940 tx=0x7f6d7c009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:50.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.535+0000 7f6d918af700 1 -- 172.21.15.64:0/872174684 shutdown_connections 2024-02-16T22:56:50.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.535+0000 7f6d918af700 1 --2- 172.21.15.64:0/872174684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d8c0fe2b0 0x7f6d8c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.535+0000 7f6d918af700 1 --2- 172.21.15.64:0/872174684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 0x7f6d8c0fb630 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.535+0000 7f6d918af700 1 -- 172.21.15.64:0/872174684 >> 172.21.15.64:0/872174684 conn(0x7f6d8c0f6ca0 msgr2=0x7f6d8c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:50.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.535+0000 7f6d918af700 1 -- 172.21.15.64:0/872174684 shutdown_connections 2024-02-16T22:56:50.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.535+0000 7f6d918af700 1 -- 172.21.15.64:0/872174684 wait complete. 2024-02-16T22:56:50.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d918af700 1 Processor -- start 2024-02-16T22:56:50.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d918af700 1 -- start start 2024-02-16T22:56:50.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d918af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 0x7f6d8c10d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:50.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d918af700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d8c0fe2b0 0x7f6d8c10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:50.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d918af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c10e4c0 con 0x7f6d8c0fb220 2024-02-16T22:56:50.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d918af700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c10ed60 con 0x7f6d8c0fe2b0 2024-02-16T22:56:50.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d8affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 0x7f6d8c10d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:50.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d8affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 0x7f6d8c10d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48024/0 (socket says 172.21.15.64:48024) 2024-02-16T22:56:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.536+0000 7f6d8affd700 1 -- 172.21.15.64:0/1023856524 learned_addr learned my addr 172.21.15.64:0/1023856524 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d8a7fc700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d8c0fe2b0 0x7f6d8c10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d8affd700 1 -- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d8c0fe2b0 msgr2=0x7f6d8c10dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d8affd700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d8c0fe2b0 0x7f6d8c10dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d8affd700 1 -- 172.21.15.64:0/1023856524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c005040 con 0x7f6d8c0fb220 2024-02-16T22:56:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d8a7fc700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d8c0fe2b0 0x7f6d8c10dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:56:50.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d8affd700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 0x7f6d8c10d980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009fd0 tx=0x7f6d7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d737fe700 1 -- 172.21.15.64:0/1023856524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7c021070 con 0x7f6d8c0fb220 2024-02-16T22:56:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d737fe700 1 -- 172.21.15.64:0/1023856524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d7c01c400 con 0x7f6d8c0fb220 2024-02-16T22:56:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d737fe700 1 -- 172.21.15.64:0/1023856524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7c00e070 con 0x7f6d8c0fb220 2024-02-16T22:56:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c10efe0 con 0x7f6d8c0fb220 2024-02-16T22:56:50.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.537+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c10f470 con 0x7f6d8c0fb220 2024-02-16T22:56:50.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.539+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c04bed0 con 0x7f6d8c0fb220 2024-02-16T22:56:50.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.539+0000 7f6d737fe700 1 -- 172.21.15.64:0/1023856524 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f6d7c01ca90 con 0x7f6d8c0fb220 2024-02-16T22:56:50.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.540+0000 7f6d737fe700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f6d7406b2e0 0x7f6d7406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.540+0000 7f6d737fe700 1 -- 172.21.15.64:0/1023856524 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f6d7c08aef0 con 0x7f6d8c0fb220 2024-02-16T22:56:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.540+0000 7f6d8a7fc700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f6d7406b2e0 0x7f6d7406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.540+0000 7f6d8a7fc700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f6d7406b2e0 0x7f6d7406d790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009b80 tx=0x7f6d8000b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:50.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.545+0000 7f6d737fe700 1 -- 172.21.15.64:0/1023856524 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f6d7c058a80 con 0x7f6d8c0fb220 2024-02-16T22:56:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.874+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6d8c10f660 con 0x7f6d8c0fb220 2024-02-16T22:56:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.874+0000 7f6d737fe700 1 -- 172.21.15.64:0/1023856524 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f6d7c058610 con 0x7f6d8c0fb220 2024-02-16T22:56:50.875 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:56:50.875 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:56:50.876 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:56:50.877 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:56:50.878 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:56:50.878 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:56:50.878 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:56:50.878 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:56:50.878 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:56:50.878 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:56:50.878 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:56:50.878 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:56:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.876+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f6d7406b2e0 msgr2=0x7f6d7406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.876+0000 7f6d918af700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f6d7406b2e0 0x7f6d7406d790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009b80 tx=0x7f6d8000b040 comp rx=0 tx=0).stop 2024-02-16T22:56:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.876+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 msgr2=0x7f6d8c10d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.876+0000 7f6d918af700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 0x7f6d8c10d980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009fd0 tx=0x7f6d7c009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.877+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 shutdown_connections 2024-02-16T22:56:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.877+0000 7f6d918af700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d8c0fb220 0x7f6d8c10d980 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.877+0000 7f6d918af700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f6d7406b2e0 0x7f6d7406d790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.877+0000 7f6d918af700 1 --2- 172.21.15.64:0/1023856524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d8c0fe2b0 0x7f6d8c10dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.877+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 >> 172.21.15.64:0/1023856524 conn(0x7f6d8c0f6ca0 msgr2=0x7f6d8c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:50.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.877+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 shutdown_connections 2024-02-16T22:56:50.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.877+0000 7f6d918af700 1 -- 172.21.15.64:0/1023856524 wait complete. 2024-02-16T22:56:50.880 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:56:50.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.997+0000 7ff484525700 1 -- 172.21.15.64:0/1783808277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 msgr2=0x7ff47c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:50.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.997+0000 7ff484525700 1 --2- 172.21.15.64:0/1783808277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 0x7ff47c0fe550 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff46c004940 tx=0x7ff46c009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:50.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.997+0000 7ff484525700 1 -- 172.21.15.64:0/1783808277 shutdown_connections 2024-02-16T22:56:50.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.997+0000 7ff484525700 1 --2- 172.21.15.64:0/1783808277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 0x7ff47c0fe550 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.997+0000 7ff484525700 1 --2- 172.21.15.64:0/1783808277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff47c0fcef0 0x7ff47c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:50.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.997+0000 7ff484525700 1 -- 172.21.15.64:0/1783808277 >> 172.21.15.64:0/1783808277 conn(0x7ff47c0f8620 msgr2=0x7ff47c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:50.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.997+0000 7ff484525700 1 -- 172.21.15.64:0/1783808277 shutdown_connections 2024-02-16T22:56:50.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.997+0000 7ff484525700 1 -- 172.21.15.64:0/1783808277 wait complete. 2024-02-16T22:56:50.999 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:50 smithi064 ceph-mon[29226]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:50.999 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:50 smithi064 ceph-mon[29226]: pgmap v30: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 40 MiB/s rd, 88 MiB/s wr, 277 op/s 2024-02-16T22:56:50.999 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:50 smithi064 ceph-mon[29226]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:51.000 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:50 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1857992095' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:56:51.000 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:50 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1023856524' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:56:51.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.998+0000 7ff484525700 1 Processor -- start 2024-02-16T22:56:51.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.998+0000 7ff484525700 1 -- start start 2024-02-16T22:56:51.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.998+0000 7ff484525700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff47c0fcef0 0x7ff47c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:51.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.998+0000 7ff484525700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 0x7ff47c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.998+0000 7ff484525700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0708e0 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.998+0000 7ff484525700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c070a50 con 0x7ff47c0fcef0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.998+0000 7ff481ac0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 0x7ff47c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff4822c1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff47c0fcef0 0x7ff47c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff481ac0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 0x7ff47c0702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48040/0 (socket says 172.21.15.64:48040) 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff481ac0700 1 -- 172.21.15.64:0/3312863431 learned_addr learned my addr 172.21.15.64:0/3312863431 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff481ac0700 1 -- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff47c0fcef0 msgr2=0x7ff47c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff481ac0700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff47c0fcef0 0x7ff47c071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff481ac0700 1 -- 172.21.15.64:0/3312863431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c005040 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff481ac0700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 0x7ff47c0702e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff46c000c00 tx=0x7ff46c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff472ffd700 1 -- 172.21.15.64:0/3312863431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46c00e070 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:50.999+0000 7ff472ffd700 1 -- 172.21.15.64:0/3312863431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff46c01c400 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.000+0000 7ff472ffd700 1 -- 172.21.15.64:0/3312863431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46c00b650 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.000+0000 7ff484525700 1 -- 172.21.15.64:0/3312863431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c070cd0 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.000+0000 7ff484525700 1 -- 172.21.15.64:0/3312863431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c115900 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.001+0000 7ff470ff9700 1 -- 172.21.15.64:0/3312863431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4640052b0 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.002+0000 7ff472ffd700 1 -- 172.21.15.64:0/3312863431 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7ff46c01ca50 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.003+0000 7ff472ffd700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff46806b2e0 0x7ff46806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:51.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.003+0000 7ff472ffd700 1 -- 172.21.15.64:0/3312863431 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff46c08b6d0 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.003+0000 7ff4822c1700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff46806b2e0 0x7ff46806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:51.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.003+0000 7ff4822c1700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff46806b2e0 0x7ff46806d790 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff47c071bf0 tx=0x7ff47800b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:51.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.008+0000 7ff472ffd700 1 -- 172.21.15.64:0/3312863431 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff46c059410 con 0x7ff47c0fe0e0 2024-02-16T22:56:51.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:50 smithi192 ceph-mon[36254]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:51.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:50 smithi192 ceph-mon[36254]: pgmap v30: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 40 MiB/s rd, 88 MiB/s wr, 277 op/s 2024-02-16T22:56:51.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:50 smithi192 ceph-mon[36254]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:51.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:50 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1857992095' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:56:51.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:50 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1023856524' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:56:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.315+0000 7ff470ff9700 1 -- 172.21.15.64:0/3312863431 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff464000bc0 con 0x7ff46806b2e0 2024-02-16T22:56:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.317+0000 7ff472ffd700 1 -- 172.21.15.64:0/3312863431 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7ff464000bc0 con 0x7ff46806b2e0 2024-02-16T22:56:51.370 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:56:51.370 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:56:51.370 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:56:51.370 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:56:51.370 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [], 2024-02-16T22:56:51.370 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "1/21 daemons upgraded", 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "", 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.323+0000 7ff470ff9700 1 -- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff46806b2e0 msgr2=0x7ff46806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.323+0000 7ff470ff9700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff46806b2e0 0x7ff46806d790 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff47c071bf0 tx=0x7ff47800b040 comp rx=0 tx=0).stop 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.323+0000 7ff470ff9700 1 -- 172.21.15.64:0/3312863431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 msgr2=0x7ff47c0702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.323+0000 7ff470ff9700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 0x7ff47c0702e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff46c000c00 tx=0x7ff46c009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.324+0000 7ff470ff9700 1 -- 172.21.15.64:0/3312863431 shutdown_connections 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.324+0000 7ff470ff9700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff46806b2e0 0x7ff46806d790 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff47c071bf0 tx=0x7ff47800b040 comp rx=0 tx=0).stop 2024-02-16T22:56:51.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.324+0000 7ff470ff9700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff47c0fcef0 0x7ff47c071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.324+0000 7ff470ff9700 1 --2- 172.21.15.64:0/3312863431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c0fe0e0 0x7ff47c0702e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.324+0000 7ff470ff9700 1 -- 172.21.15.64:0/3312863431 >> 172.21.15.64:0/3312863431 conn(0x7ff47c0f8620 msgr2=0x7ff47c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.327+0000 7ff470ff9700 1 -- 172.21.15.64:0/3312863431 shutdown_connections 2024-02-16T22:56:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.327+0000 7ff470ff9700 1 -- 172.21.15.64:0/3312863431 wait complete. 2024-02-16T22:56:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.497+0000 7f72c068e700 1 -- 172.21.15.64:0/3702954069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 msgr2=0x7f72b80ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.497+0000 7f72c068e700 1 --2- 172.21.15.64:0/3702954069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 0x7f72b80ff560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f72b4004940 tx=0x7f72b4009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.497+0000 7f72c068e700 1 -- 172.21.15.64:0/3702954069 shutdown_connections 2024-02-16T22:56:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.497+0000 7f72c068e700 1 --2- 172.21.15.64:0/3702954069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b8100360 0x7f72b81007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.497+0000 7f72c068e700 1 --2- 172.21.15.64:0/3702954069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 0x7f72b80ff560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.497+0000 7f72c068e700 1 -- 172.21.15.64:0/3702954069 >> 172.21.15.64:0/3702954069 conn(0x7f72b80fa920 msgr2=0x7f72b80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.497+0000 7f72c068e700 1 -- 172.21.15.64:0/3702954069 shutdown_connections 2024-02-16T22:56:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.497+0000 7f72c068e700 1 -- 172.21.15.64:0/3702954069 wait complete. 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.498+0000 7f72c068e700 1 Processor -- start 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.498+0000 7f72c068e700 1 -- start start 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.499+0000 7f72c068e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 0x7f72b810fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.499+0000 7f72c068e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b8100360 0x7f72b81100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.499+0000 7f72be42a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 0x7f72b810fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.499+0000 7f72c068e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b81106d0 con 0x7f72b80ff150 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.499+0000 7f72be42a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 0x7f72b810fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48056/0 (socket says 172.21.15.64:48056) 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.499+0000 7f72be42a700 1 -- 172.21.15.64:0/3869077895 learned_addr learned my addr 172.21.15.64:0/3869077895 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.499+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8110f70 con 0x7f72b8100360 2024-02-16T22:56:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.499+0000 7f72bdc29700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b8100360 0x7f72b81100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:51.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72bdc29700 1 -- 172.21.15.64:0/3869077895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 msgr2=0x7f72b810fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72bdc29700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 0x7f72b810fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72bdc29700 1 -- 172.21.15.64:0/3869077895 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4005040 con 0x7f72b8100360 2024-02-16T22:56:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72bdc29700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b8100360 0x7f72b81100d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f72a8002820 tx=0x7f72a8004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72aeffd700 1 -- 172.21.15.64:0/3869077895 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72a8007cc0 con 0x7f72b8100360 2024-02-16T22:56:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72aeffd700 1 -- 172.21.15.64:0/3869077895 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72a800f040 con 0x7f72b8100360 2024-02-16T22:56:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72aeffd700 1 -- 172.21.15.64:0/3869077895 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72a8013920 con 0x7f72b8100360 2024-02-16T22:56:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b8111250 con 0x7f72b8100360 2024-02-16T22:56:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.500+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b8111790 con 0x7f72b8100360 2024-02-16T22:56:51.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.502+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b804bed0 con 0x7f72b8100360 2024-02-16T22:56:51.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.502+0000 7f72aeffd700 1 -- 172.21.15.64:0/3869077895 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f72a801c400 con 0x7f72b8100360 2024-02-16T22:56:51.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.503+0000 7f72aeffd700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f72a406b2e0 0x7f72a406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:56:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.503+0000 7f72aeffd700 1 -- 172.21.15.64:0/3869077895 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f72a8089ac0 con 0x7f72b8100360 2024-02-16T22:56:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.503+0000 7f72be42a700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f72a406b2e0 0x7f72a406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:56:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.504+0000 7f72be42a700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f72a406b2e0 0x7f72a406d790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009fd0 tx=0x7f72b4009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:56:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.507+0000 7f72aeffd700 1 -- 172.21.15.64:0/3869077895 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f72a80575b0 con 0x7f72b8100360 2024-02-16T22:56:51.920 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:56:51.920 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:56:51.921 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:56:51.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.919+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f72b8107c30 con 0x7f72b8100360 2024-02-16T22:56:51.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.919+0000 7f72aeffd700 1 -- 172.21.15.64:0/3869077895 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f72a8057140 con 0x7f72b8100360 2024-02-16T22:56:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f72a406b2e0 msgr2=0x7f72a406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f72a406b2e0 0x7f72a406d790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009fd0 tx=0x7f72b4009e30 comp rx=0 tx=0).stop 2024-02-16T22:56:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b8100360 msgr2=0x7f72b81100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:56:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b8100360 0x7f72b81100d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f72a8002820 tx=0x7f72a8004570 comp rx=0 tx=0).stop 2024-02-16T22:56:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 shutdown_connections 2024-02-16T22:56:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b80ff150 0x7f72b810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f72a406b2e0 0x7f72a406d790 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 --2- 172.21.15.64:0/3869077895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b8100360 0x7f72b81100d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:56:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 >> 172.21.15.64:0/3869077895 conn(0x7f72b80fa920 msgr2=0x7f72b8103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:56:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 shutdown_connections 2024-02-16T22:56:51.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:56:51.922+0000 7f72c068e700 1 -- 172.21.15.64:0/3869077895 wait complete. 2024-02-16T22:56:52.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:51 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:52.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:51 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:52.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:51 smithi192 ceph-mon[36254]: Reconfiguring grafana.smithi064 (dependencies changed)... 2024-02-16T22:56:52.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:51 smithi192 ceph-mon[36254]: Reconfiguring daemon grafana.smithi064 on smithi064 2024-02-16T22:56:52.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:51 smithi192 ceph-mon[36254]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:52.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:51 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3869077895' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:56:52.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:51 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:52.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:51 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:56:52.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:51 smithi064 ceph-mon[29226]: Reconfiguring grafana.smithi064 (dependencies changed)... 2024-02-16T22:56:52.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:51 smithi064 ceph-mon[29226]: Reconfiguring daemon grafana.smithi064 on smithi064 2024-02-16T22:56:52.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:51 smithi064 ceph-mon[29226]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:56:52.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:51 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3869077895' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:56:52.963 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:52 smithi064 ceph-mon[29226]: pgmap v31: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 40 MiB/s rd, 89 MiB/s wr, 312 op/s 2024-02-16T22:56:53.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:52 smithi192 ceph-mon[36254]: pgmap v31: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 40 MiB/s rd, 89 MiB/s wr, 312 op/s 2024-02-16T22:56:55.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:54 smithi192 ceph-mon[36254]: pgmap v32: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 38 MiB/s rd, 84 MiB/s wr, 343 op/s 2024-02-16T22:56:55.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:54 smithi064 ceph-mon[29226]: pgmap v32: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 38 MiB/s rd, 84 MiB/s wr, 343 op/s 2024-02-16T22:56:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:55 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:56:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:55 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:56:57.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:56 smithi192 ceph-mon[36254]: pgmap v33: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 9.0 MiB/s rd, 36 MiB/s wr, 192 op/s 2024-02-16T22:56:57.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:56 smithi064 ceph-mon[29226]: pgmap v33: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 9.0 MiB/s rd, 36 MiB/s wr, 192 op/s 2024-02-16T22:56:58.040 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-16T22:56:58.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-16T22:56:59.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:56:58 smithi192 ceph-mon[36254]: pgmap v34: 65 pgs: 65 active+clean; 1.8 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 9.0 MiB/s rd, 37 MiB/s wr, 267 op/s 2024-02-16T22:56:59.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:56:58 smithi064 ceph-mon[29226]: pgmap v34: 65 pgs: 65 active+clean; 1.8 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 9.0 MiB/s rd, 37 MiB/s wr, 267 op/s 2024-02-16T22:57:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:00 smithi064 ceph-mon[29226]: pgmap v35: 65 pgs: 65 active+clean; 1.8 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 153 op/s 2024-02-16T22:57:01.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:00 smithi192 ceph-mon[36254]: pgmap v35: 65 pgs: 65 active+clean; 1.8 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 153 op/s 2024-02-16T22:57:03.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:02 smithi064 ceph-mon[29226]: pgmap v36: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 174 op/s 2024-02-16T22:57:03.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:02 smithi192 ceph-mon[36254]: pgmap v36: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 174 op/s 2024-02-16T22:57:05.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:04 smithi192 ceph-mon[36254]: pgmap v37: 65 pgs: 65 active+clean; 1011 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2024-02-16T22:57:05.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:04 smithi064 ceph-mon[29226]: pgmap v37: 65 pgs: 65 active+clean; 1011 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2024-02-16T22:57:07.119 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-16T22:57:07.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-16T22:57:07.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:06 smithi064 ceph-mon[29226]: pgmap v38: 65 pgs: 65 active+clean; 1011 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-02-16T22:57:07.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:06 smithi192 ceph-mon[36254]: pgmap v38: 65 pgs: 65 active+clean; 1011 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-02-16T22:57:09.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:08 smithi192 ceph-mon[36254]: pgmap v39: 65 pgs: 65 active+clean; 538 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 35 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-02-16T22:57:09.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:08 smithi064 ceph-mon[29226]: pgmap v39: 65 pgs: 65 active+clean; 538 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 35 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-02-16T22:57:11.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:10 smithi064 ceph-mon[29226]: pgmap v40: 65 pgs: 65 active+clean; 538 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-02-16T22:57:11.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:10 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:57:11.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:10 smithi192 ceph-mon[36254]: pgmap v40: 65 pgs: 65 active+clean; 538 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-02-16T22:57:11.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:10 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:57:13.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:12 smithi192 ceph-mon[36254]: pgmap v41: 65 pgs: 65 active+clean; 503 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 38 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-02-16T22:57:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:12 smithi064 ceph-mon[29226]: pgmap v41: 65 pgs: 65 active+clean; 503 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 38 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-02-16T22:57:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:14 smithi192 ceph-mon[36254]: pgmap v42: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-02-16T22:57:15.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:14 smithi064 ceph-mon[29226]: pgmap v42: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-02-16T22:57:16.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:16 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:16.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:16 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:16.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:16 smithi192 ceph-mon[36254]: Reconfiguring prometheus.smithi064 (dependencies changed)... 2024-02-16T22:57:16.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:16 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:16.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:16 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:16.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:16 smithi064 ceph-mon[29226]: Reconfiguring prometheus.smithi064 (dependencies changed)... 2024-02-16T22:57:17.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:17 smithi192 ceph-mon[36254]: Reconfiguring daemon prometheus.smithi064 on smithi064 2024-02-16T22:57:17.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:17 smithi192 ceph-mon[36254]: pgmap v43: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-02-16T22:57:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:17 smithi064 ceph-mon[29226]: Reconfiguring daemon prometheus.smithi064 on smithi064 2024-02-16T22:57:17.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:17 smithi064 ceph-mon[29226]: pgmap v43: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-02-16T22:57:19.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:19 smithi192 ceph-mon[36254]: pgmap v44: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-02-16T22:57:19.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:19 smithi064 ceph-mon[29226]: pgmap v44: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-02-16T22:57:21.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:21 smithi192 ceph-mon[36254]: pgmap v45: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-16T22:57:21.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:21 smithi064 ceph-mon[29226]: pgmap v45: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.044+0000 7ff32bf0c700 1 -- 172.21.15.64:0/188172792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fcfb0 msgr2=0x7ff3240fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.044+0000 7ff32bf0c700 1 --2- 172.21.15.64:0/188172792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fcfb0 0x7ff3240fd420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff314004940 tx=0x7ff314009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.044+0000 7ff32bf0c700 1 -- 172.21.15.64:0/188172792 shutdown_connections 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.044+0000 7ff32bf0c700 1 --2- 172.21.15.64:0/188172792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fcfb0 0x7ff3240fd420 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.044+0000 7ff32bf0c700 1 --2- 172.21.15.64:0/188172792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3240fbe40 0x7ff3240fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.044+0000 7ff32bf0c700 1 -- 172.21.15.64:0/188172792 >> 172.21.15.64:0/188172792 conn(0x7ff3240f74e0 msgr2=0x7ff3240f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.044+0000 7ff32bf0c700 1 -- 172.21.15.64:0/188172792 shutdown_connections 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.044+0000 7ff32bf0c700 1 -- 172.21.15.64:0/188172792 wait complete. 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff32bf0c700 1 Processor -- start 2024-02-16T22:57:22.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff32bf0c700 1 -- start start 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff32bf0c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fbe40 0x7ff32410fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff32bf0c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3240fcfb0 0x7ff32410fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff32bf0c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3241105f0 con 0x7ff3240fbe40 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff32bf0c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff324110e90 con 0x7ff3240fcfb0 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff329ca8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fbe40 0x7ff32410fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff329ca8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fbe40 0x7ff32410fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39342/0 (socket says 172.21.15.64:39342) 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff329ca8700 1 -- 172.21.15.64:0/2897969828 learned_addr learned my addr 172.21.15.64:0/2897969828 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff3294a7700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3240fcfb0 0x7ff32410fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff329ca8700 1 -- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3240fcfb0 msgr2=0x7ff32410fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff329ca8700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3240fcfb0 0x7ff32410fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.045+0000 7ff329ca8700 1 -- 172.21.15.64:0/2897969828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314005040 con 0x7ff3240fbe40 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.046+0000 7ff329ca8700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fbe40 0x7ff32410fab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff32000eb40 tx=0x7ff32000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:22.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.046+0000 7ff31a7fc700 1 -- 172.21.15.64:0/2897969828 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff320007cc0 con 0x7ff3240fbe40 2024-02-16T22:57:22.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.046+0000 7ff31a7fc700 1 -- 172.21.15.64:0/2897969828 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff32000f040 con 0x7ff3240fbe40 2024-02-16T22:57:22.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.046+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff324111110 con 0x7ff3240fbe40 2024-02-16T22:57:22.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.046+0000 7ff31a7fc700 1 -- 172.21.15.64:0/2897969828 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32000ca10 con 0x7ff3240fbe40 2024-02-16T22:57:22.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.046+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff324111650 con 0x7ff3240fbe40 2024-02-16T22:57:22.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.047+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32404bed0 con 0x7ff3240fbe40 2024-02-16T22:57:22.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.048+0000 7ff31a7fc700 1 -- 172.21.15.64:0/2897969828 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7ff32000cb70 con 0x7ff3240fbe40 2024-02-16T22:57:22.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.048+0000 7ff31a7fc700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff31006b2e0 0x7ff31006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.049+0000 7ff31a7fc700 1 -- 172.21.15.64:0/2897969828 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff3200898d0 con 0x7ff3240fbe40 2024-02-16T22:57:22.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.049+0000 7ff3294a7700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff31006b2e0 0x7ff31006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.049+0000 7ff3294a7700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff31006b2e0 0x7ff31006d790 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff314009b30 tx=0x7ff314009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.055+0000 7ff31a7fc700 1 -- 172.21.15.64:0/2897969828 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff32004f620 con 0x7ff3240fbe40 2024-02-16T22:57:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.352+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff324065dd0 con 0x7ff31006b2e0 2024-02-16T22:57:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.353+0000 7ff31a7fc700 1 -- 172.21.15.64:0/2897969828 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7ff324065dd0 con 0x7ff31006b2e0 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff31006b2e0 msgr2=0x7ff31006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff31006b2e0 0x7ff31006d790 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff314009b30 tx=0x7ff314009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fbe40 msgr2=0x7ff32410fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fbe40 0x7ff32410fab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff32000eb40 tx=0x7ff32000b040 comp rx=0 tx=0).stop 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 shutdown_connections 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3240fbe40 0x7ff32410fab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7ff31006b2e0 0x7ff31006d790 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 --2- 172.21.15.64:0/2897969828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3240fcfb0 0x7ff32410fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 >> 172.21.15.64:0/2897969828 conn(0x7ff3240f74e0 msgr2=0x7ff3240656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:22.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 shutdown_connections 2024-02-16T22:57:22.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.356+0000 7ff32bf0c700 1 -- 172.21.15.64:0/2897969828 wait complete. 2024-02-16T22:57:22.368 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:57:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.481+0000 7f2036fb8700 1 -- 172.21.15.64:0/2543601134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 msgr2=0x7f20300fb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.481+0000 7f2036fb8700 1 --2- 172.21.15.64:0/2543601134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 0x7f20300fb5d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2024004940 tx=0x7f2024009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.481+0000 7f2036fb8700 1 -- 172.21.15.64:0/2543601134 shutdown_connections 2024-02-16T22:57:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.481+0000 7f2036fb8700 1 --2- 172.21.15.64:0/2543601134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20300fe250 0x7f20300fe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.481+0000 7f2036fb8700 1 --2- 172.21.15.64:0/2543601134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 0x7f20300fb5d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.481+0000 7f2036fb8700 1 -- 172.21.15.64:0/2543601134 >> 172.21.15.64:0/2543601134 conn(0x7f20300f6c40 msgr2=0x7f20300f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.482+0000 7f2036fb8700 1 -- 172.21.15.64:0/2543601134 shutdown_connections 2024-02-16T22:57:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.482+0000 7f2036fb8700 1 -- 172.21.15.64:0/2543601134 wait complete. 2024-02-16T22:57:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.482+0000 7f2036fb8700 1 Processor -- start 2024-02-16T22:57:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.482+0000 7f2036fb8700 1 -- start start 2024-02-16T22:57:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2036fb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 0x7f203010d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2036fb8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20300fe250 0x7f203010dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2036fb8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203010e3e0 con 0x7f20300fb1c0 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2036fb8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203010ec80 con 0x7f20300fe250 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2034d54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 0x7f203010d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2034d54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 0x7f203010d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39352/0 (socket says 172.21.15.64:39352) 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2034d54700 1 -- 172.21.15.64:0/1377265336 learned_addr learned my addr 172.21.15.64:0/1377265336 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f202ffff700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20300fe250 0x7f203010dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2034d54700 1 -- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20300fe250 msgr2=0x7f203010dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2034d54700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20300fe250 0x7f203010dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2034d54700 1 -- 172.21.15.64:0/1377265336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2024005040 con 0x7f20300fb1c0 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f202ffff700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20300fe250 0x7f203010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.483+0000 7f2034d54700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 0x7f203010d8a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2024000c00 tx=0x7f2024009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.484+0000 7f202d7fa700 1 -- 172.21.15.64:0/1377265336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2024021070 con 0x7f20300fb1c0 2024-02-16T22:57:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.484+0000 7f202d7fa700 1 -- 172.21.15.64:0/1377265336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2024009430 con 0x7f20300fb1c0 2024-02-16T22:57:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.484+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203010ef00 con 0x7f20300fb1c0 2024-02-16T22:57:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.484+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203010f3e0 con 0x7f20300fb1c0 2024-02-16T22:57:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.484+0000 7f202d7fa700 1 -- 172.21.15.64:0/1377265336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f202400e070 con 0x7f20300fb1c0 2024-02-16T22:57:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.485+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20140052e0 con 0x7f20300fb1c0 2024-02-16T22:57:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.486+0000 7f202d7fa700 1 -- 172.21.15.64:0/1377265336 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f2024013af0 con 0x7f20300fb1c0 2024-02-16T22:57:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.486+0000 7f202d7fa700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f202006b2e0 0x7f202006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.486+0000 7f202d7fa700 1 -- 172.21.15.64:0/1377265336 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f202408aed0 con 0x7f20300fb1c0 2024-02-16T22:57:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.486+0000 7f202ffff700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f202006b2e0 0x7f202006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.487+0000 7f202ffff700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f202006b2e0 0x7f202006d790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f201c009b80 tx=0x7f201c00b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.493+0000 7f202d7fa700 1 -- 172.21.15.64:0/1377265336 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2024058b00 con 0x7f20300fb1c0 2024-02-16T22:57:22.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.772+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2014000bf0 con 0x7f202006b2e0 2024-02-16T22:57:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.774+0000 7f202d7fa700 1 -- 172.21.15.64:0/1377265336 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f2014000bf0 con 0x7f202006b2e0 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f202006b2e0 msgr2=0x7f202006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f202006b2e0 0x7f202006d790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f201c009b80 tx=0x7f201c00b040 comp rx=0 tx=0).stop 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 msgr2=0x7f203010d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 0x7f203010d8a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2024000c00 tx=0x7f2024009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 shutdown_connections 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20300fb1c0 0x7f203010d8a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f202006b2e0 0x7f202006d790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 --2- 172.21.15.64:0/1377265336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20300fe250 0x7f203010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 >> 172.21.15.64:0/1377265336 conn(0x7f20300f6c40 msgr2=0x7f2030101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 shutdown_connections 2024-02-16T22:57:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.776+0000 7f2036fb8700 1 -- 172.21.15.64:0/1377265336 wait complete. 2024-02-16T22:57:22.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.930+0000 7f1dafc6a700 1 -- 172.21.15.64:0/3560220314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da80fcfb0 msgr2=0x7f1da80fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.930+0000 7f1dafc6a700 1 --2- 172.21.15.64:0/3560220314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da80fcfb0 0x7f1da80fd420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1d98004940 tx=0x7f1d98009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.930+0000 7f1dafc6a700 1 -- 172.21.15.64:0/3560220314 shutdown_connections 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 --2- 172.21.15.64:0/3560220314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da80fcfb0 0x7f1da80fd420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 --2- 172.21.15.64:0/3560220314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80fbe40 0x7f1da80fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 -- 172.21.15.64:0/3560220314 >> 172.21.15.64:0/3560220314 conn(0x7f1da80f7500 msgr2=0x7f1da80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 -- 172.21.15.64:0/3560220314 shutdown_connections 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 -- 172.21.15.64:0/3560220314 wait complete. 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 Processor -- start 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 -- start start 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80fbe40 0x7f1da80fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da80fcfb0 0x7f1da80fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80fe980 con 0x7f1da80fcfb0 2024-02-16T22:57:22.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.931+0000 7f1dafc6a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80feaf0 con 0x7f1da80fbe40 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.932+0000 7f1dada06700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80fbe40 0x7f1da80fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.932+0000 7f1dada06700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80fbe40 0x7f1da80fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:60864/0 (socket says 172.21.15.64:60864) 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.932+0000 7f1dada06700 1 -- 172.21.15.64:0/1861434228 learned_addr learned my addr 172.21.15.64:0/1861434228 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.932+0000 7f1dad205700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da80fcfb0 0x7f1da80fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.932+0000 7f1dada06700 1 -- 172.21.15.64:0/1861434228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da80fcfb0 msgr2=0x7f1da80fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.932+0000 7f1dada06700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da80fcfb0 0x7f1da80fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.932+0000 7f1dada06700 1 -- 172.21.15.64:0/1861434228 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d98005040 con 0x7f1da80fbe40 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.932+0000 7f1dada06700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80fbe40 0x7f1da80fff00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1da4002a50 tx=0x7f1da40046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.933+0000 7f1d9e7fc700 1 -- 172.21.15.64:0/1861434228 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1da4007cc0 con 0x7f1da80fbe40 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.933+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da80fedd0 con 0x7f1da80fbe40 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.933+0000 7f1d9e7fc700 1 -- 172.21.15.64:0/1861434228 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1da400f040 con 0x7f1da80fbe40 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.933+0000 7f1d9e7fc700 1 -- 172.21.15.64:0/1861434228 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1da400da30 con 0x7f1da80fbe40 2024-02-16T22:57:22.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.933+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da8115970 con 0x7f1da80fbe40 2024-02-16T22:57:22.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.934+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da804bed0 con 0x7f1da80fbe40 2024-02-16T22:57:22.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.935+0000 7f1d9e7fc700 1 -- 172.21.15.64:0/1861434228 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f1da400db90 con 0x7f1da80fbe40 2024-02-16T22:57:22.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.935+0000 7f1d9e7fc700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f1d9406b330 0x7f1d9406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:22.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.935+0000 7f1d9e7fc700 1 -- 172.21.15.64:0/1861434228 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f1da4089c10 con 0x7f1da80fbe40 2024-02-16T22:57:22.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.936+0000 7f1dad205700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f1d9406b330 0x7f1d9406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:22.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.936+0000 7f1dad205700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f1d9406b330 0x7f1d9406d7e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800b3b0 tx=0x7f1d98009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:22.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:22.940+0000 7f1d9e7fc700 1 -- 172.21.15.64:0/1861434228 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f1da4057680 con 0x7f1da80fbe40 2024-02-16T22:57:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.200+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1da8066660 con 0x7f1d9406b330 2024-02-16T22:57:23.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.210+0000 7f1d9e7fc700 1 -- 172.21.15.64:0/1861434228 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f1da8066660 con 0x7f1d9406b330 2024-02-16T22:57:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:57:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 starting - - - - 2024-02-16T22:57:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (9m) 66s ago 9m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:57:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (8m) 69s ago 8m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:57:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 starting - - - - 2024-02-16T22:57:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (3m) 66s ago 3m 15.6M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:57:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (3m) 66s ago 3m 213M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (3m) 69s ago 3m 14.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (3m) 69s ago 3m 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:9283 running (10m) 66s ago 10m 359M - 17.2.7-513-gb88fae66 870a0944ccb8 7189c74ed7fc 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (98s) 69s ago 8m 487M - 19.0.0-1450-gf78a58c0 f081f847070a 05118581c81b 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (10m) 66s ago 10m 48.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (7m) 69s ago 7m 36.2M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 starting - - - - 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (8m) 69s ago 8m 15.4M - 1.5.0 0da6a335fe13 c32798cb2969 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (7m) 66s ago 7m 94.7M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (6m) 66s ago 6m 91.5M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (6m) 66s ago 6m 84.2M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (5m) 69s ago 5m 94.9M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:57:23.212 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (5m) 69s ago 5m 90.5M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:57:23.213 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (5m) 69s ago 5m 89.6M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:57:23.213 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (7m) 66s ago 8m 56.7M - 2.43.0 a07b618ecd1d dd20eff0542c 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f1d9406b330 msgr2=0x7f1d9406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f1d9406b330 0x7f1d9406d7e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800b3b0 tx=0x7f1d98009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80fbe40 msgr2=0x7f1da80fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80fbe40 0x7f1da80fff00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1da4002a50 tx=0x7f1da40046b0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 shutdown_connections 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f1d9406b330 0x7f1d9406d7e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80fbe40 0x7f1da80fff00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 --2- 172.21.15.64:0/1861434228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da80fcfb0 0x7f1da80fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 >> 172.21.15.64:0/1861434228 conn(0x7f1da80f7500 msgr2=0x7f1da8065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 shutdown_connections 2024-02-16T22:57:23.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.213+0000 7f1dafc6a700 1 -- 172.21.15.64:0/1861434228 wait complete. 2024-02-16T22:57:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.363+0000 7f52dad27700 1 -- 172.21.15.64:0/1898856186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 msgr2=0x7f52d40fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.363+0000 7f52dad27700 1 --2- 172.21.15.64:0/1898856186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 0x7f52d40fc250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f52c8004940 tx=0x7f52c8009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.363+0000 7f52dad27700 1 -- 172.21.15.64:0/1898856186 shutdown_connections 2024-02-16T22:57:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.363+0000 7f52dad27700 1 --2- 172.21.15.64:0/1898856186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d40fcfb0 0x7f52d40fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.363+0000 7f52dad27700 1 --2- 172.21.15.64:0/1898856186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 0x7f52d40fc250 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.363+0000 7f52dad27700 1 -- 172.21.15.64:0/1898856186 >> 172.21.15.64:0/1898856186 conn(0x7f52d40f7520 msgr2=0x7f52d40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.363+0000 7f52dad27700 1 -- 172.21.15.64:0/1898856186 shutdown_connections 2024-02-16T22:57:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.363+0000 7f52dad27700 1 -- 172.21.15.64:0/1898856186 wait complete. 2024-02-16T22:57:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52dad27700 1 Processor -- start 2024-02-16T22:57:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52dad27700 1 -- start start 2024-02-16T22:57:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52dad27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 0x7f52d4110650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52dad27700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d40fcfb0 0x7f52d4112ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52dad27700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4066cf0 con 0x7f52d40fbe40 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52dad27700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4066e30 con 0x7f52d40fcfb0 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52d8ac3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 0x7f52d4110650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52d8ac3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 0x7f52d4110650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39400/0 (socket says 172.21.15.64:39400) 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52d8ac3700 1 -- 172.21.15.64:0/3679393904 learned_addr learned my addr 172.21.15.64:0/3679393904 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.364+0000 7f52d3fff700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d40fcfb0 0x7f52d4112ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52d8ac3700 1 -- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d40fcfb0 msgr2=0x7f52d4112ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52d8ac3700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d40fcfb0 0x7f52d4112ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52d8ac3700 1 -- 172.21.15.64:0/3679393904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c8005040 con 0x7f52d40fbe40 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52d8ac3700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 0x7f52d4110650 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009fd0 tx=0x7f52c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52d17fa700 1 -- 172.21.15.64:0/3679393904 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52c800e040 con 0x7f52d40fbe40 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d41131a0 con 0x7f52d40fbe40 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52d17fa700 1 -- 172.21.15.64:0/3679393904 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52c800bcc0 con 0x7f52d40fbe40 2024-02-16T22:57:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52d17fa700 1 -- 172.21.15.64:0/3679393904 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52c801c630 con 0x7f52d40fbe40 2024-02-16T22:57:23.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.365+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d41136e0 con 0x7f52d40fbe40 2024-02-16T22:57:23.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.367+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d404bed0 con 0x7f52d40fbe40 2024-02-16T22:57:23.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.367+0000 7f52d17fa700 1 -- 172.21.15.64:0/3679393904 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f52c8013420 con 0x7f52d40fbe40 2024-02-16T22:57:23.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.368+0000 7f52d17fa700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f52c406b2e0 0x7f52c406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:23.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.368+0000 7f52d17fa700 1 -- 172.21.15.64:0/3679393904 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f52c808b210 con 0x7f52d40fbe40 2024-02-16T22:57:23.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.368+0000 7f52d3fff700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f52c406b2e0 0x7f52c406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:23.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.368+0000 7f52d3fff700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f52c406b2e0 0x7f52c406d790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52c0004f40 tx=0x7f52c000b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:23.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.375+0000 7f52d17fa700 1 -- 172.21.15.64:0/3679393904 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f52c8055270 con 0x7f52d40fbe40 2024-02-16T22:57:23.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:23 smithi192 ceph-mon[36254]: pgmap v46: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-16T22:57:23.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:23 smithi064 ceph-mon[29226]: pgmap v46: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-16T22:57:23.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.746+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f52d4113ae0 con 0x7f52d40fbe40 2024-02-16T22:57:23.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.746+0000 7f52d17fa700 1 -- 172.21.15.64:0/3679393904 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f52c8058930 con 0x7f52d40fbe40 2024-02-16T22:57:23.747 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:57:23.747 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:57:23.747 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:57:23.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:57:23.747 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 13, 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:57:23.748 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:57:23.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f52c406b2e0 msgr2=0x7f52c406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:23.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f52c406b2e0 0x7f52c406d790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52c0004f40 tx=0x7f52c000b040 comp rx=0 tx=0).stop 2024-02-16T22:57:23.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 msgr2=0x7f52d4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:23.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 0x7f52d4110650 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009fd0 tx=0x7f52c8009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:23.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 shutdown_connections 2024-02-16T22:57:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d40fbe40 0x7f52d4110650 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f52c406b2e0 0x7f52c406d790 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 --2- 172.21.15.64:0/3679393904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d40fcfb0 0x7f52d4112ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.749+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 >> 172.21.15.64:0/3679393904 conn(0x7f52d40f7520 msgr2=0x7f52d4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.750+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 shutdown_connections 2024-02-16T22:57:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.750+0000 7f52dad27700 1 -- 172.21.15.64:0/3679393904 wait complete. 2024-02-16T22:57:23.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.915+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2421683278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 msgr2=0x7f2e400fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.915+0000 7f2e46cbb700 1 --2- 172.21.15.64:0/2421683278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 0x7f2e400fe550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c004970 tx=0x7f2e2c009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.915+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2421683278 shutdown_connections 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.915+0000 7f2e46cbb700 1 --2- 172.21.15.64:0/2421683278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 0x7f2e400fe550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.915+0000 7f2e46cbb700 1 --2- 172.21.15.64:0/2421683278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e400fcef0 0x7f2e400fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.915+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2421683278 >> 172.21.15.64:0/2421683278 conn(0x7f2e400f8620 msgr2=0x7f2e400faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.915+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2421683278 shutdown_connections 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.915+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2421683278 wait complete. 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e46cbb700 1 Processor -- start 2024-02-16T22:57:23.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e46cbb700 1 -- start start 2024-02-16T22:57:23.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e46cbb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e400fcef0 0x7f2e40071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e46cbb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 0x7f2e400702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e46cbb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400708e0 con 0x7f2e400fcef0 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e46cbb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40070a50 con 0x7f2e400fe0e0 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e44a57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e400fcef0 0x7f2e40071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e44a57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e400fcef0 0x7f2e40071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39414/0 (socket says 172.21.15.64:39414) 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e44a57700 1 -- 172.21.15.64:0/2707328896 learned_addr learned my addr 172.21.15.64:0/2707328896 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.916+0000 7f2e3ffff700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 0x7f2e400702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e3ffff700 1 -- 172.21.15.64:0/2707328896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e400fcef0 msgr2=0x7f2e40071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e3ffff700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e400fcef0 0x7f2e40071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e3ffff700 1 -- 172.21.15.64:0/2707328896 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c005040 con 0x7f2e400fe0e0 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e3ffff700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 0x7f2e400702e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b370 tx=0x7f2e2c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e3d7fa700 1 -- 172.21.15.64:0/2707328896 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e2c00e070 con 0x7f2e400fe0e0 2024-02-16T22:57:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e3d7fa700 1 -- 172.21.15.64:0/2707328896 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e2c013420 con 0x7f2e400fe0e0 2024-02-16T22:57:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e3d7fa700 1 -- 172.21.15.64:0/2707328896 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e2c01c640 con 0x7f2e400fe0e0 2024-02-16T22:57:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e40070cd0 con 0x7f2e400fe0e0 2024-02-16T22:57:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.917+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e40115900 con 0x7f2e400fe0e0 2024-02-16T22:57:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.919+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4004bed0 con 0x7f2e400fe0e0 2024-02-16T22:57:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.919+0000 7f2e3d7fa700 1 -- 172.21.15.64:0/2707328896 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f2e2c013a20 con 0x7f2e400fe0e0 2024-02-16T22:57:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.919+0000 7f2e3d7fa700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2e3006b2e0 0x7f2e3006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.919+0000 7f2e3d7fa700 1 -- 172.21.15.64:0/2707328896 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f2e2c08b320 con 0x7f2e400fe0e0 2024-02-16T22:57:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.919+0000 7f2e44a57700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2e3006b2e0 0x7f2e3006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.920+0000 7f2e44a57700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2e3006b2e0 0x7f2e3006d790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2e40071bf0 tx=0x7f2e3400b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:23.924+0000 7f2e3d7fa700 1 -- 172.21.15.64:0/2707328896 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2e2c059010 con 0x7f2e400fe0e0 2024-02-16T22:57:24.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.205+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2e40105a30 con 0x7f2e400fe0e0 2024-02-16T22:57:24.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.206+0000 7f2e3d7fa700 1 -- 172.21.15.64:0/2707328896 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f2e2c058ba0 con 0x7f2e400fe0e0 2024-02-16T22:57:24.207 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:57:24.207 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:57:24.207 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:57:24.207 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:57:24.207 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:57:24.207 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:57:24.207 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:57:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:57:24.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:57:24.210 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:57:24.210 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:57:24.210 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:57:24.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2e3006b2e0 msgr2=0x7f2e3006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:24.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2e3006b2e0 0x7f2e3006d790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2e40071bf0 tx=0x7f2e3400b040 comp rx=0 tx=0).stop 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 msgr2=0x7f2e400702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 0x7f2e400702e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b370 tx=0x7f2e2c009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 shutdown_connections 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e400fcef0 0x7f2e40071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f2e3006b2e0 0x7f2e3006d790 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 --2- 172.21.15.64:0/2707328896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e400fe0e0 0x7f2e400702e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 >> 172.21.15.64:0/2707328896 conn(0x7f2e400f8620 msgr2=0x7f2e401013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 shutdown_connections 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.209+0000 7f2e46cbb700 1 -- 172.21.15.64:0/2707328896 wait complete. 2024-02-16T22:57:24.211 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:57:24.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.339+0000 7fa8a96f1700 1 -- 172.21.15.64:0/303881587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fbe40 msgr2=0x7fa8a40fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:24.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.339+0000 7fa8a96f1700 1 --2- 172.21.15.64:0/303881587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fbe40 0x7fa8a40fc250 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa894004940 tx=0x7fa894009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:24.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.339+0000 7fa8a96f1700 1 -- 172.21.15.64:0/303881587 shutdown_connections 2024-02-16T22:57:24.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.339+0000 7fa8a96f1700 1 --2- 172.21.15.64:0/303881587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40fcfb0 0x7fa8a40fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.339+0000 7fa8a96f1700 1 --2- 172.21.15.64:0/303881587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fbe40 0x7fa8a40fc250 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.339+0000 7fa8a96f1700 1 -- 172.21.15.64:0/303881587 >> 172.21.15.64:0/303881587 conn(0x7fa8a40f74e0 msgr2=0x7fa8a40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:24.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.339+0000 7fa8a96f1700 1 -- 172.21.15.64:0/303881587 shutdown_connections 2024-02-16T22:57:24.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.339+0000 7fa8a96f1700 1 -- 172.21.15.64:0/303881587 wait complete. 2024-02-16T22:57:24.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a96f1700 1 Processor -- start 2024-02-16T22:57:24.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a96f1700 1 -- start start 2024-02-16T22:57:24.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a96f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40fbe40 0x7fa8a4110610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a96f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fcfb0 0x7fa8a4112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a96f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a4066550 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a96f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a4066690 con 0x7fa8a40fbe40 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40fbe40 0x7fa8a4110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40fbe40 0x7fa8a4110610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:60926/0 (socket says 172.21.15.64:60926) 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a2ffd700 1 -- 172.21.15.64:0/1404078760 learned_addr learned my addr 172.21.15.64:0/1404078760 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a27fc700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fcfb0 0x7fa8a4112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a27fc700 1 -- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40fbe40 msgr2=0x7fa8a4110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a27fc700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40fbe40 0x7fa8a4110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.340+0000 7fa8a27fc700 1 -- 172.21.15.64:0/1404078760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa894005040 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.341+0000 7fa8a27fc700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fcfb0 0x7fa8a4112b60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa89800b580 tx=0x7fa8980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.341+0000 7fa88b7fe700 1 -- 172.21.15.64:0/1404078760 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa89800f6f0 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.341+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a4113240 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.341+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a4113780 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.341+0000 7fa88b7fe700 1 -- 172.21.15.64:0/1404078760 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa89800fd20 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.341+0000 7fa88b7fe700 1 -- 172.21.15.64:0/1404078760 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8980103a0 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.342+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8900052e0 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.343+0000 7fa88b7fe700 1 -- 172.21.15.64:0/1404078760 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7fa89800f850 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.343+0000 7fa88b7fe700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa88c06b2e0 0x7fa88c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:24.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.344+0000 7fa88b7fe700 1 -- 172.21.15.64:0/1404078760 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa898089820 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.344+0000 7fa8a2ffd700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa88c06b2e0 0x7fa88c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:24.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.344+0000 7fa8a2ffd700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa88c06b2e0 0x7fa88c06d790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa894004f90 tx=0x7fa894009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:24.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.351+0000 7fa88b7fe700 1 -- 172.21.15.64:0/1404078760 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa898053a30 con 0x7fa8a40fcfb0 2024-02-16T22:57:24.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:24 smithi192 ceph-mon[36254]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:24 smithi192 ceph-mon[36254]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:24 smithi192 ceph-mon[36254]: from='client.24519 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:24 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3679393904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:57:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:24 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2707328896' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:57:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:24 smithi064 ceph-mon[29226]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:24 smithi064 ceph-mon[29226]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:24 smithi064 ceph-mon[29226]: from='client.24519 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:24 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/3679393904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:57:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:24 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2707328896' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:57:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.618+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa890000bf0 con 0x7fa88c06b2e0 2024-02-16T22:57:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.619+0000 7fa88b7fe700 1 -- 172.21.15.64:0/1404078760 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fa890000bf0 con 0x7fa88c06b2e0 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [], 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "1/21 daemons upgraded", 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "", 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:57:24.620 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:57:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa88c06b2e0 msgr2=0x7fa88c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa88c06b2e0 0x7fa88c06d790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa894004f90 tx=0x7fa894009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fcfb0 msgr2=0x7fa8a4112b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fcfb0 0x7fa8a4112b60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa89800b580 tx=0x7fa8980091a0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 shutdown_connections 2024-02-16T22:57:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa88c06b2e0 0x7fa88c06d790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40fbe40 0x7fa8a4110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 --2- 172.21.15.64:0/1404078760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a40fcfb0 0x7fa8a4112b60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 >> 172.21.15.64:0/1404078760 conn(0x7fa8a40f74e0 msgr2=0x7fa8a4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 shutdown_connections 2024-02-16T22:57:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.622+0000 7fa8a96f1700 1 -- 172.21.15.64:0/1404078760 wait complete. 2024-02-16T22:57:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 -- 172.21.15.64:0/2653500796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 msgr2=0x7f313c0ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 --2- 172.21.15.64:0/2653500796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 0x7f313c0ffb00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f312c004940 tx=0x7f312c009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 -- 172.21.15.64:0/2653500796 shutdown_connections 2024-02-16T22:57:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 --2- 172.21.15.64:0/2653500796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f313c1000d0 0x7f313c102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 --2- 172.21.15.64:0/2653500796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 0x7f313c0ffb00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 -- 172.21.15.64:0/2653500796 >> 172.21.15.64:0/2653500796 conn(0x7f313c0f7510 msgr2=0x7f313c0f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 -- 172.21.15.64:0/2653500796 shutdown_connections 2024-02-16T22:57:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 -- 172.21.15.64:0/2653500796 wait complete. 2024-02-16T22:57:24.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.748+0000 7f31422a5700 1 Processor -- start 2024-02-16T22:57:24.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f31422a5700 1 -- start start 2024-02-16T22:57:24.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f31422a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 0x7f313c10d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:24.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f31422a5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f313c1000d0 0x7f313c10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:24.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f31422a5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c10e4c0 con 0x7f313c066c70 2024-02-16T22:57:24.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f31422a5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c10ed60 con 0x7f313c1000d0 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f313bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 0x7f313c10d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f313bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 0x7f313c10d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39454/0 (socket says 172.21.15.64:39454) 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f313bfff700 1 -- 172.21.15.64:0/1325254861 learned_addr learned my addr 172.21.15.64:0/1325254861 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f313b7fe700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f313c1000d0 0x7f313c10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f313bfff700 1 -- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f313c1000d0 msgr2=0x7f313c10dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f313bfff700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f313c1000d0 0x7f313c10dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.749+0000 7f313bfff700 1 -- 172.21.15.64:0/1325254861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c005040 con 0x7f313c066c70 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.750+0000 7f313bfff700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 0x7f313c10d980 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f312c0093b0 tx=0x7f312c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.750+0000 7f3138ff9700 1 -- 172.21.15.64:0/1325254861 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f312c00e040 con 0x7f313c066c70 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.750+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313c10efe0 con 0x7f313c066c70 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.750+0000 7f3138ff9700 1 -- 172.21.15.64:0/1325254861 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f312c00bcc0 con 0x7f313c066c70 2024-02-16T22:57:24.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.750+0000 7f3138ff9700 1 -- 172.21.15.64:0/1325254861 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f312c01c630 con 0x7f313c066c70 2024-02-16T22:57:24.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.750+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c10f520 con 0x7f313c066c70 2024-02-16T22:57:24.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.751+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c107a80 con 0x7f313c066c70 2024-02-16T22:57:24.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.752+0000 7f3138ff9700 1 -- 172.21.15.64:0/1325254861 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f312c0139f0 con 0x7f313c066c70 2024-02-16T22:57:24.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.752+0000 7f3138ff9700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f312406b2e0 0x7f312406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:24.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.752+0000 7f3138ff9700 1 -- 172.21.15.64:0/1325254861 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f312c08b150 con 0x7f313c066c70 2024-02-16T22:57:24.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.752+0000 7f313b7fe700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f312406b2e0 0x7f312406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:24.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.753+0000 7f313b7fe700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f312406b2e0 0x7f312406d790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f313c0669b0 tx=0x7f3130009ee0 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:24.760+0000 7f3138ff9700 1 -- 172.21.15.64:0/1325254861 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f312c058d80 con 0x7f313c066c70 2024-02-16T22:57:25.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.179+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f313c04bed0 con 0x7f313c066c70 2024-02-16T22:57:25.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.180+0000 7f3138ff9700 1 -- 172.21.15.64:0/1325254861 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f312c058910 con 0x7f313c066c70 2024-02-16T22:57:25.182 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:57:25.182 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:57:25.182 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f312406b2e0 msgr2=0x7f312406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f312406b2e0 0x7f312406d790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f313c0669b0 tx=0x7f3130009ee0 comp rx=0 tx=0).stop 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 msgr2=0x7f313c10d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 0x7f313c10d980 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f312c0093b0 tx=0x7f312c009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 shutdown_connections 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313c066c70 0x7f313c10d980 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f312406b2e0 0x7f312406d790 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 --2- 172.21.15.64:0/1325254861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f313c1000d0 0x7f313c10dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 >> 172.21.15.64:0/1325254861 conn(0x7f313c0f7510 msgr2=0x7f313c0f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:25.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 shutdown_connections 2024-02-16T22:57:25.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:25.183+0000 7f31422a5700 1 -- 172.21.15.64:0/1325254861 wait complete. 2024-02-16T22:57:25.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:25 smithi064 ceph-mon[29226]: pgmap v47: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 2.1 MiB/s wr, 126 op/s 2024-02-16T22:57:25.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:25 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1325254861' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:57:25.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:25 smithi192 ceph-mon[36254]: pgmap v47: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 2.1 MiB/s wr, 126 op/s 2024-02-16T22:57:25.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:25 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1325254861' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:57:26.366 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:26 smithi064 ceph-mon[29226]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:26.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:26 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:57:26.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:26 smithi192 ceph-mon[36254]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:26.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:26 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:57:27.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:27 smithi064 ceph-mon[29226]: pgmap v48: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-02-16T22:57:27.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:27 smithi192 ceph-mon[36254]: pgmap v48: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-02-16T22:57:29.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:29 smithi064 ceph-mon[29226]: pgmap v49: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.1 MiB/s wr, 127 op/s 2024-02-16T22:57:29.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:29 smithi064 ceph-mon[29226]: mgrmap e26: smithi192.pauvih(active, since 93s), standbys: smithi064.kubmlk 2024-02-16T22:57:29.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:29 smithi192 ceph-mon[36254]: pgmap v49: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.1 MiB/s wr, 127 op/s 2024-02-16T22:57:29.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:29 smithi192 ceph-mon[36254]: mgrmap e26: smithi192.pauvih(active, since 93s), standbys: smithi064.kubmlk 2024-02-16T22:57:31.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:31 smithi064 ceph-mon[29226]: pgmap v50: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-02-16T22:57:31.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:31 smithi192 ceph-mon[36254]: pgmap v50: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-02-16T22:57:33.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:33 smithi064 ceph-mon[29226]: pgmap v51: 65 pgs: 65 active+clean; 300 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-02-16T22:57:33.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:33 smithi192 ceph-mon[36254]: pgmap v51: 65 pgs: 65 active+clean; 300 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-02-16T22:57:35.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:35 smithi192 ceph-mon[36254]: pgmap v52: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 124 op/s 2024-02-16T22:57:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:35 smithi064 ceph-mon[29226]: pgmap v52: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 124 op/s 2024-02-16T22:57:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:37 smithi064 ceph-mon[29226]: pgmap v53: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-02-16T22:57:37.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:37 smithi192 ceph-mon[36254]: pgmap v53: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-02-16T22:57:39.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:39 smithi064 ceph-mon[29226]: pgmap v54: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-02-16T22:57:39.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:39 smithi192 ceph-mon[36254]: pgmap v54: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-02-16T22:57:41.429 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:41 smithi064 ceph-mon[29226]: pgmap v55: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-02-16T22:57:41.429 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:41 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:57:41.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:41 smithi192 ceph-mon[36254]: pgmap v55: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-02-16T22:57:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:41 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:57:43.599 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:43 smithi192 ceph-mon[36254]: pgmap v56: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-02-16T22:57:43.600 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:43 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:43.600 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:43 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:43.603 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:43 smithi064 ceph-mon[29226]: pgmap v56: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-02-16T22:57:43.603 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:43 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:43.603 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:43 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:44 smithi192 ceph-mon[36254]: Reconfiguring node-exporter.smithi192 (dependencies changed)... 2024-02-16T22:57:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:44 smithi192 ceph-mon[36254]: Reconfiguring daemon node-exporter.smithi192 on smithi192 2024-02-16T22:57:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:44 smithi064 ceph-mon[29226]: Reconfiguring node-exporter.smithi192 (dependencies changed)... 2024-02-16T22:57:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:44 smithi064 ceph-mon[29226]: Reconfiguring daemon node-exporter.smithi192 on smithi192 2024-02-16T22:57:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:45 smithi064 ceph-mon[29226]: pgmap v57: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.0 MiB/s wr, 133 op/s 2024-02-16T22:57:45.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:45 smithi192 ceph-mon[36254]: pgmap v57: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.0 MiB/s wr, 133 op/s 2024-02-16T22:57:46.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:46.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:46.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:57:46.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:57:46.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:57:46.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:57:46.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:57:46.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:57:46.488 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:57:46.488 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: Upgrade: Need to upgrade myself (mgr.smithi192.pauvih) 2024-02-16T22:57:46.488 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: Upgrade: Need to upgrade myself (mgr.smithi192.pauvih) 2024-02-16T22:57:46.488 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:46 smithi064 ceph-mon[29226]: pgmap v58: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-02-16T22:57:46.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: Upgrade: Need to upgrade myself (mgr.smithi192.pauvih) 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: Upgrade: Need to upgrade myself (mgr.smithi192.pauvih) 2024-02-16T22:57:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:46 smithi192 ceph-mon[36254]: pgmap v58: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-02-16T22:57:49.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:48 smithi064 ceph-mon[29226]: pgmap v59: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-02-16T22:57:49.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:48 smithi192 ceph-mon[36254]: pgmap v59: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-02-16T22:57:50.767 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:50 smithi064 ceph-mon[29226]: pgmap v60: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 252 op/s 2024-02-16T22:57:50.767 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:50 smithi064 ceph-mon[29226]: Upgrade: Updating mgr.smithi064.kubmlk 2024-02-16T22:57:50.767 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:50 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:50.767 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:50 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi064.kubmlk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:57:50.767 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:50 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi064.kubmlk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:57:50.767 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:50 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:57:50.767 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:50 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:57:50.767 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:50 smithi064 ceph-mon[29226]: Deploying daemon mgr.smithi064.kubmlk on smithi064 2024-02-16T22:57:51.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:50 smithi192 ceph-mon[36254]: pgmap v60: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 252 op/s 2024-02-16T22:57:51.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:50 smithi192 ceph-mon[36254]: Upgrade: Updating mgr.smithi064.kubmlk 2024-02-16T22:57:51.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:50 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:51.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:50 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi064.kubmlk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:57:51.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:50 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi064.kubmlk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:57:51.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:50 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:57:51.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:50 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:57:51.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:50 smithi192 ceph-mon[36254]: Deploying daemon mgr.smithi064.kubmlk on smithi064 2024-02-16T22:57:53.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:52 smithi192 ceph-mon[36254]: pgmap v61: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 2.1 MiB/s wr, 325 op/s 2024-02-16T22:57:53.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:52 smithi064 ceph-mon[29226]: pgmap v61: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 2.1 MiB/s wr, 325 op/s 2024-02-16T22:57:55.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:54 smithi064 ceph-mon[29226]: pgmap v62: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 2.4 MiB/s wr, 418 op/s 2024-02-16T22:57:55.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:54 smithi192 ceph-mon[36254]: pgmap v62: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 2.4 MiB/s wr, 418 op/s 2024-02-16T22:57:55.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 -- 172.21.15.64:0/254018341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fcef0 msgr2=0x7f19bc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 --2- 172.21.15.64:0/254018341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fcef0 0x7f19bc0fd320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f19ac004940 tx=0x7f19ac009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 -- 172.21.15.64:0/254018341 shutdown_connections 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 --2- 172.21.15.64:0/254018341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19bc0fe0e0 0x7f19bc0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 --2- 172.21.15.64:0/254018341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fcef0 0x7f19bc0fd320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 -- 172.21.15.64:0/254018341 >> 172.21.15.64:0/254018341 conn(0x7f19bc0f8620 msgr2=0x7f19bc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 -- 172.21.15.64:0/254018341 shutdown_connections 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 -- 172.21.15.64:0/254018341 wait complete. 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 Processor -- start 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.317+0000 7f19c26ee700 1 -- start start 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19c26ee700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19bc0fcef0 0x7f19bc10d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19c26ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fe0e0 0x7f19bc10de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19c26ee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc10e420 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19c26ee700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc10ecc0 con 0x7f19bc0fcef0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fe0e0 0x7f19bc10de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fe0e0 0x7f19bc10de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56334/0 (socket says 172.21.15.64:56334) 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19bb7fe700 1 -- 172.21.15.64:0/3948577320 learned_addr learned my addr 172.21.15.64:0/3948577320 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19bbfff700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19bc0fcef0 0x7f19bc10d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19bb7fe700 1 -- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19bc0fcef0 msgr2=0x7f19bc10d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19bb7fe700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19bc0fcef0 0x7f19bc10d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19bb7fe700 1 -- 172.21.15.64:0/3948577320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac005040 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19bb7fe700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fe0e0 0x7f19bc10de20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f19a400eb40 tx=0x7f19a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.318+0000 7f19b8ff9700 1 -- 172.21.15.64:0/3948577320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19a4007cc0 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.319+0000 7f19b8ff9700 1 -- 172.21.15.64:0/3948577320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19a400f040 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.319+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc10ef40 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.319+0000 7f19b8ff9700 1 -- 172.21.15.64:0/3948577320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19a400ca10 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.319+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc10f430 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.320+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc04bed0 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.320+0000 7f19b8ff9700 1 -- 172.21.15.64:0/3948577320 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 88916+0+0 (secure 0 0 0) 0x7f19a400cc80 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.320+0000 7f19b8ff9700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f19a806b1c0 0x7f19a806d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:55.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.320+0000 7f19b8ff9700 1 -- 172.21.15.64:0/3948577320 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f19a4089730 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.321+0000 7f19bbfff700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f19a806b1c0 0x7f19a806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:55.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.321+0000 7f19bbfff700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f19a806b1c0 0x7f19a806d670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000c00 tx=0x7f19ac009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:55.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.325+0000 7f19b8ff9700 1 -- 172.21.15.64:0/3948577320 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f19a404f620 con 0x7f19bc0fe0e0 2024-02-16T22:57:55.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.628+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19bc10f8d0 con 0x7f19a806b1c0 2024-02-16T22:57:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.629+0000 7f19b8ff9700 1 -- 172.21.15.64:0/3948577320 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f19bc10f8d0 con 0x7f19a806b1c0 2024-02-16T22:57:55.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.631+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f19a806b1c0 msgr2=0x7f19a806d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:55.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.631+0000 7f19c26ee700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f19a806b1c0 0x7f19a806d670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000c00 tx=0x7f19ac009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:55.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.631+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fe0e0 msgr2=0x7f19bc10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:55.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.631+0000 7f19c26ee700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fe0e0 0x7f19bc10de20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f19a400eb40 tx=0x7f19a400b040 comp rx=0 tx=0).stop 2024-02-16T22:57:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.631+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 shutdown_connections 2024-02-16T22:57:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.631+0000 7f19c26ee700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f19a806b1c0 0x7f19a806d670 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.631+0000 7f19c26ee700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19bc0fcef0 0x7f19bc10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.631+0000 7f19c26ee700 1 --2- 172.21.15.64:0/3948577320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19bc0fe0e0 0x7f19bc10de20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.632+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 >> 172.21.15.64:0/3948577320 conn(0x7f19bc0f8620 msgr2=0x7f19bc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.632+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 shutdown_connections 2024-02-16T22:57:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.632+0000 7f19c26ee700 1 -- 172.21.15.64:0/3948577320 wait complete. 2024-02-16T22:57:55.645 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:57:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.764+0000 7f0f8142f700 1 -- 172.21.15.64:0/4144090524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c1003a0 msgr2=0x7f0f7c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.764+0000 7f0f8142f700 1 --2- 172.21.15.64:0/4144090524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c1003a0 0x7f0f7c102780 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004940 tx=0x7f0f70009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.764+0000 7f0f8142f700 1 -- 172.21.15.64:0/4144090524 shutdown_connections 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.764+0000 7f0f8142f700 1 --2- 172.21.15.64:0/4144090524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c1003a0 0x7f0f7c102780 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.764+0000 7f0f8142f700 1 --2- 172.21.15.64:0/4144090524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f7c0fda80 0x7f0f7c0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.764+0000 7f0f8142f700 1 -- 172.21.15.64:0/4144090524 >> 172.21.15.64:0/4144090524 conn(0x7f0f7c0f7520 msgr2=0x7f0f7c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.764+0000 7f0f8142f700 1 -- 172.21.15.64:0/4144090524 shutdown_connections 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.764+0000 7f0f8142f700 1 -- 172.21.15.64:0/4144090524 wait complete. 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.765+0000 7f0f8142f700 1 Processor -- start 2024-02-16T22:57:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.765+0000 7f0f8142f700 1 -- start start 2024-02-16T22:57:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.765+0000 7f0f8142f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0fda80 0x7f0f7c10d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.765+0000 7f0f8142f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f7c1003a0 0x7f0f7c10de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.765+0000 7f0f8142f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c10e420 con 0x7f0f7c0fda80 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.765+0000 7f0f8142f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c10ecc0 con 0x7f0f7c1003a0 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.766+0000 7f0f7a7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f7c1003a0 0x7f0f7c10de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.766+0000 7f0f7a7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f7c1003a0 0x7f0f7c10de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:33978/0 (socket says 172.21.15.64:33978) 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.766+0000 7f0f7a7fc700 1 -- 172.21.15.64:0/1079580670 learned_addr learned my addr 172.21.15.64:0/1079580670 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.766+0000 7f0f7affd700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0fda80 0x7f0f7c10d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.766+0000 7f0f7a7fc700 1 -- 172.21.15.64:0/1079580670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0fda80 msgr2=0x7f0f7c10d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.766+0000 7f0f7a7fc700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0fda80 0x7f0f7c10d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.766+0000 7f0f7a7fc700 1 -- 172.21.15.64:0/1079580670 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f70005040 con 0x7f0f7c1003a0 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.766+0000 7f0f7a7fc700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f7c1003a0 0x7f0f7c10de20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f70000c00 tx=0x7f0f70009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.767+0000 7f0f637fe700 1 -- 172.21.15.64:0/1079580670 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f70021070 con 0x7f0f7c1003a0 2024-02-16T22:57:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.767+0000 7f0f637fe700 1 -- 172.21.15.64:0/1079580670 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f7001c400 con 0x7f0f7c1003a0 2024-02-16T22:57:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.767+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c10ef40 con 0x7f0f7c1003a0 2024-02-16T22:57:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.767+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c10f420 con 0x7f0f7c1003a0 2024-02-16T22:57:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.767+0000 7f0f637fe700 1 -- 172.21.15.64:0/1079580670 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7000e070 con 0x7f0f7c1003a0 2024-02-16T22:57:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.768+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c107a50 con 0x7f0f7c1003a0 2024-02-16T22:57:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.768+0000 7f0f637fe700 1 -- 172.21.15.64:0/1079580670 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 88916+0+0 (secure 0 0 0) 0x7f0f7000b750 con 0x7f0f7c1003a0 2024-02-16T22:57:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.769+0000 7f0f637fe700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f0f6406b1d0 0x7f0f6406d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.769+0000 7f0f637fe700 1 -- 172.21.15.64:0/1079580670 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f0f7008adc0 con 0x7f0f7c1003a0 2024-02-16T22:57:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.769+0000 7f0f7affd700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f0f6406b1d0 0x7f0f6406d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.769+0000 7f0f7affd700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f0f6406b1d0 0x7f0f6406d680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009b80 tx=0x7f0f6c00b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:55.773+0000 7f0f637fe700 1 -- 172.21.15.64:0/1079580670 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f0f7008e030 con 0x7f0f7c1003a0 2024-02-16T22:57:56.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.064+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f7c10f690 con 0x7f0f6406b1d0 2024-02-16T22:57:56.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.065+0000 7f0f637fe700 1 -- 172.21.15.64:0/1079580670 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f0f7c10f690 con 0x7f0f6406b1d0 2024-02-16T22:57:56.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.067+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f0f6406b1d0 msgr2=0x7f0f6406d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:56.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.067+0000 7f0f8142f700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f0f6406b1d0 0x7f0f6406d680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009b80 tx=0x7f0f6c00b040 comp rx=0 tx=0).stop 2024-02-16T22:57:56.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.067+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f7c1003a0 msgr2=0x7f0f7c10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.067+0000 7f0f8142f700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f7c1003a0 0x7f0f7c10de20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f70000c00 tx=0x7f0f70009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.068+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 shutdown_connections 2024-02-16T22:57:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.068+0000 7f0f8142f700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f7c0fda80 0x7f0f7c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.068+0000 7f0f8142f700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f0f6406b1d0 0x7f0f6406d680 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.068+0000 7f0f8142f700 1 --2- 172.21.15.64:0/1079580670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f7c1003a0 0x7f0f7c10de20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.068+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 >> 172.21.15.64:0/1079580670 conn(0x7f0f7c0f7520 msgr2=0x7f0f7c0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.068+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 shutdown_connections 2024-02-16T22:57:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.068+0000 7f0f8142f700 1 -- 172.21.15.64:0/1079580670 wait complete. 2024-02-16T22:57:56.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.197+0000 7fa8525f9700 1 -- 172.21.15.64:0/1552084052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fcfb0 msgr2=0x7fa84c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:56.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.197+0000 7fa8525f9700 1 --2- 172.21.15.64:0/1552084052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fcfb0 0x7fa84c0fd420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa840004940 tx=0x7fa840009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:56.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:55 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.197+0000 7fa8525f9700 1 -- 172.21.15.64:0/1552084052 shutdown_connections 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.197+0000 7fa8525f9700 1 --2- 172.21.15.64:0/1552084052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fcfb0 0x7fa84c0fd420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.197+0000 7fa8525f9700 1 --2- 172.21.15.64:0/1552084052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c0fbe40 0x7fa84c0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.197+0000 7fa8525f9700 1 -- 172.21.15.64:0/1552084052 >> 172.21.15.64:0/1552084052 conn(0x7fa84c0f7500 msgr2=0x7fa84c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.197+0000 7fa8525f9700 1 -- 172.21.15.64:0/1552084052 shutdown_connections 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.197+0000 7fa8525f9700 1 -- 172.21.15.64:0/1552084052 wait complete. 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.198+0000 7fa8525f9700 1 Processor -- start 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.198+0000 7fa8525f9700 1 -- start start 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.198+0000 7fa8525f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fbe40 0x7fa84c0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.198+0000 7fa8525f9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c0fcfb0 0x7fa84c0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.198+0000 7fa8525f9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c0fe980 con 0x7fa84c0fbe40 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.198+0000 7fa8525f9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c0feaf0 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fbe40 0x7fa84c0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fbe40 0x7fa84c0fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56354/0 (socket says 172.21.15.64:56354) 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84bfff700 1 -- 172.21.15.64:0/3450486958 learned_addr learned my addr 172.21.15.64:0/3450486958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84b7fe700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c0fcfb0 0x7fa84c0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84b7fe700 1 -- 172.21.15.64:0/3450486958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fbe40 msgr2=0x7fa84c0fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84b7fe700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fbe40 0x7fa84c0fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84b7fe700 1 -- 172.21.15.64:0/3450486958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840005040 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84bfff700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fbe40 0x7fa84c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.199+0000 7fa84b7fe700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c0fcfb0 0x7fa84c0fe380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa840000c00 tx=0x7fa840009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.200+0000 7fa848ff9700 1 -- 172.21.15.64:0/3450486958 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa84000e070 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.200+0000 7fa848ff9700 1 -- 172.21.15.64:0/3450486958 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa84001c400 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.200+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c0fed70 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.200+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c1158e0 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.200+0000 7fa848ff9700 1 -- 172.21.15.64:0/3450486958 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa84000b650 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.201+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c04bed0 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.202+0000 7fa848ff9700 1 -- 172.21.15.64:0/3450486958 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 88916+0+0 (secure 0 0 0) 0x7fa84001c9e0 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.202+0000 7fa848ff9700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa83406b1d0 0x7fa83406d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.202+0000 7fa848ff9700 1 -- 172.21.15.64:0/3450486958 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa84008b000 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.202+0000 7fa84bfff700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa83406b1d0 0x7fa83406d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.203+0000 7fa84bfff700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa83406b1d0 0x7fa83406d680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa83c004fd0 tx=0x7fa83c00b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.207+0000 7fa848ff9700 1 -- 172.21.15.64:0/3450486958 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa840058e50 con 0x7fa84c0fcfb0 2024-02-16T22:57:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:55 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:57:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.562+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa84c066660 con 0x7fa83406b1d0 2024-02-16T22:57:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.572+0000 7fa848ff9700 1 -- 172.21.15.64:0/3450486958 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fa84c066660 con 0x7fa83406b1d0 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 starting - - - - 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (9m) 99s ago 9m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (8m) 103s ago 8m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 starting - - - - 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (4m) 99s ago 4m 15.6M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (4m) 99s ago 4m 213M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (4m) 103s ago 4m 14.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (4m) 103s ago 4m 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:9283 running (10m) 99s ago 10m 359M - 17.2.7-513-gb88fae66 870a0944ccb8 7189c74ed7fc 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (2m) 103s ago 8m 487M - 19.0.0-1450-gf78a58c0 f081f847070a 05118581c81b 2024-02-16T22:57:56.573 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (10m) 99s ago 10m 48.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (8m) 103s ago 8m 36.2M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 starting - - - - 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 starting - - - - 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (7m) 99s ago 7m 94.7M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (7m) 99s ago 7m 91.5M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (6m) 99s ago 6m 84.2M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (6m) 103s ago 6m 94.9M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (6m) 103s ago 6m 90.5M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (5m) 103s ago 5m 89.6M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:57:56.574 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 starting - - - - 2024-02-16T22:57:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa83406b1d0 msgr2=0x7fa83406d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa83406b1d0 0x7fa83406d680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa83c004fd0 tx=0x7fa83c00b040 comp rx=0 tx=0).stop 2024-02-16T22:57:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c0fcfb0 msgr2=0x7fa84c0fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c0fcfb0 0x7fa84c0fe380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa840000c00 tx=0x7fa840009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 shutdown_connections 2024-02-16T22:57:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0fbe40 0x7fa84c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fa83406b1d0 0x7fa83406d680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 --2- 172.21.15.64:0/3450486958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c0fcfb0 0x7fa84c0fe380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 >> 172.21.15.64:0/3450486958 conn(0x7fa84c0f7500 msgr2=0x7fa84c065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 shutdown_connections 2024-02-16T22:57:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.574+0000 7fa8525f9700 1 -- 172.21.15.64:0/3450486958 wait complete. 2024-02-16T22:57:56.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.711+0000 7faeb93a0700 1 -- 172.21.15.64:0/729530293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fbe40 msgr2=0x7faeb40fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.711+0000 7faeb93a0700 1 --2- 172.21.15.64:0/729530293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fbe40 0x7faeb40fc250 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faea4004940 tx=0x7faea4009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.712+0000 7faeb93a0700 1 -- 172.21.15.64:0/729530293 shutdown_connections 2024-02-16T22:57:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.712+0000 7faeb93a0700 1 --2- 172.21.15.64:0/729530293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb40fcfb0 0x7faeb40fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.712+0000 7faeb93a0700 1 --2- 172.21.15.64:0/729530293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fbe40 0x7faeb40fc250 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.712+0000 7faeb93a0700 1 -- 172.21.15.64:0/729530293 >> 172.21.15.64:0/729530293 conn(0x7faeb40f74e0 msgr2=0x7faeb40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.713+0000 7faeb93a0700 1 -- 172.21.15.64:0/729530293 shutdown_connections 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.713+0000 7faeb93a0700 1 -- 172.21.15.64:0/729530293 wait complete. 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.713+0000 7faeb93a0700 1 Processor -- start 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.713+0000 7faeb93a0700 1 -- start start 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb93a0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb40fbe40 0x7faeb410d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb93a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fcfb0 0x7faeb410de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb93a0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb410e410 con 0x7faeb40fcfb0 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb93a0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb410ecb0 con 0x7faeb40fbe40 2024-02-16T22:57:56.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fcfb0 0x7faeb410de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb40fbe40 0x7faeb410d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fcfb0 0x7faeb410de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56368/0 (socket says 172.21.15.64:56368) 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb40fbe40 0x7faeb410d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:34002/0 (socket says 172.21.15.64:34002) 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb27fc700 1 -- 172.21.15.64:0/1630417181 learned_addr learned my addr 172.21.15.64:0/1630417181 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb27fc700 1 -- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb40fbe40 msgr2=0x7faeb410d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb27fc700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb40fbe40 0x7faeb410d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb27fc700 1 -- 172.21.15.64:0/1630417181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea8005040 con 0x7faeb40fcfb0 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.714+0000 7faeb2ffd700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb40fbe40 0x7faeb410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:57:56.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.715+0000 7faeb27fc700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fcfb0 0x7faeb410de10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faea80096d0 tx=0x7faea8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.715+0000 7fae9b7fe700 1 -- 172.21.15.64:0/1630417181 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faea800e840 con 0x7faeb40fcfb0 2024-02-16T22:57:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.715+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea4005040 con 0x7faeb40fcfb0 2024-02-16T22:57:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.715+0000 7fae9b7fe700 1 -- 172.21.15.64:0/1630417181 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faea800f040 con 0x7faeb40fcfb0 2024-02-16T22:57:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.715+0000 7fae9b7fe700 1 -- 172.21.15.64:0/1630417181 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faea8013dc0 con 0x7faeb40fcfb0 2024-02-16T22:57:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.715+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb410f2e0 con 0x7faeb40fcfb0 2024-02-16T22:57:56.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.716+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb404bed0 con 0x7faeb40fcfb0 2024-02-16T22:57:56.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.717+0000 7fae9b7fe700 1 -- 172.21.15.64:0/1630417181 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 88916+0+0 (secure 0 0 0) 0x7faea800e9a0 con 0x7faeb40fcfb0 2024-02-16T22:57:56.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.718+0000 7fae9b7fe700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fae9c06b1c0 0x7fae9c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:56.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.718+0000 7fae9b7fe700 1 -- 172.21.15.64:0/1630417181 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7faea8089850 con 0x7faeb40fcfb0 2024-02-16T22:57:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.718+0000 7faeb2ffd700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fae9c06b1c0 0x7fae9c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.719+0000 7faeb2ffd700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fae9c06b1c0 0x7fae9c06d670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faea4009fd0 tx=0x7faea4009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:56.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:56.724+0000 7fae9b7fe700 1 -- 172.21.15.64:0/1630417181 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7faea8057600 con 0x7faeb40fcfb0 2024-02-16T22:57:57.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:56 smithi064 ceph-mon[29226]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:57.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:56 smithi064 ceph-mon[29226]: pgmap v63: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 379 op/s 2024-02-16T22:57:57.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:56 smithi064 ceph-mon[29226]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:57.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.230+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faeb4066430 con 0x7faeb40fcfb0 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.231+0000 7fae9b7fe700 1 -- 172.21.15.64:0/1630417181 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7faea8057190 con 0x7faeb40fcfb0 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:57:57.232 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 13, 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:57:57.233 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fae9c06b1c0 msgr2=0x7fae9c06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fae9c06b1c0 0x7fae9c06d670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faea4009fd0 tx=0x7faea4009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fcfb0 msgr2=0x7faeb410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fcfb0 0x7faeb410de10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faea80096d0 tx=0x7faea8009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 shutdown_connections 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fae9c06b1c0 0x7fae9c06d670 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb40fbe40 0x7faeb410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 --2- 172.21.15.64:0/1630417181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb40fcfb0 0x7faeb410de10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 >> 172.21.15.64:0/1630417181 conn(0x7faeb40f74e0 msgr2=0x7faeb4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:57.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 shutdown_connections 2024-02-16T22:57:57.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.234+0000 7faeb93a0700 1 -- 172.21.15.64:0/1630417181 wait complete. 2024-02-16T22:57:57.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:56 smithi192 ceph-mon[36254]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:57.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:56 smithi192 ceph-mon[36254]: pgmap v63: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 379 op/s 2024-02-16T22:57:57.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:56 smithi192 ceph-mon[36254]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.402+0000 7f541d2f1700 1 -- 172.21.15.64:0/2595759273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 msgr2=0x7f5410096250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.402+0000 7f541d2f1700 1 --2- 172.21.15.64:0/2595759273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 0x7f5410096250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5404004940 tx=0x7f5404009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.402+0000 7f541d2f1700 1 -- 172.21.15.64:0/2595759273 shutdown_connections 2024-02-16T22:57:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.402+0000 7f541d2f1700 1 --2- 172.21.15.64:0/2595759273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 0x7f5410096250 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.402+0000 7f541d2f1700 1 --2- 172.21.15.64:0/2595759273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5410094bf0 0x7f5410095000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.402+0000 7f541d2f1700 1 -- 172.21.15.64:0/2595759273 >> 172.21.15.64:0/2595759273 conn(0x7f54100902b0 msgr2=0x7f54100926e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.403+0000 7f541d2f1700 1 -- 172.21.15.64:0/2595759273 shutdown_connections 2024-02-16T22:57:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.403+0000 7f541d2f1700 1 -- 172.21.15.64:0/2595759273 wait complete. 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.403+0000 7f541d2f1700 1 Processor -- start 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f541d2f1700 1 -- start start 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f541d2f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 0x7f54100a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f541d2f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54100a7c60 0x7f54100acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f541d2f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54100a81b0 con 0x7f5410095e00 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f541d2f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54100a82f0 con 0x7f54100a7c60 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f5417fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 0x7f54100a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f5417fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 0x7f54100a7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56390/0 (socket says 172.21.15.64:56390) 2024-02-16T22:57:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f54177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54100a7c60 0x7f54100acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.404+0000 7f5417fff700 1 -- 172.21.15.64:0/2705522295 learned_addr learned my addr 172.21.15.64:0/2705522295 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:57.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.405+0000 7f5417fff700 1 -- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54100a7c60 msgr2=0x7f54100acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:57.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.405+0000 7f5417fff700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54100a7c60 0x7f54100acce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.406+0000 7f5417fff700 1 -- 172.21.15.64:0/2705522295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404005040 con 0x7f5410095e00 2024-02-16T22:57:57.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.406+0000 7f5417fff700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 0x7f54100a7720 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f540c00b510 tx=0x7f540c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:57.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.407+0000 7f5414ff9700 1 -- 172.21.15.64:0/2705522295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f540c00f6f0 con 0x7f5410095e00 2024-02-16T22:57:57.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.407+0000 7f5414ff9700 1 -- 172.21.15.64:0/2705522295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f540c00fd20 con 0x7f5410095e00 2024-02-16T22:57:57.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.408+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54100ad2e0 con 0x7f5410095e00 2024-02-16T22:57:57.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.408+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54100ad820 con 0x7f5410095e00 2024-02-16T22:57:57.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.408+0000 7f5414ff9700 1 -- 172.21.15.64:0/2705522295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f540c00f6f0 con 0x7f5410095e00 2024-02-16T22:57:57.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.411+0000 7f5414ff9700 1 -- 172.21.15.64:0/2705522295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 88916+0+0 (secure 0 0 0) 0x7f540c011400 con 0x7f5410095e00 2024-02-16T22:57:57.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.411+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5410099ea0 con 0x7f5410095e00 2024-02-16T22:57:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.411+0000 7f5414ff9700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f540806b2d0 0x7f540806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.411+0000 7f5414ff9700 1 -- 172.21.15.64:0/2705522295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f540c088c60 con 0x7f5410095e00 2024-02-16T22:57:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.412+0000 7f54177fe700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f540806b2d0 0x7f540806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.413+0000 7f54177fe700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f540806b2d0 0x7f540806d780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f540400b3b0 tx=0x7f5404009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:57.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.420+0000 7f5414ff9700 1 -- 172.21.15.64:0/2705522295 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f540c04c620 con 0x7f5410095e00 2024-02-16T22:57:57.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.730+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f541009d770 con 0x7f5410095e00 2024-02-16T22:57:57.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.730+0000 7f5414ff9700 1 -- 172.21.15.64:0/2705522295 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f540c04c620 con 0x7f5410095e00 2024-02-16T22:57:57.731 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:57:57.731 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:57:57.731 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:57:57.732 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:57:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:57:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:57:57.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.732+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f540806b2d0 msgr2=0x7f540806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:57.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.732+0000 7f541d2f1700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f540806b2d0 0x7f540806d780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f540400b3b0 tx=0x7f5404009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:57.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.732+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 msgr2=0x7f54100a7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:57.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.732+0000 7f541d2f1700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 0x7f54100a7720 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f540c00b510 tx=0x7f540c009120 comp rx=0 tx=0).stop 2024-02-16T22:57:57.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.733+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 shutdown_connections 2024-02-16T22:57:57.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.733+0000 7f541d2f1700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410095e00 0x7f54100a7720 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.733+0000 7f541d2f1700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f540806b2d0 0x7f540806d780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.733+0000 7f541d2f1700 1 --2- 172.21.15.64:0/2705522295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54100a7c60 0x7f54100acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.733+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 >> 172.21.15.64:0/2705522295 conn(0x7f54100902b0 msgr2=0x7f5410097020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:57.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.733+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 shutdown_connections 2024-02-16T22:57:57.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.733+0000 7f541d2f1700 1 -- 172.21.15.64:0/2705522295 wait complete. 2024-02-16T22:57:57.736 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:57:57.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.870+0000 7f24343d7700 1 -- 172.21.15.64:0/2986508513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fb220 msgr2=0x7f242c0fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:57.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.870+0000 7f24343d7700 1 --2- 172.21.15.64:0/2986508513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fb220 0x7f242c0fb630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2428004940 tx=0x7f2428009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:57.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.870+0000 7f24343d7700 1 -- 172.21.15.64:0/2986508513 shutdown_connections 2024-02-16T22:57:57.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.870+0000 7f24343d7700 1 --2- 172.21.15.64:0/2986508513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f242c0fe2b0 0x7f242c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.870+0000 7f24343d7700 1 --2- 172.21.15.64:0/2986508513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fb220 0x7f242c0fb630 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.870+0000 7f24343d7700 1 -- 172.21.15.64:0/2986508513 >> 172.21.15.64:0/2986508513 conn(0x7f242c0f6ca0 msgr2=0x7f242c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:57.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.870+0000 7f24343d7700 1 -- 172.21.15.64:0/2986508513 shutdown_connections 2024-02-16T22:57:57.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.870+0000 7f24343d7700 1 -- 172.21.15.64:0/2986508513 wait complete. 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f24343d7700 1 Processor -- start 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f24343d7700 1 -- start start 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f24343d7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f242c0fb220 0x7f242c10fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f24343d7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fe2b0 0x7f242c1100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f24343d7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1106e0 con 0x7f242c0fe2b0 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f24343d7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c110f80 con 0x7f242c0fb220 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f2431972700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fe2b0 0x7f242c1100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f2431972700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fe2b0 0x7f242c1100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56418/0 (socket says 172.21.15.64:56418) 2024-02-16T22:57:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f2431972700 1 -- 172.21.15.64:0/1342392189 learned_addr learned my addr 172.21.15.64:0/1342392189 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:57.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.871+0000 7f2432173700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f242c0fb220 0x7f242c10fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:57.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.872+0000 7f2431972700 1 -- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f242c0fb220 msgr2=0x7f242c10fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:57.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.872+0000 7f2431972700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f242c0fb220 0x7f242c10fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:57.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.872+0000 7f2431972700 1 -- 172.21.15.64:0/1342392189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428005040 con 0x7f242c0fe2b0 2024-02-16T22:57:57.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.872+0000 7f2431972700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fe2b0 0x7f242c1100e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f241c00eb40 tx=0x7f241c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:57.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.873+0000 7f2422ffd700 1 -- 172.21.15.64:0/1342392189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241c007cc0 con 0x7f242c0fe2b0 2024-02-16T22:57:57.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.873+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c111200 con 0x7f242c0fe2b0 2024-02-16T22:57:57.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.873+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c111740 con 0x7f242c0fe2b0 2024-02-16T22:57:57.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.873+0000 7f2422ffd700 1 -- 172.21.15.64:0/1342392189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f241c00f040 con 0x7f242c0fe2b0 2024-02-16T22:57:57.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.873+0000 7f2422ffd700 1 -- 172.21.15.64:0/1342392189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241c00c9a0 con 0x7f242c0fe2b0 2024-02-16T22:57:57.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.874+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24100052e0 con 0x7f242c0fe2b0 2024-02-16T22:57:57.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.875+0000 7f2422ffd700 1 -- 172.21.15.64:0/1342392189 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 88916+0+0 (secure 0 0 0) 0x7f241c00cb00 con 0x7f242c0fe2b0 2024-02-16T22:57:57.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.876+0000 7f2422ffd700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f241806b1c0 0x7f241806d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:57.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.876+0000 7f2432173700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f241806b1c0 0x7f241806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:57.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.876+0000 7f2432173700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f241806b1c0 0x7f241806d670 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2428009fd0 tx=0x7f2428009e30 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:57.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.878+0000 7f2422ffd700 1 -- 172.21.15.64:0/1342392189 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f241c007e20 con 0x7f242c0fe2b0 2024-02-16T22:57:57.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:57.883+0000 7f2422ffd700 1 -- 172.21.15.64:0/1342392189 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f241c057930 con 0x7f242c0fe2b0 2024-02-16T22:57:58.027 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:57 smithi064 ceph-mon[29226]: from='client.24545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:58.028 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:57 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1630417181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:57:58.028 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:57 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2705522295' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:57:58.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.234+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 --> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2410000bf0 con 0x7f241806b1c0 2024-02-16T22:57:58.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.236+0000 7f2422ffd700 1 -- 172.21.15.64:0/1342392189 <== mgr.24429 v2:172.21.15.192:6828/2862437082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f2410000bf0 con 0x7f241806b1c0 2024-02-16T22:57:58.236 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:57:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:57:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:57:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:57:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [], 2024-02-16T22:57:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "1/21 daemons upgraded", 2024-02-16T22:57:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T22:57:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:57:58.237 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:57:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f241806b1c0 msgr2=0x7f241806d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f241806b1c0 0x7f241806d670 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2428009fd0 tx=0x7f2428009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fe2b0 msgr2=0x7f242c1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fe2b0 0x7f242c1100e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f241c00eb40 tx=0x7f241c00b040 comp rx=0 tx=0).stop 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 shutdown_connections 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7f241806b1c0 0x7f241806d670 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2428009fd0 tx=0x7f2428009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f242c0fb220 0x7f242c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 --2- 172.21.15.64:0/1342392189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242c0fe2b0 0x7f242c1100e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.238+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 >> 172.21.15.64:0/1342392189 conn(0x7f242c0f6ca0 msgr2=0x7f242c0f7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.239+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 shutdown_connections 2024-02-16T22:57:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.239+0000 7f24343d7700 1 -- 172.21.15.64:0/1342392189 wait complete. 2024-02-16T22:57:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:57 smithi192 ceph-mon[36254]: from='client.24545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:58.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:57 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1630417181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:57:58.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:57 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2705522295' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.409+0000 7fadcb59e700 1 -- 172.21.15.64:0/4292995976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 msgr2=0x7fadc40982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.409+0000 7fadcb59e700 1 --2- 172.21.15.64:0/4292995976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 0x7fadc40982f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fadc0004940 tx=0x7fadc0009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.409+0000 7fadcb59e700 1 -- 172.21.15.64:0/4292995976 shutdown_connections 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.410+0000 7fadcb59e700 1 --2- 172.21.15.64:0/4292995976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 0x7fadc40982f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.410+0000 7fadcb59e700 1 --2- 172.21.15.64:0/4292995976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadc4093510 0x7fadc4095930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.410+0000 7fadcb59e700 1 -- 172.21.15.64:0/4292995976 >> 172.21.15.64:0/4292995976 conn(0x7fadc408f1a0 msgr2=0x7fadc40915d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.410+0000 7fadcb59e700 1 -- 172.21.15.64:0/4292995976 shutdown_connections 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.410+0000 7fadcb59e700 1 -- 172.21.15.64:0/4292995976 wait complete. 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.410+0000 7fadcb59e700 1 Processor -- start 2024-02-16T22:57:58.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.411+0000 7fadcb59e700 1 -- start start 2024-02-16T22:57:58.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.411+0000 7fadcb59e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadc4093510 0x7fadc40bca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:58.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.411+0000 7fadcb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 0x7fadc40bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:58.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.411+0000 7fadcb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40bd5c0 con 0x7fadc4095e70 2024-02-16T22:57:58.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.411+0000 7fadcb59e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40bd730 con 0x7fadc4093510 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.411+0000 7fadc9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 0x7fadc40bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.412+0000 7fadc9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 0x7fadc40bcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56428/0 (socket says 172.21.15.64:56428) 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.412+0000 7fadc9d9b700 1 -- 172.21.15.64:0/678180893 learned_addr learned my addr 172.21.15.64:0/678180893 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.412+0000 7fadca59c700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadc4093510 0x7fadc40bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.412+0000 7fadc9d9b700 1 -- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadc4093510 msgr2=0x7fadc40bca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.412+0000 7fadc9d9b700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadc4093510 0x7fadc40bca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.412+0000 7fadc9d9b700 1 -- 172.21.15.64:0/678180893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc0005040 con 0x7fadc4095e70 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.412+0000 7fadca59c700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadc4093510 0x7fadc40bca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:57:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.413+0000 7fadc9d9b700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 0x7fadc40bcfc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000c00 tx=0x7fadc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.414+0000 7fadbaffd700 1 -- 172.21.15.64:0/678180893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc0021070 con 0x7fadc4095e70 2024-02-16T22:57:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.414+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc40c2180 con 0x7fadc4095e70 2024-02-16T22:57:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.414+0000 7fadbaffd700 1 -- 172.21.15.64:0/678180893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadc0026410 con 0x7fadc4095e70 2024-02-16T22:57:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.414+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc40c2660 con 0x7fadc4095e70 2024-02-16T22:57:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.414+0000 7fadbaffd700 1 -- 172.21.15.64:0/678180893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc000e070 con 0x7fadc4095e70 2024-02-16T22:57:58.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.415+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc40a3cd0 con 0x7fadc4095e70 2024-02-16T22:57:58.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.416+0000 7fadbaffd700 1 -- 172.21.15.64:0/678180893 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 88916+0+0 (secure 0 0 0) 0x7fadc00269d0 con 0x7fadc4095e70 2024-02-16T22:57:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.416+0000 7fadbaffd700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fadb406b230 0x7fadb406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:57:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.417+0000 7fadbaffd700 1 -- 172.21.15.64:0/678180893 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fadc008aee0 con 0x7fadc4095e70 2024-02-16T22:57:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.417+0000 7fadca59c700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fadb406b230 0x7fadb406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:57:58.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.417+0000 7fadca59c700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fadb406b230 0x7fadb406d6e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009fd0 tx=0x7fadbc00b040 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:57:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.423+0000 7fadbaffd700 1 -- 172.21.15.64:0/678180893 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fadc0058d30 con 0x7fadc4095e70 2024-02-16T22:57:58.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.876+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fadc40c28a0 con 0x7fadc4095e70 2024-02-16T22:57:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.877+0000 7fadbaffd700 1 -- 172.21.15.64:0/678180893 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fadc00588c0 con 0x7fadc4095e70 2024-02-16T22:57:58.878 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:57:58.878 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:57:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:57:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fadb406b230 msgr2=0x7fadb406d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fadb406b230 0x7fadb406d6e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009fd0 tx=0x7fadbc00b040 comp rx=0 tx=0).stop 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 msgr2=0x7fadc40bcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 0x7fadc40bcfc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000c00 tx=0x7fadc0009e30 comp rx=0 tx=0).stop 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 shutdown_connections 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:6828/2862437082,v1:172.21.15.192:6829/2862437082] conn(0x7fadb406b230 0x7fadb406d6e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadc4093510 0x7fadc40bca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 --2- 172.21.15.64:0/678180893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc4095e70 0x7fadc40bcfc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 >> 172.21.15.64:0/678180893 conn(0x7fadc408f1a0 msgr2=0x7fadc4091580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 shutdown_connections 2024-02-16T22:57:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:57:58.880+0000 7fadcb59e700 1 -- 172.21.15.64:0/678180893 wait complete. 2024-02-16T22:57:59.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:59 smithi192 ceph-mon[36254]: pgmap v64: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 478 op/s 2024-02-16T22:57:59.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:59 smithi192 ceph-mon[36254]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:59.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:59 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:59.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:59 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:59.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:59 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:57:59.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:59 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/678180893' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:57:59.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:59 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:59.437 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:57:59 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:59.578 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:59 smithi064 ceph-mon[29226]: pgmap v64: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 478 op/s 2024-02-16T22:57:59.578 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:59 smithi064 ceph-mon[29226]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:57:59.578 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:59 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:59.578 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:59 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:59.578 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:59 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:57:59.578 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:59 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/678180893' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:57:59.578 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:59 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:57:59.578 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:57:59 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:01.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:01 smithi192 ceph-mon[36254]: pgmap v65: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 277 op/s 2024-02-16T22:58:01.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:01 smithi064 ceph-mon[29226]: pgmap v65: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 277 op/s 2024-02-16T22:58:02.681 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:02 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:02.681 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:02 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:02.681 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:02 smithi064 ceph-mon[29226]: pgmap v66: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 292 op/s 2024-02-16T22:58:02.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:02 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:02.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:02 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:02.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:02 smithi192 ceph-mon[36254]: pgmap v66: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 292 op/s 2024-02-16T22:58:05.228 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:04 smithi064 ceph-mon[29226]: pgmap v67: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-02-16T22:58:05.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:04 smithi192 ceph-mon[36254]: pgmap v67: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-02-16T22:58:07.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:06 smithi064 ceph-mon[29226]: pgmap v68: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.1 MiB/s wr, 173 op/s 2024-02-16T22:58:07.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:06 smithi064 ceph-mon[29226]: Standby manager daemon smithi064.kubmlk restarted 2024-02-16T22:58:07.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:06 smithi064 ceph-mon[29226]: Standby manager daemon smithi064.kubmlk started 2024-02-16T22:58:07.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:06 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi064.kubmlk/crt"}]: dispatch 2024-02-16T22:58:07.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:06 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:58:07.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:06 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi064.kubmlk/key"}]: dispatch 2024-02-16T22:58:07.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:06 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:58:07.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:06 smithi192 ceph-mon[36254]: pgmap v68: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.1 MiB/s wr, 173 op/s 2024-02-16T22:58:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:06 smithi192 ceph-mon[36254]: Standby manager daemon smithi064.kubmlk restarted 2024-02-16T22:58:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:06 smithi192 ceph-mon[36254]: Standby manager daemon smithi064.kubmlk started 2024-02-16T22:58:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:06 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi064.kubmlk/crt"}]: dispatch 2024-02-16T22:58:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:06 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:58:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:06 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi064.kubmlk/key"}]: dispatch 2024-02-16T22:58:07.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:06 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:58:08.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:07 smithi192 ceph-mon[36254]: mgrmap e27: smithi192.pauvih(active, since 2m), standbys: smithi064.kubmlk 2024-02-16T22:58:08.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:07 smithi064 ceph-mon[29226]: mgrmap e27: smithi192.pauvih(active, since 2m), standbys: smithi064.kubmlk 2024-02-16T22:58:09.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:08 smithi192 ceph-mon[36254]: pgmap v69: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 38 KiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-02-16T22:58:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:08 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:09.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:08 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:09.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:08 smithi064 ceph-mon[29226]: pgmap v69: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 38 KiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-02-16T22:58:09.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:08 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:09.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:08 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:10.413 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:10 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:10.413 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:10 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:10.413 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:10 smithi064 ceph-mon[29226]: pgmap v70: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-02-16T22:58:10.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:10 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:10.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:10 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:10.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:10 smithi192 ceph-mon[36254]: pgmap v70: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-02-16T22:58:11.253 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-16T22:58:11.253 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-16T22:58:11.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:11 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:58:11.579 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:11 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:58:11.587 DEBUG:teuthology.parallel:result is None 2024-02-16T22:58:12.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:12 smithi192 ceph-mon[36254]: pgmap v71: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-02-16T22:58:12.818 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:12 smithi064 ceph-mon[29226]: pgmap v71: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-02-16T22:58:15.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:14 smithi192 ceph-mon[36254]: pgmap v72: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.6 MiB/s wr, 226 op/s 2024-02-16T22:58:15.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:14 smithi064 ceph-mon[29226]: pgmap v72: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.6 MiB/s wr, 226 op/s 2024-02-16T22:58:17.153 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-16T22:58:17.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-16T22:58:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:58:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr fail", "who": "smithi192.pauvih"}]: dispatch 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr fail", "who": "smithi192.pauvih"}]: dispatch 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd='[{"prefix": "mgr fail", "who": "smithi192.pauvih"}]': finished 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: mgrmap e28: smithi064.kubmlk(active, starting, since 0.00710698s) 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T22:58:17.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: Manager daemon smithi064.kubmlk is now available 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:17.287 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:17.288 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:58:17.288 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:58:17.288 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:58:17.288 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:58:17.288 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:16 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:58:17.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:17.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:17.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 172.21.15.192:0/3361871181' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr fail", "who": "smithi192.pauvih"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "mgr fail", "who": "smithi192.pauvih"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.24429 ' entity='mgr.smithi192.pauvih' cmd='[{"prefix": "mgr fail", "who": "smithi192.pauvih"}]': finished 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: mgrmap e28: smithi064.kubmlk(active, starting, since 0.00710698s) 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T22:58:17.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T22:58:17.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T22:58:17.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:58:17.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:58:17.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:58:17.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:58:17.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: Manager daemon smithi064.kubmlk is now available 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:58:17.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:58:17.372 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:16 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:58:17.600 DEBUG:teuthology.parallel:result is None 2024-02-16T22:58:17.601 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-16T22:58:17.632 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-16T22:58:17.633 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-16T22:58:17.670 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-16T22:58:17.670 DEBUG:teuthology.parallel:result is None 2024-02-16T22:58:18.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:18 smithi192 ceph-mon[36254]: [16/Feb/2024:22:58:16] ENGINE Bus STARTING 2024-02-16T22:58:18.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:18 smithi192 ceph-mon[36254]: [16/Feb/2024:22:58:16] ENGINE Serving on http://172.21.15.64:8765 2024-02-16T22:58:18.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:18 smithi192 ceph-mon[36254]: [16/Feb/2024:22:58:16] ENGINE Serving on https://172.21.15.64:7150 2024-02-16T22:58:18.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:18 smithi192 ceph-mon[36254]: [16/Feb/2024:22:58:16] ENGINE Bus STARTED 2024-02-16T22:58:18.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:18 smithi192 ceph-mon[36254]: mgrmap e29: smithi064.kubmlk(active, since 1.00887s) 2024-02-16T22:58:18.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:18 smithi192 ceph-mon[36254]: pgmap v3: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:18 smithi064 ceph-mon[29226]: [16/Feb/2024:22:58:16] ENGINE Bus STARTING 2024-02-16T22:58:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:18 smithi064 ceph-mon[29226]: [16/Feb/2024:22:58:16] ENGINE Serving on http://172.21.15.64:8765 2024-02-16T22:58:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:18 smithi064 ceph-mon[29226]: [16/Feb/2024:22:58:16] ENGINE Serving on https://172.21.15.64:7150 2024-02-16T22:58:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:18 smithi064 ceph-mon[29226]: [16/Feb/2024:22:58:16] ENGINE Bus STARTED 2024-02-16T22:58:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:18 smithi064 ceph-mon[29226]: mgrmap e29: smithi064.kubmlk(active, since 1.00887s) 2024-02-16T22:58:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:18 smithi064 ceph-mon[29226]: pgmap v3: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:19.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:19 smithi192 ceph-mon[36254]: pgmap v4: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:19.446 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:19 smithi064 ceph-mon[29226]: pgmap v4: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:20.450 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:20 smithi192 ceph-mon[36254]: mgrmap e30: smithi064.kubmlk(active, since 3s) 2024-02-16T22:58:20.450 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:20 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:20.450 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:20 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:20.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:20 smithi064 ceph-mon[29226]: mgrmap e30: smithi064.kubmlk(active, since 3s) 2024-02-16T22:58:20.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:20 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:20.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:20 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:21.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:21 smithi192 ceph-mon[36254]: pgmap v5: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:21.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:21 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:21.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:21 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:21.548 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:21 smithi064 ceph-mon[29226]: pgmap v5: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:21.548 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:21 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:21.548 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:21 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:23.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:23 smithi192 ceph-mon[36254]: pgmap v6: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:23.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:23 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:23.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:23 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:23.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:23 smithi064 ceph-mon[29226]: pgmap v6: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:23.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:23 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:23.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:23 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:24 smithi192 ceph-mon[36254]: Standby manager daemon smithi192.pauvih started 2024-02-16T22:58:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:24 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/169333798' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:58:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:24 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/169333798' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:58:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:24 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/169333798' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:58:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:24 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/169333798' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:58:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:24 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:24 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:24 smithi064 ceph-mon[29226]: Standby manager daemon smithi192.pauvih started 2024-02-16T22:58:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:24 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/169333798' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:58:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:24 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/169333798' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:58:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:24 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/169333798' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:58:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:24 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/169333798' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:58:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:24 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:24.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:24 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:25.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:25 smithi064 ceph-mon[29226]: pgmap v7: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 35 KiB/s wr, 67 op/s 2024-02-16T22:58:25.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:25 smithi064 ceph-mon[29226]: mgrmap e31: smithi064.kubmlk(active, since 8s), standbys: smithi192.pauvih 2024-02-16T22:58:25.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:25 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi192.pauvih", "id": "smithi192.pauvih"}]: dispatch 2024-02-16T22:58:25.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:25 smithi192 ceph-mon[36254]: pgmap v7: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 35 KiB/s wr, 67 op/s 2024-02-16T22:58:25.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:25 smithi192 ceph-mon[36254]: mgrmap e31: smithi064.kubmlk(active, since 8s), standbys: smithi192.pauvih 2024-02-16T22:58:25.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:25 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi192.pauvih", "id": "smithi192.pauvih"}]: dispatch 2024-02-16T22:58:27.319 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:27 smithi064 ceph-mon[29226]: pgmap v8: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 32 KiB/s rd, 27 KiB/s wr, 52 op/s 2024-02-16T22:58:27.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:27 smithi192 ceph-mon[36254]: pgmap v8: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 32 KiB/s rd, 27 KiB/s wr, 52 op/s 2024-02-16T22:58:28.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 -- 172.21.15.64:0/2045148498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 msgr2=0x7fa28c0fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:28.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 --2- 172.21.15.64:0/2045148498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 0x7fa28c0fc260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa274004940 tx=0x7fa274009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:28.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 -- 172.21.15.64:0/2045148498 shutdown_connections 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 --2- 172.21.15.64:0/2045148498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcfc0 0x7fa28c0fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 --2- 172.21.15.64:0/2045148498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 0x7fa28c0fc260 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 -- 172.21.15.64:0/2045148498 >> 172.21.15.64:0/2045148498 conn(0x7fa28c0f7510 msgr2=0x7fa28c0f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 -- 172.21.15.64:0/2045148498 shutdown_connections 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 -- 172.21.15.64:0/2045148498 wait complete. 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 Processor -- start 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.989+0000 7fa290858700 1 -- start start 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa290858700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 0x7fa28c10fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa290858700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcfc0 0x7fa28c110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa290858700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c110690 con 0x7fa28c0fbe50 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa290858700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c110f30 con 0x7fa28c0fcfc0 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa28a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 0x7fa28c10fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa28a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 0x7fa28c10fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39048/0 (socket says 172.21.15.64:39048) 2024-02-16T22:58:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa28a59c700 1 -- 172.21.15.64:0/1076237861 learned_addr learned my addr 172.21.15.64:0/1076237861 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa289d9b700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcfc0 0x7fa28c110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa28a59c700 1 -- 172.21.15.64:0/1076237861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcfc0 msgr2=0x7fa28c110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa28a59c700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcfc0 0x7fa28c110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa28a59c700 1 -- 172.21.15.64:0/1076237861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274005040 con 0x7fa28c0fbe50 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa28a59c700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 0x7fa28c10fb50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa2740093b0 tx=0x7fa274009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa282ffd700 1 -- 172.21.15.64:0/1076237861 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa27400e040 con 0x7fa28c0fbe50 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa282ffd700 1 -- 172.21.15.64:0/1076237861 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa27400bcc0 con 0x7fa28c0fbe50 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c1111b0 con 0x7fa28c0fbe50 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa282ffd700 1 -- 172.21.15.64:0/1076237861 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa27401c630 con 0x7fa28c0fbe50 2024-02-16T22:58:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.990+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c1116f0 con 0x7fa28c0fbe50 2024-02-16T22:58:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.991+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c04bed0 con 0x7fa28c0fbe50 2024-02-16T22:58:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.992+0000 7fa282ffd700 1 -- 172.21.15.64:0/1076237861 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 91309+0+0 (secure 0 0 0) 0x7fa274013980 con 0x7fa28c0fbe50 2024-02-16T22:58:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.992+0000 7fa282ffd700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7fa27806d200 0x7fa27806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.992+0000 7fa282ffd700 1 -- 172.21.15.64:0/1076237861 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fa27408da20 con 0x7fa28c0fbe50 2024-02-16T22:58:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.992+0000 7fa289d9b700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7fa27806d200 0x7fa27806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.993+0000 7fa289d9b700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7fa27806d200 0x7fa27806f6b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004f40 tx=0x7fa27c00b040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:28.997+0000 7fa282ffd700 1 -- 172.21.15.64:0/1076237861 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa27405b550 con 0x7fa28c0fbe50 2024-02-16T22:58:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:58:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:58:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:58:29.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:58:29.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:58:29.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: pgmap v9: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 52 KiB/s wr, 44 op/s 2024-02-16T22:58:29.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:29.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:28 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:29.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.219+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 --> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa28c111340 con 0x7fa27806d200 2024-02-16T22:58:29.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.222+0000 7fa282ffd700 1 -- 172.21.15.64:0/1076237861 <== mgr.14772 v2:172.21.15.64:6800/1223017301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fa28c111340 con 0x7fa27806d200 2024-02-16T22:58:29.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.224+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7fa27806d200 msgr2=0x7fa27806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.224+0000 7fa290858700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7fa27806d200 0x7fa27806f6b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004f40 tx=0x7fa27c00b040 comp rx=0 tx=0).stop 2024-02-16T22:58:29.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.224+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 msgr2=0x7fa28c10fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.224+0000 7fa290858700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 0x7fa28c10fb50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa2740093b0 tx=0x7fa274009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:29.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.225+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 shutdown_connections 2024-02-16T22:58:29.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.225+0000 7fa290858700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7fa27806d200 0x7fa27806f6b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.225+0000 7fa290858700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fbe50 0x7fa28c10fb50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.225+0000 7fa290858700 1 --2- 172.21.15.64:0/1076237861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcfc0 0x7fa28c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.225+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 >> 172.21.15.64:0/1076237861 conn(0x7fa28c0f7510 msgr2=0x7fa28c065f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:29.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.225+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 shutdown_connections 2024-02-16T22:58:29.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.225+0000 7fa290858700 1 -- 172.21.15.64:0/1076237861 wait complete. 2024-02-16T22:58:29.236 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:58:29.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: pgmap v9: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 52 KiB/s wr, 44 op/s 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:29.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:28 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.331+0000 7faacfc18700 1 -- 172.21.15.64:0/101202427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fbe40 msgr2=0x7faac80fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.331+0000 7faacfc18700 1 --2- 172.21.15.64:0/101202427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fbe40 0x7faac80fc250 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faac4004940 tx=0x7faac4009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.331+0000 7faacfc18700 1 -- 172.21.15.64:0/101202427 shutdown_connections 2024-02-16T22:58:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.331+0000 7faacfc18700 1 --2- 172.21.15.64:0/101202427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac80fcfb0 0x7faac80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.331+0000 7faacfc18700 1 --2- 172.21.15.64:0/101202427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fbe40 0x7faac80fc250 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.331+0000 7faacfc18700 1 -- 172.21.15.64:0/101202427 >> 172.21.15.64:0/101202427 conn(0x7faac80f7500 msgr2=0x7faac80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.331+0000 7faacfc18700 1 -- 172.21.15.64:0/101202427 shutdown_connections 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.331+0000 7faacfc18700 1 -- 172.21.15.64:0/101202427 wait complete. 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacfc18700 1 Processor -- start 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacfc18700 1 -- start start 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacfc18700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac80fbe40 0x7faac8112a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacfc18700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fcfb0 0x7faac8114fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacfc18700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac8066be0 con 0x7faac80fcfb0 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacfc18700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac8066d20 con 0x7faac80fbe40 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacd1b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fcfb0 0x7faac8114fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacd1b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fcfb0 0x7faac8114fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39072/0 (socket says 172.21.15.64:39072) 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacd1b3700 1 -- 172.21.15.64:0/1989906450 learned_addr learned my addr 172.21.15.64:0/1989906450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:58:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.332+0000 7faacd9b4700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac80fbe40 0x7faac8112a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:29.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faacd1b3700 1 -- 172.21.15.64:0/1989906450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac80fbe40 msgr2=0x7faac8112a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faacd1b3700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac80fbe40 0x7faac8112a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faacd1b3700 1 -- 172.21.15.64:0/1989906450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac4005040 con 0x7faac80fcfb0 2024-02-16T22:58:29.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faacd1b3700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fcfb0 0x7faac8114fb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faab800c830 tx=0x7faab8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:29.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faabe7fc700 1 -- 172.21.15.64:0/1989906450 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab80106f0 con 0x7faac80fcfb0 2024-02-16T22:58:29.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faabe7fc700 1 -- 172.21.15.64:0/1989906450 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faab8010d20 con 0x7faac80fcfb0 2024-02-16T22:58:29.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac81154f0 con 0x7faac80fcfb0 2024-02-16T22:58:29.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faabe7fc700 1 -- 172.21.15.64:0/1989906450 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab80113a0 con 0x7faac80fcfb0 2024-02-16T22:58:29.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.333+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac81158f0 con 0x7faac80fcfb0 2024-02-16T22:58:29.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.334+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac804bed0 con 0x7faac80fcfb0 2024-02-16T22:58:29.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.335+0000 7faabe7fc700 1 -- 172.21.15.64:0/1989906450 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 91309+0+0 (secure 0 0 0) 0x7faab8012400 con 0x7faac80fcfb0 2024-02-16T22:58:29.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.335+0000 7faabe7fc700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7faab406d200 0x7faab406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:29.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.335+0000 7faabe7fc700 1 -- 172.21.15.64:0/1989906450 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5980+0+0 (secure 0 0 0) 0x7faab808c5d0 con 0x7faac80fcfb0 2024-02-16T22:58:29.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.335+0000 7faacd9b4700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7faab406d200 0x7faab406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:29.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.336+0000 7faacd9b4700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7faab406d200 0x7faab406f6b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faac4004890 tx=0x7faac4009e30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:29.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.344+0000 7faabe7fc700 1 -- 172.21.15.64:0/1989906450 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7faab8056640 con 0x7faac80fcfb0 2024-02-16T22:58:29.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.570+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 --> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faac8101990 con 0x7faab406d200 2024-02-16T22:58:29.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.571+0000 7faabe7fc700 1 -- 172.21.15.64:0/1989906450 <== mgr.14772 v2:172.21.15.64:6800/1223017301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7faac8101990 con 0x7faab406d200 2024-02-16T22:58:29.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7faab406d200 msgr2=0x7faab406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7faab406d200 0x7faab406f6b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faac4004890 tx=0x7faac4009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:29.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fcfb0 msgr2=0x7faac8114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fcfb0 0x7faac8114fb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faab800c830 tx=0x7faab8004470 comp rx=0 tx=0).stop 2024-02-16T22:58:29.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 shutdown_connections 2024-02-16T22:58:29.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac80fbe40 0x7faac8112a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7faab406d200 0x7faab406f6b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 --2- 172.21.15.64:0/1989906450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac80fcfb0 0x7faac8114fb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.574+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 >> 172.21.15.64:0/1989906450 conn(0x7faac80f7500 msgr2=0x7faac8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:29.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.575+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 shutdown_connections 2024-02-16T22:58:29.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.575+0000 7faacfc18700 1 -- 172.21.15.64:0/1989906450 wait complete. 2024-02-16T22:58:29.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.683+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2271492161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e381003a0 msgr2=0x7f8e38102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.683+0000 7f8e3f63e700 1 --2- 172.21.15.64:0/2271492161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e381003a0 0x7f8e38102780 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8e28004880 tx=0x7f8e28009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.683+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2271492161 shutdown_connections 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.683+0000 7f8e3f63e700 1 --2- 172.21.15.64:0/2271492161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e381003a0 0x7f8e38102780 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.683+0000 7f8e3f63e700 1 --2- 172.21.15.64:0/2271492161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e380fda80 0x7f8e380ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.683+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2271492161 >> 172.21.15.64:0/2271492161 conn(0x7f8e380f7500 msgr2=0x7f8e380f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.683+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2271492161 shutdown_connections 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.683+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2271492161 wait complete. 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3f63e700 1 Processor -- start 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3f63e700 1 -- start start 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3f63e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380fda80 0x7f8e380fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3f63e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e381003a0 0x7f8e380fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3f63e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380fbf90 con 0x7f8e380fda80 2024-02-16T22:58:29.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3f63e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380fc100 con 0x7f8e381003a0 2024-02-16T22:58:29.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3d3da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380fda80 0x7f8e380fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:29.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3d3da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380fda80 0x7f8e380fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39080/0 (socket says 172.21.15.64:39080) 2024-02-16T22:58:29.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3d3da700 1 -- 172.21.15.64:0/2062930756 learned_addr learned my addr 172.21.15.64:0/2062930756 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:58:29.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3cbd9700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e381003a0 0x7f8e380fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:29.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3d3da700 1 -- 172.21.15.64:0/2062930756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e381003a0 msgr2=0x7f8e380fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3d3da700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e381003a0 0x7f8e380fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3d3da700 1 -- 172.21.15.64:0/2062930756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e34005040 con 0x7f8e380fda80 2024-02-16T22:58:29.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e3d3da700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380fda80 0x7f8e380fd510 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e340096b0 tx=0x7f8e34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:29.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.684+0000 7f8e2dffb700 1 -- 172.21.15.64:0/2062930756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e34017070 con 0x7f8e380fda80 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.685+0000 7f8e2dffb700 1 -- 172.21.15.64:0/2062930756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e3400f040 con 0x7f8e380fda80 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.685+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e28005040 con 0x7f8e380fda80 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.685+0000 7f8e2dffb700 1 -- 172.21.15.64:0/2062930756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e34013be0 con 0x7f8e380fda80 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.685+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e380fc730 con 0x7f8e380fda80 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.686+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3804bed0 con 0x7f8e380fda80 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.686+0000 7f8e2dffb700 1 -- 172.21.15.64:0/2062930756 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f8e3400e840 con 0x7f8e380fda80 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.687+0000 7f8e2dffb700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8e2406d200 0x7f8e2406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.687+0000 7f8e2dffb700 1 -- 172.21.15.64:0/2062930756 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f8e3408c520 con 0x7f8e380fda80 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.687+0000 7f8e3cbd9700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8e2406d200 0x7f8e2406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.687+0000 7f8e3cbd9700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8e2406d200 0x7f8e2406f6b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009fd0 tx=0x7f8e28009e30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:29.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.691+0000 7f8e2dffb700 1 -- 172.21.15.64:0/2062930756 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f8e34059fb0 con 0x7f8e380fda80 2024-02-16T22:58:29.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.908+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 --> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e38101000 con 0x7f8e2406d200 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.918+0000 7f8e2dffb700 1 -- 172.21.15.64:0/2062930756 <== mgr.14772 v2:172.21.15.64:6800/1223017301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8e38101000 con 0x7f8e2406d200 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (99s) 6s ago 10m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (10m) 6s ago 10m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (9m) 10s ago 9m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (78s) 6s ago 9m 79.8M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (4m) 6s ago 4m 16.6M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (4m) 6s ago 4m 179M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:58:29.919 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (4m) 10s ago 4m 15.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (4m) 10s ago 4m 15.3M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (36s) 6s ago 11m 474M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283 running (2m) 10s ago 9m 74.7M - 19.0.0-1450-gf78a58c0 f081f847070a 05118581c81b 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (11m) 6s ago 11m 50.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (9m) 10s ago 9m 39.1M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (115s) 6s ago 10m 15.2M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (44s) 10s ago 9m 8564k - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (8m) 6s ago 8m 209M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (7m) 6s ago 7m 214M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (7m) 6s ago 7m 174M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (7m) 10s ago 7m 232M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (6m) 10s ago 6m 209M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (6m) 10s ago 6m 185M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:58:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (47s) 6s ago 9m 44.7M - 2.43.0 a07b618ecd1d 436bfb06b95a 2024-02-16T22:58:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8e2406d200 msgr2=0x7f8e2406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8e2406d200 0x7f8e2406f6b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009fd0 tx=0x7f8e28009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380fda80 msgr2=0x7f8e380fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380fda80 0x7f8e380fd510 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e340096b0 tx=0x7f8e34009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 shutdown_connections 2024-02-16T22:58:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8e2406d200 0x7f8e2406f6b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380fda80 0x7f8e380fd510 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 --2- 172.21.15.64:0/2062930756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e381003a0 0x7f8e380fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:29.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 >> 172.21.15.64:0/2062930756 conn(0x7f8e380f7500 msgr2=0x7f8e380f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:29.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 shutdown_connections 2024-02-16T22:58:29.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:29.920+0000 7f8e3f63e700 1 -- 172.21.15.64:0/2062930756 wait complete. 2024-02-16T22:58:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2708716485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fbe40 msgr2=0x7f6d080fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 --2- 172.21.15.64:0/2708716485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fbe40 0x7f6d080fc250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc004940 tx=0x7f6cfc009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2708716485 shutdown_connections 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 --2- 172.21.15.64:0/2708716485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d080fcfb0 0x7f6d080fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 --2- 172.21.15.64:0/2708716485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fbe40 0x7f6d080fc250 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2708716485 >> 172.21.15.64:0/2708716485 conn(0x7f6d080f7500 msgr2=0x7f6d080f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2708716485 shutdown_connections 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2708716485 wait complete. 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 Processor -- start 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.024+0000 7f6d0ee78700 1 -- start start 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d0ee78700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d080fbe40 0x7f6d08110660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d0ee78700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fcfb0 0x7f6d08112bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d0ee78700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08066cf0 con 0x7f6d080fcfb0 2024-02-16T22:58:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d0ee78700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08066e30 con 0x7f6d080fbe40 2024-02-16T22:58:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d07fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fcfb0 0x7f6d08112bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d07fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fcfb0 0x7f6d08112bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39106/0 (socket says 172.21.15.64:39106) 2024-02-16T22:58:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d07fff700 1 -- 172.21.15.64:0/2397895114 learned_addr learned my addr 172.21.15.64:0/2397895114 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:58:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d0cc14700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d080fbe40 0x7f6d08110660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d07fff700 1 -- 172.21.15.64:0/2397895114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d080fbe40 msgr2=0x7f6d08110660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d07fff700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d080fbe40 0x7f6d08110660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d07fff700 1 -- 172.21.15.64:0/2397895114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc005040 con 0x7f6d080fcfb0 2024-02-16T22:58:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d07fff700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fcfb0 0x7f6d08112bb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4002a50 tx=0x7f6cf40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d057fa700 1 -- 172.21.15.64:0/2397895114 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf4007cc0 con 0x7f6d080fcfb0 2024-02-16T22:58:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d057fa700 1 -- 172.21.15.64:0/2397895114 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cf400f040 con 0x7f6d080fcfb0 2024-02-16T22:58:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d081131b0 con 0x7f6d080fcfb0 2024-02-16T22:58:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d057fa700 1 -- 172.21.15.64:0/2397895114 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf400da30 con 0x7f6d080fcfb0 2024-02-16T22:58:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.025+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d081136f0 con 0x7f6d080fcfb0 2024-02-16T22:58:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.026+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0804bed0 con 0x7f6d080fcfb0 2024-02-16T22:58:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.027+0000 7f6d057fa700 1 -- 172.21.15.64:0/2397895114 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f6cf400dbf0 con 0x7f6d080fcfb0 2024-02-16T22:58:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.027+0000 7f6d057fa700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f6cf806d200 0x7f6cf806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.027+0000 7f6d057fa700 1 -- 172.21.15.64:0/2397895114 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f6cf408c550 con 0x7f6d080fcfb0 2024-02-16T22:58:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.027+0000 7f6d0cc14700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f6cf806d200 0x7f6cf806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.028+0000 7f6d0cc14700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f6cf806d200 0x7f6cf806f6b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc004890 tx=0x7f6cfc009e30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:30.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.032+0000 7f6d057fa700 1 -- 172.21.15.64:0/2397895114 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f6cf4056430 con 0x7f6d080fcfb0 2024-02-16T22:58:30.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:58:30.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:58:30.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:58:30.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:58:30.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:30.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:30.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:30.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: Reconfiguring prometheus.smithi064 (dependencies changed)... 2024-02-16T22:58:30.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:29 smithi064 ceph-mon[29226]: Reconfiguring daemon prometheus.smithi064 on smithi064 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: Reconfiguring prometheus.smithi064 (dependencies changed)... 2024-02-16T22:58:30.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:29 smithi192 ceph-mon[36254]: Reconfiguring daemon prometheus.smithi064 on smithi064 2024-02-16T22:58:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.325+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d08104930 con 0x7f6d080fcfb0 2024-02-16T22:58:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.326+0000 7f6d057fa700 1 -- 172.21.15.64:0/2397895114 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f6cf400dea0 con 0x7f6d080fcfb0 2024-02-16T22:58:30.327 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:58:30.327 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:58:30.328 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T22:58:30.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:58:30.328 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:58:30.328 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:58:30.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:58:30.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:58:30.329 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:58:30.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:58:30.329 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:58:30.329 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:58:30.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:58:30.329 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:58:30.330 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 12, 2024-02-16T22:58:30.330 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:58:30.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:58:30.330 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:58:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f6cf806d200 msgr2=0x7f6cf806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f6cf806d200 0x7f6cf806f6b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc004890 tx=0x7f6cfc009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fcfb0 msgr2=0x7f6d08112bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fcfb0 0x7f6d08112bb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4002a50 tx=0x7f6cf40046b0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 shutdown_connections 2024-02-16T22:58:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d080fbe40 0x7f6d08110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f6cf806d200 0x7f6cf806f6b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 --2- 172.21.15.64:0/2397895114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d080fcfb0 0x7f6d08112bb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 >> 172.21.15.64:0/2397895114 conn(0x7f6d080f7500 msgr2=0x7f6d08100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:30.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 shutdown_connections 2024-02-16T22:58:30.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.329+0000 7f6d0ee78700 1 -- 172.21.15.64:0/2397895114 wait complete. 2024-02-16T22:58:30.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.431+0000 7f87a11c6700 1 -- 172.21.15.64:0/4265338735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fe2b0 msgr2=0x7f879c0fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.431+0000 7f87a11c6700 1 --2- 172.21.15.64:0/4265338735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fe2b0 0x7f879c0fe700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8790004940 tx=0x7f8790009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:30.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.432+0000 7f87a11c6700 1 -- 172.21.15.64:0/4265338735 shutdown_connections 2024-02-16T22:58:30.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.432+0000 7f87a11c6700 1 --2- 172.21.15.64:0/4265338735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fe2b0 0x7f879c0fe700 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.432+0000 7f87a11c6700 1 --2- 172.21.15.64:0/4265338735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f879c0fb220 0x7f879c0fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.432+0000 7f87a11c6700 1 -- 172.21.15.64:0/4265338735 >> 172.21.15.64:0/4265338735 conn(0x7f879c0f6ca0 msgr2=0x7f879c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:30.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.432+0000 7f87a11c6700 1 -- 172.21.15.64:0/4265338735 shutdown_connections 2024-02-16T22:58:30.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.432+0000 7f87a11c6700 1 -- 172.21.15.64:0/4265338735 wait complete. 2024-02-16T22:58:30.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.432+0000 7f87a11c6700 1 Processor -- start 2024-02-16T22:58:30.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.432+0000 7f87a11c6700 1 -- start start 2024-02-16T22:58:30.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87a11c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fb220 0x7f879c10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87a11c6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f879c0fe2b0 0x7f879c10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87a11c6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c10e5e0 con 0x7f879c0fb220 2024-02-16T22:58:30.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87a11c6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c10ee20 con 0x7f879c0fe2b0 2024-02-16T22:58:30.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f879ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fb220 0x7f879c10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f879ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fb220 0x7f879c10da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39126/0 (socket says 172.21.15.64:39126) 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f879ad9d700 1 -- 172.21.15.64:0/1855027489 learned_addr learned my addr 172.21.15.64:0/1855027489 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f879a59c700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f879c0fe2b0 0x7f879c10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f879ad9d700 1 -- 172.21.15.64:0/1855027489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f879c0fe2b0 msgr2=0x7f879c10df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f879ad9d700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f879c0fe2b0 0x7f879c10df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f879ad9d700 1 -- 172.21.15.64:0/1855027489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790005040 con 0x7f879c0fb220 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f879ad9d700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fb220 0x7f879c10da10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b790 tx=0x7f878c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87837fe700 1 -- 172.21.15.64:0/1855027489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878c00f6f0 con 0x7f879c0fb220 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87837fe700 1 -- 172.21.15.64:0/1855027489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f878c00fd20 con 0x7f879c0fb220 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c10f100 con 0x7f879c0fb220 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87837fe700 1 -- 172.21.15.64:0/1855027489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878c0103a0 con 0x7f879c0fb220 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.433+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c10f640 con 0x7f879c0fb220 2024-02-16T22:58:30.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.434+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c04bed0 con 0x7f879c0fb220 2024-02-16T22:58:30.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.435+0000 7f87837fe700 1 -- 172.21.15.64:0/1855027489 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f878c011400 con 0x7f879c0fb220 2024-02-16T22:58:30.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.435+0000 7f87837fe700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f878406d200 0x7f878406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.435+0000 7f87837fe700 1 -- 172.21.15.64:0/1855027489 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f878c08c350 con 0x7f879c0fb220 2024-02-16T22:58:30.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.436+0000 7f879a59c700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f878406d200 0x7f878406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.436+0000 7f879a59c700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f878406d200 0x7f878406f6b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8790004f90 tx=0x7f8790009e30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:30.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.440+0000 7f87837fe700 1 -- 172.21.15.64:0/1855027489 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f878c059f90 con 0x7f879c0fb220 2024-02-16T22:58:30.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.688+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f879c10fba0 con 0x7f879c0fb220 2024-02-16T22:58:30.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.689+0000 7f87837fe700 1 -- 172.21.15.64:0/1855027489 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f878c059b20 con 0x7f879c0fb220 2024-02-16T22:58:30.689 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:58:30.689 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:58:30.690 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:58:30.691 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:58:30.692 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:58:30.692 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:58:30.692 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:58:30.692 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:58:30.692 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:58:30.692 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:58:30.692 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:58:30.692 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:58:30.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.691+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f878406d200 msgr2=0x7f878406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.691+0000 7f87a11c6700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f878406d200 0x7f878406f6b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8790004f90 tx=0x7f8790009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.691+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fb220 msgr2=0x7f879c10da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.691+0000 7f87a11c6700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fb220 0x7f879c10da10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b790 tx=0x7f878c009220 comp rx=0 tx=0).stop 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.691+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 shutdown_connections 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.691+0000 7f87a11c6700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f878406d200 0x7f878406f6b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.691+0000 7f87a11c6700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c0fb220 0x7f879c10da10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.691+0000 7f87a11c6700 1 --2- 172.21.15.64:0/1855027489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f879c0fe2b0 0x7f879c10df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.692+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 >> 172.21.15.64:0/1855027489 conn(0x7f879c0f6ca0 msgr2=0x7f879c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.692+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 shutdown_connections 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.692+0000 7f87a11c6700 1 -- 172.21.15.64:0/1855027489 wait complete. 2024-02-16T22:58:30.694 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:58:30.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.797+0000 7f055f87a700 1 -- 172.21.15.64:0/1020393867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fe2b0 msgr2=0x7f05580fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.797+0000 7f055f87a700 1 --2- 172.21.15.64:0/1020393867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fe2b0 0x7f05580fe700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0548004940 tx=0x7f0548009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.797+0000 7f055f87a700 1 -- 172.21.15.64:0/1020393867 shutdown_connections 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.797+0000 7f055f87a700 1 --2- 172.21.15.64:0/1020393867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fe2b0 0x7f05580fe700 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.797+0000 7f055f87a700 1 --2- 172.21.15.64:0/1020393867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fb220 0x7f05580fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.797+0000 7f055f87a700 1 -- 172.21.15.64:0/1020393867 >> 172.21.15.64:0/1020393867 conn(0x7f05580f6ca0 msgr2=0x7f05580f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055f87a700 1 -- 172.21.15.64:0/1020393867 shutdown_connections 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055f87a700 1 -- 172.21.15.64:0/1020393867 wait complete. 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055f87a700 1 Processor -- start 2024-02-16T22:58:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055f87a700 1 -- start start 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055f87a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fb220 0x7f055810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055f87a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fe2b0 0x7f055810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055f87a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055810e550 con 0x7f05580fb220 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055f87a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055810edf0 con 0x7f05580fe2b0 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055d616700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fb220 0x7f055810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055d616700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fb220 0x7f055810da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43890/0 (socket says 172.21.15.64:43890) 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.798+0000 7f055d616700 1 -- 172.21.15.64:0/229220158 learned_addr learned my addr 172.21.15.64:0/229220158 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f055ce15700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fe2b0 0x7f055810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f055d616700 1 -- 172.21.15.64:0/229220158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fe2b0 msgr2=0x7f055810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f055d616700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fe2b0 0x7f055810df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:30.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f055d616700 1 -- 172.21.15.64:0/229220158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0548005040 con 0x7f05580fb220 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f055ce15700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fe2b0 0x7f055810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f055d616700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fb220 0x7f055810da10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f055400c8e0 tx=0x7f05540044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f054dffb700 1 -- 172.21.15.64:0/229220158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055400a940 con 0x7f05580fb220 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055810f0d0 con 0x7f05580fb220 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f054dffb700 1 -- 172.21.15.64:0/229220158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f055400ce80 con 0x7f05580fb220 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f054dffb700 1 -- 172.21.15.64:0/229220158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05540113a0 con 0x7f05580fb220 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.799+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055810f610 con 0x7f05580fb220 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.800+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055804bed0 con 0x7f05580fb220 2024-02-16T22:58:30.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.801+0000 7f054dffb700 1 -- 172.21.15.64:0/229220158 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f0554012400 con 0x7f05580fb220 2024-02-16T22:58:30.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.801+0000 7f054dffb700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f054406d200 0x7f054406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:30.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.801+0000 7f054dffb700 1 -- 172.21.15.64:0/229220158 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f055408c6d0 con 0x7f05580fb220 2024-02-16T22:58:30.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.801+0000 7f055ce15700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f054406d200 0x7f054406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:30.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.801+0000 7f055ce15700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f054406d200 0x7f054406f6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0548004560 tx=0x7f0548009e30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:30.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:30.806+0000 7f054dffb700 1 -- 172.21.15.64:0/229220158 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f055405a270 con 0x7f05580fb220 2024-02-16T22:58:30.947 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:30 smithi064 ceph-mon[29226]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:58:30.947 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:30 smithi064 ceph-mon[29226]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:58:30.947 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:30 smithi064 ceph-mon[29226]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:58:30.947 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:30 smithi064 ceph-mon[29226]: pgmap v10: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 48 KiB/s wr, 40 op/s 2024-02-16T22:58:30.947 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:30 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/2397895114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:58:30.947 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:30 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1855027489' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:58:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.028+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 --> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f055810fa50 con 0x7f054406d200 2024-02-16T22:58:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.029+0000 7f054dffb700 1 -- 172.21.15.64:0/229220158 <== mgr.14772 v2:172.21.15.64:6800/1223017301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f055810fa50 con 0x7f054406d200 2024-02-16T22:58:31.029 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:58:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr" 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "2/21 daemons upgraded", 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "", 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:58:31.030 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f054406d200 msgr2=0x7f054406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f054406d200 0x7f054406f6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0548004560 tx=0x7f0548009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fb220 msgr2=0x7f055810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fb220 0x7f055810da10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f055400c8e0 tx=0x7f05540044c0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 shutdown_connections 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f054406d200 0x7f054406f6b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fb220 0x7f055810da10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 --2- 172.21.15.64:0/229220158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fe2b0 0x7f055810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 >> 172.21.15.64:0/229220158 conn(0x7f05580f6ca0 msgr2=0x7f05581014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 shutdown_connections 2024-02-16T22:58:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.031+0000 7f055f87a700 1 -- 172.21.15.64:0/229220158 wait complete. 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.133+0000 7f692e812700 1 -- 172.21.15.64:0/1752205902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 msgr2=0x7f69280fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.133+0000 7f692e812700 1 --2- 172.21.15.64:0/1752205902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 0x7f69280fc250 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6918004940 tx=0x7f6918009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 -- 172.21.15.64:0/1752205902 shutdown_connections 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 --2- 172.21.15.64:0/1752205902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69280fcfb0 0x7f69280fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 --2- 172.21.15.64:0/1752205902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 0x7f69280fc250 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 -- 172.21.15.64:0/1752205902 >> 172.21.15.64:0/1752205902 conn(0x7f69280f74e0 msgr2=0x7f69280f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 -- 172.21.15.64:0/1752205902 shutdown_connections 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 -- 172.21.15.64:0/1752205902 wait complete. 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 Processor -- start 2024-02-16T22:58:31.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 -- start start 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 0x7f692810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69280fcfb0 0x7f692810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692810e480 con 0x7f69280fbe40 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.134+0000 7f692e812700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692810ed20 con 0x7f69280fcfb0 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6927fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 0x7f692810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6927fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 0x7f692810d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43910/0 (socket says 172.21.15.64:43910) 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6927fff700 1 -- 172.21.15.64:0/1811427293 learned_addr learned my addr 172.21.15.64:0/1811427293 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f69277fe700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69280fcfb0 0x7f692810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6927fff700 1 -- 172.21.15.64:0/1811427293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69280fcfb0 msgr2=0x7f692810de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6927fff700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69280fcfb0 0x7f692810de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6927fff700 1 -- 172.21.15.64:0/1811427293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6918005040 con 0x7f69280fbe40 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f69277fe700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69280fcfb0 0x7f692810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6927fff700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 0x7f692810d940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f69180093b0 tx=0x7f6918009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6924ff9700 1 -- 172.21.15.64:0/1811427293 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691800e040 con 0x7f69280fbe40 2024-02-16T22:58:31.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6924ff9700 1 -- 172.21.15.64:0/1811427293 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f691800bcc0 con 0x7f69280fbe40 2024-02-16T22:58:31.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f6924ff9700 1 -- 172.21.15.64:0/1811427293 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691801c630 con 0x7f69280fbe40 2024-02-16T22:58:31.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692810efa0 con 0x7f69280fbe40 2024-02-16T22:58:31.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.135+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692810f4e0 con 0x7f69280fbe40 2024-02-16T22:58:31.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.136+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692804bed0 con 0x7f69280fbe40 2024-02-16T22:58:31.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.137+0000 7f6924ff9700 1 -- 172.21.15.64:0/1811427293 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f69180139f0 con 0x7f69280fbe40 2024-02-16T22:58:31.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.137+0000 7f6924ff9700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f691006d200 0x7f691006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:58:31.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.137+0000 7f6924ff9700 1 -- 172.21.15.64:0/1811427293 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f691808dbe0 con 0x7f69280fbe40 2024-02-16T22:58:31.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.137+0000 7f69277fe700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f691006d200 0x7f691006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:58:31.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.138+0000 7f69277fe700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f691006d200 0x7f691006f6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f691c009fd0 tx=0x7f691c009ee0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:58:31.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.142+0000 7f6924ff9700 1 -- 172.21.15.64:0/1811427293 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f691805b670 con 0x7f69280fbe40 2024-02-16T22:58:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:30 smithi192 ceph-mon[36254]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:58:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:30 smithi192 ceph-mon[36254]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:58:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:30 smithi192 ceph-mon[36254]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:58:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:30 smithi192 ceph-mon[36254]: pgmap v10: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 48 KiB/s wr, 40 op/s 2024-02-16T22:58:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:30 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2397895114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:58:31.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:30 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1855027489' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:58:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.435+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f692810fb60 con 0x7f69280fbe40 2024-02-16T22:58:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.435+0000 7f6924ff9700 1 -- 172.21.15.64:0/1811427293 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f691805b200 con 0x7f69280fbe40 2024-02-16T22:58:31.437 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:58:31.437 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:58:31.437 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f691006d200 msgr2=0x7f691006f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f691006d200 0x7f691006f6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f691c009fd0 tx=0x7f691c009ee0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 msgr2=0x7f692810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 0x7f692810d940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f69180093b0 tx=0x7f6918009e30 comp rx=0 tx=0).stop 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 shutdown_connections 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f691006d200 0x7f691006f6b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69280fbe40 0x7f692810d940 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 --2- 172.21.15.64:0/1811427293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69280fcfb0 0x7f692810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:58:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.437+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 >> 172.21.15.64:0/1811427293 conn(0x7f69280f74e0 msgr2=0x7f69280656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:58:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.438+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 shutdown_connections 2024-02-16T22:58:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:58:31.438+0000 7f692e812700 1 -- 172.21.15.64:0/1811427293 wait complete. 2024-02-16T22:58:31.964 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:31 smithi064 ceph-mon[29226]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:58:31.964 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:31 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:58:31.964 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:31 smithi064 ceph-mon[29226]: from='client.? 172.21.15.64:0/1811427293' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:58:32.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:31 smithi192 ceph-mon[36254]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:58:32.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:31 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:58:32.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:31 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1811427293' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:58:33.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:32 smithi064 ceph-mon[29226]: pgmap v11: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 48 KiB/s wr, 40 op/s 2024-02-16T22:58:33.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:32 smithi192 ceph-mon[36254]: pgmap v11: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 48 KiB/s wr, 40 op/s 2024-02-16T22:58:35.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:35 smithi192 ceph-mon[36254]: pgmap v12: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 48 KiB/s wr, 41 op/s 2024-02-16T22:58:35.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:35 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:35.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:35 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:35.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:35 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:58:35.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:35 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:35 smithi064 ceph-mon[29226]: pgmap v12: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 48 KiB/s wr, 41 op/s 2024-02-16T22:58:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:35 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:35 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:35 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:58:35.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:35 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:36.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:36 smithi192 ceph-mon[36254]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:58:36.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:36 smithi064 ceph-mon[29226]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:58:37.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:37 smithi192 ceph-mon[36254]: pgmap v13: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 28 KiB/s wr, 1 op/s 2024-02-16T22:58:37.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:37 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:58:37.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:37 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:58:37.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:37 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:58:37.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:37 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:37 smithi064 ceph-mon[29226]: pgmap v13: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 28 KiB/s wr, 1 op/s 2024-02-16T22:58:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:37 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:58:37.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:37 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:58:37.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:37 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:58:37.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:37 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:38.465 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:38 smithi192 ceph-mon[36254]: Upgrade: Updating mgr.smithi192.pauvih 2024-02-16T22:58:38.465 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:38 smithi192 ceph-mon[36254]: Deploying daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:58:38.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:38 smithi064 ceph-mon[29226]: Upgrade: Updating mgr.smithi192.pauvih 2024-02-16T22:58:38.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:38 smithi064 ceph-mon[29226]: Deploying daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:58:39.404 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:39 smithi192 ceph-mon[36254]: pgmap v14: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 28 KiB/s wr, 1 op/s 2024-02-16T22:58:39.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:39 smithi064 ceph-mon[29226]: pgmap v14: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 28 KiB/s wr, 1 op/s 2024-02-16T22:58:41.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:41 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:41.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:41 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:41.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:41 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:41.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:41 smithi064 ceph-mon[29226]: pgmap v15: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:58:41.203 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:41 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:41.203 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:41 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:41.203 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:41 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:41.203 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:41 smithi192 ceph-mon[36254]: pgmap v15: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:58:43.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:43 smithi064 ceph-mon[29226]: pgmap v16: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:58:43.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:43 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:43.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:43 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:43.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:43 smithi192 ceph-mon[36254]: pgmap v16: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:58:43.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:43 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:43.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:43 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:45 smithi192 ceph-mon[36254]: pgmap v17: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:58:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:45 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:45 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:45 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:45 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:45 smithi064 ceph-mon[29226]: pgmap v17: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:58:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:45 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:45 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:45 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:45.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:45 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:47 smithi192 ceph-mon[36254]: pgmap v18: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:47 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:58:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:47 smithi192 ceph-mon[36254]: Standby manager daemon smithi192.pauvih restarted 2024-02-16T22:58:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:47 smithi192 ceph-mon[36254]: Standby manager daemon smithi192.pauvih started 2024-02-16T22:58:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:47 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/4024930474' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:58:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:47 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/4024930474' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:58:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:47 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/4024930474' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:58:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:47 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/4024930474' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:58:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:47 smithi064 ceph-mon[29226]: pgmap v18: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:47 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:58:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:47 smithi064 ceph-mon[29226]: Standby manager daemon smithi192.pauvih restarted 2024-02-16T22:58:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:47 smithi064 ceph-mon[29226]: Standby manager daemon smithi192.pauvih started 2024-02-16T22:58:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:47 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/4024930474' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:58:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:47 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/4024930474' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:58:47.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:47 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/4024930474' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:58:47.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:47 smithi064 ceph-mon[29226]: from='mgr.? 172.21.15.192:0/4024930474' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:58:48.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:48 smithi192 ceph-mon[36254]: mgrmap e32: smithi064.kubmlk(active, since 31s), standbys: smithi192.pauvih 2024-02-16T22:58:48.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:48 smithi192 ceph-mon[36254]: pgmap v19: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:48.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:48 smithi064 ceph-mon[29226]: mgrmap e32: smithi064.kubmlk(active, since 31s), standbys: smithi192.pauvih 2024-02-16T22:58:48.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:48 smithi064 ceph-mon[29226]: pgmap v19: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: Upgrade: Setting container_image for all mgr 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:49.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi064.kubmlk"}]: dispatch 2024-02-16T22:58:49.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi064.kubmlk"}]: dispatch 2024-02-16T22:58:49.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi064.kubmlk"}]': finished 2024-02-16T22:58:49.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.pauvih"}]: dispatch 2024-02-16T22:58:49.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.pauvih"}]: dispatch 2024-02-16T22:58:49.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.pauvih"}]': finished 2024-02-16T22:58:49.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:49 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T22:58:49.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:49.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:49.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:49.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:58:49.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: Upgrade: Setting container_image for all mgr 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi064.kubmlk"}]: dispatch 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi064.kubmlk"}]: dispatch 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi064.kubmlk"}]': finished 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.pauvih"}]: dispatch 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.pauvih"}]: dispatch 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.pauvih"}]': finished 2024-02-16T22:58:49.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:49 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T22:58:51.334 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:51 smithi064 ceph-mon[29226]: Upgrade: Updating mon.smithi064 2024-02-16T22:58:51.334 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:51 smithi064 ceph-mon[29226]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:51.334 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:51 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:58:51.334 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:51 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:58:51.334 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:51 smithi064 ceph-mon[29226]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:51.334 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:51 smithi064 ceph-mon[29226]: Deploying daemon mon.smithi064 on smithi064 2024-02-16T22:58:51.334 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:51 smithi064 ceph-mon[29226]: pgmap v20: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:51 smithi192 ceph-mon[36254]: Upgrade: Updating mon.smithi064 2024-02-16T22:58:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:51 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='mgr.smithi064.kubmlk' 2024-02-16T22:58:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:51 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:58:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:51 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:58:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:51 smithi192 ceph-mon[36254]: from='mgr.14772 172.21.15.64:0/4174409103' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:58:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:51 smithi192 ceph-mon[36254]: Deploying daemon mon.smithi064 on smithi064 2024-02-16T22:58:51.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:51 smithi192 ceph-mon[36254]: pgmap v20: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:52.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:52 smithi064 systemd[1]: Stopping Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T22:58:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:52 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[29222]: 2024-02-16T22:58:52.322+0000 7f29b81c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi064 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:58:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:52 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[29222]: 2024-02-16T22:58:52.322+0000 7f29b81c1700 -1 mon.smithi064@0(leader) e2 *** Got Signal Terminated *** 2024-02-16T22:58:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:52 smithi064 podman[93594]: 2024-02-16 22:58:52.384349005 +0000 UTC m=+0.217790671 container stop ad935b7a6a7f2d4d9e693d448a725c3ff820ce0c6324682bc03a860c8f225ecd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=quincy-b88fae6) 2024-02-16T22:58:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:52 smithi064 podman[93594]: 2024-02-16 22:58:52.40995052 +0000 UTC m=+0.243392186 container died ad935b7a6a7f2d4d9e693d448a725c3ff820ce0c6324682bc03a860c8f225ecd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=quincy-b88fae6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-16T22:58:52.976 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:52 smithi064 podman[93594]: 2024-02-16 22:58:52.80602703 +0000 UTC m=+0.639468736 container cleanup ad935b7a6a7f2d4d9e693d448a725c3ff820ce0c6324682bc03a860c8f225ecd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=quincy-b88fae6) 2024-02-16T22:58:52.976 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:52 smithi064 bash[93594]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064 2024-02-16T22:58:53.235 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:52 smithi064 podman[93609]: 2024-02-16 22:58:52.973867916 +0000 UTC m=+0.584566395 container remove ad935b7a6a7f2d4d9e693d448a725c3ff820ce0c6324682bc03a860c8f225ecd (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=quincy-b88fae6) 2024-02-16T22:58:53.236 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:53 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service: Deactivated successfully. 2024-02-16T22:58:53.236 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:53 smithi064 systemd[1]: Stopped Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T22:58:53.236 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:53 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service: Consumed 11.622s CPU time. 2024-02-16T22:58:53.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:53 smithi064 systemd[1]: Starting Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T22:58:54.020 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:53 smithi064 podman[93738]: 2024-02-16 22:58:53.658441129 +0000 UTC m=+0.106299752 container create a19a307284664cb3a9e8202f3ba0310e0276082286160bf572cf3bc53d50ae88 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-16T22:58:54.020 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:53 smithi064 podman[93738]: 2024-02-16 22:58:53.573391095 +0000 UTC m=+0.021249727 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:58:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 podman[93738]: 2024-02-16 22:58:54.020207431 +0000 UTC m=+0.468066061 container init a19a307284664cb3a9e8202f3ba0310e0276082286160bf572cf3bc53d50ae88 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-16T22:58:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 podman[93738]: 2024-02-16 22:58:54.023652037 +0000 UTC m=+0.471510661 container start a19a307284664cb3a9e8202f3ba0310e0276082286160bf572cf3bc53d50ae88 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-16T22:58:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T22:58:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-16T22:58:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: pidfile_write: ignore empty --pid-file 2024-02-16T22:58:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: load: jerasure load: lrc 2024-02-16T22:58:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: RocksDB version: 7.9.2 2024-02-16T22:58:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Git sha 0 2024-02-16T22:58:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: DB SUMMARY 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: DB Session ID: 04R4CBPXY2N7ENWSI2Y0 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: CURRENT file: CURRENT 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1170 Bytes 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi064/store.db dir, Total Num: 1, files: 000033.sst 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi064/store.db: 000031.log size: 2090527 ; 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.error_if_exists: 0 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.create_if_missing: 0 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.paranoid_checks: 1 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.env: 0x555742daeae0 2024-02-16T22:58:54.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T22:58:54.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.info_log: 0x555744977760 2024-02-16T22:58:54.371 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T22:58:54.372 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.statistics: (nil) 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.use_fsync: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_log_file_size: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.allow_fallocate: 1 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.use_direct_reads: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.db_log_dir: 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.wal_dir: 2024-02-16T22:58:54.373 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.write_buffer_manager: 0x55574494b400 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T22:58:54.374 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.unordered_write: 0 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.row_cache: None 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.wal_filter: None 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.two_write_queues: 0 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.wal_compression: 0 2024-02-16T22:58:54.375 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.atomic_flush: 0 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.log_readahead_size: 0 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_background_jobs: 2 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_background_compactions: -1 2024-02-16T22:58:54.376 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_subcompactions: 1 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_open_files: -1 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_background_flushes: -1 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Compression algorithms supported: 2024-02-16T22:58:54.377 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: kZSTD supported: 0 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: kXpressCompression supported: 0 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: kZlibCompression supported: 1 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: kSnappyCompression supported: 1 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: kLZ4Compression supported: 1 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: kBZip2Compression supported: 0 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi064/store.db/MANIFEST-000015 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.merge_operator: 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_filter: None 2024-02-16T22:58:54.378 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555744977880) 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: index_type: 0 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: data_block_index_type: 0 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: index_shortening: 1 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: checksum: 4 2024-02-16T22:58:54.379 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: no_block_cache: 0 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_cache: 0x55574497b090 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_cache_options: 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: capacity : 536870912 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: num_shard_bits : 4 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: strict_capacity_limit : 0 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_cache_compressed: (nil) 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: persistent_cache: (nil) 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_size: 4096 2024-02-16T22:58:54.380 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_size_deviation: 10 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_restart_interval: 16 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: index_block_restart_interval: 1 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: metadata_block_size: 4096 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: partition_filters: 0 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: use_delta_encoding: 1 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: filter_policy: bloomfilter 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: whole_key_filtering: 1 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: verify_compression: 0 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: format_version: 5 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: enable_index_compression: 1 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: block_align: 0 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: max_auto_readahead_size: 262144 2024-02-16T22:58:54.381 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: prepopulate_block_cache: 0 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: initial_auto_readahead_size: 8192 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression: NoCompression 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.num_levels: 7 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T22:58:54.382 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T22:58:54.383 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T22:58:54.384 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T22:58:54.385 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.table_properties_collectors: 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.inplace_update_support: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.bloom_locality: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.max_successive_merges: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T22:58:54.386 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.ttl: 2592000 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.enable_blob_files: false 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.min_blob_size: 0 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-02-16T22:58:54.387 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi064/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12021, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d239cc3d-9382-4b73-a2cc-a577319bcd91 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708124334053404, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708124334059563, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1896396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12015, "largest_seqno": 12924, "table_properties": {"data_size": 1891631, "index_size": 2727, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1029, "raw_key_size": 9473, "raw_average_key_size": 23, "raw_value_size": 1882765, "raw_average_value_size": 4648, "num_data_blocks": 125, "num_entries": 405, "num_filter_entries": 405, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708124334, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d239cc3d-9382-4b73-a2cc-a577319bcd91", "db_session_id": "04R4CBPXY2N7ENWSI2Y0", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708124334059661, "job": 1, "event": "recovery_finished"} 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi064/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555744a6a000 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: DB pointer 0x555744a56000 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T22:58:54.388 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** DB Stats ** 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** Compaction Stats [default] ** 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: L0 1/0 1.81 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 571.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: L6 1/0 5.06 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Sum 2/0 6.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 571.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-02-16T22:58:54.389 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 571.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** Compaction Stats [default] ** 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 571.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Cumulative compaction: 0.00 GB write, 173.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:58:54.390 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Interval compaction: 0.00 GB write, 173.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Block cache BinnedLRUCache@0x55574497b090#2 capacity: 512.00 MB usage: 3.98 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.16 KB,0.000220537%) IndexBlock(1,2.83 KB,0.000539422%) Misc(1,0.00 KB,0%) 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: starting mon.smithi064 rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi064 fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: mon.smithi064@-1(???) e2 preinit fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: mon.smithi064@-1(???).mds e13 new map 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: mon.smithi064@-1(???).mds e13 print_map 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: e13 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: legacy client fscid: 1 2024-02-16T22:58:54.391 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Filesystem 'cephfs' (1) 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: fs_name cephfs 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: epoch 10 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: tableserver 0 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: root 0 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: session_timeout 60 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: session_autoclose 300 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: max_file_size 1099511627776 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: max_xattr_size 65536 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: required_client_features {} 2024-02-16T22:58:54.392 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: last_failure 0 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: last_failure_osd_epoch 0 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: max_mds 1 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: in 0 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: up {0=14474} 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: failed 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: damaged 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: stopped 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: data_pools [3] 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: metadata_pool 2 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: inline_data enabled 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: balancer 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: bal_rank_mask -1 2024-02-16T22:58:54.393 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: standby_count_wanted 1 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: [mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: Standby daemons: 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: [mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: [mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout: [mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: mon.smithi064@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: mon.smithi064@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: mon.smithi064@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: mon.smithi064@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 ceph-mon[93757]: mon.smithi064@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 bash[93738]: a19a307284664cb3a9e8202f3ba0310e0276082286160bf572cf3bc53d50ae88 2024-02-16T22:58:54.394 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:54 smithi064 systemd[1]: Started Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T22:58:55.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: pgmap v22: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: mon.smithi064 calling monitor election 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: mon.smithi064 is new leader, mons smithi064,smithi192 in quorum (ranks 0,1) 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: monmap e2: 2 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: mgrmap e32: smithi064.kubmlk(active, since 39s), standbys: smithi192.pauvih 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: from='mgr.14772 ' entity='' 2024-02-16T22:58:55.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:58:55 smithi192 ceph-mon[36254]: mgrmap e33: smithi064.kubmlk(active, since 39s), standbys: smithi192.pauvih 2024-02-16T22:58:55.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: pgmap v22: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: mon.smithi064 calling monitor election 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: mon.smithi064 is new leader, mons smithi064,smithi192 in quorum (ranks 0,1) 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: monmap e2: 2 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: mgrmap e32: smithi064.kubmlk(active, since 39s), standbys: smithi192.pauvih 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: from='mgr.14772 ' entity='' 2024-02-16T22:58:55.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:58:55 smithi064 ceph-mon[93757]: mgrmap e33: smithi064.kubmlk(active, since 39s), standbys: smithi192.pauvih 2024-02-16T22:59:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 -- 172.21.15.64:0/1440109746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fcfb0 msgr2=0x7f8f0c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 --2- 172.21.15.64:0/1440109746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fcfb0 0x7f8f0c0fd420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8efc004940 tx=0x7f8efc009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 -- 172.21.15.64:0/1440109746 shutdown_connections 2024-02-16T22:59:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 --2- 172.21.15.64:0/1440109746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fcfb0 0x7f8f0c0fd420 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 --2- 172.21.15.64:0/1440109746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f0c0fbe40 0x7f8f0c0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 -- 172.21.15.64:0/1440109746 >> 172.21.15.64:0/1440109746 conn(0x7f8f0c0f74e0 msgr2=0x7f8f0c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 -- 172.21.15.64:0/1440109746 shutdown_connections 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 -- 172.21.15.64:0/1440109746 wait complete. 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 Processor -- start 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.546+0000 7f8f1122c700 1 -- start start 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f1122c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fbe40 0x7f8f0c0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f1122c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f0c0fcfb0 0x7f8f0c0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f1122c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0fe980 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f1122c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0feaf0 con 0x7f8f0c0fcfb0 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f0ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fbe40 0x7f8f0c0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f0ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fbe40 0x7f8f0c0fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35940/0 (socket says 172.21.15.64:35940) 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f0ad9d700 1 -- 172.21.15.64:0/2290257471 learned_addr learned my addr 172.21.15.64:0/2290257471 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f0a59c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f0c0fcfb0 0x7f8f0c0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f0ad9d700 1 -- 172.21.15.64:0/2290257471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f0c0fcfb0 msgr2=0x7f8f0c0fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f0ad9d700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f0c0fcfb0 0x7f8f0c0fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f0ad9d700 1 -- 172.21.15.64:0/2290257471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc005040 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.547+0000 7f8f0ad9d700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fbe40 0x7f8f0c0fff00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f00002a50 tx=0x7f8f000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.548+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f00007cc0 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.548+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c0fedd0 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.548+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c115990 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.548+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f0000f040 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.548+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0000da30 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.549+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c04bed0 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.549+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f8f0000dbb0 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.550+0000 7f8ef37fe700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 0x7f8ef406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.550+0000 7f8f0a59c700 1 -- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 msgr2=0x7f8ef406fdb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1223017301 2024-02-16T22:59:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.550+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f8f0008cd40 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.550+0000 7f8f0a59c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 0x7f8ef406fdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:59:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.555+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f0005a050 con 0x7f8f0c0fbe40 2024-02-16T22:59:01.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.751+0000 7f8f0a59c700 1 -- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 msgr2=0x7f8ef406fdb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1223017301 2024-02-16T22:59:01.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.751+0000 7f8f0a59c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 0x7f8ef406fdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:59:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:01.778+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 --> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f0c065dd0 con 0x7f8ef406d900 2024-02-16T22:59:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:02.152+0000 7f8f0a59c700 1 -- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 msgr2=0x7f8ef406fdb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1223017301 2024-02-16T22:59:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:02.152+0000 7f8f0a59c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 0x7f8ef406fdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T22:59:02.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:02.366+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 34) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f8f0008b140 con 0x7f8f0c0fbe40 2024-02-16T22:59:02.589 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:02 smithi064 ceph-mon[93757]: Standby manager daemon smithi192.pauvih restarted 2024-02-16T22:59:02.589 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:02 smithi064 ceph-mon[93757]: Standby manager daemon smithi192.pauvih started 2024-02-16T22:59:02.589 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:02 smithi064 ceph-mon[93757]: from='mgr.? 172.21.15.192:0/947714587' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:59:02.589 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:02 smithi064 ceph-mon[93757]: from='mgr.? 172.21.15.192:0/947714587' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:59:02.589 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:02 smithi064 ceph-mon[93757]: from='mgr.? 172.21.15.192:0/947714587' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:59:02.589 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:02 smithi064 ceph-mon[93757]: from='mgr.? 172.21.15.192:0/947714587' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:59:02.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:02 smithi192 ceph-mon[36254]: Standby manager daemon smithi192.pauvih restarted 2024-02-16T22:59:02.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:02 smithi192 ceph-mon[36254]: Standby manager daemon smithi192.pauvih started 2024-02-16T22:59:02.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:02 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/947714587' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/crt"}]: dispatch 2024-02-16T22:59:02.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:02 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/947714587' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:59:02.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:02 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/947714587' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.pauvih/key"}]: dispatch 2024-02-16T22:59:02.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:02 smithi192 ceph-mon[36254]: from='mgr.? 172.21.15.192:0/947714587' entity='mgr.smithi192.pauvih' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:59:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:02.605+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 35) v1 ==== 91118+0+0 (secure 0 0 0) 0x7f8f000568f0 con 0x7f8f0c0fbe40 2024-02-16T22:59:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:02.606+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 msgr2=0x7f8ef406fdb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:59:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:02.606+0000 7f8ef37fe700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 0x7f8ef406fdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:02 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192[36235]: 2024-02-16T22:59:02.604+0000 7fba4cb63700 -1 mon.smithi192@1(peon).osd e50 update_from_paxos full map CRC mismatch, resetting to canonical 2024-02-16T22:59:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.607+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mon.0 v2:172.21.15.64:3300/0 9 ==== mgrmap(e 36) v1 ==== 91245+0+0 (secure 0 0 0) 0x7f8f000571c0 con 0x7f8f0c0fbe40 2024-02-16T22:59:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.608+0000 7f8ef37fe700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ef401cf00 0x7f8ef401d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.608+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f0c065dd0 con 0x7f8ef401cf00 2024-02-16T22:59:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.609+0000 7f8f0a59c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ef401cf00 0x7f8ef401d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:03.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.609+0000 7f8f0a59c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ef401cf00 0x7f8ef401d2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00b3b0 tx=0x7f8efc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:03.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.612+0000 7f8ef37fe700 1 -- 172.21.15.64:0/2290257471 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f8f0c065dd0 con 0x7f8ef401cf00 2024-02-16T22:59:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ef401cf00 msgr2=0x7f8ef401d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ef401cf00 0x7f8ef401d2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00b3b0 tx=0x7f8efc009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fbe40 msgr2=0x7f8f0c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fbe40 0x7f8f0c0fff00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f00002a50 tx=0x7f8f000046b0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 shutdown_connections 2024-02-16T22:59:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ef401cf00 0x7f8ef401d2d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:6800/1223017301,v1:172.21.15.64:6801/1223017301] conn(0x7f8ef406d900 0x7f8ef406fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f0c0fbe40 0x7f8f0c0fff00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 --2- 172.21.15.64:0/2290257471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f0c0fcfb0 0x7f8f0c0fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.615+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 >> 172.21.15.64:0/2290257471 conn(0x7f8f0c0f74e0 msgr2=0x7f8f0c0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.616+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 shutdown_connections 2024-02-16T22:59:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.616+0000 7f8f1122c700 1 -- 172.21.15.64:0/2290257471 wait complete. 2024-02-16T22:59:03.626 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: mgrmap e34: smithi064.kubmlk(active, since 46s), standbys: smithi192.pauvih 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: Active manager daemon smithi064.kubmlk restarted 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: Activating manager daemon smithi064.kubmlk 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: osdmap e50: 6 total, 6 up, 6 in 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: mgrmap e35: smithi064.kubmlk(active, starting, since 0.0059725s), standbys: smithi192.pauvih 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: failed to encode map e50 with expected crc 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:59:03.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi192.pauvih", "id": "smithi192.pauvih"}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:59:03.628 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:59:03.629 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: Manager daemon smithi064.kubmlk is now available 2024-02-16T22:59:03.629 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:03.629 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:03.629 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:59:03.629 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:59:03.629 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 -- 172.21.15.64:0/3694351948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fe640 msgr2=0x7f63bc0fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 --2- 172.21.15.64:0/3694351948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fe640 0x7f63bc0fea50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f63ac004940 tx=0x7f63ac009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 -- 172.21.15.64:0/3694351948 shutdown_connections 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 --2- 172.21.15.64:0/3694351948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63bc0fef90 0x7f63bc0fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 --2- 172.21.15.64:0/3694351948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fe640 0x7f63bc0fea50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 -- 172.21.15.64:0/3694351948 >> 172.21.15.64:0/3694351948 conn(0x7f63bc0f6d20 msgr2=0x7f63bc0f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 -- 172.21.15.64:0/3694351948 shutdown_connections 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 -- 172.21.15.64:0/3694351948 wait complete. 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 Processor -- start 2024-02-16T22:59:03.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.721+0000 7f63c21cf700 1 -- start start 2024-02-16T22:59:03.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63c21cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63bc0fe640 0x7f63bc10dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:03.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63c21cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fef90 0x7f63bc10e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:03.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63c21cf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc10e750 con 0x7f63bc0fef90 2024-02-16T22:59:03.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63c21cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc10eff0 con 0x7f63bc0fe640 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fef90 0x7f63bc10e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fef90 0x7f63bc10e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36026/0 (socket says 172.21.15.64:36026) 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63baffd700 1 -- 172.21.15.64:0/1414612109 learned_addr learned my addr 172.21.15.64:0/1414612109 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63bb7fe700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63bc0fe640 0x7f63bc10dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63baffd700 1 -- 172.21.15.64:0/1414612109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63bc0fe640 msgr2=0x7f63bc10dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63baffd700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63bc0fe640 0x7f63bc10dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63baffd700 1 -- 172.21.15.64:0/1414612109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac005040 con 0x7f63bc0fef90 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63bb7fe700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63bc0fe640 0x7f63bc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63baffd700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fef90 0x7f63bc10e150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f63b000eb40 tx=0x7f63b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63a3fff700 1 -- 172.21.15.64:0/1414612109 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b0007cc0 con 0x7f63bc0fef90 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63a3fff700 1 -- 172.21.15.64:0/1414612109 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63b000f040 con 0x7f63bc0fef90 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc10f270 con 0x7f63bc0fef90 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63a3fff700 1 -- 172.21.15.64:0/1414612109 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b000ca10 con 0x7f63bc0fef90 2024-02-16T22:59:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.722+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc10f7b0 con 0x7f63bc0fef90 2024-02-16T22:59:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.723+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc107a90 con 0x7f63bc0fef90 2024-02-16T22:59:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.724+0000 7f63a3fff700 1 -- 172.21.15.64:0/1414612109 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 36) v1 ==== 91245+0+0 (secure 0 0 0) 0x7f63b000cc80 con 0x7f63bc0fef90 2024-02-16T22:59:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.724+0000 7f63a3fff700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f63a406d4e0 0x7f63a406f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.724+0000 7f63a3fff700 1 -- 172.21.15.64:0/1414612109 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f63b008c580 con 0x7f63bc0fef90 2024-02-16T22:59:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.724+0000 7f63bb7fe700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f63a406d4e0 0x7f63a406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.725+0000 7f63bb7fe700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f63a406d4e0 0x7f63a406f990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009b30 tx=0x7f63ac009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:03.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.729+0000 7f63a3fff700 1 -- 172.21.15.64:0/1414612109 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63b0056130 con 0x7f63bc0fef90 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: mgrmap e34: smithi064.kubmlk(active, since 46s), standbys: smithi192.pauvih 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: Active manager daemon smithi064.kubmlk restarted 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: Activating manager daemon smithi064.kubmlk 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: osdmap e50: 6 total, 6 up, 6 in 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: mgrmap e35: smithi064.kubmlk(active, starting, since 0.0059725s), standbys: smithi192.pauvih 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: failed to encode map e50 with expected crc 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T22:59:03.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi064.kubmlk", "id": "smithi064.kubmlk"}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr metadata", "who": "smithi192.pauvih", "id": "smithi192.pauvih"}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:59:03.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: Manager daemon smithi064.kubmlk is now available 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:59:03.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:03 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.kubmlk/trash_purge_schedule"}]: dispatch 2024-02-16T22:59:03.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.954+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63bc105ac0 con 0x7f63a406d4e0 2024-02-16T22:59:03.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.956+0000 7f63a3fff700 1 -- 172.21.15.64:0/1414612109 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f63bc105ac0 con 0x7f63a406d4e0 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f63a406d4e0 msgr2=0x7f63a406f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f63a406d4e0 0x7f63a406f990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009b30 tx=0x7f63ac009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fef90 msgr2=0x7f63bc10e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fef90 0x7f63bc10e150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f63b000eb40 tx=0x7f63b000b040 comp rx=0 tx=0).stop 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 shutdown_connections 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f63a406d4e0 0x7f63a406f990 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63bc0fe640 0x7f63bc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 --2- 172.21.15.64:0/1414612109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63bc0fef90 0x7f63bc10e150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.958+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 >> 172.21.15.64:0/1414612109 conn(0x7f63bc0f6d20 msgr2=0x7f63bc101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.959+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 shutdown_connections 2024-02-16T22:59:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:03.959+0000 7f63c21cf700 1 -- 172.21.15.64:0/1414612109 wait complete. 2024-02-16T22:59:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.060+0000 7f6a2c681700 1 -- 172.21.15.64:0/1461860753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 msgr2=0x7f6a24070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.060+0000 7f6a2c681700 1 --2- 172.21.15.64:0/1461860753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 0x7f6a24070300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6a20004940 tx=0x7f6a20009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.060+0000 7f6a2c681700 1 -- 172.21.15.64:0/1461860753 shutdown_connections 2024-02-16T22:59:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.060+0000 7f6a2c681700 1 --2- 172.21.15.64:0/1461860753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24070840 0x7f6a24070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.060+0000 7f6a2c681700 1 --2- 172.21.15.64:0/1461860753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 0x7f6a24070300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.060+0000 7f6a2c681700 1 -- 172.21.15.64:0/1461860753 >> 172.21.15.64:0/1461860753 conn(0x7f6a240f8620 msgr2=0x7f6a240faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.060+0000 7f6a2c681700 1 -- 172.21.15.64:0/1461860753 shutdown_connections 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.060+0000 7f6a2c681700 1 -- 172.21.15.64:0/1461860753 wait complete. 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a2c681700 1 Processor -- start 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a2c681700 1 -- start start 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a2c681700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24070840 0x7f6a2410fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a2c681700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 0x7f6a24110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a2c681700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24110710 con 0x7f6a24071e90 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a2c681700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24110f50 con 0x7f6a24070840 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a29c1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 0x7f6a24110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a29c1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 0x7f6a24110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36052/0 (socket says 172.21.15.64:36052) 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a29c1c700 1 -- 172.21.15.64:0/1591729826 learned_addr learned my addr 172.21.15.64:0/1591729826 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.061+0000 7f6a2a41d700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24070840 0x7f6a2410fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a29c1c700 1 -- 172.21.15.64:0/1591729826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24070840 msgr2=0x7f6a2410fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a29c1c700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24070840 0x7f6a2410fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a29c1c700 1 -- 172.21.15.64:0/1591729826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a20005040 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a29c1c700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 0x7f6a24110080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400b790 tx=0x7f6a14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a1affd700 1 -- 172.21.15.64:0/1591729826 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a1400f770 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a24111230 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a24111770 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a1affd700 1 -- 172.21.15.64:0/1591729826 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a1400fda0 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.062+0000 7f6a1affd700 1 -- 172.21.15.64:0/1591729826 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a14010420 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.063+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a080052e0 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.063+0000 7f6a1affd700 1 -- 172.21.15.64:0/1591729826 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 36) v1 ==== 91245+0+0 (secure 0 0 0) 0x7f6a1400f8d0 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.064+0000 7f6a1affd700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f6a1006d4e0 0x7f6a1006f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.064+0000 7f6a2a41d700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f6a1006d4e0 0x7f6a1006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.064+0000 7f6a1affd700 1 -- 172.21.15.64:0/1591729826 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6a1408c920 con 0x7f6a24071e90 2024-02-16T22:59:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.064+0000 7f6a2a41d700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f6a1006d4e0 0x7f6a1006f990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b020 tx=0x7f6a20009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.069+0000 7f6a1affd700 1 -- 172.21.15.64:0/1591729826 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a140564a0 con 0x7f6a24071e90 2024-02-16T22:59:04.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.282+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a08000bf0 con 0x7f6a1006d4e0 2024-02-16T22:59:04.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.293+0000 7f6a1affd700 1 -- 172.21.15.64:0/1591729826 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6a08000bf0 con 0x7f6a1006d4e0 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (2m) 19s ago 10m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (10m) 19s ago 10m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (9m) 21s ago 9m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (113s) 19s ago 10m 81.4M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (5m) 19s ago 5m 16.8M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (5m) 19s ago 5m 155M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (5m) 21s ago 5m 15.7M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:59:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (5m) 21s ago 5m 15.6M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (70s) 19s ago 11m 491M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (24s) 21s ago 9m 67.9M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (12m) 19s ago 12m 51.8M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 ad935b7a6a7f 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (9m) 21s ago 9m 40.6M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 cc9609dd6ec5 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (2m) 19s ago 10m 15.7M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (79s) 21s ago 9m 13.6M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (8m) 19s ago 8m 209M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (8m) 19s ago 8m 215M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (7m) 19s ago 7m 175M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (7m) 21s ago 7m 232M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (7m) 21s ago 7m 209M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (6m) 21s ago 6m 185M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:59:04.295 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (30s) 19s ago 10m 35.8M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T22:59:04.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f6a1006d4e0 msgr2=0x7f6a1006f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f6a1006d4e0 0x7f6a1006f990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b020 tx=0x7f6a20009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 msgr2=0x7f6a24110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 0x7f6a24110080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400b790 tx=0x7f6a14009250 comp rx=0 tx=0).stop 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 shutdown_connections 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f6a1006d4e0 0x7f6a1006f990 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24070840 0x7f6a2410fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 --2- 172.21.15.64:0/1591729826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a24071e90 0x7f6a24110080 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 >> 172.21.15.64:0/1591729826 conn(0x7f6a240f8620 msgr2=0x7f6a24102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 shutdown_connections 2024-02-16T22:59:04.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.295+0000 7f6a2c681700 1 -- 172.21.15.64:0/1591729826 wait complete. 2024-02-16T22:59:04.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.423+0000 7fc1b4c36700 1 -- 172.21.15.64:0/2970917573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b01000c0 msgr2=0x7fc1b0102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.423+0000 7fc1b4c36700 1 --2- 172.21.15.64:0/2970917573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b01000c0 0x7fc1b0102520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4004940 tx=0x7fc1a4009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 -- 172.21.15.64:0/2970917573 shutdown_connections 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 --2- 172.21.15.64:0/2970917573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b01000c0 0x7fc1b0102520 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 --2- 172.21.15.64:0/2970917573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b0066c70 0x7fc1b00ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 -- 172.21.15.64:0/2970917573 >> 172.21.15.64:0/2970917573 conn(0x7fc1b00f7520 msgr2=0x7fc1b00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 -- 172.21.15.64:0/2970917573 shutdown_connections 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 -- 172.21.15.64:0/2970917573 wait complete. 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 Processor -- start 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 -- start start 2024-02-16T22:59:04.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b0066c70 0x7fc1b00fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b01000c0 0x7fc1b00fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00fbff0 con 0x7fc1b0066c70 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.424+0000 7fc1b4c36700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00fc160 con 0x7fc1b01000c0 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1ae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b0066c70 0x7fc1b00fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1ae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b0066c70 0x7fc1b00fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36060/0 (socket says 172.21.15.64:36060) 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1ae59c700 1 -- 172.21.15.64:0/1395633740 learned_addr learned my addr 172.21.15.64:0/1395633740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1add9b700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b01000c0 0x7fc1b00fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1ae59c700 1 -- 172.21.15.64:0/1395633740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b01000c0 msgr2=0x7fc1b00fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1ae59c700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b01000c0 0x7fc1b00fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1ae59c700 1 -- 172.21.15.64:0/1395633740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a4005040 con 0x7fc1b0066c70 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1add9b700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b01000c0 0x7fc1b00fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:59:04.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1ae59c700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b0066c70 0x7fc1b00fd570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000eb70 tx=0x7fc1a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:04.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc19effd700 1 -- 172.21.15.64:0/1395633740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a0007cc0 con 0x7fc1b0066c70 2024-02-16T22:59:04.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b00fc3e0 con 0x7fc1b0066c70 2024-02-16T22:59:04.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc19effd700 1 -- 172.21.15.64:0/1395633740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1a000f040 con 0x7fc1b0066c70 2024-02-16T22:59:04.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc19effd700 1 -- 172.21.15.64:0/1395633740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a000ca90 con 0x7fc1b0066c70 2024-02-16T22:59:04.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.425+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b0115940 con 0x7fc1b0066c70 2024-02-16T22:59:04.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.426+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b004bed0 con 0x7fc1b0066c70 2024-02-16T22:59:04.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.427+0000 7fc19effd700 1 -- 172.21.15.64:0/1395633740 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 36) v1 ==== 91245+0+0 (secure 0 0 0) 0x7fc1a000cbf0 con 0x7fc1b0066c70 2024-02-16T22:59:04.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.427+0000 7fc19effd700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc19806d4e0 0x7fc19806f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.427+0000 7fc19effd700 1 -- 172.21.15.64:0/1395633740 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc1a008c580 con 0x7fc1b0066c70 2024-02-16T22:59:04.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.427+0000 7fc1add9b700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc19806d4e0 0x7fc19806f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.428+0000 7fc1add9b700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc19806d4e0 0x7fc19806f990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4000c00 tx=0x7fc1a4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:04.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.432+0000 7fc19effd700 1 -- 172.21.15.64:0/1395633740 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1a0056080 con 0x7fc1b0066c70 2024-02-16T22:59:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:04 smithi192 ceph-mon[36254]: failed to encode map e50 with expected crc 2024-02-16T22:59:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:04 smithi192 ceph-mon[36254]: [16/Feb/2024:22:59:03] ENGINE Bus STARTING 2024-02-16T22:59:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:04 smithi192 ceph-mon[36254]: [16/Feb/2024:22:59:03] ENGINE Serving on http://172.21.15.64:8765 2024-02-16T22:59:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:04 smithi192 ceph-mon[36254]: [16/Feb/2024:22:59:03] ENGINE Serving on https://172.21.15.64:7150 2024-02-16T22:59:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:04 smithi192 ceph-mon[36254]: [16/Feb/2024:22:59:03] ENGINE Bus STARTED 2024-02-16T22:59:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:04 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:04 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:04.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:04 smithi192 ceph-mon[36254]: mgrmap e36: smithi064.kubmlk(active, since 1.00988s), standbys: smithi192.pauvih 2024-02-16T22:59:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:04 smithi064 ceph-mon[93757]: failed to encode map e50 with expected crc 2024-02-16T22:59:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:04 smithi064 ceph-mon[93757]: [16/Feb/2024:22:59:03] ENGINE Bus STARTING 2024-02-16T22:59:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:04 smithi064 ceph-mon[93757]: [16/Feb/2024:22:59:03] ENGINE Serving on http://172.21.15.64:8765 2024-02-16T22:59:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:04 smithi064 ceph-mon[93757]: [16/Feb/2024:22:59:03] ENGINE Serving on https://172.21.15.64:7150 2024-02-16T22:59:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:04 smithi064 ceph-mon[93757]: [16/Feb/2024:22:59:03] ENGINE Bus STARTED 2024-02-16T22:59:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:04.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:04 smithi064 ceph-mon[93757]: mgrmap e36: smithi064.kubmlk(active, since 1.00988s), standbys: smithi192.pauvih 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.753+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc1b0105a30 con 0x7fc1b0066c70 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.753+0000 7fc19effd700 1 -- 172.21.15.64:0/1395633740 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7fc1a0059740 con 0x7fc1b0066c70 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:59:04.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 11, 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:59:04.755 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:59:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc19806d4e0 msgr2=0x7fc19806f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc19806d4e0 0x7fc19806f990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4000c00 tx=0x7fc1a4009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b0066c70 msgr2=0x7fc1b00fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b0066c70 0x7fc1b00fd570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000eb70 tx=0x7fc1a000b040 comp rx=0 tx=0).stop 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 shutdown_connections 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc19806d4e0 0x7fc19806f990 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b0066c70 0x7fc1b00fd570 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 --2- 172.21.15.64:0/1395633740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b01000c0 0x7fc1b00fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 >> 172.21.15.64:0/1395633740 conn(0x7fc1b00f7520 msgr2=0x7fc1b00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 shutdown_connections 2024-02-16T22:59:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.756+0000 7fc1b4c36700 1 -- 172.21.15.64:0/1395633740 wait complete. 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.871+0000 7f8cff3f8700 1 -- 172.21.15.64:0/1115873587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 msgr2=0x7f8cf80fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.871+0000 7f8cff3f8700 1 --2- 172.21.15.64:0/1115873587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 0x7f8cf80fc290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4004940 tx=0x7f8cf4009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.871+0000 7f8cff3f8700 1 -- 172.21.15.64:0/1115873587 shutdown_connections 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.871+0000 7f8cff3f8700 1 --2- 172.21.15.64:0/1115873587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf80fd080 0x7f8cf80fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.871+0000 7f8cff3f8700 1 --2- 172.21.15.64:0/1115873587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 0x7f8cf80fc290 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.871+0000 7f8cff3f8700 1 -- 172.21.15.64:0/1115873587 >> 172.21.15.64:0/1115873587 conn(0x7f8cf80f7590 msgr2=0x7f8cf80f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cff3f8700 1 -- 172.21.15.64:0/1115873587 shutdown_connections 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cff3f8700 1 -- 172.21.15.64:0/1115873587 wait complete. 2024-02-16T22:59:04.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cff3f8700 1 Processor -- start 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cff3f8700 1 -- start start 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cff3f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 0x7f8cf810c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cff3f8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf80fd080 0x7f8cf810ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cff3f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf810d460 con 0x7f8cf80fbe80 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cff3f8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf810dd00 con 0x7f8cf80fd080 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cfd194700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 0x7f8cf810c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cfd194700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 0x7f8cf810c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36098/0 (socket says 172.21.15.64:36098) 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.872+0000 7f8cfd194700 1 -- 172.21.15.64:0/2077146000 learned_addr learned my addr 172.21.15.64:0/2077146000 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cfc993700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf80fd080 0x7f8cf810ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cfd194700 1 -- 172.21.15.64:0/2077146000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf80fd080 msgr2=0x7f8cf810ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cfd194700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf80fd080 0x7f8cf810ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cfd194700 1 -- 172.21.15.64:0/2077146000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf4005040 con 0x7f8cf80fbe80 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cfc993700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf80fd080 0x7f8cf810ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:59:04.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cfd194700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 0x7f8cf810c920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4000c00 tx=0x7f8cf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:04.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cedffb700 1 -- 172.21.15.64:0/2077146000 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf400e040 con 0x7f8cf80fbe80 2024-02-16T22:59:04.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cedffb700 1 -- 172.21.15.64:0/2077146000 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cf400bcc0 con 0x7f8cf80fbe80 2024-02-16T22:59:04.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf810df80 con 0x7f8cf80fbe80 2024-02-16T22:59:04.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cedffb700 1 -- 172.21.15.64:0/2077146000 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf401c630 con 0x7f8cf80fbe80 2024-02-16T22:59:04.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.873+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf810e4c0 con 0x7f8cf80fbe80 2024-02-16T22:59:04.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.874+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf804bed0 con 0x7f8cf80fbe80 2024-02-16T22:59:04.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.875+0000 7f8cedffb700 1 -- 172.21.15.64:0/2077146000 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 36) v1 ==== 91245+0+0 (secure 0 0 0) 0x7f8cf4013980 con 0x7f8cf80fbe80 2024-02-16T22:59:04.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.875+0000 7f8cedffb700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ce406d4e0 0x7f8ce406f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:04.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.875+0000 7f8cedffb700 1 -- 172.21.15.64:0/2077146000 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8cf408de50 con 0x7f8cf80fbe80 2024-02-16T22:59:04.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.875+0000 7f8cfc993700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ce406d4e0 0x7f8ce406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:04.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.876+0000 7f8cfc993700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ce406d4e0 0x7f8ce406f990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007fd0 tx=0x7f8ce8009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:04.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:04.880+0000 7f8cedffb700 1 -- 172.21.15.64:0/2077146000 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8cf4057950 con 0x7f8cf80fbe80 2024-02-16T22:59:05.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.138+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8cf810eaa0 con 0x7f8cf80fbe80 2024-02-16T22:59:05.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.138+0000 7f8cedffb700 1 -- 172.21.15.64:0/2077146000 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f8cf405b010 con 0x7f8cf80fbe80 2024-02-16T22:59:05.139 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:59:05.139 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:59:05.139 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:59:05.140 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:59:05.140 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:59:05.140 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:59:05.140 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:59:05.140 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:59:05.140 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:59:05.141 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:59:05.141 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:59:05.141 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:59:05.141 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:59:05.141 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:59:05.141 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:59:05.141 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:59:05.142 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T22:59:05.142 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:59:05.142 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:59:05.142 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:59:05.142 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:59:05.142 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:59:05.142 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:59:05.143 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:59:05.143 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:59:05.143 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:59:05.143 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:59:05.143 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:59:05.143 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:59:05.143 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:59:05.144 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:59:05.144 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T22:59:05.144 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:59:05.144 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:05.144 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:59:05.144 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:59:05.144 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:59:05.145 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:59:05.145 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:05.145 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:05.145 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:05.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ce406d4e0 msgr2=0x7f8ce406f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ce406d4e0 0x7f8ce406f990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007fd0 tx=0x7f8ce8009040 comp rx=0 tx=0).stop 2024-02-16T22:59:05.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 msgr2=0x7f8cf810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 0x7f8cf810c920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4000c00 tx=0x7f8cf4009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:05.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 shutdown_connections 2024-02-16T22:59:05.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ce406d4e0 0x7f8ce406f990 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf80fbe80 0x7f8cf810c920 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 --2- 172.21.15.64:0/2077146000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf80fd080 0x7f8cf810ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 >> 172.21.15.64:0/2077146000 conn(0x7f8cf80f7590 msgr2=0x7f8cf81002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:05.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 shutdown_connections 2024-02-16T22:59:05.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.140+0000 7f8cff3f8700 1 -- 172.21.15.64:0/2077146000 wait complete. 2024-02-16T22:59:05.148 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:59:05.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.250+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/3877183998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4100100 msgr2=0x7fb0e4102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.250+0000 7fb0eb0d2700 1 --2- 172.21.15.64:0/3877183998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4100100 0x7fb0e4102520 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0004940 tx=0x7fb0d0009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:05.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.250+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/3877183998 shutdown_connections 2024-02-16T22:59:05.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.250+0000 7fb0eb0d2700 1 --2- 172.21.15.64:0/3877183998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4100100 0x7fb0e4102520 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.250+0000 7fb0eb0d2700 1 --2- 172.21.15.64:0/3877183998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e40663a0 0x7fb0e40ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.250+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/3877183998 >> 172.21.15.64:0/3877183998 conn(0x7fb0e40f74e0 msgr2=0x7fb0e40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.250+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/3877183998 shutdown_connections 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.250+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/3877183998 wait complete. 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0eb0d2700 1 Processor -- start 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0eb0d2700 1 -- start start 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0eb0d2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e40663a0 0x7fb0e410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0eb0d2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e4100100 0x7fb0e410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0eb0d2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e410e550 con 0x7fb0e40663a0 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0eb0d2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e410ed90 con 0x7fb0e4100100 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0e8e6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e40663a0 0x7fb0e410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0e8e6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e40663a0 0x7fb0e410d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36118/0 (socket says 172.21.15.64:36118) 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0e8e6e700 1 -- 172.21.15.64:0/1994007099 learned_addr learned my addr 172.21.15.64:0/1994007099 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.251+0000 7fb0e3fff700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e4100100 0x7fb0e410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0e8e6e700 1 -- 172.21.15.64:0/1994007099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e4100100 msgr2=0x7fb0e410dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0e8e6e700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e4100100 0x7fb0e410dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0e8e6e700 1 -- 172.21.15.64:0/1994007099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0005040 con 0x7fb0e40663a0 2024-02-16T22:59:05.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0e8e6e700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e40663a0 0x7fb0e410d980 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800b790 tx=0x7fb0d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0e17fa700 1 -- 172.21.15.64:0/1994007099 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d800f6f0 con 0x7fb0e40663a0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0e17fa700 1 -- 172.21.15.64:0/1994007099 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0d800fd20 con 0x7fb0e40663a0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0e17fa700 1 -- 172.21.15.64:0/1994007099 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d80103a0 con 0x7fb0e40663a0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e410f070 con 0x7fb0e40663a0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.252+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e410f5b0 con 0x7fb0e40663a0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.253+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e404bed0 con 0x7fb0e40663a0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.253+0000 7fb0e17fa700 1 -- 172.21.15.64:0/1994007099 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 36) v1 ==== 91245+0+0 (secure 0 0 0) 0x7fb0d8011400 con 0x7fb0e40663a0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.254+0000 7fb0e17fa700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0d406d4e0 0x7fb0d406f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.254+0000 7fb0e3fff700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0d406d4e0 0x7fb0d406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.254+0000 7fb0e17fa700 1 -- 172.21.15.64:0/1994007099 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb0d808c7d0 con 0x7fb0e40663a0 2024-02-16T22:59:05.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.254+0000 7fb0e3fff700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0d406d4e0 0x7fb0d406f990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0004890 tx=0x7fb0d0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:05.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.259+0000 7fb0e17fa700 1 -- 172.21.15.64:0/1994007099 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0d8056350 con 0x7fb0e40663a0 2024-02-16T22:59:05.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.485+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0e4100d60 con 0x7fb0d406d4e0 2024-02-16T22:59:05.486 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:05 smithi064 ceph-mon[93757]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:05.486 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:05 smithi064 ceph-mon[93757]: pgmap v3: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:05.486 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:05 smithi064 ceph-mon[93757]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:05.486 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:05 smithi064 ceph-mon[93757]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:05.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:05 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1395633740' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:05.487 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:05 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2077146000' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:59:05.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.486+0000 7fb0e17fa700 1 -- 172.21.15.64:0/1994007099 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fb0e4100d60 con 0x7fb0d406d4e0 2024-02-16T22:59:05.487 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:59:05.487 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:59:05.487 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:59:05.487 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:59:05.488 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T22:59:05.488 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr" 2024-02-16T22:59:05.488 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T22:59:05.488 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "2/21 daemons upgraded", 2024-02-16T22:59:05.488 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "", 2024-02-16T22:59:05.488 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:59:05.488 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:59:05.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.489+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0d406d4e0 msgr2=0x7fb0d406f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.489+0000 7fb0eb0d2700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0d406d4e0 0x7fb0d406f990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0004890 tx=0x7fb0d0009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:05.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e40663a0 msgr2=0x7fb0e410d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e40663a0 0x7fb0e410d980 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800b790 tx=0x7fb0d8009250 comp rx=0 tx=0).stop 2024-02-16T22:59:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 shutdown_connections 2024-02-16T22:59:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0d406d4e0 0x7fb0d406f990 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e40663a0 0x7fb0e410d980 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 --2- 172.21.15.64:0/1994007099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e4100100 0x7fb0e410dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 >> 172.21.15.64:0/1994007099 conn(0x7fb0e40f74e0 msgr2=0x7fb0e40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 shutdown_connections 2024-02-16T22:59:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.490+0000 7fb0eb0d2700 1 -- 172.21.15.64:0/1994007099 wait complete. 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.592+0000 7f7dd2045700 1 -- 172.21.15.64:0/235128237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 msgr2=0x7f7dcc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.592+0000 7f7dd2045700 1 --2- 172.21.15.64:0/235128237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 0x7f7dcc0fd320 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc004940 tx=0x7f7dbc009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.592+0000 7f7dd2045700 1 -- 172.21.15.64:0/235128237 shutdown_connections 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.592+0000 7f7dd2045700 1 --2- 172.21.15.64:0/235128237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dcc0fe0e0 0x7f7dcc0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.592+0000 7f7dd2045700 1 --2- 172.21.15.64:0/235128237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 0x7f7dcc0fd320 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.592+0000 7f7dd2045700 1 -- 172.21.15.64:0/235128237 >> 172.21.15.64:0/235128237 conn(0x7f7dcc0f8620 msgr2=0x7f7dcc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.592+0000 7f7dd2045700 1 -- 172.21.15.64:0/235128237 shutdown_connections 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.592+0000 7f7dd2045700 1 -- 172.21.15.64:0/235128237 wait complete. 2024-02-16T22:59:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dd2045700 1 Processor -- start 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dd2045700 1 -- start start 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dd2045700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 0x7f7dcc10fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dd2045700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dcc0fe0e0 0x7f7dcc110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dd2045700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc110700 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dd2045700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc110fa0 con 0x7f7dcc0fe0e0 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dcb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 0x7f7dcc10fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dcb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 0x7f7dcc10fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36142/0 (socket says 172.21.15.64:36142) 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dcb7fe700 1 -- 172.21.15.64:0/3788857990 learned_addr learned my addr 172.21.15.64:0/3788857990 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.593+0000 7f7dcaffd700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dcc0fe0e0 0x7f7dcc110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7dcb7fe700 1 -- 172.21.15.64:0/3788857990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dcc0fe0e0 msgr2=0x7f7dcc110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7dcb7fe700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dcc0fe0e0 0x7f7dcc110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7dcb7fe700 1 -- 172.21.15.64:0/3788857990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc005040 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7dcb7fe700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 0x7f7dcc10fbc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009380 tx=0x7f7dbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7db3fff700 1 -- 172.21.15.64:0/3788857990 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dbc00e040 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc111220 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7db3fff700 1 -- 172.21.15.64:0/3788857990 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dbc004290 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7db3fff700 1 -- 172.21.15.64:0/3788857990 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dbc01c3f0 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.594+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc111760 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.595+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc04bed0 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.595+0000 7f7db3fff700 1 -- 172.21.15.64:0/3788857990 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 36) v1 ==== 91245+0+0 (secure 0 0 0) 0x7f7dbc013420 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.596+0000 7f7db3fff700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7db406d4e0 0x7f7db406f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.596+0000 7f7dcaffd700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7db406d4e0 0x7f7db406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.596+0000 7f7db3fff700 1 -- 172.21.15.64:0/3788857990 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7dbc08dd40 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.596+0000 7f7dcaffd700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7db406d4e0 0x7f7db406f990 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0004f60 tx=0x7f7dc000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.601+0000 7f7db3fff700 1 -- 172.21.15.64:0/3788857990 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7dbc091030 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.606 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:05 smithi192 ceph-mon[36254]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:05.606 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:05 smithi192 ceph-mon[36254]: pgmap v3: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:05.606 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:05 smithi192 ceph-mon[36254]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:05.606 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:05 smithi192 ceph-mon[36254]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:05.606 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:05 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/1395633740' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:05.606 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:05 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/2077146000' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:59:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.617+0000 7f7db3fff700 1 -- 172.21.15.64:0/3788857990 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 37) v1 ==== 91539+0+0 (secure 0 0 0) 0x7f7dbc057c10 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.897+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7dcc070f70 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.897+0000 7f7db3fff700 1 -- 172.21.15.64:0/3788857990 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f7dbc071290 con 0x7f7dcc0fcef0 2024-02-16T22:59:05.898 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:59:05.898 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:59:05.898 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:59:05.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7db406d4e0 msgr2=0x7f7db406f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7db406d4e0 0x7f7db406f990 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0004f60 tx=0x7f7dc000b040 comp rx=0 tx=0).stop 2024-02-16T22:59:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 msgr2=0x7f7dcc10fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 0x7f7dcc10fbc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009380 tx=0x7f7dbc009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 shutdown_connections 2024-02-16T22:59:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7db406d4e0 0x7f7db406f990 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dcc0fcef0 0x7f7dcc10fbc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 --2- 172.21.15.64:0/3788857990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dcc0fe0e0 0x7f7dcc110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 >> 172.21.15.64:0/3788857990 conn(0x7f7dcc0f8620 msgr2=0x7f7dcc0fa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 shutdown_connections 2024-02-16T22:59:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:05.900+0000 7f7dd2045700 1 -- 172.21.15.64:0/3788857990 wait complete. 2024-02-16T22:59:06.557 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:06 smithi192 ceph-mon[36254]: pgmap v4: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:06.557 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:06 smithi192 ceph-mon[36254]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:06.557 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:06 smithi192 ceph-mon[36254]: mgrmap e37: smithi064.kubmlk(active, since 3s), standbys: smithi192.pauvih 2024-02-16T22:59:06.557 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:06 smithi192 ceph-mon[36254]: from='client.? 172.21.15.64:0/3788857990' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:59:06.581 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:06 smithi064 ceph-mon[93757]: pgmap v4: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:06.581 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:06 smithi064 ceph-mon[93757]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:06.582 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:06 smithi064 ceph-mon[93757]: mgrmap e37: smithi064.kubmlk(active, since 3s), standbys: smithi192.pauvih 2024-02-16T22:59:06.582 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:06 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3788857990' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:59:07.676 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:07 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:07.677 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:07 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:07.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:07 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:07.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:07 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:08.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:08 smithi192 ceph-mon[36254]: pgmap v5: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:08.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:08 smithi192 ceph-mon[36254]: mgrmap e38: smithi064.kubmlk(active, since 4s), standbys: smithi192.pauvih 2024-02-16T22:59:08.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:08 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:08.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:08 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:08.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:08 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:08.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:08 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:08.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:08 smithi064 ceph-mon[93757]: pgmap v5: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:08.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:08 smithi064 ceph-mon[93757]: mgrmap e38: smithi064.kubmlk(active, since 4s), standbys: smithi192.pauvih 2024-02-16T22:59:08.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:08.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:08.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:08.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:10.614 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:10 smithi192 ceph-mon[36254]: pgmap v6: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:10.615 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:10 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:10.615 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:10 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:10.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:10 smithi064 ceph-mon[93757]: pgmap v6: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:10.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:10 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:10.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:10 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:12.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: pgmap v7: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:59:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:12 smithi192 ceph-mon[36254]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:59:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: pgmap v7: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-16T22:59:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:59:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: Updating smithi064:/etc/ceph/ceph.conf 2024-02-16T22:59:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: Updating smithi192:/etc/ceph/ceph.conf 2024-02-16T22:59:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:59:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:59:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:59:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:12 smithi064 ceph-mon[93757]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:59:13.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:13 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T22:59:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: Updating smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:59:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: Updating smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:59:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:13.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:13.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:13.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T22:59:14.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:14 smithi064 ceph-mon[93757]: pgmap v8: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-16T22:59:14.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:14.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:59:14.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:59:14.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:14.483 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:14 smithi192 ceph-mon[36254]: pgmap v8: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-16T22:59:14.483 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:14 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:14.483 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:14 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:59:14.483 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:14 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:59:14.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:14 smithi192 ceph-mon[36254]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:15.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:15 smithi064 ceph-mon[93757]: Upgrade: Updating mon.smithi192 2024-02-16T22:59:15.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:15 smithi064 ceph-mon[93757]: Deploying daemon mon.smithi192 on smithi192 2024-02-16T22:59:15.456 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:15 smithi192 ceph-mon[36254]: Upgrade: Updating mon.smithi192 2024-02-16T22:59:15.456 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:15 smithi192 ceph-mon[36254]: Deploying daemon mon.smithi192 on smithi192 2024-02-16T22:59:16.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:15 smithi192 systemd[1]: Stopping Ceph mon.smithi192 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T22:59:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:15 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192[36235]: 2024-02-16T22:59:15.968+0000 7fba5236e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi192 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:59:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:15 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192[36235]: 2024-02-16T22:59:15.968+0000 7fba5236e700 -1 mon.smithi192@1(peon) e2 *** Got Signal Terminated *** 2024-02-16T22:59:16.493 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:16 smithi192 podman[82483]: 2024-02-16 22:59:16.172872711 +0000 UTC m=+0.299129871 container died cc9609dd6ec5bcdb427741c3bd63f53a3ab9cd02d8952711a45a7c5613736952 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-b88fae6, org.label-schema.license=GPLv2) 2024-02-16T22:59:16.493 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:16 smithi192 podman[82483]: 2024-02-16 22:59:16.493409882 +0000 UTC m=+0.619667039 container cleanup cc9609dd6ec5bcdb427741c3bd63f53a3ab9cd02d8952711a45a7c5613736952 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=quincy-b88fae6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-16T22:59:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:16 smithi192 bash[82483]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192 2024-02-16T22:59:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:16 smithi192 podman[82497]: 2024-02-16 22:59:16.66969264 +0000 UTC m=+0.499506939 container remove cc9609dd6ec5bcdb427741c3bd63f53a3ab9cd02d8952711a45a7c5613736952 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-b88fae6, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-16T22:59:17.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:16 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi192.service: Deactivated successfully. 2024-02-16T22:59:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:16 smithi192 systemd[1]: Stopped Ceph mon.smithi192 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T22:59:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:16 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi192.service: Consumed 6.362s CPU time. 2024-02-16T22:59:17.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 systemd[1]: Starting Ceph mon.smithi192 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T22:59:17.568 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 podman[82633]: 2024-02-16 22:59:17.22030754 +0000 UTC m=+0.018672312 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:59:17.568 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 podman[82633]: 2024-02-16 22:59:17.320862261 +0000 UTC m=+0.119227066 container create eb1d711c4dcd147eae17f04420f53e97f0cae3fbb708e143d97fb2fdb73441fa (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-02-16T22:59:17.568 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 podman[82633]: 2024-02-16 22:59:17.531857479 +0000 UTC m=+0.330222244 container init eb1d711c4dcd147eae17f04420f53e97f0cae3fbb708e143d97fb2fdb73441fa (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 podman[82633]: 2024-02-16 22:59:17.535134123 +0000 UTC m=+0.333498887 container start eb1d711c4dcd147eae17f04420f53e97f0cae3fbb708e143d97fb2fdb73441fa (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: pidfile_write: ignore empty --pid-file 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: load: jerasure load: lrc 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: RocksDB version: 7.9.2 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Git sha 0 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: DB SUMMARY 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: DB Session ID: FB98IX6VDQ9FRQ1TE2JD 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: CURRENT file: CURRENT 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1095 Bytes 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi192/store.db dir, Total Num: 1, files: 000027.sst 2024-02-16T22:59:17.569 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi192/store.db: 000025.log size: 6021222 ; 2024-02-16T22:59:17.570 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.error_if_exists: 0 2024-02-16T22:59:17.570 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.create_if_missing: 0 2024-02-16T22:59:17.570 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.paranoid_checks: 1 2024-02-16T22:59:17.570 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T22:59:17.570 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T22:59:17.570 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.env: 0x564080cc4ae0 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.info_log: 0x564081c21760 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.statistics: (nil) 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.use_fsync: 0 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_log_file_size: 0 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.allow_fallocate: 1 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T22:59:17.571 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T22:59:17.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.use_direct_reads: 0 2024-02-16T22:59:17.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T22:59:17.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T22:59:17.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.db_log_dir: 2024-02-16T22:59:17.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.wal_dir: 2024-02-16T22:59:17.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T22:59:17.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T22:59:17.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T22:59:17.573 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T22:59:17.573 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T22:59:17.573 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T22:59:17.573 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T22:59:17.574 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T22:59:17.574 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.write_buffer_manager: 0x564081bf5400 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.unordered_write: 0 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T22:59:17.575 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.row_cache: None 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.wal_filter: None 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.two_write_queues: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.wal_compression: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.atomic_flush: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T22:59:17.576 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.log_readahead_size: 0 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_background_jobs: 2 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_background_compactions: -1 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_subcompactions: 1 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T22:59:17.577 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T22:59:17.578 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T22:59:17.578 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T22:59:17.578 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T22:59:17.578 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T22:59:17.578 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T22:59:17.578 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T22:59:17.578 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_open_files: -1 2024-02-16T22:59:17.579 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T22:59:17.579 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T22:59:17.579 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T22:59:17.579 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T22:59:17.579 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_background_flushes: -1 2024-02-16T22:59:17.579 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Compression algorithms supported: 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: kZSTD supported: 0 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: kXpressCompression supported: 0 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: kZlibCompression supported: 1 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: kSnappyCompression supported: 1 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: kLZ4Compression supported: 1 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: kBZip2Compression supported: 0 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi192/store.db/MANIFEST-000009 2024-02-16T22:59:17.580 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.merge_operator: 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_filter: None 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564081c21880) 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: cache_index_and_filter_blocks: 1 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: pin_top_level_index_and_filter: 1 2024-02-16T22:59:17.581 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_type: 0 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_block_index_type: 0 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_shortening: 1 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: checksum: 4 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: no_block_cache: 0 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache: 0x564081c25090 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_name: BinnedLRUCache 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_options: 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: capacity : 536870912 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: num_shard_bits : 4 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: strict_capacity_limit : 0 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: high_pri_pool_ratio: 0.000 2024-02-16T22:59:17.582 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_compressed: (nil) 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: persistent_cache: (nil) 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_size: 4096 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_size_deviation: 10 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_restart_interval: 16 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_block_restart_interval: 1 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: metadata_block_size: 4096 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: partition_filters: 0 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: use_delta_encoding: 1 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: filter_policy: bloomfilter 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: whole_key_filtering: 1 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: verify_compression: 0 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: read_amp_bytes_per_bit: 0 2024-02-16T22:59:17.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: format_version: 5 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: enable_index_compression: 1 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_align: 0 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_auto_readahead_size: 262144 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: prepopulate_block_cache: 0 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: initial_auto_readahead_size: 8192 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression: NoCompression 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T22:59:17.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.num_levels: 7 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:59:17.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T22:59:17.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T22:59:17.587 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T22:59:17.588 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.table_properties_collectors: 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.inplace_update_support: 0 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.bloom_locality: 0 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.max_successive_merges: 0 2024-02-16T22:59:17.589 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.ttl: 2592000 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.enable_blob_files: false 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.min_blob_size: 0 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T22:59:17.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi192/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12160, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c7903066-6769-4878-a246-d0dab5379c8c 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708124357565058, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-02-16T22:59:17.591 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-02-16T22:59:18.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708124357581527, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3613729, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12157, "largest_seqno": 13666, "table_properties": {"data_size": 3606875, "index_size": 4235, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1605, "raw_key_size": 14409, "raw_average_key_size": 23, "raw_value_size": 3593515, "raw_average_value_size": 5814, "num_data_blocks": 193, "num_entries": 618, "num_filter_entries": 618, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708124357, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c7903066-6769-4878-a246-d0dab5379c8c", "db_session_id": "FB98IX6VDQ9FRQ1TE2JD", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-02-16T22:59:18.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708124357581642, "job": 1, "event": "recovery_finished"} 2024-02-16T22:59:18.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-02-16T22:59:18.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi192/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564081d14000 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: DB pointer 0x564081d00000 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** DB Stats ** 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: L0 1/0 3.45 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 492.3 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: L6 1/0 5.06 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Sum 2/0 8.51 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 492.3 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 492.3 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 492.3 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:59:18.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative compaction: 0.00 GB write, 168.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval compaction: 0.00 GB write, 168.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Block cache BinnedLRUCache@0x564081c25090#2 capacity: 512.00 MB usage: 6.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.58 KB,0.000873208%) Misc(1,0.00 KB,0%) 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: starting mon.smithi192 rank 1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi192 fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: mon.smithi192@-1(???) e2 preinit fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: mon.smithi192@-1(???).mds e13 new map 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: mon.smithi192@-1(???).mds e13 print_map 2024-02-16T22:59:18.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: e13 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: legacy client fscid: 1 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Filesystem 'cephfs' (1) 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: fs_name cephfs 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: epoch 10 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: tableserver 0 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: root 0 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: session_timeout 60 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: session_autoclose 300 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_file_size 1099511627776 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_xattr_size 65536 2024-02-16T22:59:18.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: required_client_features {} 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: last_failure 0 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: last_failure_osd_epoch 0 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_mds 1 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: in 0 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: up {0=14474} 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: failed 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: damaged 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: stopped 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_pools [3] 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: metadata_pool 2 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: inline_data enabled 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: balancer 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: bal_rank_mask -1 2024-02-16T22:59:18.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: standby_count_wanted 1 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Standby daemons: 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: mon.smithi192@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: mon.smithi192@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: mon.smithi192@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: mon.smithi192@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 ceph-mon[82652]: mon.smithi192@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 bash[82633]: eb1d711c4dcd147eae17f04420f53e97f0cae3fbb708e143d97fb2fdb73441fa 2024-02-16T22:59:18.041 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:17 smithi192 systemd[1]: Started Ceph mon.smithi192 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T22:59:19.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:19 smithi064 ceph-mon[93757]: pgmap v9: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-16T22:59:19.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:19 smithi192 ceph-mon[82652]: pgmap v9: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-16T22:59:20.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: pgmap v11: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: mon.smithi064 calling monitor election 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: mon.smithi192 calling monitor election 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: mon.smithi064 is new leader, mons smithi064,smithi192 in quorum (ranks 0,1) 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: monmap e3: 2 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: osdmap e50: 6 total, 6 up, 6 in 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: mgrmap e38: smithi064.kubmlk(active, since 16s), standbys: smithi192.pauvih 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:20.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:20.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:20.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: pgmap v11: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T22:59:20.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-16T22:59:20.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: mon.smithi064 calling monitor election 2024-02-16T22:59:20.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: mon.smithi192 calling monitor election 2024-02-16T22:59:20.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: mon.smithi064 is new leader, mons smithi064,smithi192 in quorum (ranks 0,1) 2024-02-16T22:59:20.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: monmap e3: 2 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:59:20.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T22:59:20.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: osdmap e50: 6 total, 6 up, 6 in 2024-02-16T22:59:20.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: mgrmap e38: smithi064.kubmlk(active, since 16s), standbys: smithi192.pauvih 2024-02-16T22:59:20.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:59:20.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:59:20.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:59:20.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-02-16T22:59:20.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:20.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:20.370 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:22.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:22 smithi064 ceph-mon[93757]: pgmap v12: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T22:59:22.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:22.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:22.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:22 smithi192 ceph-mon[82652]: pgmap v12: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T22:59:22.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:22.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:24.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:24 smithi064 ceph-mon[93757]: pgmap v13: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:24.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:24 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:24.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:24 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:24.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:24 smithi192 ceph-mon[82652]: pgmap v13: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:24 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:24.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:24 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:26.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:26 smithi064 ceph-mon[93757]: pgmap v14: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:26.480 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:26 smithi192 ceph-mon[82652]: pgmap v14: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:27.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:27.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:27.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:27.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:59:27.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:27.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:59:27.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:59:27.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:59:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:59:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:59:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:28 smithi064 ceph-mon[93757]: pgmap v15: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:28 smithi064 ceph-mon[93757]: Reconfiguring mon.smithi064 (monmap changed)... 2024-02-16T22:59:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:28 smithi064 ceph-mon[93757]: Reconfiguring daemon mon.smithi064 on smithi064 2024-02-16T22:59:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi064.kubmlk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:59:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:59:28.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:28 smithi192 ceph-mon[82652]: pgmap v15: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:28 smithi192 ceph-mon[82652]: Reconfiguring mon.smithi064 (monmap changed)... 2024-02-16T22:59:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:28 smithi192 ceph-mon[82652]: Reconfiguring daemon mon.smithi064 on smithi064 2024-02-16T22:59:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi064.kubmlk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:59:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:59:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:29.863 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:29 smithi064 ceph-mon[93757]: Reconfiguring mgr.smithi064.kubmlk (monmap changed)... 2024-02-16T22:59:29.864 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:29 smithi064 ceph-mon[93757]: Reconfiguring daemon mgr.smithi064.kubmlk on smithi064 2024-02-16T22:59:30.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:29 smithi192 ceph-mon[82652]: Reconfiguring mgr.smithi064.kubmlk (monmap changed)... 2024-02-16T22:59:30.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:29 smithi192 ceph-mon[82652]: Reconfiguring daemon mgr.smithi064.kubmlk on smithi064 2024-02-16T22:59:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:30 smithi064 ceph-mon[93757]: pgmap v16: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:59:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:30 smithi192 ceph-mon[82652]: pgmap v16: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:31.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:31.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:31.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:59:31.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:31.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:31 smithi064 ceph-mon[93757]: Reconfiguring crash.smithi064 (monmap changed)... 2024-02-16T22:59:31.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:31 smithi064 ceph-mon[93757]: Reconfiguring daemon crash.smithi064 on smithi064 2024-02-16T22:59:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:31 smithi192 ceph-mon[82652]: Reconfiguring crash.smithi064 (monmap changed)... 2024-02-16T22:59:32.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:31 smithi192 ceph-mon[82652]: Reconfiguring daemon crash.smithi064 on smithi064 2024-02-16T22:59:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:32 smithi064 ceph-mon[93757]: pgmap v17: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:32 smithi064 ceph-mon[93757]: Reconfiguring osd.0 (monmap changed)... 2024-02-16T22:59:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:59:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:32 smithi064 ceph-mon[93757]: Reconfiguring daemon osd.0 on smithi064 2024-02-16T22:59:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:32 smithi192 ceph-mon[82652]: pgmap v17: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:32 smithi192 ceph-mon[82652]: Reconfiguring osd.0 (monmap changed)... 2024-02-16T22:59:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:59:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:32 smithi192 ceph-mon[82652]: Reconfiguring daemon osd.0 on smithi064 2024-02-16T22:59:33.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:33 smithi064 ceph-mon[93757]: pgmap v18: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:33.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:59:33.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:33.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:33.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:33 smithi064 ceph-mon[93757]: Reconfiguring osd.1 (monmap changed)... 2024-02-16T22:59:33.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:59:33.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:33.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:33 smithi064 ceph-mon[93757]: Reconfiguring daemon osd.1 on smithi064 2024-02-16T22:59:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:33 smithi192 ceph-mon[82652]: pgmap v18: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:59:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:33 smithi192 ceph-mon[82652]: Reconfiguring osd.1 (monmap changed)... 2024-02-16T22:59:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:59:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:33 smithi192 ceph-mon[82652]: Reconfiguring daemon osd.1 on smithi064 2024-02-16T22:59:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.010+0000 7f8790dcc700 1 -- 172.21.15.64:0/3161028129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fb2e0 msgr2=0x7f878c0fb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.010+0000 7f8790dcc700 1 --2- 172.21.15.64:0/3161028129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fb2e0 0x7f878c0fb6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f877c004940 tx=0x7f877c009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 -- 172.21.15.64:0/3161028129 shutdown_connections 2024-02-16T22:59:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 --2- 172.21.15.64:0/3161028129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f878c0fbcc0 0x7f878c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 --2- 172.21.15.64:0/3161028129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fb2e0 0x7f878c0fb6f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 -- 172.21.15.64:0/3161028129 >> 172.21.15.64:0/3161028129 conn(0x7f878c0f6ca0 msgr2=0x7f878c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 -- 172.21.15.64:0/3161028129 shutdown_connections 2024-02-16T22:59:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 -- 172.21.15.64:0/3161028129 wait complete. 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 Processor -- start 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 -- start start 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f878c0fb2e0 0x7f878c0fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fc2c0 0x7f878c0fc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0fccc0 con 0x7f878c0fc2c0 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8790dcc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0fce00 con 0x7f878c0fb2e0 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8789d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fc2c0 0x7f878c0fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8789d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fc2c0 0x7f878c0fc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48902/0 (socket says 172.21.15.64:48902) 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.011+0000 7f8789d9b700 1 -- 172.21.15.64:0/745125553 learned_addr learned my addr 172.21.15.64:0/745125553 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f878a59c700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f878c0fb2e0 0x7f878c0fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8789d9b700 1 -- 172.21.15.64:0/745125553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f878c0fb2e0 msgr2=0x7f878c0fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8789d9b700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f878c0fb2e0 0x7f878c0fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8789d9b700 1 -- 172.21.15.64:0/745125553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c005040 con 0x7f878c0fc2c0 2024-02-16T22:59:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8789d9b700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fc2c0 0x7f878c0fc730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8774002a50 tx=0x7f8774004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:36.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8782ffd700 1 -- 172.21.15.64:0/745125553 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8774007cc0 con 0x7f878c0fc2c0 2024-02-16T22:59:36.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8782ffd700 1 -- 172.21.15.64:0/745125553 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f877400f040 con 0x7f878c0fc2c0 2024-02-16T22:59:36.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c0fd0e0 con 0x7f878c0fc2c0 2024-02-16T22:59:36.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8782ffd700 1 -- 172.21.15.64:0/745125553 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f877400d9c0 con 0x7f878c0fc2c0 2024-02-16T22:59:36.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.012+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c115d90 con 0x7f878c0fc2c0 2024-02-16T22:59:36.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.013+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c107ab0 con 0x7f878c0fc2c0 2024-02-16T22:59:36.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.014+0000 7f8782ffd700 1 -- 172.21.15.64:0/745125553 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f8774009730 con 0x7f878c0fc2c0 2024-02-16T22:59:36.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.014+0000 7f8782ffd700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f877806d900 0x7f877806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.014+0000 7f8782ffd700 1 -- 172.21.15.64:0/745125553 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f877408ce70 con 0x7f878c0fc2c0 2024-02-16T22:59:36.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.014+0000 7f878a59c700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f877806d900 0x7f877806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:36.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.015+0000 7f878a59c700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f877806d900 0x7f877806fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f877c00b3b0 tx=0x7f877c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:36.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.019+0000 7f8782ffd700 1 -- 172.21.15.64:0/745125553 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f877405a020 con 0x7f878c0fc2c0 2024-02-16T22:59:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.243+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f878c0fbdd0 con 0x7f877806d900 2024-02-16T22:59:36.244 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:36 smithi064 ceph-mon[93757]: pgmap v19: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:36.244 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:36 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:36.244 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:36 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:36.244 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:36 smithi064 ceph-mon[93757]: Reconfiguring osd.2 (monmap changed)... 2024-02-16T22:59:36.244 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:36 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:59:36.244 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:36 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:36.245 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:36 smithi064 ceph-mon[93757]: Reconfiguring daemon osd.2 on smithi064 2024-02-16T22:59:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.244+0000 7f8782ffd700 1 -- 172.21.15.64:0/745125553 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f878c0fbdd0 con 0x7f877806d900 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f877806d900 msgr2=0x7f877806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f877806d900 0x7f877806fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f877c00b3b0 tx=0x7f877c009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fc2c0 msgr2=0x7f878c0fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fc2c0 0x7f878c0fc730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8774002a50 tx=0x7f8774004600 comp rx=0 tx=0).stop 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 shutdown_connections 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f877806d900 0x7f877806fdb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f878c0fb2e0 0x7f878c0fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 --2- 172.21.15.64:0/745125553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f878c0fc2c0 0x7f878c0fc730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 >> 172.21.15.64:0/745125553 conn(0x7f878c0f6ca0 msgr2=0x7f878c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 shutdown_connections 2024-02-16T22:59:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.246+0000 7f8790dcc700 1 -- 172.21.15.64:0/745125553 wait complete. 2024-02-16T22:59:36.257 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.350+0000 7fbeada45700 1 -- 172.21.15.64:0/3372170036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 msgr2=0x7fbea80fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.350+0000 7fbeada45700 1 --2- 172.21.15.64:0/3372170036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 0x7fbea80fc250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbe98004940 tx=0x7fbe98009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.350+0000 7fbeada45700 1 -- 172.21.15.64:0/3372170036 shutdown_connections 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.350+0000 7fbeada45700 1 --2- 172.21.15.64:0/3372170036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea80fcfb0 0x7fbea80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.350+0000 7fbeada45700 1 --2- 172.21.15.64:0/3372170036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 0x7fbea80fc250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.350+0000 7fbeada45700 1 -- 172.21.15.64:0/3372170036 >> 172.21.15.64:0/3372170036 conn(0x7fbea80f7520 msgr2=0x7fbea80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.350+0000 7fbeada45700 1 -- 172.21.15.64:0/3372170036 shutdown_connections 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.350+0000 7fbeada45700 1 -- 172.21.15.64:0/3372170036 wait complete. 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbeada45700 1 Processor -- start 2024-02-16T22:59:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbeada45700 1 -- start start 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbeada45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 0x7fbea810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbeada45700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea80fcfb0 0x7fbea810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbeada45700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea810e490 con 0x7fbea80fbe40 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbeada45700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea810ed30 con 0x7fbea80fcfb0 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbea6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 0x7fbea810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbea6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 0x7fbea810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48920/0 (socket says 172.21.15.64:48920) 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.351+0000 7fbea6ffd700 1 -- 172.21.15.64:0/459855639 learned_addr learned my addr 172.21.15.64:0/459855639 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbea6ffd700 1 -- 172.21.15.64:0/459855639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea80fcfb0 msgr2=0x7fbea810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbea67fc700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea80fcfb0 0x7fbea810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbea6ffd700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea80fcfb0 0x7fbea810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbea6ffd700 1 -- 172.21.15.64:0/459855639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe98005040 con 0x7fbea80fbe40 2024-02-16T22:59:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbea67fc700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea80fcfb0 0x7fbea810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:59:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbea6ffd700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 0x7fbea810d950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009fd0 tx=0x7fbe98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbe8f7fe700 1 -- 172.21.15.64:0/459855639 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe9800e040 con 0x7fbea80fbe40 2024-02-16T22:59:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea810efb0 con 0x7fbea80fbe40 2024-02-16T22:59:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbe8f7fe700 1 -- 172.21.15.64:0/459855639 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe9800bcc0 con 0x7fbea80fbe40 2024-02-16T22:59:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbe8f7fe700 1 -- 172.21.15.64:0/459855639 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe9801c630 con 0x7fbea80fbe40 2024-02-16T22:59:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.352+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea810f4f0 con 0x7fbea80fbe40 2024-02-16T22:59:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.353+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea804bed0 con 0x7fbea80fbe40 2024-02-16T22:59:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.354+0000 7fbe8f7fe700 1 -- 172.21.15.64:0/459855639 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fbe98013980 con 0x7fbea80fbe40 2024-02-16T22:59:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.354+0000 7fbe8f7fe700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fbe9006d900 0x7fbe9006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.354+0000 7fbe8f7fe700 1 -- 172.21.15.64:0/459855639 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbe9808e590 con 0x7fbea80fbe40 2024-02-16T22:59:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.354+0000 7fbea67fc700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fbe9006d900 0x7fbe9006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.355+0000 7fbea67fc700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fbe9006d900 0x7fbe9006fdb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009fd0 tx=0x7fbe9c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:36.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.359+0000 7fbe8f7fe700 1 -- 172.21.15.64:0/459855639 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe9805b7c0 con 0x7fbea80fbe40 2024-02-16T22:59:36.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:36 smithi192 ceph-mon[82652]: pgmap v19: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:36 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:36 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:36 smithi192 ceph-mon[82652]: Reconfiguring osd.2 (monmap changed)... 2024-02-16T22:59:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:36 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:59:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:36 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:36.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:36 smithi192 ceph-mon[82652]: Reconfiguring daemon osd.2 on smithi064 2024-02-16T22:59:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.586+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbea810fb60 con 0x7fbe9006d900 2024-02-16T22:59:36.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.588+0000 7fbe8f7fe700 1 -- 172.21.15.64:0/459855639 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fbea810fb60 con 0x7fbe9006d900 2024-02-16T22:59:36.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fbe9006d900 msgr2=0x7fbe9006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fbe9006d900 0x7fbe9006fdb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009fd0 tx=0x7fbe9c00b040 comp rx=0 tx=0).stop 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 msgr2=0x7fbea810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 0x7fbea810d950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009fd0 tx=0x7fbe98009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 shutdown_connections 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fbe9006d900 0x7fbe9006fdb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea80fbe40 0x7fbea810d950 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 --2- 172.21.15.64:0/459855639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea80fcfb0 0x7fbea810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 >> 172.21.15.64:0/459855639 conn(0x7fbea80f7520 msgr2=0x7fbea8065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 shutdown_connections 2024-02-16T22:59:36.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.590+0000 7fbeada45700 1 -- 172.21.15.64:0/459855639 wait complete. 2024-02-16T22:59:36.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.692+0000 7fc6d6eec700 1 -- 172.21.15.64:0/2948094476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 msgr2=0x7fc6d00fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.692+0000 7fc6d6eec700 1 --2- 172.21.15.64:0/2948094476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 0x7fc6d00fd420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc004940 tx=0x7fc6bc009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.692+0000 7fc6d6eec700 1 -- 172.21.15.64:0/2948094476 shutdown_connections 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.692+0000 7fc6d6eec700 1 --2- 172.21.15.64:0/2948094476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 0x7fc6d00fd420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.692+0000 7fc6d6eec700 1 --2- 172.21.15.64:0/2948094476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d00fbe40 0x7fc6d00fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.692+0000 7fc6d6eec700 1 -- 172.21.15.64:0/2948094476 >> 172.21.15.64:0/2948094476 conn(0x7fc6d00f74e0 msgr2=0x7fc6d00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.692+0000 7fc6d6eec700 1 -- 172.21.15.64:0/2948094476 shutdown_connections 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.692+0000 7fc6d6eec700 1 -- 172.21.15.64:0/2948094476 wait complete. 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6d6eec700 1 Processor -- start 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6d6eec700 1 -- start start 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6d6eec700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d00fbe40 0x7fc6d010c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6d6eec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 0x7fc6d010cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6d6eec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d010d420 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6d6eec700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d010dc60 con 0x7fc6d00fbe40 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 0x7fc6d010cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 0x7fc6d010cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48942/0 (socket says 172.21.15.64:48942) 2024-02-16T22:59:36.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6cffff700 1 -- 172.21.15.64:0/140226008 learned_addr learned my addr 172.21.15.64:0/140226008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6cffff700 1 -- 172.21.15.64:0/140226008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d00fbe40 msgr2=0x7fc6d010c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6cffff700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d00fbe40 0x7fc6d010c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6cffff700 1 -- 172.21.15.64:0/140226008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc005040 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.693+0000 7fc6cffff700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 0x7fc6d010cd90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0093b0 tx=0x7fc6bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.694+0000 7fc6cd7fa700 1 -- 172.21.15.64:0/140226008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6bc00e040 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.694+0000 7fc6cd7fa700 1 -- 172.21.15.64:0/140226008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6bc00bcc0 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.694+0000 7fc6cd7fa700 1 -- 172.21.15.64:0/140226008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6bc01c630 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.694+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d010dee0 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.694+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d010e420 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.695+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d004bed0 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.695+0000 7fc6cd7fa700 1 -- 172.21.15.64:0/140226008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fc6bc0139f0 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.696+0000 7fc6cd7fa700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc6c006d900 0x7fc6c006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:36.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.696+0000 7fc6cd7fa700 1 -- 172.21.15.64:0/140226008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc6bc08e560 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.696+0000 7fc6d4c88700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc6c006d900 0x7fc6c006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:36.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.697+0000 7fc6d4c88700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc6c006d900 0x7fc6c006fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4006db0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.701+0000 7fc6cd7fa700 1 -- 172.21.15.64:0/140226008 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6bc05b790 con 0x7fc6d00fcfb0 2024-02-16T22:59:36.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.918+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6d0101990 con 0x7fc6c006d900 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.929+0000 7fc6cd7fa700 1 -- 172.21.15.64:0/140226008 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc6d0101990 con 0x7fc6c006d900 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (2m) 28s ago 11m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (11m) 28s ago 11m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 85668a27934b 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (10m) 14s ago 10m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 6f67704fa297 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (2m) 28s ago 10m 81.5M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (5m) 28s ago 5m 17.1M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (5m) 28s ago 5m 154M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (5m) 14s ago 5m 16.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (5m) 14s ago 5m 16.0M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T22:59:36.930 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (103s) 28s ago 12m 473M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (57s) 14s ago 10m 447M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (43s) 28s ago 12m 28.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (19s) 14s ago 10m 25.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (3m) 28s ago 11m 15.6M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (111s) 14s ago 10m 15.4M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (9m) 28s ago 9m 210M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (8m) 28s ago 8m 216M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (8m) 28s ago 8m 175M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (8m) 14s ago 8m 233M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (7m) 14s ago 7m 209M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (7m) 14s ago 7m 185M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T22:59:36.931 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (62s) 28s ago 10m 47.0M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T22:59:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc6c006d900 msgr2=0x7fc6c006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc6c006d900 0x7fc6c006fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4006db0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 msgr2=0x7fc6d010cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 0x7fc6d010cd90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0093b0 tx=0x7fc6bc009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 shutdown_connections 2024-02-16T22:59:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc6c006d900 0x7fc6c006fdb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d00fbe40 0x7fc6d010c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 --2- 172.21.15.64:0/140226008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d00fcfb0 0x7fc6d010cd90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 >> 172.21.15.64:0/140226008 conn(0x7fc6d00f74e0 msgr2=0x7fc6d0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 shutdown_connections 2024-02-16T22:59:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:36.931+0000 7fc6d6eec700 1 -- 172.21.15.64:0/140226008 wait complete. 2024-02-16T22:59:37.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.035+0000 7fb0cb85e700 1 -- 172.21.15.64:0/297207055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 msgr2=0x7fb0c40fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.035+0000 7fb0cb85e700 1 --2- 172.21.15.64:0/297207055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 0x7fb0c40fe550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4004940 tx=0x7fb0b4009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:37.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.035+0000 7fb0cb85e700 1 -- 172.21.15.64:0/297207055 shutdown_connections 2024-02-16T22:59:37.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.035+0000 7fb0cb85e700 1 --2- 172.21.15.64:0/297207055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 0x7fb0c40fe550 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.035+0000 7fb0cb85e700 1 --2- 172.21.15.64:0/297207055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40fcef0 0x7fb0c40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.035+0000 7fb0cb85e700 1 -- 172.21.15.64:0/297207055 >> 172.21.15.64:0/297207055 conn(0x7fb0c40f8620 msgr2=0x7fb0c40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:37.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.036+0000 7fb0cb85e700 1 -- 172.21.15.64:0/297207055 shutdown_connections 2024-02-16T22:59:37.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.036+0000 7fb0cb85e700 1 -- 172.21.15.64:0/297207055 wait complete. 2024-02-16T22:59:37.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.036+0000 7fb0cb85e700 1 Processor -- start 2024-02-16T22:59:37.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.036+0000 7fb0cb85e700 1 -- start start 2024-02-16T22:59:37.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.036+0000 7fb0cb85e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40fcef0 0x7fb0c410d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0cb85e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 0x7fb0c410de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0cb85e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c410e410 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0cb85e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c410ecb0 con 0x7fb0c40fcef0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0c8df9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 0x7fb0c410de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0c8df9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 0x7fb0c410de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48952/0 (socket says 172.21.15.64:48952) 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0c8df9700 1 -- 172.21.15.64:0/87405413 learned_addr learned my addr 172.21.15.64:0/87405413 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0c95fa700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40fcef0 0x7fb0c410d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0c8df9700 1 -- 172.21.15.64:0/87405413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40fcef0 msgr2=0x7fb0c410d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0c8df9700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40fcef0 0x7fb0c410d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0c8df9700 1 -- 172.21.15.64:0/87405413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4005040 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0c8df9700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 0x7fb0c410de10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000c00 tx=0x7fb0b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0b9ffb700 1 -- 172.21.15.64:0/87405413 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0b400e070 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0b9ffb700 1 -- 172.21.15.64:0/87405413 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0b4009430 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c410ef30 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.037+0000 7fb0b9ffb700 1 -- 172.21.15.64:0/87405413 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0b400b410 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.038+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c410f350 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.039+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c404bed0 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.039+0000 7fb0b9ffb700 1 -- 172.21.15.64:0/87405413 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fb0b401c820 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.040+0000 7fb0b9ffb700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0b006d900 0x7fb0b006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.040+0000 7fb0b9ffb700 1 -- 172.21.15.64:0/87405413 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb0b408e660 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.040+0000 7fb0c95fa700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0b006d900 0x7fb0b006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.040+0000 7fb0c95fa700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0b006d900 0x7fb0b006fdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009b80 tx=0x7fb0c000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:37.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.044+0000 7fb0b9ffb700 1 -- 172.21.15.64:0/87405413 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0b405b930 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.338 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:37 smithi064 ceph-mon[93757]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:37.339 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:37 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:37.339 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:37 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:37.339 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:37 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.tqesxx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:59:37.339 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:37 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:37.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.338+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb0c4105a30 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.338+0000 7fb0b9ffb700 1 -- 172.21.15.64:0/87405413 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fb0b405b4c0 con 0x7fb0c40fe0e0 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:59:37.340 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T22:59:37.341 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T22:59:37.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T22:59:37.341 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T22:59:37.341 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 10, 2024-02-16T22:59:37.341 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:59:37.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T22:59:37.341 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0b006d900 msgr2=0x7fb0b006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0b006d900 0x7fb0b006fdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009b80 tx=0x7fb0c000b040 comp rx=0 tx=0).stop 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 msgr2=0x7fb0c410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 0x7fb0c410de10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000c00 tx=0x7fb0b4009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 shutdown_connections 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb0b006d900 0x7fb0b006fdb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40fcef0 0x7fb0c410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 --2- 172.21.15.64:0/87405413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40fe0e0 0x7fb0c410de10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 >> 172.21.15.64:0/87405413 conn(0x7fb0c40f8620 msgr2=0x7fb0c41013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 shutdown_connections 2024-02-16T22:59:37.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.340+0000 7fb0cb85e700 1 -- 172.21.15.64:0/87405413 wait complete. 2024-02-16T22:59:37.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.442+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/2411191283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 msgr2=0x7fa8f40fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.442+0000 7fa8fa7bf700 1 --2- 172.21.15.64:0/2411191283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 0x7fa8f40fd320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4004940 tx=0x7fa8e4009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:37.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.442+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/2411191283 shutdown_connections 2024-02-16T22:59:37.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.442+0000 7fa8fa7bf700 1 --2- 172.21.15.64:0/2411191283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f40fe0e0 0x7fa8f40fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.442+0000 7fa8fa7bf700 1 --2- 172.21.15.64:0/2411191283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 0x7fa8f40fd320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.442+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/2411191283 >> 172.21.15.64:0/2411191283 conn(0x7fa8f40f8620 msgr2=0x7fa8f40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:37.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.442+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/2411191283 shutdown_connections 2024-02-16T22:59:37.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.442+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/2411191283 wait complete. 2024-02-16T22:59:37.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8fa7bf700 1 Processor -- start 2024-02-16T22:59:37.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8fa7bf700 1 -- start start 2024-02-16T22:59:37.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8fa7bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 0x7fa8f410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8fa7bf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f40fe0e0 0x7fa8f410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8fa7bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f410e560 con 0x7fa8f40fcef0 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8fa7bf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f410eda0 con 0x7fa8f40fe0e0 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 0x7fa8f410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 0x7fa8f410d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48976/0 (socket says 172.21.15.64:48976) 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.443+0000 7fa8f3fff700 1 -- 172.21.15.64:0/3670086765 learned_addr learned my addr 172.21.15.64:0/3670086765 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f37fe700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f40fe0e0 0x7fa8f410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f3fff700 1 -- 172.21.15.64:0/3670086765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f40fe0e0 msgr2=0x7fa8f410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f3fff700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f40fe0e0 0x7fa8f410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f3fff700 1 -- 172.21.15.64:0/3670086765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e4005040 con 0x7fa8f40fcef0 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f37fe700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f40fe0e0 0x7fa8f410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f3fff700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 0x7fa8f410d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4000c00 tx=0x7fa8e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f0ff9700 1 -- 172.21.15.64:0/3670086765 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8e400e040 con 0x7fa8f40fcef0 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f0ff9700 1 -- 172.21.15.64:0/3670086765 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8e400bcc0 con 0x7fa8f40fcef0 2024-02-16T22:59:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f410f020 con 0x7fa8f40fcef0 2024-02-16T22:59:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8f0ff9700 1 -- 172.21.15.64:0/3670086765 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8e401c630 con 0x7fa8f40fcef0 2024-02-16T22:59:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.444+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f410f560 con 0x7fa8f40fcef0 2024-02-16T22:59:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.445+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f404bed0 con 0x7fa8f40fcef0 2024-02-16T22:59:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.446+0000 7fa8f0ff9700 1 -- 172.21.15.64:0/3670086765 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fa8e4013420 con 0x7fa8f40fcef0 2024-02-16T22:59:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.446+0000 7fa8f0ff9700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa8dc06d900 0x7fa8dc06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.446+0000 7fa8f0ff9700 1 -- 172.21.15.64:0/3670086765 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa8e408e540 con 0x7fa8f40fcef0 2024-02-16T22:59:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.446+0000 7fa8f37fe700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa8dc06d900 0x7fa8dc06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.447+0000 7fa8f37fe700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa8dc06d900 0x7fa8dc06fdb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009b10 tx=0x7fa8e8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.451+0000 7fa8f0ff9700 1 -- 172.21.15.64:0/3670086765 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8e405b6f0 con 0x7fa8f40fcef0 2024-02-16T22:59:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:37 smithi192 ceph-mon[82652]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:37 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:37 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:37 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.tqesxx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:59:37.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:37 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:37.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.703+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa8f4105ab0 con 0x7fa8f40fcef0 2024-02-16T22:59:37.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.704+0000 7fa8f0ff9700 1 -- 172.21.15.64:0/3670086765 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7fa8e405b280 con 0x7fa8f40fcef0 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T22:59:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T22:59:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T22:59:37.709 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T22:59:37.710 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:37.710 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:37.710 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:59:37.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa8dc06d900 msgr2=0x7fa8dc06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa8dc06d900 0x7fa8dc06fdb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009b10 tx=0x7fa8e8008040 comp rx=0 tx=0).stop 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 msgr2=0x7fa8f410d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 0x7fa8f410d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4000c00 tx=0x7fa8e4009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 shutdown_connections 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa8dc06d900 0x7fa8dc06fdb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8f40fcef0 0x7fa8f410d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 --2- 172.21.15.64:0/3670086765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f40fe0e0 0x7fa8f410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 >> 172.21.15.64:0/3670086765 conn(0x7fa8f40f8620 msgr2=0x7fa8f41013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 shutdown_connections 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.706+0000 7fa8fa7bf700 1 -- 172.21.15.64:0/3670086765 wait complete. 2024-02-16T22:59:37.711 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T22:59:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.812+0000 7f7e3b779700 1 -- 172.21.15.64:0/623610311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e34066c70 msgr2=0x7f7e340ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.812+0000 7f7e3b779700 1 --2- 172.21.15.64:0/623610311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e34066c70 0x7f7e340ffb00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e30004940 tx=0x7f7e30009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.812+0000 7f7e3b779700 1 -- 172.21.15.64:0/623610311 shutdown_connections 2024-02-16T22:59:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.812+0000 7f7e3b779700 1 --2- 172.21.15.64:0/623610311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e341000d0 0x7f7e34102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.812+0000 7f7e3b779700 1 --2- 172.21.15.64:0/623610311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e34066c70 0x7f7e340ffb00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.812+0000 7f7e3b779700 1 -- 172.21.15.64:0/623610311 >> 172.21.15.64:0/623610311 conn(0x7f7e340f7510 msgr2=0x7f7e340f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.812+0000 7f7e3b779700 1 -- 172.21.15.64:0/623610311 shutdown_connections 2024-02-16T22:59:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.812+0000 7f7e3b779700 1 -- 172.21.15.64:0/623610311 wait complete. 2024-02-16T22:59:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e3b779700 1 Processor -- start 2024-02-16T22:59:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e3b779700 1 -- start start 2024-02-16T22:59:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e3b779700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e34066c70 0x7f7e3410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e3b779700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e341000d0 0x7f7e3410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e3b779700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3410e4c0 con 0x7f7e341000d0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e3b779700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3410ed60 con 0x7f7e34066c70 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e38d14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e341000d0 0x7f7e3410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e38d14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e341000d0 0x7f7e3410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48986/0 (socket says 172.21.15.64:48986) 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e39515700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e34066c70 0x7f7e3410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e38d14700 1 -- 172.21.15.64:0/172757440 learned_addr learned my addr 172.21.15.64:0/172757440 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e38d14700 1 -- 172.21.15.64:0/172757440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e34066c70 msgr2=0x7f7e3410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e38d14700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e34066c70 0x7f7e3410d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.813+0000 7f7e38d14700 1 -- 172.21.15.64:0/172757440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30005040 con 0x7f7e341000d0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.814+0000 7f7e38d14700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e341000d0 0x7f7e3410dec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e24002a50 tx=0x7f7e24004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.814+0000 7f7e29ffb700 1 -- 172.21.15.64:0/172757440 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e24007cc0 con 0x7f7e341000d0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.814+0000 7f7e29ffb700 1 -- 172.21.15.64:0/172757440 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e2400f040 con 0x7f7e341000d0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.814+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3410f040 con 0x7f7e341000d0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.814+0000 7f7e29ffb700 1 -- 172.21.15.64:0/172757440 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e2400d9c0 con 0x7f7e341000d0 2024-02-16T22:59:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.814+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3410f580 con 0x7f7e341000d0 2024-02-16T22:59:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.815+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e34107a80 con 0x7f7e341000d0 2024-02-16T22:59:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.815+0000 7f7e29ffb700 1 -- 172.21.15.64:0/172757440 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f7e24009730 con 0x7f7e341000d0 2024-02-16T22:59:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.816+0000 7f7e29ffb700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7e2006d900 0x7f7e2006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.816+0000 7f7e29ffb700 1 -- 172.21.15.64:0/172757440 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7e2408ce80 con 0x7f7e341000d0 2024-02-16T22:59:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.816+0000 7f7e39515700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7e2006d900 0x7f7e2006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.816+0000 7f7e39515700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7e2006d900 0x7f7e2006fdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000b3b0 tx=0x7f7e30009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:37.820+0000 7f7e29ffb700 1 -- 172.21.15.64:0/172757440 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e2405a030 con 0x7f7e341000d0 2024-02-16T22:59:38.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.049+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e3410f810 con 0x7f7e2006d900 2024-02-16T22:59:38.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.050+0000 7f7e29ffb700 1 -- 172.21.15.64:0/172757440 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f7e3410f810 con 0x7f7e2006d900 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr" 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "4/21 daemons upgraded", 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading mon daemons", 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T22:59:38.051 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T22:59:38.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7e2006d900 msgr2=0x7f7e2006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:38.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7e2006d900 0x7f7e2006fdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000b3b0 tx=0x7f7e30009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:38.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e341000d0 msgr2=0x7f7e3410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:38.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e341000d0 0x7f7e3410dec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e24002a50 tx=0x7f7e24004600 comp rx=0 tx=0).stop 2024-02-16T22:59:38.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 shutdown_connections 2024-02-16T22:59:38.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7e2006d900 0x7f7e2006fdb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e34066c70 0x7f7e3410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 --2- 172.21.15.64:0/172757440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e341000d0 0x7f7e3410dec0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 >> 172.21.15.64:0/172757440 conn(0x7f7e340f7510 msgr2=0x7f7e340f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:38.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 shutdown_connections 2024-02-16T22:59:38.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.052+0000 7f7e3b779700 1 -- 172.21.15.64:0/172757440 wait complete. 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.157+0000 7fe945a57700 1 -- 172.21.15.64:0/1276413355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fcf30 msgr2=0x7fe9400fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.157+0000 7fe945a57700 1 --2- 172.21.15.64:0/1276413355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fcf30 0x7fe9400fd340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe930004940 tx=0x7fe930009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.157+0000 7fe945a57700 1 -- 172.21.15.64:0/1276413355 shutdown_connections 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.157+0000 7fe945a57700 1 --2- 172.21.15.64:0/1276413355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9400fe1e0 0x7fe9400fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.157+0000 7fe945a57700 1 --2- 172.21.15.64:0/1276413355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fcf30 0x7fe9400fd340 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.157+0000 7fe945a57700 1 -- 172.21.15.64:0/1276413355 >> 172.21.15.64:0/1276413355 conn(0x7fe9400f86b0 msgr2=0x7fe9400faac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.157+0000 7fe945a57700 1 -- 172.21.15.64:0/1276413355 shutdown_connections 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.157+0000 7fe945a57700 1 -- 172.21.15.64:0/1276413355 wait complete. 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe945a57700 1 Processor -- start 2024-02-16T22:59:38.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe945a57700 1 -- start start 2024-02-16T22:59:38.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe945a57700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9400fcf30 0x7fe94010d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:38.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe945a57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fe1e0 0x7fe94010ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:38.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe945a57700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94010e3d0 con 0x7fe9400fe1e0 2024-02-16T22:59:38.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe945a57700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94010ec70 con 0x7fe9400fcf30 2024-02-16T22:59:38.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe93e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fe1e0 0x7fe94010ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:38.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe93e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fe1e0 0x7fe94010ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48998/0 (socket says 172.21.15.64:48998) 2024-02-16T22:59:38.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe93e7fc700 1 -- 172.21.15.64:0/1295390075 learned_addr learned my addr 172.21.15.64:0/1295390075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T22:59:38.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.158+0000 7fe93effd700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9400fcf30 0x7fe94010d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:38.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe93e7fc700 1 -- 172.21.15.64:0/1295390075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9400fcf30 msgr2=0x7fe94010d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:38.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe93e7fc700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9400fcf30 0x7fe94010d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe93e7fc700 1 -- 172.21.15.64:0/1295390075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe930005040 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe93e7fc700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fe1e0 0x7fe94010ddd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe934002a80 tx=0x7fe9340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe9277fe700 1 -- 172.21.15.64:0/1295390075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe934007cc0 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe9277fe700 1 -- 172.21.15.64:0/1295390075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe93400f040 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94010ef50 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe9277fe700 1 -- 172.21.15.64:0/1295390075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe93400dac0 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.159+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94010f490 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.160+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94004bed0 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.161+0000 7fe9277fe700 1 -- 172.21.15.64:0/1295390075 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fe93400dcc0 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.161+0000 7fe9277fe700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe92806d900 0x7fe92806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.161+0000 7fe9277fe700 1 -- 172.21.15.64:0/1295390075 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe93408cdb0 con 0x7fe9400fe1e0 2024-02-16T22:59:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.161+0000 7fe93effd700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe92806d900 0x7fe92806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:59:38.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.162+0000 7fe93effd700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe92806d900 0x7fe92806fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9300043d0 tx=0x7fe930009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:59:38.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.166+0000 7fe9277fe700 1 -- 172.21.15.64:0/1295390075 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe934059fe0 con 0x7fe9400fe1e0 2024-02-16T22:59:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:38 smithi064 ceph-mon[93757]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:38 smithi064 ceph-mon[93757]: pgmap v20: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:38 smithi064 ceph-mon[93757]: Reconfiguring mds.cephfs.smithi064.tqesxx (monmap changed)... 2024-02-16T22:59:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:38 smithi064 ceph-mon[93757]: Reconfiguring daemon mds.cephfs.smithi064.tqesxx on smithi064 2024-02-16T22:59:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:38 smithi064 ceph-mon[93757]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:38 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/87405413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:38 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3670086765' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:59:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.460+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe940105a20 con 0x7fe9400fe1e0 2024-02-16T22:59:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.460+0000 7fe9277fe700 1 -- 172.21.15.64:0/1295390075 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe934059b70 con 0x7fe9400fe1e0 2024-02-16T22:59:38.461 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:59:38.461 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:59:38.461 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe92806d900 msgr2=0x7fe92806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe92806d900 0x7fe92806fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9300043d0 tx=0x7fe930009e30 comp rx=0 tx=0).stop 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fe1e0 msgr2=0x7fe94010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fe1e0 0x7fe94010ddd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe934002a80 tx=0x7fe9340046b0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 shutdown_connections 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe92806d900 0x7fe92806fdb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9400fcf30 0x7fe94010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 --2- 172.21.15.64:0/1295390075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9400fe1e0 0x7fe94010ddd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 >> 172.21.15.64:0/1295390075 conn(0x7fe9400f86b0 msgr2=0x7fe940101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 shutdown_connections 2024-02-16T22:59:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T22:59:38.463+0000 7fe945a57700 1 -- 172.21.15.64:0/1295390075 wait complete. 2024-02-16T22:59:38.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:38 smithi192 ceph-mon[82652]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:38 smithi192 ceph-mon[82652]: pgmap v20: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:38 smithi192 ceph-mon[82652]: Reconfiguring mds.cephfs.smithi064.tqesxx (monmap changed)... 2024-02-16T22:59:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:38 smithi192 ceph-mon[82652]: Reconfiguring daemon mds.cephfs.smithi064.tqesxx on smithi064 2024-02-16T22:59:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:38 smithi192 ceph-mon[82652]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:38 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/87405413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:38.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:38 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3670086765' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:59:39.358 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:39 smithi064 ceph-mon[93757]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:39.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:39 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1295390075' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:59:39.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:39.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:39.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:39 smithi064 ceph-mon[93757]: Reconfiguring mds.cephfs.smithi064.bmcyuf (monmap changed)... 2024-02-16T22:59:39.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.bmcyuf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:59:39.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:39.359 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:39 smithi064 ceph-mon[93757]: Reconfiguring daemon mds.cephfs.smithi064.bmcyuf on smithi064 2024-02-16T22:59:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:39 smithi192 ceph-mon[82652]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:59:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:39 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1295390075' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:59:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:39 smithi192 ceph-mon[82652]: Reconfiguring mds.cephfs.smithi064.bmcyuf (monmap changed)... 2024-02-16T22:59:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.bmcyuf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:59:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:39.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:39 smithi192 ceph-mon[82652]: Reconfiguring daemon mds.cephfs.smithi064.bmcyuf on smithi064 2024-02-16T22:59:40.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:40 smithi192 ceph-mon[82652]: pgmap v21: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:40.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:40 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:40.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:40 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:40.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:40 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:59:40.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:40 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:40.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:40 smithi064 ceph-mon[93757]: pgmap v21: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:40.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:40 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:40.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:40 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:40.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:40 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:59:40.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:40 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:41.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:41 smithi192 ceph-mon[82652]: Reconfiguring crash.smithi192 (monmap changed)... 2024-02-16T22:59:41.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:41 smithi192 ceph-mon[82652]: Reconfiguring daemon crash.smithi192 on smithi192 2024-02-16T22:59:41.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:41 smithi064 ceph-mon[93757]: Reconfiguring crash.smithi192 (monmap changed)... 2024-02-16T22:59:41.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:41 smithi064 ceph-mon[93757]: Reconfiguring daemon crash.smithi192 on smithi192 2024-02-16T22:59:42.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:42 smithi064 ceph-mon[93757]: pgmap v22: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:42.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:42 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:42.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:42 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:42.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:42 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:59:42.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:42 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:59:42.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:42 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:42.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:42 smithi192 ceph-mon[82652]: pgmap v22: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:42.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:42 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:42.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:42 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:42.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:42 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.pauvih", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:59:42.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:42 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:59:42.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:42 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:43.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:43 smithi192 ceph-mon[82652]: Reconfiguring mgr.smithi192.pauvih (monmap changed)... 2024-02-16T22:59:43.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:43 smithi192 ceph-mon[82652]: Reconfiguring daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:59:43.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:43 smithi064 ceph-mon[93757]: Reconfiguring mgr.smithi192.pauvih (monmap changed)... 2024-02-16T22:59:43.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:43 smithi064 ceph-mon[93757]: Reconfiguring daemon mgr.smithi192.pauvih on smithi192 2024-02-16T22:59:44.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:44 smithi192 ceph-mon[82652]: pgmap v23: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:44.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:44.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:44.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:59:44.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:59:44.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:44.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:44 smithi064 ceph-mon[93757]: pgmap v23: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:44.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:44.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:44.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:59:44.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:59:44.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:45.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:45 smithi064 ceph-mon[93757]: Reconfiguring mon.smithi192 (monmap changed)... 2024-02-16T22:59:45.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:45 smithi064 ceph-mon[93757]: Reconfiguring daemon mon.smithi192 on smithi192 2024-02-16T22:59:45.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:45 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:45.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:45 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:45.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:45 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:59:45.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:45 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:45.897 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:45 smithi192 ceph-mon[82652]: Reconfiguring mon.smithi192 (monmap changed)... 2024-02-16T22:59:45.897 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:45 smithi192 ceph-mon[82652]: Reconfiguring daemon mon.smithi192 on smithi192 2024-02-16T22:59:45.897 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:45 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:45.898 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:45 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:45.898 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:45 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:59:45.898 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:45 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:46.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:46 smithi192 ceph-mon[82652]: pgmap v24: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:46 smithi192 ceph-mon[82652]: Reconfiguring osd.3 (monmap changed)... 2024-02-16T22:59:46.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:46 smithi192 ceph-mon[82652]: Reconfiguring daemon osd.3 on smithi192 2024-02-16T22:59:46.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:46 smithi064 ceph-mon[93757]: pgmap v24: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:46.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:46 smithi064 ceph-mon[93757]: Reconfiguring osd.3 (monmap changed)... 2024-02-16T22:59:46.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:46 smithi064 ceph-mon[93757]: Reconfiguring daemon osd.3 on smithi192 2024-02-16T22:59:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:47 smithi192 ceph-mon[82652]: pgmap v25: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:47 smithi192 ceph-mon[82652]: Reconfiguring osd.4 (monmap changed)... 2024-02-16T22:59:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:59:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:47 smithi192 ceph-mon[82652]: Reconfiguring daemon osd.4 on smithi192 2024-02-16T22:59:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:59:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:47 smithi064 ceph-mon[93757]: pgmap v25: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:47 smithi064 ceph-mon[93757]: Reconfiguring osd.4 (monmap changed)... 2024-02-16T22:59:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:59:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:47 smithi064 ceph-mon[93757]: Reconfiguring daemon osd.4 on smithi192 2024-02-16T22:59:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:59:49.736 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:49 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:49.736 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:49 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:49.736 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:49 smithi192 ceph-mon[82652]: Reconfiguring osd.5 (monmap changed)... 2024-02-16T22:59:49.736 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:49 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:59:49.736 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:49 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:49.736 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:49 smithi192 ceph-mon[82652]: Reconfiguring daemon osd.5 on smithi192 2024-02-16T22:59:49.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:49 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:49.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:49 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:49.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:49 smithi064 ceph-mon[93757]: Reconfiguring osd.5 (monmap changed)... 2024-02-16T22:59:49.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:49 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:59:49.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:49 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:49.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:49 smithi064 ceph-mon[93757]: Reconfiguring daemon osd.5 on smithi192 2024-02-16T22:59:50.750 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:50 smithi192 ceph-mon[82652]: pgmap v26: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:50.750 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:50 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:50.750 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:50 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:50.750 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:50 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.tnppti", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:59:50.750 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:50 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:50.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:50 smithi064 ceph-mon[93757]: pgmap v26: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:50.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:50 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:50.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:50 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:50.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:50 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.tnppti", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:59:50.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:50 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:51.701 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:51 smithi192 ceph-mon[82652]: Reconfiguring mds.cephfs.smithi192.tnppti (monmap changed)... 2024-02-16T22:59:51.701 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:51 smithi192 ceph-mon[82652]: Reconfiguring daemon mds.cephfs.smithi192.tnppti on smithi192 2024-02-16T22:59:51.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:51 smithi064 ceph-mon[93757]: Reconfiguring mds.cephfs.smithi192.tnppti (monmap changed)... 2024-02-16T22:59:51.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:51 smithi064 ceph-mon[93757]: Reconfiguring daemon mds.cephfs.smithi192.tnppti on smithi192 2024-02-16T22:59:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:52 smithi064 ceph-mon[93757]: pgmap v27: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:52 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:52 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:52 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.luiybj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:59:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:52 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:52.637 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:52 smithi192 ceph-mon[82652]: pgmap v27: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:52.638 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:52 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:52.638 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:52 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:52.638 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:52 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.luiybj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:59:52.638 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:52 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:53.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: Reconfiguring mds.cephfs.smithi192.luiybj (monmap changed)... 2024-02-16T22:59:53.446 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: Reconfiguring daemon mds.cephfs.smithi192.luiybj on smithi192 2024-02-16T22:59:53.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:53.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:53.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:53.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:53.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:53.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:53.448 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi064"}]: dispatch 2024-02-16T22:59:53.448 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi064"}]': finished 2024-02-16T22:59:53.448 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]: dispatch 2024-02-16T22:59:53.448 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]': finished 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: Reconfiguring mds.cephfs.smithi192.luiybj (monmap changed)... 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: Reconfiguring daemon mds.cephfs.smithi192.luiybj on smithi192 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi064"}]: dispatch 2024-02-16T22:59:53.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi064"}]': finished 2024-02-16T22:59:53.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]: dispatch 2024-02-16T22:59:53.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]': finished 2024-02-16T22:59:54.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:54 smithi064 ceph-mon[93757]: pgmap v28: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:54.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:54 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all mon 2024-02-16T22:59:54.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:54 smithi192 ceph-mon[82652]: pgmap v28: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:54.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:54 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all mon 2024-02-16T22:59:56.110 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:55 smithi064 ceph-mon[93757]: pgmap v29: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:56.111 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:55 smithi064 ceph-mon[93757]: Upgrade: Updating crash.smithi064 (1/2) 2024-02-16T22:59:56.111 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:55 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:56.111 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:55 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:59:56.111 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:55 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:56.111 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:55 smithi064 ceph-mon[93757]: Deploying daemon crash.smithi064 on smithi064 2024-02-16T22:59:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:55 smithi192 ceph-mon[82652]: pgmap v29: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:55 smithi192 ceph-mon[82652]: Upgrade: Updating crash.smithi064 (1/2) 2024-02-16T22:59:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:55 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T22:59:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:55 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:59:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:55 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:59:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:55 smithi192 ceph-mon[82652]: Deploying daemon crash.smithi064 on smithi064 2024-02-16T22:59:58.237 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:57 smithi064 ceph-mon[93757]: pgmap v30: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T22:59:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:57 smithi192 ceph-mon[82652]: pgmap v30: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:00.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:59 smithi192 ceph-mon[82652]: pgmap v31: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:00.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:59 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:00.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 22:59:59 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:59 smithi064 ceph-mon[93757]: pgmap v31: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:59 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 22:59:59 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:01.027 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:00 smithi192 ceph-mon[82652]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:00:01.027 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:00 smithi192 ceph-mon[82652]: Upgrade: Updating crash.smithi192 (2/2) 2024-02-16T23:00:01.027 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:00 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:01.027 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:00 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T23:00:01.027 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:00 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:01.027 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:00 smithi192 ceph-mon[82652]: Deploying daemon crash.smithi192 on smithi192 2024-02-16T23:00:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:00 smithi064 ceph-mon[93757]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:00:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:00 smithi064 ceph-mon[93757]: Upgrade: Updating crash.smithi192 (2/2) 2024-02-16T23:00:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:00 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:00 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T23:00:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:00 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:01.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:00 smithi064 ceph-mon[93757]: Deploying daemon crash.smithi192 on smithi192 2024-02-16T23:00:02.454 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:02 smithi192 ceph-mon[82652]: pgmap v32: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:02.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:02 smithi064 ceph-mon[93757]: pgmap v32: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:03.425 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:00:03.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:00:04.590 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:04 smithi064 ceph-mon[93757]: pgmap v33: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:04.590 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:04.590 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:04.590 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:00:04.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:04 smithi192 ceph-mon[82652]: pgmap v33: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:04.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:04.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:04.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:00:06.589 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:06 smithi064 ceph-mon[93757]: pgmap v34: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:06.622 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:06 smithi192 ceph-mon[82652]: pgmap v34: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:08.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:08 smithi064 ceph-mon[93757]: pgmap v35: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:08.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:08.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:08 smithi192 ceph-mon[82652]: pgmap v35: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:08 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:08 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:08.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.571+0000 7f16af1c4700 1 -- 172.21.15.64:0/1638644989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fcef0 msgr2=0x7f16a80fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:08.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.571+0000 7f16af1c4700 1 --2- 172.21.15.64:0/1638644989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fcef0 0x7f16a80fd320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f169c004940 tx=0x7f169c009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.571+0000 7f16af1c4700 1 -- 172.21.15.64:0/1638644989 shutdown_connections 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.571+0000 7f16af1c4700 1 --2- 172.21.15.64:0/1638644989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a80fe0e0 0x7f16a80fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.571+0000 7f16af1c4700 1 --2- 172.21.15.64:0/1638644989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fcef0 0x7f16a80fd320 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.571+0000 7f16af1c4700 1 -- 172.21.15.64:0/1638644989 >> 172.21.15.64:0/1638644989 conn(0x7f16a80f8620 msgr2=0x7f16a80faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.571+0000 7f16af1c4700 1 -- 172.21.15.64:0/1638644989 shutdown_connections 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.571+0000 7f16af1c4700 1 -- 172.21.15.64:0/1638644989 wait complete. 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16af1c4700 1 Processor -- start 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16af1c4700 1 -- start start 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16af1c4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a80fcef0 0x7f16a8071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16af1c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fe0e0 0x7f16a80702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16af1c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80708e0 con 0x7f16a80fe0e0 2024-02-16T23:00:08.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16af1c4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a8070a50 con 0x7f16a80fcef0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fe0e0 0x7f16a80702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fe0e0 0x7f16a80702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47220/0 (socket says 172.21.15.64:47220) 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16a7fff700 1 -- 172.21.15.64:0/2407364303 learned_addr learned my addr 172.21.15.64:0/2407364303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.572+0000 7f16acf60700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a80fcef0 0x7f16a8071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16a7fff700 1 -- 172.21.15.64:0/2407364303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a80fcef0 msgr2=0x7f16a8071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16a7fff700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a80fcef0 0x7f16a8071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16a7fff700 1 -- 172.21.15.64:0/2407364303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c005040 con 0x7f16a80fe0e0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16a7fff700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fe0e0 0x7f16a80702e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f169400eb70 tx=0x7f169400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16a57fa700 1 -- 172.21.15.64:0/2407364303 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1694007cc0 con 0x7f16a80fe0e0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a8070cd0 con 0x7f16a80fe0e0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16a57fa700 1 -- 172.21.15.64:0/2407364303 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f169400f040 con 0x7f16a80fe0e0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16a57fa700 1 -- 172.21.15.64:0/2407364303 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f169400ca90 con 0x7f16a80fe0e0 2024-02-16T23:00:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.573+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a8115940 con 0x7f16a80fe0e0 2024-02-16T23:00:08.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.574+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a804bed0 con 0x7f16a80fe0e0 2024-02-16T23:00:08.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.575+0000 7f16a57fa700 1 -- 172.21.15.64:0/2407364303 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f169400cbf0 con 0x7f16a80fe0e0 2024-02-16T23:00:08.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.575+0000 7f16a57fa700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f169806d900 0x7f169806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:08.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.575+0000 7f16a57fa700 1 -- 172.21.15.64:0/2407364303 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f169408cc90 con 0x7f16a80fe0e0 2024-02-16T23:00:08.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.575+0000 7f16acf60700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f169806d900 0x7f169806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:08.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.576+0000 7f16acf60700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f169806d900 0x7f169806fdb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f169c013740 tx=0x7f169c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:08.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.580+0000 7f16a57fa700 1 -- 172.21.15.64:0/2407364303 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1694059e40 con 0x7f16a80fe0e0 2024-02-16T23:00:08.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.805+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16a8102ac0 con 0x7f169806d900 2024-02-16T23:00:08.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.806+0000 7f16a57fa700 1 -- 172.21.15.64:0/2407364303 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f16a8102ac0 con 0x7f169806d900 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f169806d900 msgr2=0x7f169806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f169806d900 0x7f169806fdb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f169c013740 tx=0x7f169c009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fe0e0 msgr2=0x7f16a80702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fe0e0 0x7f16a80702e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f169400eb70 tx=0x7f169400b040 comp rx=0 tx=0).stop 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 shutdown_connections 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f169806d900 0x7f169806fdb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a80fcef0 0x7f16a8071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 --2- 172.21.15.64:0/2407364303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16a80fe0e0 0x7f16a80702e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 >> 172.21.15.64:0/2407364303 conn(0x7f16a80f8620 msgr2=0x7f16a81013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 shutdown_connections 2024-02-16T23:00:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.808+0000 7f16af1c4700 1 -- 172.21.15.64:0/2407364303 wait complete. 2024-02-16T23:00:08.819 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 -- 172.21.15.64:0/855619229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fef80 msgr2=0x7fcf500fb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 --2- 172.21.15.64:0/855619229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fef80 0x7fcf500fb030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf40004940 tx=0x7fcf40009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 -- 172.21.15.64:0/855619229 shutdown_connections 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 --2- 172.21.15.64:0/855619229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fef80 0x7fcf500fb030 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 --2- 172.21.15.64:0/855619229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf500fe5a0 0x7fcf500fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 -- 172.21.15.64:0/855619229 >> 172.21.15.64:0/855619229 conn(0x7fcf500f6ca0 msgr2=0x7fcf500f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 -- 172.21.15.64:0/855619229 shutdown_connections 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 -- 172.21.15.64:0/855619229 wait complete. 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 Processor -- start 2024-02-16T23:00:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.914+0000 7fcf5739a700 1 -- start start 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf5739a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fe5a0 0x7fcf5010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf5739a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf5010def0 0x7fcf5010ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf5739a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5010e3b0 con 0x7fcf500fe5a0 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf5739a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5010e4f0 con 0x7fcf5010def0 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf55136700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fe5a0 0x7fcf5010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf55136700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fe5a0 0x7fcf5010d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47244/0 (socket says 172.21.15.64:47244) 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf55136700 1 -- 172.21.15.64:0/1441345900 learned_addr learned my addr 172.21.15.64:0/1441345900 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf54935700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf5010def0 0x7fcf5010ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf55136700 1 -- 172.21.15.64:0/1441345900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf5010def0 msgr2=0x7fcf5010ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf55136700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf5010def0 0x7fcf5010ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf55136700 1 -- 172.21.15.64:0/1441345900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40005040 con 0x7fcf500fe5a0 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf55136700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fe5a0 0x7fcf5010d9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00eb70 tx=0x7fcf4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:08.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.915+0000 7fcf45ffb700 1 -- 172.21.15.64:0/1441345900 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4c007cc0 con 0x7fcf500fe5a0 2024-02-16T23:00:08.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.916+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5010f390 con 0x7fcf500fe5a0 2024-02-16T23:00:08.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.916+0000 7fcf45ffb700 1 -- 172.21.15.64:0/1441345900 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf4c00f040 con 0x7fcf500fe5a0 2024-02-16T23:00:08.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.916+0000 7fcf45ffb700 1 -- 172.21.15.64:0/1441345900 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4c00cb20 con 0x7fcf500fe5a0 2024-02-16T23:00:08.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.916+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5010f8a0 con 0x7fcf500fe5a0 2024-02-16T23:00:08.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.917+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5004bed0 con 0x7fcf500fe5a0 2024-02-16T23:00:08.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.917+0000 7fcf45ffb700 1 -- 172.21.15.64:0/1441345900 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fcf4c00cc80 con 0x7fcf500fe5a0 2024-02-16T23:00:08.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.917+0000 7fcf45ffb700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf3c06d900 0x7fcf3c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:08.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.918+0000 7fcf45ffb700 1 -- 172.21.15.64:0/1441345900 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcf4c08cb30 con 0x7fcf500fe5a0 2024-02-16T23:00:08.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.918+0000 7fcf54935700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf3c06d900 0x7fcf3c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:08.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.918+0000 7fcf54935700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf3c06d900 0x7fcf3c06fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009b30 tx=0x7fcf40009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:08.923+0000 7fcf45ffb700 1 -- 172.21.15.64:0/1441345900 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf4c059d90 con 0x7fcf500fe5a0 2024-02-16T23:00:09.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.188+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf500feff0 con 0x7fcf3c06d900 2024-02-16T23:00:09.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.189+0000 7fcf45ffb700 1 -- 172.21.15.64:0/1441345900 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fcf500feff0 con 0x7fcf3c06d900 2024-02-16T23:00:09.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf3c06d900 msgr2=0x7fcf3c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf3c06d900 0x7fcf3c06fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009b30 tx=0x7fcf40009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fe5a0 msgr2=0x7fcf5010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fe5a0 0x7fcf5010d9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00eb70 tx=0x7fcf4c00b040 comp rx=0 tx=0).stop 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 shutdown_connections 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf3c06d900 0x7fcf3c06fdb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf500fe5a0 0x7fcf5010d9b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 --2- 172.21.15.64:0/1441345900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf5010def0 0x7fcf5010ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 >> 172.21.15.64:0/1441345900 conn(0x7fcf500f6ca0 msgr2=0x7fcf50101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 shutdown_connections 2024-02-16T23:00:09.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.192+0000 7fcf5739a700 1 -- 172.21.15.64:0/1441345900 wait complete. 2024-02-16T23:00:09.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.298+0000 7fdbee0c7700 1 -- 172.21.15.64:0/774327167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8070840 msgr2=0x7fdbe8070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.298+0000 7fdbee0c7700 1 --2- 172.21.15.64:0/774327167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8070840 0x7fdbe8070cb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc004940 tx=0x7fdbdc009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.298+0000 7fdbee0c7700 1 -- 172.21.15.64:0/774327167 shutdown_connections 2024-02-16T23:00:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.298+0000 7fdbee0c7700 1 --2- 172.21.15.64:0/774327167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8070840 0x7fdbe8070cb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.298+0000 7fdbee0c7700 1 --2- 172.21.15.64:0/774327167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbe8071e90 0x7fdbe8070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.298+0000 7fdbee0c7700 1 -- 172.21.15.64:0/774327167 >> 172.21.15.64:0/774327167 conn(0x7fdbe80f8620 msgr2=0x7fdbe80faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.298+0000 7fdbee0c7700 1 -- 172.21.15.64:0/774327167 shutdown_connections 2024-02-16T23:00:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.298+0000 7fdbee0c7700 1 -- 172.21.15.64:0/774327167 wait complete. 2024-02-16T23:00:09.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbee0c7700 1 Processor -- start 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbee0c7700 1 -- start start 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbee0c7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbe8070840 0x7fdbe810fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbee0c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8071e90 0x7fdbe81100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbee0c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81106f0 con 0x7fdbe8071e90 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbee0c7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8110f90 con 0x7fdbe8070840 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8071e90 0x7fdbe81100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8071e90 0x7fdbe81100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47256/0 (socket says 172.21.15.64:47256) 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbe6ffd700 1 -- 172.21.15.64:0/4186195279 learned_addr learned my addr 172.21.15.64:0/4186195279 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbe77fe700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbe8070840 0x7fdbe810fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbe6ffd700 1 -- 172.21.15.64:0/4186195279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbe8070840 msgr2=0x7fdbe810fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.299+0000 7fdbe6ffd700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbe8070840 0x7fdbe810fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.300+0000 7fdbe6ffd700 1 -- 172.21.15.64:0/4186195279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc005040 con 0x7fdbe8071e90 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.300+0000 7fdbe6ffd700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8071e90 0x7fdbe81100f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009180 tx=0x7fdbdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.300+0000 7fdbcffff700 1 -- 172.21.15.64:0/4186195279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbdc00e040 con 0x7fdbe8071e90 2024-02-16T23:00:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.300+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe8111210 con 0x7fdbe8071e90 2024-02-16T23:00:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.300+0000 7fdbcffff700 1 -- 172.21.15.64:0/4186195279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbdc004290 con 0x7fdbe8071e90 2024-02-16T23:00:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.300+0000 7fdbcffff700 1 -- 172.21.15.64:0/4186195279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbdc013ea0 con 0x7fdbe8071e90 2024-02-16T23:00:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.300+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe8111750 con 0x7fdbe8071e90 2024-02-16T23:00:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.301+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe804bed0 con 0x7fdbe8071e90 2024-02-16T23:00:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.302+0000 7fdbcffff700 1 -- 172.21.15.64:0/4186195279 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fdbdc00bbf0 con 0x7fdbe8071e90 2024-02-16T23:00:09.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.302+0000 7fdbcffff700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdbd006d900 0x7fdbd006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:09.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.302+0000 7fdbcffff700 1 -- 172.21.15.64:0/4186195279 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdbdc08e410 con 0x7fdbe8071e90 2024-02-16T23:00:09.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.302+0000 7fdbe77fe700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdbd006d900 0x7fdbd006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:09.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.303+0000 7fdbe77fe700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdbd006d900 0x7fdbd006fdb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8007fd0 tx=0x7fdbd8009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:09.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.307+0000 7fdbcffff700 1 -- 172.21.15.64:0/4186195279 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdbdc05b6e0 con 0x7fdbe8071e90 2024-02-16T23:00:09.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.522+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbe8104570 con 0x7fdbd006d900 2024-02-16T23:00:09.523 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:09.523 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:09.523 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:09.523 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.532+0000 7fdbcffff700 1 -- 172.21.15.64:0/4186195279 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdbe8104570 con 0x7fdbd006d900 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (3m) 0s ago 11m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (11s) 0s ago 11m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (5s) 2s ago 10m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (2m) 0s ago 11m 84.3M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (6m) 0s ago 6m 17.5M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (6m) 0s ago 6m 153M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (6m) 2s ago 6m 16.5M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (6m) 2s ago 6m 16.2M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (2m) 0s ago 13m 494M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (90s) 2s ago 10m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:00:09.533 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (75s) 0s ago 13m 41.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (52s) 2s ago 10m 35.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (3m) 0s ago 11m 15.5M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (2m) 2s ago 10m 15.8M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (10m) 0s ago 9m 210M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 ece82d3c0508 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (9m) 0s ago 9m 216M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (9m) 0s ago 9m 176M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (8m) 2s ago 8m 234M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (8m) 2s ago 8m 210M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (7m) 2s ago 7m 186M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T23:00:09.534 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (95s) 0s ago 11m 47.6M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:00:09.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:09.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:09.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:09.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdbd006d900 msgr2=0x7fdbd006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdbd006d900 0x7fdbd006fdb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8007fd0 tx=0x7fdbd8009040 comp rx=0 tx=0).stop 2024-02-16T23:00:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8071e90 msgr2=0x7fdbe81100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8071e90 0x7fdbe81100f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009180 tx=0x7fdbdc009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 shutdown_connections 2024-02-16T23:00:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdbd006d900 0x7fdbd006fdb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbe8070840 0x7fdbe810fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 --2- 172.21.15.64:0/4186195279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe8071e90 0x7fdbe81100f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.534+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 >> 172.21.15.64:0/4186195279 conn(0x7fdbe80f8620 msgr2=0x7fdbe8102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.535+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 shutdown_connections 2024-02-16T23:00:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.535+0000 7fdbee0c7700 1 -- 172.21.15.64:0/4186195279 wait complete. 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 -- 172.21.15.64:0/1234274223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fb220 msgr2=0x7f5c2c0fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 --2- 172.21.15.64:0/1234274223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fb220 0x7f5c2c0fb630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c004940 tx=0x7f5c1c009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 -- 172.21.15.64:0/1234274223 shutdown_connections 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 --2- 172.21.15.64:0/1234274223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c2c0fe2b0 0x7f5c2c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 --2- 172.21.15.64:0/1234274223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fb220 0x7f5c2c0fb630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 -- 172.21.15.64:0/1234274223 >> 172.21.15.64:0/1234274223 conn(0x7f5c2c0f6ca0 msgr2=0x7f5c2c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 -- 172.21.15.64:0/1234274223 shutdown_connections 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 -- 172.21.15.64:0/1234274223 wait complete. 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 Processor -- start 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.639+0000 7f5c32959700 1 -- start start 2024-02-16T23:00:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c32959700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c2c0fb220 0x7f5c2c0fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c32959700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fe2b0 0x7f5c2c0fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c32959700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0fc7c0 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c32959700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0fc930 con 0x7f5c2c0fb220 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c2b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fe2b0 0x7f5c2c0fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c2b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fe2b0 0x7f5c2c0fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47284/0 (socket says 172.21.15.64:47284) 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c2b7fe700 1 -- 172.21.15.64:0/996420525 learned_addr learned my addr 172.21.15.64:0/996420525 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c2bfff700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c2c0fb220 0x7f5c2c0fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c2b7fe700 1 -- 172.21.15.64:0/996420525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c2c0fb220 msgr2=0x7f5c2c0fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c2b7fe700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c2c0fb220 0x7f5c2c0fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c2b7fe700 1 -- 172.21.15.64:0/996420525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c005040 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.640+0000 7f5c2b7fe700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fe2b0 0x7f5c2c0fc1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c20002a50 tx=0x7f5c200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.641+0000 7f5c28ff9700 1 -- 172.21.15.64:0/996420525 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c20007cc0 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.641+0000 7f5c28ff9700 1 -- 172.21.15.64:0/996420525 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c2000f040 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.641+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c0fcc10 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.641+0000 7f5c28ff9700 1 -- 172.21.15.64:0/996420525 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c20009640 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.641+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c115a70 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.642+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2c04bed0 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.642+0000 7f5c28ff9700 1 -- 172.21.15.64:0/996420525 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f5c200097a0 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.643+0000 7f5c28ff9700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5c1406d900 0x7f5c1406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.643+0000 7f5c28ff9700 1 -- 172.21.15.64:0/996420525 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5c2008ce00 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.643+0000 7f5c2bfff700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5c1406d900 0x7f5c1406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.643+0000 7f5c2bfff700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5c1406d900 0x7f5c1406fdb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00b3b0 tx=0x7f5c1c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.648+0000 7f5c28ff9700 1 -- 172.21.15.64:0/996420525 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c2005a030 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.938+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5c2c105a40 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.938+0000 7f5c28ff9700 1 -- 172.21.15.64:0/996420525 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f5c20059bc0 con 0x7f5c2c0fe2b0 2024-02-16T23:00:09.939 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:00:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:00:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:00:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:00:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:00:09.939 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 10, 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:00:09.940 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:00:09.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5c1406d900 msgr2=0x7f5c1406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5c1406d900 0x7f5c1406fdb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00b3b0 tx=0x7f5c1c009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fe2b0 msgr2=0x7f5c2c0fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fe2b0 0x7f5c2c0fc1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c20002a50 tx=0x7f5c200046b0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 shutdown_connections 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5c1406d900 0x7f5c1406fdb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c2c0fb220 0x7f5c2c0fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 --2- 172.21.15.64:0/996420525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2c0fe2b0 0x7f5c2c0fc1c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 >> 172.21.15.64:0/996420525 conn(0x7f5c2c0f6ca0 msgr2=0x7f5c2c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 shutdown_connections 2024-02-16T23:00:09.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:09.941+0000 7f5c32959700 1 -- 172.21.15.64:0/996420525 wait complete. 2024-02-16T23:00:10.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.045+0000 7ff524c47700 1 -- 172.21.15.64:0/340836721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fcef0 msgr2=0x7ff5200fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.045+0000 7ff524c47700 1 --2- 172.21.15.64:0/340836721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fcef0 0x7ff5200fd320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff510004940 tx=0x7ff510009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:10.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.045+0000 7ff524c47700 1 -- 172.21.15.64:0/340836721 shutdown_connections 2024-02-16T23:00:10.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.045+0000 7ff524c47700 1 --2- 172.21.15.64:0/340836721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5200fe0e0 0x7ff5200fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.045+0000 7ff524c47700 1 --2- 172.21.15.64:0/340836721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fcef0 0x7ff5200fd320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.045+0000 7ff524c47700 1 -- 172.21.15.64:0/340836721 >> 172.21.15.64:0/340836721 conn(0x7ff5200f8620 msgr2=0x7ff5200faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff524c47700 1 -- 172.21.15.64:0/340836721 shutdown_connections 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff524c47700 1 -- 172.21.15.64:0/340836721 wait complete. 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff524c47700 1 Processor -- start 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff524c47700 1 -- start start 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff524c47700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5200fcef0 0x7ff52010fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff524c47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fe0e0 0x7ff520110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff524c47700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520110610 con 0x7ff5200fe0e0 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff524c47700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520110eb0 con 0x7ff5200fcef0 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff51dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fe0e0 0x7ff520110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff51dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fe0e0 0x7ff520110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47296/0 (socket says 172.21.15.64:47296) 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.046+0000 7ff51dd9b700 1 -- 172.21.15.64:0/4028449551 learned_addr learned my addr 172.21.15.64:0/4028449551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff51e59c700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5200fcef0 0x7ff52010fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff51dd9b700 1 -- 172.21.15.64:0/4028449551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5200fcef0 msgr2=0x7ff52010fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff51dd9b700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5200fcef0 0x7ff52010fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff51dd9b700 1 -- 172.21.15.64:0/4028449551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff510005040 con 0x7ff5200fe0e0 2024-02-16T23:00:10.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff51dd9b700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fe0e0 0x7ff520110010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff51400eb40 tx=0x7ff51400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff50effd700 1 -- 172.21.15.64:0/4028449551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff514007cc0 con 0x7ff5200fe0e0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff50effd700 1 -- 172.21.15.64:0/4028449551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff51400f040 con 0x7ff5200fe0e0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff520111130 con 0x7ff5200fe0e0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff50effd700 1 -- 172.21.15.64:0/4028449551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51400ca10 con 0x7ff5200fe0e0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.047+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520111670 con 0x7ff5200fe0e0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.048+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520102210 con 0x7ff5200fe0e0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.049+0000 7ff50effd700 1 -- 172.21.15.64:0/4028449551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7ff51400cc80 con 0x7ff5200fe0e0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.049+0000 7ff50effd700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff50806d900 0x7ff50806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.049+0000 7ff50effd700 1 -- 172.21.15.64:0/4028449551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff51408ccf0 con 0x7ff5200fe0e0 2024-02-16T23:00:10.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.049+0000 7ff51e59c700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff50806d900 0x7ff50806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.050+0000 7ff51e59c700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff50806d900 0x7ff50806fdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff510000c00 tx=0x7ff510009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:10.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.054+0000 7ff50effd700 1 -- 172.21.15.64:0/4028449551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff51405aea0 con 0x7ff5200fe0e0 2024-02-16T23:00:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.312+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff52004bed0 con 0x7ff5200fe0e0 2024-02-16T23:00:10.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:10 smithi064 ceph-mon[93757]: pgmap v36: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:10.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:10 smithi064 ceph-mon[93757]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:10.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:10 smithi064 ceph-mon[93757]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:10.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:10 smithi064 ceph-mon[93757]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:10.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:10 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/996420525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:10.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.312+0000 7ff50effd700 1 -- 172.21.15.64:0/4028449551 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7ff51405aa30 con 0x7ff5200fe0e0 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T23:00:10.314 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:00:10.315 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:00:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:00:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff50806d900 msgr2=0x7ff50806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff50806d900 0x7ff50806fdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff510000c00 tx=0x7ff510009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fe0e0 msgr2=0x7ff520110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fe0e0 0x7ff520110010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff51400eb40 tx=0x7ff51400b040 comp rx=0 tx=0).stop 2024-02-16T23:00:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 shutdown_connections 2024-02-16T23:00:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff50806d900 0x7ff50806fdb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5200fcef0 0x7ff52010fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 --2- 172.21.15.64:0/4028449551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5200fe0e0 0x7ff520110010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 >> 172.21.15.64:0/4028449551 conn(0x7ff5200f8620 msgr2=0x7ff5200ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 shutdown_connections 2024-02-16T23:00:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.315+0000 7ff524c47700 1 -- 172.21.15.64:0/4028449551 wait complete. 2024-02-16T23:00:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T23:00:10.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:10 smithi192 ceph-mon[82652]: pgmap v36: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:10.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:10 smithi192 ceph-mon[82652]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:10.364 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:10 smithi192 ceph-mon[82652]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:10.364 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:10 smithi192 ceph-mon[82652]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:10.364 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:10 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/996420525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 -- 172.21.15.64:0/1048709837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fe0e0 msgr2=0x7f7b780fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 --2- 172.21.15.64:0/1048709837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fe0e0 0x7f7b780fe550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b64004940 tx=0x7f7b64009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 -- 172.21.15.64:0/1048709837 shutdown_connections 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 --2- 172.21.15.64:0/1048709837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fe0e0 0x7f7b780fe550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 --2- 172.21.15.64:0/1048709837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b780fcef0 0x7f7b780fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 -- 172.21.15.64:0/1048709837 >> 172.21.15.64:0/1048709837 conn(0x7f7b780f8620 msgr2=0x7f7b780faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 -- 172.21.15.64:0/1048709837 shutdown_connections 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 -- 172.21.15.64:0/1048709837 wait complete. 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.422+0000 7f7b7eaba700 1 Processor -- start 2024-02-16T23:00:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7eaba700 1 -- start start 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7eaba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fcef0 0x7f7b7810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7eaba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b780fe0e0 0x7f7b7810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7eaba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7810e4b0 con 0x7f7b780fcef0 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7eaba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7810ed50 con 0x7f7b780fe0e0 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7c856700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fcef0 0x7f7b7810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7c856700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fcef0 0x7f7b7810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47322/0 (socket says 172.21.15.64:47322) 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7c856700 1 -- 172.21.15.64:0/4243611569 learned_addr learned my addr 172.21.15.64:0/4243611569 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b77fff700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b780fe0e0 0x7f7b7810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7c856700 1 -- 172.21.15.64:0/4243611569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b780fe0e0 msgr2=0x7f7b7810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7c856700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b780fe0e0 0x7f7b7810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7c856700 1 -- 172.21.15.64:0/4243611569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64005040 con 0x7f7b780fcef0 2024-02-16T23:00:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.423+0000 7f7b7c856700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fcef0 0x7f7b7810d970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c002a80 tx=0x7f7b6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.424+0000 7f7b757fa700 1 -- 172.21.15.64:0/4243611569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6c007cc0 con 0x7f7b780fcef0 2024-02-16T23:00:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.424+0000 7f7b757fa700 1 -- 172.21.15.64:0/4243611569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b6c00f040 con 0x7f7b780fcef0 2024-02-16T23:00:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.424+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7810f030 con 0x7f7b780fcef0 2024-02-16T23:00:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.424+0000 7f7b757fa700 1 -- 172.21.15.64:0/4243611569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6c00da30 con 0x7f7b780fcef0 2024-02-16T23:00:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.424+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7810f570 con 0x7f7b780fcef0 2024-02-16T23:00:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.425+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7804bed0 con 0x7f7b780fcef0 2024-02-16T23:00:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.425+0000 7f7b757fa700 1 -- 172.21.15.64:0/4243611569 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f7b6c00db90 con 0x7f7b780fcef0 2024-02-16T23:00:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.426+0000 7f7b757fa700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7b6806d900 0x7f7b6806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.426+0000 7f7b757fa700 1 -- 172.21.15.64:0/4243611569 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7b6c08ce80 con 0x7f7b780fcef0 2024-02-16T23:00:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.426+0000 7f7b77fff700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7b6806d900 0x7f7b6806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.426+0000 7f7b77fff700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7b6806d900 0x7f7b6806fdb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b7a0 tx=0x7f7b64009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:10.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.431+0000 7f7b757fa700 1 -- 172.21.15.64:0/4243611569 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b6c05a030 con 0x7f7b780fcef0 2024-02-16T23:00:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.653+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b7810fa10 con 0x7f7b6806d900 2024-02-16T23:00:10.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.655+0000 7f7b757fa700 1 -- 172.21.15.64:0/4243611569 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f7b7810fa10 con 0x7f7b6806d900 2024-02-16T23:00:10.655 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "6/21 daemons upgraded", 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading crash daemons", 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:00:10.656 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7b6806d900 msgr2=0x7f7b6806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7b6806d900 0x7f7b6806fdb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b7a0 tx=0x7f7b64009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fcef0 msgr2=0x7f7b7810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fcef0 0x7f7b7810d970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c002a80 tx=0x7f7b6c004600 comp rx=0 tx=0).stop 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 shutdown_connections 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7b6806d900 0x7f7b6806fdb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780fcef0 0x7f7b7810d970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 --2- 172.21.15.64:0/4243611569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b780fe0e0 0x7f7b7810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 >> 172.21.15.64:0/4243611569 conn(0x7f7b780f8620 msgr2=0x7f7b781013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 shutdown_connections 2024-02-16T23:00:10.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.657+0000 7f7b7eaba700 1 -- 172.21.15.64:0/4243611569 wait complete. 2024-02-16T23:00:10.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.768+0000 7f1945682700 1 -- 172.21.15.64:0/1948287979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 msgr2=0x7f19400fd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.768+0000 7f1945682700 1 --2- 172.21.15.64:0/1948287979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 0x7f19400fd4f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1930004940 tx=0x7f1930009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:10.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.768+0000 7f1945682700 1 -- 172.21.15.64:0/1948287979 shutdown_connections 2024-02-16T23:00:10.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.768+0000 7f1945682700 1 --2- 172.21.15.64:0/1948287979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 0x7f19400fd4f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.768+0000 7f1945682700 1 --2- 172.21.15.64:0/1948287979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19400fbe80 0x7f19400fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.768+0000 7f1945682700 1 -- 172.21.15.64:0/1948287979 >> 172.21.15.64:0/1948287979 conn(0x7f19400f7590 msgr2=0x7f19400f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:10.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.768+0000 7f1945682700 1 -- 172.21.15.64:0/1948287979 shutdown_connections 2024-02-16T23:00:10.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.768+0000 7f1945682700 1 -- 172.21.15.64:0/1948287979 wait complete. 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.769+0000 7f1945682700 1 Processor -- start 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.769+0000 7f1945682700 1 -- start start 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.769+0000 7f1945682700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19400fbe80 0x7f194010fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.769+0000 7f1945682700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 0x7f1940110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.769+0000 7f1945682700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940110600 con 0x7f19400fd080 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.769+0000 7f1945682700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940110ea0 con 0x7f19400fbe80 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.769+0000 7f193e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 0x7f1940110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f193e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 0x7f1940110000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59762/0 (socket says 172.21.15.64:59762) 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f193effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19400fbe80 0x7f194010fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f193e7fc700 1 -- 172.21.15.64:0/1218424672 learned_addr learned my addr 172.21.15.64:0/1218424672 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f193effd700 1 -- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 msgr2=0x7f1940110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f193effd700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 0x7f1940110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f193effd700 1 -- 172.21.15.64:0/1218424672 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1930005040 con 0x7f19400fbe80 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f193e7fc700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 0x7f1940110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f193effd700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19400fbe80 0x7f194010fac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f19400fcbf0 tx=0x7f193400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f19277fe700 1 -- 172.21.15.64:0/1218424672 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1934004e90 con 0x7f19400fbe80 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f19277fe700 1 -- 172.21.15.64:0/1218424672 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f193400f040 con 0x7f19400fbe80 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1940111120 con 0x7f19400fbe80 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f19277fe700 1 -- 172.21.15.64:0/1218424672 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f193400c980 con 0x7f19400fbe80 2024-02-16T23:00:10.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.770+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1940111660 con 0x7f19400fbe80 2024-02-16T23:00:10.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.771+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194004bed0 con 0x7f19400fbe80 2024-02-16T23:00:10.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.772+0000 7f19277fe700 1 -- 172.21.15.64:0/1218424672 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f19340090d0 con 0x7f19400fbe80 2024-02-16T23:00:10.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.772+0000 7f19277fe700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f192806d900 0x7f192806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:10.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.773+0000 7f19277fe700 1 -- 172.21.15.64:0/1218424672 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f193408c9c0 con 0x7f19400fbe80 2024-02-16T23:00:10.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.773+0000 7f193e7fc700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f192806d900 0x7f192806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.773+0000 7f193e7fc700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f192806d900 0x7f192806fdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f193000b0d0 tx=0x7f1930009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:10.777+0000 7f19277fe700 1 -- 172.21.15.64:0/1218424672 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1934059ca0 con 0x7f19400fbe80 2024-02-16T23:00:11.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.081+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f19401119f0 con 0x7f19400fbe80 2024-02-16T23:00:11.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.082+0000 7f19277fe700 1 -- 172.21.15.64:0/1218424672 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1934059830 con 0x7f19400fbe80 2024-02-16T23:00:11.083 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:00:11.083 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:00:11.083 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:00:11.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f192806d900 msgr2=0x7f192806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:11.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f192806d900 0x7f192806fdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f193000b0d0 tx=0x7f1930009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19400fbe80 msgr2=0x7f194010fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19400fbe80 0x7f194010fac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f19400fcbf0 tx=0x7f193400b040 comp rx=0 tx=0).stop 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 shutdown_connections 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f192806d900 0x7f192806fdb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19400fbe80 0x7f194010fac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 --2- 172.21.15.64:0/1218424672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19400fd080 0x7f1940110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 >> 172.21.15.64:0/1218424672 conn(0x7f19400f7590 msgr2=0x7f1940065f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 shutdown_connections 2024-02-16T23:00:11.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:11.084+0000 7f1945682700 1 -- 172.21.15.64:0/1218424672 wait complete. 2024-02-16T23:00:11.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:11.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:11.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:11 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/4028449551' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:00:11.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:11 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1218424672' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:00:11.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:11.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:11.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:11 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/4028449551' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:00:11.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:11 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1218424672' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:00:12.581 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:12 smithi064 ceph-mon[93757]: pgmap v37: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:12.581 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:12 smithi064 ceph-mon[93757]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:12.581 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:12.581 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:12.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:12 smithi192 ceph-mon[82652]: pgmap v37: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:12.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:12 smithi192 ceph-mon[82652]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: pgmap v38: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi064"}]: dispatch 2024-02-16T23:00:14.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi064"}]': finished 2024-02-16T23:00:14.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]: dispatch 2024-02-16T23:00:14.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]': finished 2024-02-16T23:00:14.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: pgmap v38: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi064"}]: dispatch 2024-02-16T23:00:14.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi064"}]': finished 2024-02-16T23:00:14.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]: dispatch 2024-02-16T23:00:14.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]': finished 2024-02-16T23:00:14.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T23:00:15.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:15 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all crash 2024-02-16T23:00:15.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:15 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T23:00:15.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:15 smithi064 ceph-mon[93757]: Upgrade: osd.0 is safe to restart 2024-02-16T23:00:15.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:15 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all crash 2024-02-16T23:00:15.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:15 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T23:00:15.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:15 smithi192 ceph-mon[82652]: Upgrade: osd.0 is safe to restart 2024-02-16T23:00:16.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:16 smithi064 ceph-mon[93757]: pgmap v39: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:16.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:16.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T23:00:16.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:16 smithi192 ceph-mon[82652]: pgmap v39: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T23:00:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:17.360 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:17 smithi064 ceph-mon[93757]: Upgrade: Updating osd.0 2024-02-16T23:00:17.360 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:17 smithi064 ceph-mon[93757]: Deploying daemon osd.0 on smithi064 2024-02-16T23:00:17.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:17 smithi192 ceph-mon[82652]: Upgrade: Updating osd.0 2024-02-16T23:00:17.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:17 smithi192 ceph-mon[82652]: Deploying daemon osd.0 on smithi064 2024-02-16T23:00:18.409 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:18 smithi064 ceph-mon[93757]: pgmap v40: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:18.409 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:00:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:18 smithi192 ceph-mon[82652]: pgmap v40: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:00:19.315 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:19 smithi064 systemd[1]: Stopping Ceph osd.0 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:00:19.618 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:19 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[47946]: 2024-02-16T23:00:19.314+0000 7fed1357e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:00:19.618 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:19 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[47946]: 2024-02-16T23:00:19.314+0000 7fed1357e700 -1 osd.0 50 *** Got signal Terminated *** 2024-02-16T23:00:19.618 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:19 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[47946]: 2024-02-16T23:00:19.314+0000 7fed1357e700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:00:20.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:20 smithi064 ceph-mon[93757]: pgmap v41: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:20.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:20 smithi064 ceph-mon[93757]: osd.0 marked itself down and dead 2024-02-16T23:00:20.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:20 smithi192 ceph-mon[82652]: pgmap v41: 65 pgs: 65 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:20.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:20 smithi192 ceph-mon[82652]: osd.0 marked itself down and dead 2024-02-16T23:00:21.016 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:20 smithi064 podman[102139]: 2024-02-16 23:00:20.733738292 +0000 UTC m=+1.529531901 container died ece82d3c0508ad44f9915aed0d113b3c4c4194e347b352f436ee3f1791931127 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=quincy-b88fae6, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-16T23:00:21.308 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:21 smithi064 podman[102139]: 2024-02-16 23:00:21.106309796 +0000 UTC m=+1.902103404 container cleanup ece82d3c0508ad44f9915aed0d113b3c4c4194e347b352f436ee3f1791931127 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=quincy-b88fae6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-16T23:00:21.308 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:21 smithi064 bash[102139]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0 2024-02-16T23:00:21.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:21 smithi064 ceph-mon[93757]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:00:21.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:21 smithi064 ceph-mon[93757]: failed to encode map e51 with expected crc 2024-02-16T23:00:21.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:21 smithi064 ceph-mon[93757]: osdmap e51: 6 total, 5 up, 6 in 2024-02-16T23:00:21.618 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:21 smithi064 podman[102156]: 2024-02-16 23:00:21.382812358 +0000 UTC m=+0.646500441 container remove ece82d3c0508ad44f9915aed0d113b3c4c4194e347b352f436ee3f1791931127 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=quincy-b88fae6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-16T23:00:21.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:21 smithi192 ceph-mon[82652]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:00:21.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:21 smithi192 ceph-mon[82652]: failed to encode map e51 with expected crc 2024-02-16T23:00:21.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:21 smithi192 ceph-mon[82652]: osdmap e51: 6 total, 5 up, 6 in 2024-02-16T23:00:22.028 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:21 smithi064 podman[102249]: 2024-02-16 23:00:21.622527491 +0000 UTC m=+0.017883983 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:00:22.028 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:21 smithi064 podman[102249]: 2024-02-16 23:00:21.742275694 +0000 UTC m=+0.137632228 container create c9c2ef0b1acebb42f4c69a3cbc81f018117ec4feb30accf75565a6a8411aeada (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-16T23:00:22.028 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:21 smithi064 podman[102249]: 2024-02-16 23:00:21.936398912 +0000 UTC m=+0.331755404 container init c9c2ef0b1acebb42f4c69a3cbc81f018117ec4feb30accf75565a6a8411aeada (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T23:00:22.028 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:21 smithi064 podman[102249]: 2024-02-16 23:00:21.939825168 +0000 UTC m=+0.335181669 container start c9c2ef0b1acebb42f4c69a3cbc81f018117ec4feb30accf75565a6a8411aeada (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-16T23:00:22.028 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:22 smithi064 podman[102249]: 2024-02-16 23:00:22.028383862 +0000 UTC m=+0.423740362 container attach c9c2ef0b1acebb42f4c69a3cbc81f018117ec4feb30accf75565a6a8411aeada (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-02-16T23:00:22.293 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:22 smithi064 podman[102249]: 2024-02-16 23:00:22.241357148 +0000 UTC m=+0.636713650 container died c9c2ef0b1acebb42f4c69a3cbc81f018117ec4feb30accf75565a6a8411aeada (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-16T23:00:22.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:22 smithi064 ceph-mon[93757]: pgmap v43: 65 pgs: 9 stale+active+clean, 56 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:22.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:22 smithi064 ceph-mon[93757]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:22 smithi064 ceph-mon[93757]: osdmap e52: 6 total, 5 up, 6 in 2024-02-16T23:00:22.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:22 smithi064 ceph-mon[93757]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:22 smithi064 ceph-mon[93757]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:22 smithi064 ceph-mon[93757]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:22 smithi064 ceph-mon[93757]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:22 smithi064 ceph-mon[93757]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:22 smithi192 ceph-mon[82652]: pgmap v43: 65 pgs: 9 stale+active+clean, 56 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:22.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:22 smithi192 ceph-mon[82652]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:22 smithi192 ceph-mon[82652]: osdmap e52: 6 total, 5 up, 6 in 2024-02-16T23:00:22.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:22 smithi192 ceph-mon[82652]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:22 smithi192 ceph-mon[82652]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:22 smithi192 ceph-mon[82652]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:22 smithi192 ceph-mon[82652]: failed to encode map e52 with expected crc 2024-02-16T23:00:22.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:22 smithi192 ceph-mon[82652]: failed to encode map e52 with expected crc 2024-02-16T23:00:23.118 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:22 smithi064 podman[102275]: 2024-02-16 23:00:22.815890602 +0000 UTC m=+0.566042245 container remove c9c2ef0b1acebb42f4c69a3cbc81f018117ec4feb30accf75565a6a8411aeada (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2) 2024-02-16T23:00:23.118 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:22 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.0.service: Deactivated successfully. 2024-02-16T23:00:23.118 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:22 smithi064 systemd[1]: Stopped Ceph osd.0 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:00:23.118 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:22 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.0.service: Consumed 52.123s CPU time. 2024-02-16T23:00:23.118 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 systemd[1]: Starting Ceph osd.0 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:00:23.536 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 podman[102377]: 2024-02-16 23:00:23.162551469 +0000 UTC m=+0.018007523 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:00:23.536 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 podman[102377]: 2024-02-16 23:00:23.275379335 +0000 UTC m=+0.130835388 container create 61cca415d6fdd0fc0c20092b2a169de25c4332c04036fac5fa9f6266a028bfb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:00:23.536 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 podman[102377]: 2024-02-16 23:00:23.536151447 +0000 UTC m=+0.391607506 container init 61cca415d6fdd0fc0c20092b2a169de25c4332c04036fac5fa9f6266a028bfb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:00:23.868 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 podman[102377]: 2024-02-16 23:00:23.53971251 +0000 UTC m=+0.395168564 container start 61cca415d6fdd0fc0c20092b2a169de25c4332c04036fac5fa9f6266a028bfb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-16T23:00:23.868 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 podman[102377]: 2024-02-16 23:00:23.628261683 +0000 UTC m=+0.483717766 container attach 61cca415d6fdd0fc0c20092b2a169de25c4332c04036fac5fa9f6266a028bfb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-02-16T23:00:24.319 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate[102392]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T23:00:24.319 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate[102392]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T23:00:24.319 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 bash[102377]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 bash[102377]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 bash[102377]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 bash[102377]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 bash[102377]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 bash[102377]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 bash[102377]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate[102392]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate[102392]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate[102392]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate[102392]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate[102392]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-16T23:00:24.320 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:24 smithi064 podman[102377]: 2024-02-16 23:00:24.010733147 +0000 UTC m=+0.866189201 container died 61cca415d6fdd0fc0c20092b2a169de25c4332c04036fac5fa9f6266a028bfb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-16T23:00:24.615 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:24 smithi064 ceph-mon[93757]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:24.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:24 smithi192 ceph-mon[82652]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:24.868 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:24 smithi064 podman[102525]: 2024-02-16 23:00:24.615638579 +0000 UTC m=+0.599711753 container remove 61cca415d6fdd0fc0c20092b2a169de25c4332c04036fac5fa9f6266a028bfb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0) 2024-02-16T23:00:24.868 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:24 smithi064 podman[102590]: 2024-02-16 23:00:24.73157559 +0000 UTC m=+0.017997826 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:00:25.186 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:24 smithi064 podman[102590]: 2024-02-16 23:00:24.875087308 +0000 UTC m=+0.161509582 container create 3d93cd9ed3d9e8a3e2a6bd5264902706e2cf3cb84bcc5b69f287d9a00b4615fc (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-16T23:00:25.186 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:25 smithi064 podman[102590]: 2024-02-16 23:00:25.186464992 +0000 UTC m=+0.472887236 container init 3d93cd9ed3d9e8a3e2a6bd5264902706e2cf3cb84bcc5b69f287d9a00b4615fc (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:00:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:25 smithi064 ceph-mon[93757]: Health check failed: Degraded data redundancy: 49/261 objects degraded (18.774%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T23:00:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:25 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:25 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:25.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:25 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:00:25.618 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:25 smithi064 podman[102590]: 2024-02-16 23:00:25.189832572 +0000 UTC m=+0.476254802 container start 3d93cd9ed3d9e8a3e2a6bd5264902706e2cf3cb84bcc5b69f287d9a00b4615fc (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:00:25.618 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:25 smithi064 ceph-osd[102609]: -- 172.21.15.64:0/3549424209 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556c013831e0 con 0x556c01354c00 2024-02-16T23:00:25.618 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:25 smithi064 bash[102590]: 3d93cd9ed3d9e8a3e2a6bd5264902706e2cf3cb84bcc5b69f287d9a00b4615fc 2024-02-16T23:00:25.618 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:25 smithi064 systemd[1]: Started Ceph osd.0 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:00:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:25 smithi192 ceph-mon[82652]: Health check failed: Degraded data redundancy: 49/261 objects degraded (18.774%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T23:00:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:25 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:25 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:25 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:00:26.118 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:25 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[102605]: 2024-02-16T23:00:25.736+0000 7f2af9ed47c0 -1 Falling back to public interface 2024-02-16T23:00:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:26 smithi192 ceph-mon[82652]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:26.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:26 smithi064 ceph-mon[93757]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:28.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:28 smithi064 ceph-mon[93757]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:28.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:28 smithi192 ceph-mon[82652]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:30.544 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:30 smithi064 ceph-mon[93757]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:30.545 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:30 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[102605]: 2024-02-16T23:00:30.180+0000 7f2af9ed47c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-02-16T23:00:30.545 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:30 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[102605]: 2024-02-16T23:00:30.543+0000 7f2af9ed47c0 -1 osd.0 50 log_to_monitors true 2024-02-16T23:00:30.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:30 smithi192 ceph-mon[82652]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:31.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:31 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:31.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:31 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:31.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:31 smithi192 ceph-mon[82652]: from='osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T23:00:31.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:31 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:31.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:31 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:31.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:31 smithi064 ceph-mon[93757]: from='osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T23:00:32.586 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[102605]: 2024-02-16T23:00:32.419+0000 7f2ae7324700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T23:00:32.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-mon[93757]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:32.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-mon[93757]: from='osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T23:00:32.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-mon[93757]: failed to encode map e53 with expected crc 2024-02-16T23:00:32.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-mon[93757]: osdmap e53: 6 total, 5 up, 6 in 2024-02-16T23:00:32.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-mon[93757]: from='osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T23:00:32.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:32.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:32.587 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:32 smithi064 ceph-mon[93757]: from='osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107]' entity='osd.0' 2024-02-16T23:00:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:32 smithi192 ceph-mon[82652]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:32 smithi192 ceph-mon[82652]: from='osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T23:00:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:32 smithi192 ceph-mon[82652]: failed to encode map e53 with expected crc 2024-02-16T23:00:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:32 smithi192 ceph-mon[82652]: osdmap e53: 6 total, 5 up, 6 in 2024-02-16T23:00:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:32 smithi192 ceph-mon[82652]: from='osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T23:00:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:33.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:32 smithi192 ceph-mon[82652]: from='osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107]' entity='osd.0' 2024-02-16T23:00:33.758 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:33 smithi064 ceph-mon[93757]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:33.758 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:33.758 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:00:33.758 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:33 smithi064 ceph-mon[93757]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:00:33.758 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:33 smithi064 ceph-mon[93757]: osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107] boot 2024-02-16T23:00:33.758 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:33 smithi064 ceph-mon[93757]: osdmap e54: 6 total, 6 up, 6 in 2024-02-16T23:00:33.758 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T23:00:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:33 smithi192 ceph-mon[82652]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 49/261 objects degraded (18.774%) 2024-02-16T23:00:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:00:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:33 smithi192 ceph-mon[82652]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:00:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:33 smithi192 ceph-mon[82652]: osd.0 [v2:172.21.15.64:6802/3681251107,v1:172.21.15.64:6803/3681251107] boot 2024-02-16T23:00:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:33 smithi192 ceph-mon[82652]: osdmap e54: 6 total, 6 up, 6 in 2024-02-16T23:00:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T23:00:34.995 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:34 smithi064 ceph-mon[93757]: failed to encode map e54 with expected crc 2024-02-16T23:00:34.995 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:34 smithi064 ceph-mon[93757]: mgrmap e39: smithi064.kubmlk(active, since 91s), standbys: smithi192.pauvih 2024-02-16T23:00:34.995 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:34 smithi064 ceph-mon[93757]: osdmap e55: 6 total, 6 up, 6 in 2024-02-16T23:00:34.995 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:34 smithi064 ceph-mon[93757]: failed to encode map e55 with expected crc 2024-02-16T23:00:34.995 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:34 smithi064 ceph-mon[93757]: failed to encode map e55 with expected crc 2024-02-16T23:00:35.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:34 smithi192 ceph-mon[82652]: failed to encode map e54 with expected crc 2024-02-16T23:00:35.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:34 smithi192 ceph-mon[82652]: mgrmap e39: smithi064.kubmlk(active, since 91s), standbys: smithi192.pauvih 2024-02-16T23:00:35.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:34 smithi192 ceph-mon[82652]: osdmap e55: 6 total, 6 up, 6 in 2024-02-16T23:00:35.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:34 smithi192 ceph-mon[82652]: failed to encode map e55 with expected crc 2024-02-16T23:00:35.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:34 smithi192 ceph-mon[82652]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 6/261 objects degraded (2.299%) 2024-02-16T23:00:36.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: Health check update: Degraded data redundancy: 6/261 objects degraded (2.299%), 3 pgs degraded (PG_DEGRADED) 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T23:00:36.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:35 smithi192 ceph-mon[82652]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: failed to encode map e55 with expected crc 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 6/261 objects degraded (2.299%) 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: Health check update: Degraded data redundancy: 6/261 objects degraded (2.299%), 3 pgs degraded (PG_DEGRADED) 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T23:00:36.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:35 smithi064 ceph-mon[93757]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-16T23:00:37.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:36 smithi192 ceph-mon[82652]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/261 objects degraded (2.299%), 3 pgs degraded) 2024-02-16T23:00:37.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:36 smithi064 ceph-mon[93757]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/261 objects degraded (2.299%), 3 pgs degraded) 2024-02-16T23:00:38.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:37 smithi192 ceph-mon[82652]: pgmap v55: 65 pgs: 25 peering, 40 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T23:00:38.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:37 smithi064 ceph-mon[93757]: pgmap v55: 65 pgs: 25 peering, 40 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T23:00:40.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:39 smithi192 ceph-mon[82652]: pgmap v56: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 432 B/s wr, 0 op/s 2024-02-16T23:00:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:39 smithi064 ceph-mon[93757]: pgmap v56: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 432 B/s wr, 0 op/s 2024-02-16T23:00:41.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 -- 172.21.15.64:0/1576017537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fb220 msgr2=0x7f50c80fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 --2- 172.21.15.64:0/1576017537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fb220 0x7f50c80fb630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50b8004940 tx=0x7f50b8009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 -- 172.21.15.64:0/1576017537 shutdown_connections 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 --2- 172.21.15.64:0/1576017537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c80fe2b0 0x7f50c80fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 --2- 172.21.15.64:0/1576017537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fb220 0x7f50c80fb630 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 -- 172.21.15.64:0/1576017537 >> 172.21.15.64:0/1576017537 conn(0x7f50c80f6ca0 msgr2=0x7f50c80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 -- 172.21.15.64:0/1576017537 shutdown_connections 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 -- 172.21.15.64:0/1576017537 wait complete. 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.201+0000 7f50ce11c700 1 Processor -- start 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50ce11c700 1 -- start start 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50ce11c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c80fb220 0x7f50c80fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50ce11c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fe2b0 0x7f50c80fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50ce11c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80fc7c0 con 0x7f50c80fe2b0 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50ce11c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80fc930 con 0x7f50c80fb220 2024-02-16T23:00:41.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fe2b0 0x7f50c80fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fe2b0 0x7f50c80fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50612/0 (socket says 172.21.15.64:50612) 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50c6ffd700 1 -- 172.21.15.64:0/684396726 learned_addr learned my addr 172.21.15.64:0/684396726 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50c77fe700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c80fb220 0x7f50c80fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50c6ffd700 1 -- 172.21.15.64:0/684396726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c80fb220 msgr2=0x7f50c80fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50c6ffd700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c80fb220 0x7f50c80fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50c6ffd700 1 -- 172.21.15.64:0/684396726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8005040 con 0x7f50c80fe2b0 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.202+0000 7f50c6ffd700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fe2b0 0x7f50c80fc1c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00c8c0 tx=0x7f50bc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.203+0000 7f50affff700 1 -- 172.21.15.64:0/684396726 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50bc00a760 con 0x7f50c80fe2b0 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.203+0000 7f50affff700 1 -- 172.21.15.64:0/684396726 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50bc00ad90 con 0x7f50c80fe2b0 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.203+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c80fcc10 con 0x7f50c80fe2b0 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.203+0000 7f50affff700 1 -- 172.21.15.64:0/684396726 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50bc0113a0 con 0x7f50c80fe2b0 2024-02-16T23:00:41.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.203+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c8115ab0 con 0x7f50c80fe2b0 2024-02-16T23:00:41.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.204+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c8107a80 con 0x7f50c80fe2b0 2024-02-16T23:00:41.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.204+0000 7f50affff700 1 -- 172.21.15.64:0/684396726 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f50bc012400 con 0x7f50c80fe2b0 2024-02-16T23:00:41.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.205+0000 7f50affff700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f50b006d7e0 0x7f50b006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.205+0000 7f50affff700 1 -- 172.21.15.64:0/684396726 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f50bc08cde0 con 0x7f50c80fe2b0 2024-02-16T23:00:41.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.205+0000 7f50c77fe700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f50b006d7e0 0x7f50b006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.205+0000 7f50c77fe700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f50b006d7e0 0x7f50b006fc90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50b8004890 tx=0x7f50b8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:41.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.210+0000 7f50affff700 1 -- 172.21.15.64:0/684396726 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f50bc05a2d0 con 0x7f50c80fe2b0 2024-02-16T23:00:41.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.435+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50c802b310 con 0x7f50b006d7e0 2024-02-16T23:00:41.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.437+0000 7f50affff700 1 -- 172.21.15.64:0/684396726 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f50c802b310 con 0x7f50b006d7e0 2024-02-16T23:00:41.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.438+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f50b006d7e0 msgr2=0x7f50b006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.438+0000 7f50ce11c700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f50b006d7e0 0x7f50b006fc90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50b8004890 tx=0x7f50b8009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:41.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.438+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fe2b0 msgr2=0x7f50c80fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.438+0000 7f50ce11c700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fe2b0 0x7f50c80fc1c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00c8c0 tx=0x7f50bc0044c0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.439+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 shutdown_connections 2024-02-16T23:00:41.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.439+0000 7f50ce11c700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f50b006d7e0 0x7f50b006fc90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.439+0000 7f50ce11c700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c80fb220 0x7f50c80fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.439+0000 7f50ce11c700 1 --2- 172.21.15.64:0/684396726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50c80fe2b0 0x7f50c80fc1c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.439+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 >> 172.21.15.64:0/684396726 conn(0x7f50c80f6ca0 msgr2=0x7f50c80f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:41.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.439+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 shutdown_connections 2024-02-16T23:00:41.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.439+0000 7f50ce11c700 1 -- 172.21.15.64:0/684396726 wait complete. 2024-02-16T23:00:41.449 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:00:41.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.542+0000 7fb2e4925700 1 -- 172.21.15.64:0/368854522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e0066c50 msgr2=0x7fb2e00ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.542+0000 7fb2e4925700 1 --2- 172.21.15.64:0/368854522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e0066c50 0x7fb2e00ffb10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0004940 tx=0x7fb2d0009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:41.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 -- 172.21.15.64:0/368854522 shutdown_connections 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 --2- 172.21.15.64:0/368854522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2e01000e0 0x7fb2e0102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 --2- 172.21.15.64:0/368854522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e0066c50 0x7fb2e00ffb10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 -- 172.21.15.64:0/368854522 >> 172.21.15.64:0/368854522 conn(0x7fb2e00f7520 msgr2=0x7fb2e00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 -- 172.21.15.64:0/368854522 shutdown_connections 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 -- 172.21.15.64:0/368854522 wait complete. 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 Processor -- start 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 -- start start 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2e0066c50 0x7fb2e00fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e01000e0 0x7fb2e00fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e00fc010 con 0x7fb2e01000e0 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.543+0000 7fb2e4925700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e00fc180 con 0x7fb2e0066c50 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ddd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e01000e0 0x7fb2e00fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ddd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e01000e0 0x7fb2e00fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50630/0 (socket says 172.21.15.64:50630) 2024-02-16T23:00:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2de59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2e0066c50 0x7fb2e00fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ddd9b700 1 -- 172.21.15.64:0/203204542 learned_addr learned my addr 172.21.15.64:0/203204542 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ddd9b700 1 -- 172.21.15.64:0/203204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2e0066c50 msgr2=0x7fb2e00fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ddd9b700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2e0066c50 0x7fb2e00fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ddd9b700 1 -- 172.21.15.64:0/203204542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0005040 con 0x7fb2e01000e0 2024-02-16T23:00:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ddd9b700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e01000e0 0x7fb2e00fba10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4002a50 tx=0x7fb2d4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ceffd700 1 -- 172.21.15.64:0/203204542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d4007cc0 con 0x7fb2e01000e0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e00fc460 con 0x7fb2e01000e0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ceffd700 1 -- 172.21.15.64:0/203204542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2d400f040 con 0x7fb2e01000e0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2ceffd700 1 -- 172.21.15.64:0/203204542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d400d9c0 con 0x7fb2e01000e0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.544+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e0115970 con 0x7fb2e01000e0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.546+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e0107a70 con 0x7fb2e01000e0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.546+0000 7fb2ceffd700 1 -- 172.21.15.64:0/203204542 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb2d4009730 con 0x7fb2e01000e0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.547+0000 7fb2ceffd700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2c806d7f0 0x7fb2c806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.547+0000 7fb2ceffd700 1 -- 172.21.15.64:0/203204542 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb2d408cbf0 con 0x7fb2e01000e0 2024-02-16T23:00:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.547+0000 7fb2de59c700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2c806d7f0 0x7fb2c806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.547+0000 7fb2de59c700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2c806d7f0 0x7fb2c806fca0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000b840 tx=0x7fb2d0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.551+0000 7fb2ceffd700 1 -- 172.21.15.64:0/203204542 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2d4059fc0 con 0x7fb2e01000e0 2024-02-16T23:00:41.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.776+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2e002b310 con 0x7fb2c806d7f0 2024-02-16T23:00:41.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.777+0000 7fb2ceffd700 1 -- 172.21.15.64:0/203204542 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb2e002b310 con 0x7fb2c806d7f0 2024-02-16T23:00:41.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2c806d7f0 msgr2=0x7fb2c806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2c806d7f0 0x7fb2c806fca0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000b840 tx=0x7fb2d0009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:41.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e01000e0 msgr2=0x7fb2e00fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e01000e0 0x7fb2e00fba10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4002a50 tx=0x7fb2d4004600 comp rx=0 tx=0).stop 2024-02-16T23:00:41.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 shutdown_connections 2024-02-16T23:00:41.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2c806d7f0 0x7fb2c806fca0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2e0066c50 0x7fb2e00fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 --2- 172.21.15.64:0/203204542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2e01000e0 0x7fb2e00fba10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 >> 172.21.15.64:0/203204542 conn(0x7fb2e00f7520 msgr2=0x7fb2e00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:41.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 shutdown_connections 2024-02-16T23:00:41.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.779+0000 7fb2e4925700 1 -- 172.21.15.64:0/203204542 wait complete. 2024-02-16T23:00:41.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:41 smithi064 ceph-mon[93757]: pgmap v57: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 383 B/s wr, 0 op/s 2024-02-16T23:00:41.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:41 smithi064 ceph-mon[93757]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.881+0000 7f360d7ed700 1 -- 172.21.15.64:0/4038291320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 msgr2=0x7f36080ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.881+0000 7f360d7ed700 1 --2- 172.21.15.64:0/4038291320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 0x7f36080ffb10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f35f0004940 tx=0x7f35f0009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.881+0000 7f360d7ed700 1 -- 172.21.15.64:0/4038291320 shutdown_connections 2024-02-16T23:00:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.881+0000 7f360d7ed700 1 --2- 172.21.15.64:0/4038291320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36081000e0 0x7f3608102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.881+0000 7f360d7ed700 1 --2- 172.21.15.64:0/4038291320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 0x7f36080ffb10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.881+0000 7f360d7ed700 1 -- 172.21.15.64:0/4038291320 >> 172.21.15.64:0/4038291320 conn(0x7f36080f7520 msgr2=0x7f36080f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.881+0000 7f360d7ed700 1 -- 172.21.15.64:0/4038291320 shutdown_connections 2024-02-16T23:00:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.881+0000 7f360d7ed700 1 -- 172.21.15.64:0/4038291320 wait complete. 2024-02-16T23:00:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f360d7ed700 1 Processor -- start 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f360d7ed700 1 -- start start 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f360d7ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 0x7f36080fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f360d7ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36081000e0 0x7f36080fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f360d7ed700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36080fc0a0 con 0x7f3608066c50 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f360d7ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36080fc210 con 0x7f36081000e0 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f3606ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 0x7f36080fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f3606ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 0x7f36080fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50658/0 (socket says 172.21.15.64:50658) 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f3606ffd700 1 -- 172.21.15.64:0/624645615 learned_addr learned my addr 172.21.15.64:0/624645615 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.882+0000 7f36067fc700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36081000e0 0x7f36080fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f3606ffd700 1 -- 172.21.15.64:0/624645615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36081000e0 msgr2=0x7f36080fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f3606ffd700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36081000e0 0x7f36080fba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f3606ffd700 1 -- 172.21.15.64:0/624645615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0005040 con 0x7f3608066c50 2024-02-16T23:00:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f3606ffd700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 0x7f36080fd590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f35f00093b0 tx=0x7f35f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f35ff7fe700 1 -- 172.21.15.64:0/624645615 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f000e040 con 0x7f3608066c50 2024-02-16T23:00:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f35ff7fe700 1 -- 172.21.15.64:0/624645615 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35f000bcc0 con 0x7f3608066c50 2024-02-16T23:00:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36080fc490 con 0x7f3608066c50 2024-02-16T23:00:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f35ff7fe700 1 -- 172.21.15.64:0/624645615 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f001c630 con 0x7f3608066c50 2024-02-16T23:00:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.883+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3608115980 con 0x7f3608066c50 2024-02-16T23:00:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.884+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3608107af0 con 0x7f3608066c50 2024-02-16T23:00:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.884+0000 7f35ff7fe700 1 -- 172.21.15.64:0/624645615 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f35f00139f0 con 0x7f3608066c50 2024-02-16T23:00:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.885+0000 7f35ff7fe700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f35f406d7e0 0x7f35f406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.885+0000 7f35ff7fe700 1 -- 172.21.15.64:0/624645615 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f35f008e2e0 con 0x7f3608066c50 2024-02-16T23:00:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.885+0000 7f36067fc700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f35f406d7e0 0x7f35f406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.885+0000 7f36067fc700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f35f406d7e0 0x7f35f406fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35f8006fd0 tx=0x7f35f8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:41.889+0000 7f35ff7fe700 1 -- 172.21.15.64:0/624645615 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f35f005b730 con 0x7f3608066c50 2024-02-16T23:00:42.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:41 smithi192 ceph-mon[82652]: pgmap v57: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 383 B/s wr, 0 op/s 2024-02-16T23:00:42.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:41 smithi192 ceph-mon[82652]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.101+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f360802b340 con 0x7f35f406d7e0 2024-02-16T23:00:42.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.111+0000 7f35ff7fe700 1 -- 172.21.15.64:0/624645615 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f360802b340 con 0x7f35f406d7e0 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (3m) 11s ago 12m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (43s) 11s ago 12m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (38s) 35s ago 11m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (3m) 11s ago 12m 84.3M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (6m) 11s ago 6m 17.6M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (6m) 11s ago 6m 153M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (6m) 35s ago 6m 16.5M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (6m) 35s ago 6m 16.2M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (2m) 11s ago 13m 497M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:00:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (2m) 35s ago 11m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (108s) 11s ago 13m 42.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (84s) 35s ago 11m 35.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (4m) 11s ago 12m 15.5M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (2m) 35s ago 11m 15.8M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (17s) 11s ago 10m 12.6M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (10m) 11s ago 10m 216M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 f078acb0ea2f 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (9m) 11s ago 9m 176M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (9m) 35s ago 9m 234M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (8m) 35s ago 8m 210M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (8m) 35s ago 8m 186M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T23:00:42.113 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (2m) 11s ago 11m 57.1M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:00:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f35f406d7e0 msgr2=0x7f35f406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f35f406d7e0 0x7f35f406fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35f8006fd0 tx=0x7f35f8008040 comp rx=0 tx=0).stop 2024-02-16T23:00:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 msgr2=0x7f36080fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 0x7f36080fd590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f35f00093b0 tx=0x7f35f0009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 shutdown_connections 2024-02-16T23:00:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f35f406d7e0 0x7f35f406fc90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3608066c50 0x7f36080fd590 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 --2- 172.21.15.64:0/624645615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36081000e0 0x7f36080fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 >> 172.21.15.64:0/624645615 conn(0x7f36080f7520 msgr2=0x7f36080f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 shutdown_connections 2024-02-16T23:00:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.113+0000 7f360d7ed700 1 -- 172.21.15.64:0/624645615 wait complete. 2024-02-16T23:00:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.215+0000 7f20b2e1d700 1 -- 172.21.15.64:0/2422634374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 msgr2=0x7f20ac0fb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.215+0000 7f20b2e1d700 1 --2- 172.21.15.64:0/2422634374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 0x7f20ac0fb790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f209c004940 tx=0x7f209c009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.215+0000 7f20b2e1d700 1 -- 172.21.15.64:0/2422634374 shutdown_connections 2024-02-16T23:00:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.215+0000 7f20b2e1d700 1 --2- 172.21.15.64:0/2422634374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0fbcd0 0x7f20ac102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.215+0000 7f20b2e1d700 1 --2- 172.21.15.64:0/2422634374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 0x7f20ac0fb790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.215+0000 7f20b2e1d700 1 -- 172.21.15.64:0/2422634374 >> 172.21.15.64:0/2422634374 conn(0x7f20ac0f6d20 msgr2=0x7f20ac0f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.215+0000 7f20b2e1d700 1 -- 172.21.15.64:0/2422634374 shutdown_connections 2024-02-16T23:00:42.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.215+0000 7f20b2e1d700 1 -- 172.21.15.64:0/2422634374 wait complete. 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b2e1d700 1 Processor -- start 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b2e1d700 1 -- start start 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b2e1d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 0x7f20ac10dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b2e1d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0fbcd0 0x7f20ac10e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b2e1d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac10eff0 con 0x7f20ac0fb380 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b2e1d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac10f160 con 0x7f20ac0fbcd0 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b0bb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 0x7f20ac10dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b0bb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 0x7f20ac10dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50682/0 (socket says 172.21.15.64:50682) 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.216+0000 7f20b0bb9700 1 -- 172.21.15.64:0/319026142 learned_addr learned my addr 172.21.15.64:0/319026142 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20abfff700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0fbcd0 0x7f20ac10e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20b0bb9700 1 -- 172.21.15.64:0/319026142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0fbcd0 msgr2=0x7f20ac10e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20b0bb9700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0fbcd0 0x7f20ac10e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20b0bb9700 1 -- 172.21.15.64:0/319026142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c005040 con 0x7f20ac0fb380 2024-02-16T23:00:42.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20b0bb9700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 0x7f20ac10dda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f209c0093b0 tx=0x7f209c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:42.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20a97fa700 1 -- 172.21.15.64:0/319026142 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209c00e040 con 0x7f20ac0fb380 2024-02-16T23:00:42.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac10f3e0 con 0x7f20ac0fb380 2024-02-16T23:00:42.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20a97fa700 1 -- 172.21.15.64:0/319026142 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f209c00bcc0 con 0x7f20ac0fb380 2024-02-16T23:00:42.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20a97fa700 1 -- 172.21.15.64:0/319026142 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209c01c630 con 0x7f20ac0fb380 2024-02-16T23:00:42.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.217+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac10f920 con 0x7f20ac0fb380 2024-02-16T23:00:42.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.218+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac107b00 con 0x7f20ac0fb380 2024-02-16T23:00:42.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.219+0000 7f20a97fa700 1 -- 172.21.15.64:0/319026142 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f209c0139f0 con 0x7f20ac0fb380 2024-02-16T23:00:42.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.219+0000 7f20a97fa700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f209406d7e0 0x7f209406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.220+0000 7f20a97fa700 1 -- 172.21.15.64:0/319026142 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f209c08e2e0 con 0x7f20ac0fb380 2024-02-16T23:00:42.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.220+0000 7f20abfff700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f209406d7e0 0x7f209406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.220+0000 7f20abfff700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f209406d7e0 0x7f209406fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006fd0 tx=0x7f20a0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:42.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.224+0000 7f20a97fa700 1 -- 172.21.15.64:0/319026142 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f209c05b730 con 0x7f20ac0fb380 2024-02-16T23:00:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.512+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f20ac04bed0 con 0x7f20ac0fb380 2024-02-16T23:00:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.513+0000 7f20a97fa700 1 -- 172.21.15.64:0/319026142 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f209c05b2c0 con 0x7f20ac0fb380 2024-02-16T23:00:42.514 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:00:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:00:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:00:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:00:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:00:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:00:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:00:42.514 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:00:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 5, 2024-02-16T23:00:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T23:00:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:00:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:00:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T23:00:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:00:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:00:42.515 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 9, 2024-02-16T23:00:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-16T23:00:42.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:00:42.516 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:00:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f209406d7e0 msgr2=0x7f209406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f209406d7e0 0x7f209406fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006fd0 tx=0x7f20a0008040 comp rx=0 tx=0).stop 2024-02-16T23:00:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 msgr2=0x7f20ac10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 0x7f20ac10dda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f209c0093b0 tx=0x7f209c009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 shutdown_connections 2024-02-16T23:00:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f209406d7e0 0x7f209406fc90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20ac0fb380 0x7f20ac10dda0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 --2- 172.21.15.64:0/319026142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0fbcd0 0x7f20ac10e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 >> 172.21.15.64:0/319026142 conn(0x7f20ac0f6d20 msgr2=0x7f20ac100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 shutdown_connections 2024-02-16T23:00:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.515+0000 7f20b2e1d700 1 -- 172.21.15.64:0/319026142 wait complete. 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.617+0000 7f55a3949700 1 -- 172.21.15.64:0/889968795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fb220 msgr2=0x7f559c0fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.617+0000 7f55a3949700 1 --2- 172.21.15.64:0/889968795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fb220 0x7f559c0fb630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5590004940 tx=0x7f5590009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.617+0000 7f55a3949700 1 -- 172.21.15.64:0/889968795 shutdown_connections 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.617+0000 7f55a3949700 1 --2- 172.21.15.64:0/889968795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559c0fe2b0 0x7f559c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.617+0000 7f55a3949700 1 --2- 172.21.15.64:0/889968795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fb220 0x7f559c0fb630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.617+0000 7f55a3949700 1 -- 172.21.15.64:0/889968795 >> 172.21.15.64:0/889968795 conn(0x7f559c0f6ca0 msgr2=0x7f559c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.617+0000 7f55a3949700 1 -- 172.21.15.64:0/889968795 shutdown_connections 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.617+0000 7f55a3949700 1 -- 172.21.15.64:0/889968795 wait complete. 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a3949700 1 Processor -- start 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a3949700 1 -- start start 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a3949700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559c0fb220 0x7f559c0fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a3949700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fe2b0 0x7f559c0fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a3949700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c0fc7c0 con 0x7f559c0fe2b0 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a3949700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c0fc930 con 0x7f559c0fb220 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a0ee4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fe2b0 0x7f559c0fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a0ee4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fe2b0 0x7f559c0fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50694/0 (socket says 172.21.15.64:50694) 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a0ee4700 1 -- 172.21.15.64:0/2768219331 learned_addr learned my addr 172.21.15.64:0/2768219331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a16e5700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559c0fb220 0x7f559c0fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a0ee4700 1 -- 172.21.15.64:0/2768219331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559c0fb220 msgr2=0x7f559c0fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.618+0000 7f55a0ee4700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559c0fb220 0x7f559c0fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.619+0000 7f55a0ee4700 1 -- 172.21.15.64:0/2768219331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5590005040 con 0x7f559c0fe2b0 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.619+0000 7f55a0ee4700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fe2b0 0x7f559c0fc1c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5598002a50 tx=0x7f55980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.619+0000 7f558dffb700 1 -- 172.21.15.64:0/2768219331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5598007cc0 con 0x7f559c0fe2b0 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.619+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c0fcc10 con 0x7f559c0fe2b0 2024-02-16T23:00:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.619+0000 7f558dffb700 1 -- 172.21.15.64:0/2768219331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f559800f040 con 0x7f559c0fe2b0 2024-02-16T23:00:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.619+0000 7f558dffb700 1 -- 172.21.15.64:0/2768219331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559800da30 con 0x7f559c0fe2b0 2024-02-16T23:00:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.619+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c115ab0 con 0x7f559c0fe2b0 2024-02-16T23:00:42.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.620+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559c107a80 con 0x7f559c0fe2b0 2024-02-16T23:00:42.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.620+0000 7f558dffb700 1 -- 172.21.15.64:0/2768219331 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f559800db90 con 0x7f559c0fe2b0 2024-02-16T23:00:42.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.621+0000 7f558dffb700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f558806d7e0 0x7f558806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.621+0000 7f558dffb700 1 -- 172.21.15.64:0/2768219331 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f559808cb10 con 0x7f559c0fe2b0 2024-02-16T23:00:42.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.621+0000 7f55a16e5700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f558806d7e0 0x7f558806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.621+0000 7f55a16e5700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f558806d7e0 0x7f558806fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5590004890 tx=0x7f5590009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:42.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.626+0000 7f558dffb700 1 -- 172.21.15.64:0/2768219331 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5598059f60 con 0x7f559c0fe2b0 2024-02-16T23:00:42.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.877+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f559c105a40 con 0x7f559c0fe2b0 2024-02-16T23:00:42.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.878+0000 7f558dffb700 1 -- 172.21.15.64:0/2768219331 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f5598059af0 con 0x7f559c0fe2b0 2024-02-16T23:00:42.878 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T23:00:42.878 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:00:42.878 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:00:42.878 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:00:42.879 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:00:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:00:42.881 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:00:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:00:42.881 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:00:42.881 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:00:42.881 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f558806d7e0 msgr2=0x7f558806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f558806d7e0 0x7f558806fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5590004890 tx=0x7f5590009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fe2b0 msgr2=0x7f559c0fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fe2b0 0x7f559c0fc1c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5598002a50 tx=0x7f55980046b0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 shutdown_connections 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f558806d7e0 0x7f558806fc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559c0fb220 0x7f559c0fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 --2- 172.21.15.64:0/2768219331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559c0fe2b0 0x7f559c0fc1c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 >> 172.21.15.64:0/2768219331 conn(0x7f559c0f6ca0 msgr2=0x7f559c0f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 shutdown_connections 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.880+0000 7f55a3949700 1 -- 172.21.15.64:0/2768219331 wait complete. 2024-02-16T23:00:42.882 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T23:00:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.983+0000 7fc626115700 1 -- 172.21.15.64:0/3503083324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 msgr2=0x7fc6200ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.983+0000 7fc626115700 1 --2- 172.21.15.64:0/3503083324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 0x7fc6200ffe60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc614004940 tx=0x7fc614009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:42.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.983+0000 7fc626115700 1 -- 172.21.15.64:0/3503083324 shutdown_connections 2024-02-16T23:00:42.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.983+0000 7fc626115700 1 --2- 172.21.15.64:0/3503083324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6201003a0 0x7fc620102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.983+0000 7fc626115700 1 --2- 172.21.15.64:0/3503083324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 0x7fc6200ffe60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.983+0000 7fc626115700 1 -- 172.21.15.64:0/3503083324 >> 172.21.15.64:0/3503083324 conn(0x7fc6200f74e0 msgr2=0x7fc6200f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:42.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.983+0000 7fc626115700 1 -- 172.21.15.64:0/3503083324 shutdown_connections 2024-02-16T23:00:42.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.983+0000 7fc626115700 1 -- 172.21.15.64:0/3503083324 wait complete. 2024-02-16T23:00:42.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc626115700 1 Processor -- start 2024-02-16T23:00:42.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc626115700 1 -- start start 2024-02-16T23:00:42.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc626115700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 0x7fc62010fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc626115700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6201003a0 0x7fc620110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc626115700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201106d0 con 0x7fc6200fda80 2024-02-16T23:00:42.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc626115700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc620110f10 con 0x7fc6201003a0 2024-02-16T23:00:42.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc61f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 0x7fc62010fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc61f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 0x7fc62010fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50720/0 (socket says 172.21.15.64:50720) 2024-02-16T23:00:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.984+0000 7fc61f7fe700 1 -- 172.21.15.64:0/3521452509 learned_addr learned my addr 172.21.15.64:0/3521452509 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc61effd700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6201003a0 0x7fc620110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc61f7fe700 1 -- 172.21.15.64:0/3521452509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6201003a0 msgr2=0x7fc620110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc61f7fe700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6201003a0 0x7fc620110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc61f7fe700 1 -- 172.21.15.64:0/3521452509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614005040 con 0x7fc6200fda80 2024-02-16T23:00:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc61effd700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6201003a0 0x7fc620110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:00:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc61f7fe700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 0x7fc62010fb00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6140093b0 tx=0x7fc614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:42.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc607fff700 1 -- 172.21.15.64:0/3521452509 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61400e040 con 0x7fc6200fda80 2024-02-16T23:00:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc620111190 con 0x7fc6200fda80 2024-02-16T23:00:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc607fff700 1 -- 172.21.15.64:0/3521452509 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc61400bcc0 con 0x7fc6200fda80 2024-02-16T23:00:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc607fff700 1 -- 172.21.15.64:0/3521452509 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61401c630 con 0x7fc6200fda80 2024-02-16T23:00:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.985+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6201116d0 con 0x7fc6200fda80 2024-02-16T23:00:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.986+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62004bed0 con 0x7fc6200fda80 2024-02-16T23:00:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.987+0000 7fc607fff700 1 -- 172.21.15.64:0/3521452509 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc6140139f0 con 0x7fc6200fda80 2024-02-16T23:00:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.987+0000 7fc607fff700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc60806d7e0 0x7fc60806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.987+0000 7fc607fff700 1 -- 172.21.15.64:0/3521452509 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc61408e2d0 con 0x7fc6200fda80 2024-02-16T23:00:42.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.987+0000 7fc61effd700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc60806d7e0 0x7fc60806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:42.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.987+0000 7fc61effd700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc60806d7e0 0x7fc60806fc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc610006fd0 tx=0x7fc610008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:42.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:42.992+0000 7fc607fff700 1 -- 172.21.15.64:0/3521452509 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc61405b720 con 0x7fc6200fda80 2024-02-16T23:00:43.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:42 smithi192 ceph-mon[82652]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:43.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:42 smithi192 ceph-mon[82652]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:43.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:42 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/319026142' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:43.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:42 smithi064 ceph-mon[93757]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:43.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:42 smithi064 ceph-mon[93757]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:43.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:42 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/319026142' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:00:43.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.212+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc620111320 con 0x7fc60806d7e0 2024-02-16T23:00:43.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.213+0000 7fc607fff700 1 -- 172.21.15.64:0/3521452509 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc620111320 con 0x7fc60806d7e0 2024-02-16T23:00:43.214 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:00:43.214 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:00:43.214 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:00:43.214 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:00:43.214 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:00:43.214 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:00:43.214 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:00:43.215 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:00:43.215 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:00:43.215 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "7/21 daemons upgraded", 2024-02-16T23:00:43.215 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T23:00:43.215 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:00:43.215 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:00:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc60806d7e0 msgr2=0x7fc60806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc60806d7e0 0x7fc60806fc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc610006fd0 tx=0x7fc610008040 comp rx=0 tx=0).stop 2024-02-16T23:00:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 msgr2=0x7fc62010fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 0x7fc62010fb00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6140093b0 tx=0x7fc614009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 shutdown_connections 2024-02-16T23:00:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc60806d7e0 0x7fc60806fc90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6200fda80 0x7fc62010fb00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 --2- 172.21.15.64:0/3521452509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6201003a0 0x7fc620110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.215+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 >> 172.21.15.64:0/3521452509 conn(0x7fc6200f74e0 msgr2=0x7fc6200fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.216+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 shutdown_connections 2024-02-16T23:00:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.216+0000 7fc626115700 1 -- 172.21.15.64:0/3521452509 wait complete. 2024-02-16T23:00:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.317+0000 7f74d2ff5700 1 -- 172.21.15.64:0/1584382163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 msgr2=0x7f74cc0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.317+0000 7f74d2ff5700 1 --2- 172.21.15.64:0/1584382163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 0x7f74cc0ffe60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74c0004940 tx=0x7f74c0009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.317+0000 7f74d2ff5700 1 -- 172.21.15.64:0/1584382163 shutdown_connections 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.317+0000 7f74d2ff5700 1 --2- 172.21.15.64:0/1584382163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74cc1003a0 0x7f74cc102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.317+0000 7f74d2ff5700 1 --2- 172.21.15.64:0/1584382163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 0x7f74cc0ffe60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.317+0000 7f74d2ff5700 1 -- 172.21.15.64:0/1584382163 >> 172.21.15.64:0/1584382163 conn(0x7f74cc0f7520 msgr2=0x7f74cc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.318+0000 7f74d2ff5700 1 -- 172.21.15.64:0/1584382163 shutdown_connections 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.318+0000 7f74d2ff5700 1 -- 172.21.15.64:0/1584382163 wait complete. 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.318+0000 7f74d2ff5700 1 Processor -- start 2024-02-16T23:00:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.318+0000 7f74d2ff5700 1 -- start start 2024-02-16T23:00:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d2ff5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 0x7f74cc10d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d2ff5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74cc1003a0 0x7f74cc10de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d2ff5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc10e430 con 0x7f74cc0fda80 2024-02-16T23:00:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d2ff5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc10ecd0 con 0x7f74cc1003a0 2024-02-16T23:00:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d0d91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 0x7f74cc10d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d0d91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 0x7f74cc10d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50730/0 (socket says 172.21.15.64:50730) 2024-02-16T23:00:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d0d91700 1 -- 172.21.15.64:0/2620030381 learned_addr learned my addr 172.21.15.64:0/2620030381 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:00:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74cbfff700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74cc1003a0 0x7f74cc10de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d0d91700 1 -- 172.21.15.64:0/2620030381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74cc1003a0 msgr2=0x7f74cc10de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d0d91700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74cc1003a0 0x7f74cc10de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d0d91700 1 -- 172.21.15.64:0/2620030381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0005040 con 0x7f74cc0fda80 2024-02-16T23:00:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d0d91700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 0x7f74cc10d8f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f74c00093b0 tx=0x7f74c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74c97fa700 1 -- 172.21.15.64:0/2620030381 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74c000e040 con 0x7f74cc0fda80 2024-02-16T23:00:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74c97fa700 1 -- 172.21.15.64:0/2620030381 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74c000bcc0 con 0x7f74cc0fda80 2024-02-16T23:00:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.319+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc10ef50 con 0x7f74cc0fda80 2024-02-16T23:00:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.320+0000 7f74c97fa700 1 -- 172.21.15.64:0/2620030381 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74c001c630 con 0x7f74cc0fda80 2024-02-16T23:00:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.320+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc10f490 con 0x7f74cc0fda80 2024-02-16T23:00:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.321+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc04bed0 con 0x7f74cc0fda80 2024-02-16T23:00:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.321+0000 7f74c97fa700 1 -- 172.21.15.64:0/2620030381 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f74c00139f0 con 0x7f74cc0fda80 2024-02-16T23:00:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.321+0000 7f74c97fa700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f74bc06d7e0 0x7f74bc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:00:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.322+0000 7f74c97fa700 1 -- 172.21.15.64:0/2620030381 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f74c008e2e0 con 0x7f74cc0fda80 2024-02-16T23:00:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.322+0000 7f74cbfff700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f74bc06d7e0 0x7f74bc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:00:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.322+0000 7f74cbfff700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f74bc06d7e0 0x7f74bc06fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009fd0 tx=0x7f74b8009f20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:00:43.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.326+0000 7f74c97fa700 1 -- 172.21.15.64:0/2620030381 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74c005b730 con 0x7f74cc0fda80 2024-02-16T23:00:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.612+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f74cc113f10 con 0x7f74cc0fda80 2024-02-16T23:00:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.613+0000 7f74c97fa700 1 -- 172.21.15.64:0/2620030381 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f74c005b2c0 con 0x7f74cc0fda80 2024-02-16T23:00:43.614 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:00:43.614 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:00:43.614 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:00:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f74bc06d7e0 msgr2=0x7f74bc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f74bc06d7e0 0x7f74bc06fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009fd0 tx=0x7f74b8009f20 comp rx=0 tx=0).stop 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 msgr2=0x7f74cc10d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 0x7f74cc10d8f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f74c00093b0 tx=0x7f74c0009e30 comp rx=0 tx=0).stop 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 shutdown_connections 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f74bc06d7e0 0x7f74bc06fc90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc0fda80 0x7f74cc10d8f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 --2- 172.21.15.64:0/2620030381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74cc1003a0 0x7f74cc10de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 >> 172.21.15.64:0/2620030381 conn(0x7f74cc0f7520 msgr2=0x7f74cc0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 shutdown_connections 2024-02-16T23:00:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:00:43.615+0000 7f74d2ff5700 1 -- 172.21.15.64:0/2620030381 wait complete. 2024-02-16T23:00:43.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:43 smithi064 ceph-mon[93757]: pgmap v58: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 333 B/s wr, 0 op/s 2024-02-16T23:00:43.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:43 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2768219331' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:00:43.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:43 smithi064 ceph-mon[93757]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:43.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:43 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2620030381' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:00:44.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:43 smithi192 ceph-mon[82652]: pgmap v58: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 333 B/s wr, 0 op/s 2024-02-16T23:00:44.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:43 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2768219331' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:00:44.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:43 smithi192 ceph-mon[82652]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:00:44.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:43 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2620030381' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:00:46.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:45 smithi192 ceph-mon[82652]: pgmap v59: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:45 smithi064 ceph-mon[93757]: pgmap v59: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:48.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:47 smithi192 ceph-mon[82652]: pgmap v60: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:48.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:48.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:00:48.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:47 smithi064 ceph-mon[93757]: pgmap v60: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:00:48.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:50.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:49 smithi192 ceph-mon[82652]: pgmap v61: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:50.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:49 smithi064 ceph-mon[93757]: pgmap v61: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:51.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:50 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T23:00:51.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:50 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T23:00:51.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:50 smithi192 ceph-mon[82652]: Upgrade: osd.1 is safe to restart 2024-02-16T23:00:51.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:50 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T23:00:51.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:50 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T23:00:51.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:50 smithi064 ceph-mon[93757]: Upgrade: osd.1 is safe to restart 2024-02-16T23:00:52.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:51 smithi192 ceph-mon[82652]: pgmap v62: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:52.036 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:51 smithi064 ceph-mon[93757]: pgmap v62: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:53.343 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:53 smithi064 ceph-mon[93757]: Upgrade: Updating osd.1 2024-02-16T23:00:53.344 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:53.344 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T23:00:53.344 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:53.344 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:53 smithi064 ceph-mon[93757]: Deploying daemon osd.1 on smithi064 2024-02-16T23:00:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:53 smithi192 ceph-mon[82652]: Upgrade: Updating osd.1 2024-02-16T23:00:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:00:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T23:00:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:00:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:53 smithi192 ceph-mon[82652]: Deploying daemon osd.1 on smithi064 2024-02-16T23:00:54.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:54 smithi064 ceph-mon[93757]: pgmap v63: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:54.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:54 smithi192 ceph-mon[82652]: pgmap v63: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:55.548 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:55 smithi064 systemd[1]: Stopping Ceph osd.1 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:00:55.868 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:55 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[55345]: 2024-02-16T23:00:55.546+0000 7f2ac4a4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:00:55.868 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:55 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[55345]: 2024-02-16T23:00:55.546+0000 7f2ac4a4b700 -1 osd.1 55 *** Got signal Terminated *** 2024-02-16T23:00:55.868 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:55 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[55345]: 2024-02-16T23:00:55.546+0000 7f2ac4a4b700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:00:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:56 smithi064 ceph-mon[93757]: pgmap v64: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:56 smithi064 ceph-mon[93757]: osd.1 marked itself down and dead 2024-02-16T23:00:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:56 smithi192 ceph-mon[82652]: pgmap v64: 65 pgs: 65 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:56 smithi192 ceph-mon[82652]: osd.1 marked itself down and dead 2024-02-16T23:00:56.766 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:56 smithi064 podman[107794]: 2024-02-16 23:00:56.507759503 +0000 UTC m=+1.083040331 container died f078acb0ea2f938644b1b81dc01f34de63673d073f17fd8ede6609cb77f6f5c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=quincy-b88fae6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-16T23:00:57.116 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:56 smithi064 podman[107794]: 2024-02-16 23:00:56.873453125 +0000 UTC m=+1.448733952 container cleanup f078acb0ea2f938644b1b81dc01f34de63673d073f17fd8ede6609cb77f6f5c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=quincy-b88fae6, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-16T23:00:57.116 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:56 smithi064 bash[107794]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1 2024-02-16T23:00:57.368 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:57 smithi064 podman[107811]: 2024-02-16 23:00:57.116252814 +0000 UTC m=+0.608260833 container remove f078acb0ea2f938644b1b81dc01f34de63673d073f17fd8ede6609cb77f6f5c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-b88fae6, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-16T23:00:57.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:57 smithi064 ceph-mon[93757]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:00:57.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:57 smithi064 ceph-mon[93757]: osdmap e56: 6 total, 5 up, 6 in 2024-02-16T23:00:57.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:57 smithi064 ceph-mon[93757]: failed to encode map e56 with expected crc 2024-02-16T23:00:57.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:57 smithi064 ceph-mon[93757]: failed to encode map e56 with expected crc 2024-02-16T23:00:57.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:57 smithi064 ceph-mon[93757]: failed to encode map e56 with expected crc 2024-02-16T23:00:57.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:57 smithi064 ceph-mon[93757]: failed to encode map e56 with expected crc 2024-02-16T23:00:57.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:57 smithi192 ceph-mon[82652]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:00:57.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:57 smithi192 ceph-mon[82652]: osdmap e56: 6 total, 5 up, 6 in 2024-02-16T23:00:57.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:57 smithi192 ceph-mon[82652]: failed to encode map e56 with expected crc 2024-02-16T23:00:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:57 smithi192 ceph-mon[82652]: failed to encode map e56 with expected crc 2024-02-16T23:00:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:57 smithi192 ceph-mon[82652]: failed to encode map e56 with expected crc 2024-02-16T23:00:57.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:57 smithi192 ceph-mon[82652]: failed to encode map e56 with expected crc 2024-02-16T23:00:57.779 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:57 smithi064 podman[107901]: 2024-02-16 23:00:57.360828425 +0000 UTC m=+0.018315807 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:00:57.779 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:57 smithi064 podman[107901]: 2024-02-16 23:00:57.49249364 +0000 UTC m=+0.149981063 container create e3ada10f7058686ee223e11704c971ce1b34d6a2e7e0e317c8dbcc56d934f46b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-16T23:00:57.779 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:57 smithi064 podman[107901]: 2024-02-16 23:00:57.779251222 +0000 UTC m=+0.436738613 container init e3ada10f7058686ee223e11704c971ce1b34d6a2e7e0e317c8dbcc56d934f46b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-02-16T23:00:58.052 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:57 smithi064 podman[107901]: 2024-02-16 23:00:57.784313567 +0000 UTC m=+0.441800960 container start e3ada10f7058686ee223e11704c971ce1b34d6a2e7e0e317c8dbcc56d934f46b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-16T23:00:58.052 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:57 smithi064 podman[107901]: 2024-02-16 23:00:57.887199669 +0000 UTC m=+0.544687080 container attach e3ada10f7058686ee223e11704c971ce1b34d6a2e7e0e317c8dbcc56d934f46b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:00:58.368 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:58 smithi064 podman[107901]: 2024-02-16 23:00:58.0966485 +0000 UTC m=+0.754135873 container died e3ada10f7058686ee223e11704c971ce1b34d6a2e7e0e317c8dbcc56d934f46b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: pgmap v66: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: osdmap e57: 6 total, 5 up, 6 in 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:58 smithi064 ceph-mon[93757]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: pgmap v66: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:00:58.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-16T23:00:58.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: osdmap e57: 6 total, 5 up, 6 in 2024-02-16T23:00:58.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:58 smithi192 ceph-mon[82652]: failed to encode map e57 with expected crc 2024-02-16T23:00:58.632 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:58 smithi064 podman[107935]: 2024-02-16 23:00:58.615884657 +0000 UTC m=+0.514534632 container remove e3ada10f7058686ee223e11704c971ce1b34d6a2e7e0e317c8dbcc56d934f46b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-16T23:00:59.059 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:58 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-16T23:00:59.059 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:58 smithi064 systemd[1]: Stopped Ceph osd.1 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:00:59.059 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:58 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.1.service: Consumed 57.009s CPU time. 2024-02-16T23:00:59.059 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:58 smithi064 systemd[1]: Starting Ceph osd.1 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:00:59.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:00:59 smithi064 ceph-mon[93757]: Health check failed: Degraded data redundancy: 32/261 objects degraded (12.261%), 11 pgs degraded (PG_DEGRADED) 2024-02-16T23:00:59.368 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 podman[108040]: 2024-02-16 23:00:59.058688617 +0000 UTC m=+0.103508905 container create a23a2b973da78d9569573f8fdcc1485c8d3d525e1a9647bbfa898cb2c9eba970 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-16T23:00:59.368 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 podman[108040]: 2024-02-16 23:00:58.973583575 +0000 UTC m=+0.018403825 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:00:59.368 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 podman[108040]: 2024-02-16 23:00:59.286437497 +0000 UTC m=+0.331257756 container init a23a2b973da78d9569573f8fdcc1485c8d3d525e1a9647bbfa898cb2c9eba970 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-02-16T23:00:59.368 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 podman[108040]: 2024-02-16 23:00:59.289922771 +0000 UTC m=+0.334743022 container start a23a2b973da78d9569573f8fdcc1485c8d3d525e1a9647bbfa898cb2c9eba970 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-16T23:00:59.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:00:59 smithi192 ceph-mon[82652]: Health check failed: Degraded data redundancy: 32/261 objects degraded (12.261%), 11 pgs degraded (PG_DEGRADED) 2024-02-16T23:00:59.737 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 podman[108040]: 2024-02-16 23:00:59.395000254 +0000 UTC m=+0.439820522 container attach a23a2b973da78d9569573f8fdcc1485c8d3d525e1a9647bbfa898cb2c9eba970 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-02-16T23:01:00.107 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate[108055]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T23:01:00.107 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate[108055]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 bash[108040]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 bash[108040]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 bash[108040]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 bash[108040]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 bash[108040]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 bash[108040]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 bash[108040]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate[108055]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate[108055]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate[108055]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate[108055]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate[108055]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-16T23:01:00.108 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:00:59 smithi064 podman[108040]: 2024-02-16 23:00:59.762575304 +0000 UTC m=+0.807395548 container died a23a2b973da78d9569573f8fdcc1485c8d3d525e1a9647bbfa898cb2c9eba970 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-16T23:01:00.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:00 smithi064 ceph-mon[93757]: pgmap v68: 65 pgs: 11 active+undersized+degraded, 5 peering, 18 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 32/261 objects degraded (12.261%) 2024-02-16T23:01:00.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:00 smithi192 ceph-mon[82652]: pgmap v68: 65 pgs: 11 active+undersized+degraded, 5 peering, 18 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 32/261 objects degraded (12.261%) 2024-02-16T23:01:00.722 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:00 smithi064 podman[108178]: 2024-02-16 23:01:00.382382237 +0000 UTC m=+0.614927835 container remove a23a2b973da78d9569573f8fdcc1485c8d3d525e1a9647bbfa898cb2c9eba970 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-activate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T23:01:00.722 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:00 smithi064 podman[108243]: 2024-02-16 23:01:00.496868399 +0000 UTC m=+0.018186221 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:01:00.723 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:00 smithi064 podman[108243]: 2024-02-16 23:01:00.60841748 +0000 UTC m=+0.129735329 container create 7044a8328d9a666a4164d60801330c6d4795cef8e2050baeabef0cb0ebb298ba (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-16T23:01:01.118 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:00 smithi064 podman[108243]: 2024-02-16 23:01:00.819170567 +0000 UTC m=+0.340488386 container init 7044a8328d9a666a4164d60801330c6d4795cef8e2050baeabef0cb0ebb298ba (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-16T23:01:01.118 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:00 smithi064 podman[108243]: 2024-02-16 23:01:00.82237213 +0000 UTC m=+0.343689938 container start 7044a8328d9a666a4164d60801330c6d4795cef8e2050baeabef0cb0ebb298ba (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:01:01.118 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:00 smithi064 ceph-osd[108262]: -- 172.21.15.64:0/511815555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bd5075d1e0 con 0x55bd5072ec00 2024-02-16T23:01:01.118 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:00 smithi064 bash[108243]: 7044a8328d9a666a4164d60801330c6d4795cef8e2050baeabef0cb0ebb298ba 2024-02-16T23:01:01.118 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:00 smithi064 systemd[1]: Started Ceph osd.1 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:01:01.618 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:01 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[108258]: 2024-02-16T23:01:01.360+0000 7f2d7421b7c0 -1 Falling back to public interface 2024-02-16T23:01:02.064 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:01 smithi064 ceph-mon[93757]: pgmap v69: 65 pgs: 11 active+undersized+degraded, 5 peering, 18 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 32/261 objects degraded (12.261%) 2024-02-16T23:01:02.065 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:01 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:02.065 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:01 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:02.065 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:01 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:01:02.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:01 smithi192 ceph-mon[82652]: pgmap v69: 65 pgs: 11 active+undersized+degraded, 5 peering, 18 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 32/261 objects degraded (12.261%) 2024-02-16T23:01:02.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:01 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:02.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:01 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:02.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:01 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:01:03.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:03 smithi064 ceph-mon[93757]: pgmap v70: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:03.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:03.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:01:03.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:03 smithi064 ceph-mon[93757]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-02-16T23:01:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:03 smithi192 ceph-mon[82652]: pgmap v70: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:01:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:03 smithi192 ceph-mon[82652]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-02-16T23:01:06.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:05 smithi192 ceph-mon[82652]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:06.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:05 smithi064 ceph-mon[93757]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:06.118 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:05 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[108258]: 2024-02-16T23:01:05.836+0000 7f2d7421b7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-02-16T23:01:06.867 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:06 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[108258]: 2024-02-16T23:01:06.383+0000 7f2d7421b7c0 -1 osd.1 55 log_to_monitors true 2024-02-16T23:01:07.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:07 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:07.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:07 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:07.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:07 smithi192 ceph-mon[82652]: from='osd.1 [v2:172.21.15.64:6810/1635803238,v1:172.21.15.64:6811/1635803238]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T23:01:07.618 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:01:07 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[108258]: 2024-02-16T23:01:07.248+0000 7f2d65673700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T23:01:07.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:07 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:07.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:07 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:07.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:07 smithi064 ceph-mon[93757]: from='osd.1 [v2:172.21.15.64:6810/1635803238,v1:172.21.15.64:6811/1635803238]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T23:01:08.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:08 smithi064 ceph-mon[93757]: pgmap v72: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:08.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:08 smithi064 ceph-mon[93757]: from='osd.1 [v2:172.21.15.64:6810/1635803238,v1:172.21.15.64:6811/1635803238]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T23:01:08.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:08 smithi064 ceph-mon[93757]: osdmap e58: 6 total, 5 up, 6 in 2024-02-16T23:01:08.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:08 smithi064 ceph-mon[93757]: from='osd.1 [v2:172.21.15.64:6810/1635803238,v1:172.21.15.64:6811/1635803238]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T23:01:08.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:08.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:08.247 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:08 smithi064 ceph-mon[93757]: failed to encode map e58 with expected crc 2024-02-16T23:01:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:08 smithi192 ceph-mon[82652]: pgmap v72: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 251 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:08 smithi192 ceph-mon[82652]: from='osd.1 [v2:172.21.15.64:6810/1635803238,v1:172.21.15.64:6811/1635803238]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T23:01:08.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:08 smithi192 ceph-mon[82652]: osdmap e58: 6 total, 5 up, 6 in 2024-02-16T23:01:08.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:08 smithi192 ceph-mon[82652]: from='osd.1 [v2:172.21.15.64:6810/1635803238,v1:172.21.15.64:6811/1635803238]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T23:01:08.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:08 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:08.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:08 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:08.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:08 smithi192 ceph-mon[82652]: failed to encode map e58 with expected crc 2024-02-16T23:01:09.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:09 smithi064 ceph-mon[93757]: failed to encode map e58 with expected crc 2024-02-16T23:01:09.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:09 smithi064 ceph-mon[93757]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:01:09.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:09 smithi064 ceph-mon[93757]: failed to encode map e58 with expected crc 2024-02-16T23:01:09.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:09 smithi064 ceph-mon[93757]: osd.1 [v2:172.21.15.64:6810/1635803238,v1:172.21.15.64:6811/1635803238] boot 2024-02-16T23:01:09.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:09 smithi064 ceph-mon[93757]: osdmap e59: 6 total, 6 up, 6 in 2024-02-16T23:01:09.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T23:01:09.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:09 smithi064 ceph-mon[93757]: Health check update: Degraded data redundancy: 34/261 objects degraded (13.027%), 13 pgs degraded (PG_DEGRADED) 2024-02-16T23:01:09.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:09 smithi192 ceph-mon[82652]: failed to encode map e58 with expected crc 2024-02-16T23:01:09.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:09 smithi192 ceph-mon[82652]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:01:09.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:09 smithi192 ceph-mon[82652]: failed to encode map e58 with expected crc 2024-02-16T23:01:09.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:09 smithi192 ceph-mon[82652]: osd.1 [v2:172.21.15.64:6810/1635803238,v1:172.21.15.64:6811/1635803238] boot 2024-02-16T23:01:09.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:09 smithi192 ceph-mon[82652]: osdmap e59: 6 total, 6 up, 6 in 2024-02-16T23:01:09.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T23:01:09.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:09 smithi192 ceph-mon[82652]: Health check update: Degraded data redundancy: 34/261 objects degraded (13.027%), 13 pgs degraded (PG_DEGRADED) 2024-02-16T23:01:10.497 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: pgmap v75: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/261 objects degraded (6.897%) 2024-02-16T23:01:10.497 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.497 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.497 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.497 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.498 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: osdmap e60: 6 total, 6 up, 6 in 2024-02-16T23:01:10.498 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.498 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.498 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.498 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.498 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:10 smithi064 ceph-mon[93757]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: pgmap v75: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/261 objects degraded (6.897%) 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: osdmap e60: 6 total, 6 up, 6 in 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:10.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:10 smithi192 ceph-mon[82652]: failed to encode map e60 with expected crc 2024-02-16T23:01:12.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: pgmap v77: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/261 objects degraded (6.897%) 2024-02-16T23:01:12.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T23:01:12.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:11 smithi192 ceph-mon[82652]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: pgmap v77: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/261 objects degraded (6.897%) 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T23:01:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T23:01:12.295 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:11 smithi064 ceph-mon[93757]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-16T23:01:13.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:12 smithi192 ceph-mon[82652]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/261 objects degraded (6.897%), 7 pgs degraded) 2024-02-16T23:01:13.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:12 smithi064 ceph-mon[93757]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/261 objects degraded (6.897%), 7 pgs degraded) 2024-02-16T23:01:13.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.733+0000 7f0560378700 1 -- 172.21.15.64:0/1001035456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fbe40 msgr2=0x7f05580fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:13.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.733+0000 7f0560378700 1 --2- 172.21.15.64:0/1001035456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fbe40 0x7f05580fc250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f054c004940 tx=0x7f054c009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:13.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.733+0000 7f0560378700 1 -- 172.21.15.64:0/1001035456 shutdown_connections 2024-02-16T23:01:13.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.733+0000 7f0560378700 1 --2- 172.21.15.64:0/1001035456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fcfb0 0x7f05580fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:13.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.733+0000 7f0560378700 1 --2- 172.21.15.64:0/1001035456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fbe40 0x7f05580fc250 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:13.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.733+0000 7f0560378700 1 -- 172.21.15.64:0/1001035456 >> 172.21.15.64:0/1001035456 conn(0x7f05580f74e0 msgr2=0x7f05580f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.733+0000 7f0560378700 1 -- 172.21.15.64:0/1001035456 shutdown_connections 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.733+0000 7f0560378700 1 -- 172.21.15.64:0/1001035456 wait complete. 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f0560378700 1 Processor -- start 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f0560378700 1 -- start start 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f0560378700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fbe40 0x7f055810fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f0560378700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fcfb0 0x7f0558110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f0560378700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0558110680 con 0x7f05580fcfb0 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f0560378700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0558110f20 con 0x7f05580fbe40 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f055d913700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fcfb0 0x7f0558110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f055d913700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fcfb0 0x7f0558110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51850/0 (socket says 172.21.15.64:51850) 2024-02-16T23:01:13.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f055d913700 1 -- 172.21.15.64:0/3085209924 learned_addr learned my addr 172.21.15.64:0/3085209924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.734+0000 7f055e114700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fbe40 0x7f055810fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f055d913700 1 -- 172.21.15.64:0/3085209924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fbe40 msgr2=0x7f055810fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f055d913700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fbe40 0x7f055810fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f055d913700 1 -- 172.21.15.64:0/3085209924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c005040 con 0x7f05580fcfb0 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f055e114700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fbe40 0x7f055810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f055d913700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fcfb0 0x7f0558110080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0554002a50 tx=0x7f05540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f054affd700 1 -- 172.21.15.64:0/3085209924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0554007cc0 con 0x7f05580fcfb0 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f054affd700 1 -- 172.21.15.64:0/3085209924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f055400f040 con 0x7f05580fcfb0 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0558111200 con 0x7f05580fcfb0 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f054affd700 1 -- 172.21.15.64:0/3085209924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055400da30 con 0x7f05580fcfb0 2024-02-16T23:01:13.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.735+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0558111740 con 0x7f05580fcfb0 2024-02-16T23:01:13.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.736+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055804bed0 con 0x7f05580fcfb0 2024-02-16T23:01:13.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.737+0000 7f054affd700 1 -- 172.21.15.64:0/3085209924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f055400db90 con 0x7f05580fcfb0 2024-02-16T23:01:13.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.737+0000 7f054affd700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f054406d7f0 0x7f054406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:13.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.737+0000 7f055e114700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f054406d7f0 0x7f054406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:13.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.737+0000 7f054affd700 1 -- 172.21.15.64:0/3085209924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f055408cc20 con 0x7f05580fcfb0 2024-02-16T23:01:13.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.738+0000 7f055e114700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f054406d7f0 0x7f054406fca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f054c00b3b0 tx=0x7f054c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:13.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.742+0000 7f054affd700 1 -- 172.21.15.64:0/3085209924 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0554059ff0 con 0x7f05580fcfb0 2024-02-16T23:01:13.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.964+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0558111b90 con 0x7f054406d7f0 2024-02-16T23:01:13.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.966+0000 7f054affd700 1 -- 172.21.15.64:0/3085209924 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0558111b90 con 0x7f054406d7f0 2024-02-16T23:01:13.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.967+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f054406d7f0 msgr2=0x7f054406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:13.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.967+0000 7f0560378700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f054406d7f0 0x7f054406fca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f054c00b3b0 tx=0x7f054c009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:13.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.967+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fcfb0 msgr2=0x7f0558110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:13.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.967+0000 7f0560378700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fcfb0 0x7f0558110080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0554002a50 tx=0x7f05540046b0 comp rx=0 tx=0).stop 2024-02-16T23:01:13.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.968+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 shutdown_connections 2024-02-16T23:01:13.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.968+0000 7f0560378700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f054406d7f0 0x7f054406fca0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:13.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.968+0000 7f0560378700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05580fbe40 0x7f055810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:13.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.968+0000 7f0560378700 1 --2- 172.21.15.64:0/3085209924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05580fcfb0 0x7f0558110080 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:13.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.968+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 >> 172.21.15.64:0/3085209924 conn(0x7f05580f74e0 msgr2=0x7f05580656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:13.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.968+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 shutdown_connections 2024-02-16T23:01:13.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:13.968+0000 7f0560378700 1 -- 172.21.15.64:0/3085209924 wait complete. 2024-02-16T23:01:13.978 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:01:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 -- 172.21.15.64:0/2199329933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc0fda80 msgr2=0x7f3ebc0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 --2- 172.21.15.64:0/2199329933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc0fda80 0x7f3ebc0ffe60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4004940 tx=0x7f3eb4009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 -- 172.21.15.64:0/2199329933 shutdown_connections 2024-02-16T23:01:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 --2- 172.21.15.64:0/2199329933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc1003a0 0x7f3ebc102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 --2- 172.21.15.64:0/2199329933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc0fda80 0x7f3ebc0ffe60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 -- 172.21.15.64:0/2199329933 >> 172.21.15.64:0/2199329933 conn(0x7f3ebc0f74e0 msgr2=0x7f3ebc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 -- 172.21.15.64:0/2199329933 shutdown_connections 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 -- 172.21.15.64:0/2199329933 wait complete. 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 Processor -- start 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.074+0000 7f3ec1a22700 1 -- start start 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3ec1a22700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0fda80 0x7f3ebc10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3ec1a22700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1003a0 0x7f3ebc10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3ec1a22700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc10e470 con 0x7f3ebc1003a0 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3ec1a22700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc10ed10 con 0x7f3ebc0fda80 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eba7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1003a0 0x7f3ebc10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eba7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1003a0 0x7f3ebc10de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51882/0 (socket says 172.21.15.64:51882) 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eba7fc700 1 -- 172.21.15.64:0/3373251566 learned_addr learned my addr 172.21.15.64:0/3373251566 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3ebaffd700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0fda80 0x7f3ebc10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eba7fc700 1 -- 172.21.15.64:0/3373251566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0fda80 msgr2=0x7f3ebc10d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eba7fc700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0fda80 0x7f3ebc10d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eba7fc700 1 -- 172.21.15.64:0/3373251566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb4005040 con 0x7f3ebc1003a0 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eba7fc700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1003a0 0x7f3ebc10de70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4002a50 tx=0x7f3ea40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:14.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eb37fe700 1 -- 172.21.15.64:0/3373251566 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea4007cc0 con 0x7f3ebc1003a0 2024-02-16T23:01:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eb37fe700 1 -- 172.21.15.64:0/3373251566 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ea400f040 con 0x7f3ebc1003a0 2024-02-16T23:01:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.075+0000 7f3eb37fe700 1 -- 172.21.15.64:0/3373251566 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea4009640 con 0x7f3ebc1003a0 2024-02-16T23:01:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.076+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc10eff0 con 0x7f3ebc1003a0 2024-02-16T23:01:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.076+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc10f4e0 con 0x7f3ebc1003a0 2024-02-16T23:01:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.077+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc04bed0 con 0x7f3ebc1003a0 2024-02-16T23:01:14.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.077+0000 7f3eb37fe700 1 -- 172.21.15.64:0/3373251566 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3ea40097a0 con 0x7f3ebc1003a0 2024-02-16T23:01:14.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.078+0000 7f3eb37fe700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3ea806d7e0 0x7f3ea806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.078+0000 7f3eb37fe700 1 -- 172.21.15.64:0/3373251566 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3ea408cc40 con 0x7f3ebc1003a0 2024-02-16T23:01:14.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.078+0000 7f3ebaffd700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3ea806d7e0 0x7f3ea806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.078+0000 7f3ebaffd700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3ea806d7e0 0x7f3ea806fc90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4004f90 tx=0x7f3eb4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:14.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.082+0000 7f3eb37fe700 1 -- 172.21.15.64:0/3373251566 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ea405a010 con 0x7f3ebc1003a0 2024-02-16T23:01:14.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:13 smithi192 ceph-mon[82652]: pgmap v78: 65 pgs: 20 peering, 45 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:14.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.303+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ebc10f980 con 0x7f3ea806d7e0 2024-02-16T23:01:14.303 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:13 smithi064 ceph-mon[93757]: pgmap v78: 65 pgs: 20 peering, 45 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:14.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.304+0000 7f3eb37fe700 1 -- 172.21.15.64:0/3373251566 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3ebc10f980 con 0x7f3ea806d7e0 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3ea806d7e0 msgr2=0x7f3ea806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3ea806d7e0 0x7f3ea806fc90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4004f90 tx=0x7f3eb4009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1003a0 msgr2=0x7f3ebc10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1003a0 0x7f3ebc10de70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4002a50 tx=0x7f3ea40046b0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 shutdown_connections 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3ea806d7e0 0x7f3ea806fc90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0fda80 0x7f3ebc10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 --2- 172.21.15.64:0/3373251566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1003a0 0x7f3ebc10de70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 >> 172.21.15.64:0/3373251566 conn(0x7f3ebc0f74e0 msgr2=0x7f3ebc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 shutdown_connections 2024-02-16T23:01:14.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.306+0000 7f3ec1a22700 1 -- 172.21.15.64:0/3373251566 wait complete. 2024-02-16T23:01:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.409+0000 7f23ce8f7700 1 -- 172.21.15.64:0/843996102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 msgr2=0x7f23c8102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.409+0000 7f23ce8f7700 1 --2- 172.21.15.64:0/843996102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 0x7f23c8102520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f23b8004940 tx=0x7f23b8009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.409+0000 7f23ce8f7700 1 -- 172.21.15.64:0/843996102 shutdown_connections 2024-02-16T23:01:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.409+0000 7f23ce8f7700 1 --2- 172.21.15.64:0/843996102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 0x7f23c8102520 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.409+0000 7f23ce8f7700 1 --2- 172.21.15.64:0/843996102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c8066c70 0x7f23c80ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.409+0000 7f23ce8f7700 1 -- 172.21.15.64:0/843996102 >> 172.21.15.64:0/843996102 conn(0x7f23c80f7520 msgr2=0x7f23c80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.409+0000 7f23ce8f7700 1 -- 172.21.15.64:0/843996102 shutdown_connections 2024-02-16T23:01:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.409+0000 7f23ce8f7700 1 -- 172.21.15.64:0/843996102 wait complete. 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23ce8f7700 1 Processor -- start 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23ce8f7700 1 -- start start 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23ce8f7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c8066c70 0x7f23c80fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23ce8f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 0x7f23c80fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23ce8f7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c80fbff0 con 0x7f23c81000c0 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23ce8f7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c80fc160 con 0x7f23c8066c70 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 0x7f23c80fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 0x7f23c80fb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51906/0 (socket says 172.21.15.64:51906) 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23c77fe700 1 -- 172.21.15.64:0/3779985564 learned_addr learned my addr 172.21.15.64:0/3779985564 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23c7fff700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c8066c70 0x7f23c80fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23c77fe700 1 -- 172.21.15.64:0/3779985564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c8066c70 msgr2=0x7f23c80fd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23c77fe700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c8066c70 0x7f23c80fd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23c77fe700 1 -- 172.21.15.64:0/3779985564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8005040 con 0x7f23c81000c0 2024-02-16T23:01:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.410+0000 7f23c7fff700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c8066c70 0x7f23c80fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:01:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.411+0000 7f23c77fe700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 0x7f23c80fb9f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23b8000c00 tx=0x7f23b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.411+0000 7f23c4ff9700 1 -- 172.21.15.64:0/3779985564 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b800e040 con 0x7f23c81000c0 2024-02-16T23:01:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.411+0000 7f23c4ff9700 1 -- 172.21.15.64:0/3779985564 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23b800bcc0 con 0x7f23c81000c0 2024-02-16T23:01:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.411+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c80fc3e0 con 0x7f23c81000c0 2024-02-16T23:01:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.411+0000 7f23c4ff9700 1 -- 172.21.15.64:0/3779985564 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b8013ea0 con 0x7f23c81000c0 2024-02-16T23:01:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.411+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c8115960 con 0x7f23c81000c0 2024-02-16T23:01:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.412+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c8107a50 con 0x7f23c81000c0 2024-02-16T23:01:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.412+0000 7f23c4ff9700 1 -- 172.21.15.64:0/3779985564 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f23b8013420 con 0x7f23c81000c0 2024-02-16T23:01:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.413+0000 7f23c4ff9700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f23b006d7f0 0x7f23b006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.413+0000 7f23c4ff9700 1 -- 172.21.15.64:0/3779985564 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f23b808df20 con 0x7f23c81000c0 2024-02-16T23:01:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.413+0000 7f23c7fff700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f23b006d7f0 0x7f23b006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.413+0000 7f23c7fff700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f23b006d7f0 0x7f23b006fca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23bc007fd0 tx=0x7f23bc009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.418+0000 7f23c4ff9700 1 -- 172.21.15.64:0/3779985564 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23b80c1d20 con 0x7f23c81000c0 2024-02-16T23:01:14.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.629+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f23c805e3d0 con 0x7f23b006d7f0 2024-02-16T23:01:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.640+0000 7f23c4ff9700 1 -- 172.21.15.64:0/3779985564 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f23c805e3d0 con 0x7f23b006d7f0 2024-02-16T23:01:14.640 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:01:14.640 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (4m) 8s ago 12m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:01:14.640 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (76s) 8s ago 13m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:01:14.640 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (71s) 67s ago 12m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (4m) 8s ago 12m 84.3M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (7m) 8s ago 7m 17.7M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (7m) 8s ago 7m 153M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (7m) 67s ago 7m 16.5M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (7m) 67s ago 7m 16.2M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (3m) 8s ago 14m 498M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (2m) 67s ago 11m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (2m) 8s ago 14m 44.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (117s) 67s ago 11m 35.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (4m) 8s ago 12m 15.6M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (3m) 67s ago 11m 15.8M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (49s) 8s ago 11m 95.7M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (14s) 8s ago 10m 12.6M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (10m) 8s ago 10m 177M 1557M 17.2.7-513-gb88fae66 870a0944ccb8 a3b3c9bcf868 2024-02-16T23:01:14.641 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (9m) 67s ago 9m 234M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T23:01:14.642 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (9m) 67s ago 9m 210M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T23:01:14.642 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (8m) 67s ago 8m 186M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T23:01:14.642 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (2m) 8s ago 12m 59.3M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f23b006d7f0 msgr2=0x7f23b006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f23b006d7f0 0x7f23b006fca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23bc007fd0 tx=0x7f23bc009040 comp rx=0 tx=0).stop 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 msgr2=0x7f23c80fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 0x7f23c80fb9f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23b8000c00 tx=0x7f23b8009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 shutdown_connections 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f23b006d7f0 0x7f23b006fca0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c8066c70 0x7f23c80fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 --2- 172.21.15.64:0/3779985564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c81000c0 0x7f23c80fb9f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 >> 172.21.15.64:0/3779985564 conn(0x7f23c80f7520 msgr2=0x7f23c80f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 shutdown_connections 2024-02-16T23:01:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.642+0000 7f23ce8f7700 1 -- 172.21.15.64:0/3779985564 wait complete. 2024-02-16T23:01:14.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1691673737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fe0e0 msgr2=0x7fcc780fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 --2- 172.21.15.64:0/1691673737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fe0e0 0x7fcc780fe550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcc60004940 tx=0x7fcc60009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:14.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1691673737 shutdown_connections 2024-02-16T23:01:14.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 --2- 172.21.15.64:0/1691673737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fe0e0 0x7fcc780fe550 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 --2- 172.21.15.64:0/1691673737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc780fcef0 0x7fcc780fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1691673737 >> 172.21.15.64:0/1691673737 conn(0x7fcc780f8620 msgr2=0x7fcc780faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1691673737 shutdown_connections 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1691673737 wait complete. 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.744+0000 7fcc7cfe9700 1 Processor -- start 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7cfe9700 1 -- start start 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7cfe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fcef0 0x7fcc7810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7cfe9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc780fe0e0 0x7fcc7810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7cfe9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7810e4b0 con 0x7fcc780fcef0 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7cfe9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7810ed50 con 0x7fcc780fe0e0 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fcef0 0x7fcc7810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fcef0 0x7fcc7810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51924/0 (socket says 172.21.15.64:51924) 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7659c700 1 -- 172.21.15.64:0/1108716594 learned_addr learned my addr 172.21.15.64:0/1108716594 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc75d9b700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc780fe0e0 0x7fcc7810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7659c700 1 -- 172.21.15.64:0/1108716594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc780fe0e0 msgr2=0x7fcc7810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7659c700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc780fe0e0 0x7fcc7810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:14.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7659c700 1 -- 172.21.15.64:0/1108716594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc60005040 con 0x7fcc780fcef0 2024-02-16T23:01:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc7659c700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fcef0 0x7fcc7810d970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcc68001ce0 tx=0x7fcc68009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.745+0000 7fcc6effd700 1 -- 172.21.15.64:0/1108716594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc68007b30 con 0x7fcc780fcef0 2024-02-16T23:01:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.746+0000 7fcc6effd700 1 -- 172.21.15.64:0/1108716594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc6800f040 con 0x7fcc780fcef0 2024-02-16T23:01:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.746+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7810f030 con 0x7fcc780fcef0 2024-02-16T23:01:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.746+0000 7fcc6effd700 1 -- 172.21.15.64:0/1108716594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc680096d0 con 0x7fcc780fcef0 2024-02-16T23:01:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.746+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7810f570 con 0x7fcc780fcef0 2024-02-16T23:01:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.746+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7804bed0 con 0x7fcc780fcef0 2024-02-16T23:01:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.747+0000 7fcc6effd700 1 -- 172.21.15.64:0/1108716594 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcc68007c90 con 0x7fcc780fcef0 2024-02-16T23:01:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.747+0000 7fcc6effd700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc6406d7e0 0x7fcc6406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.748+0000 7fcc6effd700 1 -- 172.21.15.64:0/1108716594 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcc6808c820 con 0x7fcc780fcef0 2024-02-16T23:01:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.748+0000 7fcc75d9b700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc6406d7e0 0x7fcc6406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.748+0000 7fcc75d9b700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc6406d7e0 0x7fcc6406fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000b3b0 tx=0x7fcc60009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:14.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:14.752+0000 7fcc6effd700 1 -- 172.21.15.64:0/1108716594 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc68059ca0 con 0x7fcc780fcef0 2024-02-16T23:01:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.045+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcc7810f990 con 0x7fcc780fcef0 2024-02-16T23:01:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.045+0000 7fcc6effd700 1 -- 172.21.15.64:0/1108716594 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fcc68059830 con 0x7fcc780fcef0 2024-02-16T23:01:15.046 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:01:15.046 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:01:15.046 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:01:15.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:01:15.046 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4, 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 8, 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:01:15.047 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:01:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc6406d7e0 msgr2=0x7fcc6406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc6406d7e0 0x7fcc6406fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000b3b0 tx=0x7fcc60009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fcef0 msgr2=0x7fcc7810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fcef0 0x7fcc7810d970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcc68001ce0 tx=0x7fcc68009e60 comp rx=0 tx=0).stop 2024-02-16T23:01:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 shutdown_connections 2024-02-16T23:01:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc6406d7e0 0x7fcc6406fc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc780fcef0 0x7fcc7810d970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 --2- 172.21.15.64:0/1108716594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc780fe0e0 0x7fcc7810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 >> 172.21.15.64:0/1108716594 conn(0x7fcc780f8620 msgr2=0x7fcc781013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 shutdown_connections 2024-02-16T23:01:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.048+0000 7fcc7cfe9700 1 -- 172.21.15.64:0/1108716594 wait complete. 2024-02-16T23:01:15.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.151+0000 7fe0f7dba700 1 -- 172.21.15.64:0/1108612042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fcfc0 msgr2=0x7fe0f00fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.151+0000 7fe0f7dba700 1 --2- 172.21.15.64:0/1108612042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fcfc0 0x7fe0f00fd430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec004940 tx=0x7fe0ec009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:15.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.151+0000 7fe0f7dba700 1 -- 172.21.15.64:0/1108612042 shutdown_connections 2024-02-16T23:01:15.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.151+0000 7fe0f7dba700 1 --2- 172.21.15.64:0/1108612042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fcfc0 0x7fe0f00fd430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.151+0000 7fe0f7dba700 1 --2- 172.21.15.64:0/1108612042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f00fbe50 0x7fe0f00fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.151+0000 7fe0f7dba700 1 -- 172.21.15.64:0/1108612042 >> 172.21.15.64:0/1108612042 conn(0x7fe0f00f7510 msgr2=0x7fe0f00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:15.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.151+0000 7fe0f7dba700 1 -- 172.21.15.64:0/1108612042 shutdown_connections 2024-02-16T23:01:15.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.151+0000 7fe0f7dba700 1 -- 172.21.15.64:0/1108612042 wait complete. 2024-02-16T23:01:15.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f7dba700 1 Processor -- start 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f7dba700 1 -- start start 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f7dba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fbe50 0x7fe0f0110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f7dba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f00fcfc0 0x7fe0f0112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f7dba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f0066d70 con 0x7fe0f00fbe50 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f7dba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f0066eb0 con 0x7fe0f00fcfc0 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f5b56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fbe50 0x7fe0f0110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f5b56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fbe50 0x7fe0f0110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51932/0 (socket says 172.21.15.64:51932) 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f5b56700 1 -- 172.21.15.64:0/825183068 learned_addr learned my addr 172.21.15.64:0/825183068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f5355700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f00fcfc0 0x7fe0f0112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f5b56700 1 -- 172.21.15.64:0/825183068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f00fcfc0 msgr2=0x7fe0f0112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f5b56700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f00fcfc0 0x7fe0f0112b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f5b56700 1 -- 172.21.15.64:0/825183068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec005040 con 0x7fe0f00fbe50 2024-02-16T23:01:15.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.152+0000 7fe0f5b56700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fbe50 0x7fe0f0110640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400c8c0 tx=0x7fe0e40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:15.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.153+0000 7fe0e27fc700 1 -- 172.21.15.64:0/825183068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e400a980 con 0x7fe0f00fbe50 2024-02-16T23:01:15.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.153+0000 7fe0e27fc700 1 -- 172.21.15.64:0/825183068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0e400ce60 con 0x7fe0f00fbe50 2024-02-16T23:01:15.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.153+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f01131f0 con 0x7fe0f00fbe50 2024-02-16T23:01:15.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.153+0000 7fe0e27fc700 1 -- 172.21.15.64:0/825183068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e40115e0 con 0x7fe0f00fbe50 2024-02-16T23:01:15.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.153+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f01136e0 con 0x7fe0f00fbe50 2024-02-16T23:01:15.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.154+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f004bed0 con 0x7fe0f00fbe50 2024-02-16T23:01:15.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.154+0000 7fe0e27fc700 1 -- 172.21.15.64:0/825183068 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe0e4012400 con 0x7fe0f00fbe50 2024-02-16T23:01:15.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.155+0000 7fe0e27fc700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe0dc06d7e0 0x7fe0dc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.155+0000 7fe0e27fc700 1 -- 172.21.15.64:0/825183068 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe0e408cf60 con 0x7fe0f00fbe50 2024-02-16T23:01:15.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.155+0000 7fe0f5355700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe0dc06d7e0 0x7fe0dc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.155+0000 7fe0f5355700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe0dc06d7e0 0x7fe0dc06fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec004f90 tx=0x7fe0ec009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.159+0000 7fe0e27fc700 1 -- 172.21.15.64:0/825183068 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0e40c0d20 con 0x7fe0f00fbe50 2024-02-16T23:01:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:15 smithi192 ceph-mon[82652]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:15 smithi192 ceph-mon[82652]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:15.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:15 smithi064 ceph-mon[93757]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:15.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:15 smithi064 ceph-mon[93757]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.409+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe0f0113c80 con 0x7fe0f00fbe50 2024-02-16T23:01:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.409+0000 7fe0e27fc700 1 -- 172.21.15.64:0/825183068 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7fe0e4056880 con 0x7fe0f00fbe50 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:01:15.410 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:01:15.411 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:01:15.412 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:01:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe0dc06d7e0 msgr2=0x7fe0dc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe0dc06d7e0 0x7fe0dc06fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec004f90 tx=0x7fe0ec009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fbe50 msgr2=0x7fe0f0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fbe50 0x7fe0f0110640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400c8c0 tx=0x7fe0e40044c0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 shutdown_connections 2024-02-16T23:01:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe0dc06d7e0 0x7fe0dc06fc90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f00fbe50 0x7fe0f0110640 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 --2- 172.21.15.64:0/825183068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f00fcfc0 0x7fe0f0112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 >> 172.21.15.64:0/825183068 conn(0x7fe0f00f7510 msgr2=0x7fe0f0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 shutdown_connections 2024-02-16T23:01:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.411+0000 7fe0f7dba700 1 -- 172.21.15.64:0/825183068 wait complete. 2024-02-16T23:01:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T23:01:15.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.514+0000 7f227b04d700 1 -- 172.21.15.64:0/1703891006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 msgr2=0x7f22740fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.514+0000 7f227b04d700 1 --2- 172.21.15.64:0/1703891006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 0x7f22740fe700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2260004940 tx=0x7f2260009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:15.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.514+0000 7f227b04d700 1 -- 172.21.15.64:0/1703891006 shutdown_connections 2024-02-16T23:01:15.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.514+0000 7f227b04d700 1 --2- 172.21.15.64:0/1703891006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 0x7f22740fe700 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.514+0000 7f227b04d700 1 --2- 172.21.15.64:0/1703891006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22740fb220 0x7f22740fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.514+0000 7f227b04d700 1 -- 172.21.15.64:0/1703891006 >> 172.21.15.64:0/1703891006 conn(0x7f22740f6ca0 msgr2=0x7f22740f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:15.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.514+0000 7f227b04d700 1 -- 172.21.15.64:0/1703891006 shutdown_connections 2024-02-16T23:01:15.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.514+0000 7f227b04d700 1 -- 172.21.15.64:0/1703891006 wait complete. 2024-02-16T23:01:15.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f227b04d700 1 Processor -- start 2024-02-16T23:01:15.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f227b04d700 1 -- start start 2024-02-16T23:01:15.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f227b04d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22740fb220 0x7f227410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f227b04d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 0x7f227410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f227b04d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227410e4e0 con 0x7f22740fe2b0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f227b04d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227410ed80 con 0x7f22740fb220 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f2273fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 0x7f227410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f2273fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 0x7f227410dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51944/0 (socket says 172.21.15.64:51944) 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f2273fff700 1 -- 172.21.15.64:0/2662384650 learned_addr learned my addr 172.21.15.64:0/2662384650 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.515+0000 7f2278de9700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22740fb220 0x7f227410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f2273fff700 1 -- 172.21.15.64:0/2662384650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22740fb220 msgr2=0x7f227410d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f2273fff700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22740fb220 0x7f227410d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f2273fff700 1 -- 172.21.15.64:0/2662384650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2260005040 con 0x7f22740fe2b0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f2273fff700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 0x7f227410dee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2260009180 tx=0x7f2260009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f22717fa700 1 -- 172.21.15.64:0/2662384650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f226000e040 con 0x7f22740fe2b0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f22717fa700 1 -- 172.21.15.64:0/2662384650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2260009410 con 0x7f22740fe2b0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227410f000 con 0x7f22740fe2b0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f22717fa700 1 -- 172.21.15.64:0/2662384650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f226001c630 con 0x7f22740fe2b0 2024-02-16T23:01:15.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.516+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227410f540 con 0x7f22740fe2b0 2024-02-16T23:01:15.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.517+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227404bed0 con 0x7f22740fe2b0 2024-02-16T23:01:15.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.517+0000 7f22717fa700 1 -- 172.21.15.64:0/2662384650 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2260013420 con 0x7f22740fe2b0 2024-02-16T23:01:15.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.518+0000 7f22717fa700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f226406d7e0 0x7f226406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.518+0000 7f22717fa700 1 -- 172.21.15.64:0/2662384650 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f226008e0f0 con 0x7f22740fe2b0 2024-02-16T23:01:15.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.518+0000 7f2278de9700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f226406d7e0 0x7f226406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.518+0000 7f2278de9700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f226406d7e0 0x7f226406fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2268009fd0 tx=0x7f226800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:15.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.523+0000 7f22717fa700 1 -- 172.21.15.64:0/2662384650 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f226005b5e0 con 0x7f22740fe2b0 2024-02-16T23:01:15.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.745+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f227410f9e0 con 0x7f226406d7e0 2024-02-16T23:01:15.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.746+0000 7f22717fa700 1 -- 172.21.15.64:0/2662384650 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f227410f9e0 con 0x7f226406d7e0 2024-02-16T23:01:15.746 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:01:15.746 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:01:15.746 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "8/21 daemons upgraded", 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:01:15.747 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:01:15.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f226406d7e0 msgr2=0x7f226406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f226406d7e0 0x7f226406fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2268009fd0 tx=0x7f226800b040 comp rx=0 tx=0).stop 2024-02-16T23:01:15.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 msgr2=0x7f227410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 0x7f227410dee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2260009180 tx=0x7f2260009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 shutdown_connections 2024-02-16T23:01:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f226406d7e0 0x7f226406fc90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22740fb220 0x7f227410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 --2- 172.21.15.64:0/2662384650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22740fe2b0 0x7f227410dee0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 >> 172.21.15.64:0/2662384650 conn(0x7f22740f6ca0 msgr2=0x7f22741014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 shutdown_connections 2024-02-16T23:01:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.748+0000 7f227b04d700 1 -- 172.21.15.64:0/2662384650 wait complete. 2024-02-16T23:01:15.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 -- 172.21.15.64:0/2800768789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fe0e0 msgr2=0x7f790c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 --2- 172.21.15.64:0/2800768789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fe0e0 0x7f790c0fe550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f78f4004940 tx=0x7f78f4009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:15.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 -- 172.21.15.64:0/2800768789 shutdown_connections 2024-02-16T23:01:15.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 --2- 172.21.15.64:0/2800768789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fe0e0 0x7f790c0fe550 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 --2- 172.21.15.64:0/2800768789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f790c0fcef0 0x7f790c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 -- 172.21.15.64:0/2800768789 >> 172.21.15.64:0/2800768789 conn(0x7f790c0f8620 msgr2=0x7f790c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 -- 172.21.15.64:0/2800768789 shutdown_connections 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 -- 172.21.15.64:0/2800768789 wait complete. 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.851+0000 7f791261c700 1 Processor -- start 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f791261c700 1 -- start start 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f791261c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fcef0 0x7f790c10fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f791261c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f790c0fe0e0 0x7f790c1100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f791261c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c1106a0 con 0x7f790c0fcef0 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f791261c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c110f40 con 0x7f790c0fe0e0 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f790bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fcef0 0x7f790c10fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f790bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fcef0 0x7f790c10fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51958/0 (socket says 172.21.15.64:51958) 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f790bfff700 1 -- 172.21.15.64:0/4068363731 learned_addr learned my addr 172.21.15.64:0/4068363731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f7903fff700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f790c0fe0e0 0x7f790c1100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f790bfff700 1 -- 172.21.15.64:0/4068363731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f790c0fe0e0 msgr2=0x7f790c1100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:15.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f790bfff700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f790c0fe0e0 0x7f790c1100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:15.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f790bfff700 1 -- 172.21.15.64:0/4068363731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4005040 con 0x7f790c0fcef0 2024-02-16T23:01:15.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f7903fff700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f790c0fe0e0 0x7f790c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:01:15.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f790bfff700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fcef0 0x7f790c10fb60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78fc002a80 tx=0x7f78fc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:15.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f79097fa700 1 -- 172.21.15.64:0/4068363731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78fc007cc0 con 0x7f790c0fcef0 2024-02-16T23:01:15.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f79097fa700 1 -- 172.21.15.64:0/4068363731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78fc00f040 con 0x7f790c0fcef0 2024-02-16T23:01:15.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c111220 con 0x7f790c0fcef0 2024-02-16T23:01:15.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.852+0000 7f79097fa700 1 -- 172.21.15.64:0/4068363731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78fc00dad0 con 0x7f790c0fcef0 2024-02-16T23:01:15.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.853+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c111760 con 0x7f790c0fcef0 2024-02-16T23:01:15.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.853+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c102210 con 0x7f790c0fcef0 2024-02-16T23:01:15.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.854+0000 7f79097fa700 1 -- 172.21.15.64:0/4068363731 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f78fc00dc30 con 0x7f790c0fcef0 2024-02-16T23:01:15.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.854+0000 7f79097fa700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f78f806d7e0 0x7f78f806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:15.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.855+0000 7f7903fff700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f78f806d7e0 0x7f78f806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:15.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.855+0000 7f79097fa700 1 -- 172.21.15.64:0/4068363731 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f78fc08cc40 con 0x7f790c0fcef0 2024-02-16T23:01:15.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.855+0000 7f7903fff700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f78f806d7e0 0x7f78f806fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78f4004560 tx=0x7f78f4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:15.859+0000 7f79097fa700 1 -- 172.21.15.64:0/4068363731 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78fc05a010 con 0x7f790c0fcef0 2024-02-16T23:01:16.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.146+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f790c04bed0 con 0x7f790c0fcef0 2024-02-16T23:01:16.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.147+0000 7f79097fa700 1 -- 172.21.15.64:0/4068363731 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f78fc059ba0 con 0x7f790c0fcef0 2024-02-16T23:01:16.147 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:01:16.147 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:01:16.147 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:01:16.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f78f806d7e0 msgr2=0x7f78f806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:16.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f78f806d7e0 0x7f78f806fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78f4004560 tx=0x7f78f4009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:16.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fcef0 msgr2=0x7f790c10fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:16.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fcef0 0x7f790c10fb60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78fc002a80 tx=0x7f78fc004600 comp rx=0 tx=0).stop 2024-02-16T23:01:16.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 shutdown_connections 2024-02-16T23:01:16.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f78f806d7e0 0x7f78f806fc90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:16.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c0fcef0 0x7f790c10fb60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:16.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 --2- 172.21.15.64:0/4068363731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f790c0fe0e0 0x7f790c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:16.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 >> 172.21.15.64:0/4068363731 conn(0x7f790c0f8620 msgr2=0x7f790c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:16.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 shutdown_connections 2024-02-16T23:01:16.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:16.149+0000 7f791261c700 1 -- 172.21.15.64:0/4068363731 wait complete. 2024-02-16T23:01:16.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:16 smithi192 ceph-mon[82652]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:16.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:16 smithi192 ceph-mon[82652]: pgmap v79: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:16.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:16 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1108716594' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:16.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:16 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/825183068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:01:16.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:16 smithi064 ceph-mon[93757]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:16.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:16 smithi064 ceph-mon[93757]: pgmap v79: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:16.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:16 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1108716594' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:16.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:16 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/825183068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:01:17.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:17 smithi192 ceph-mon[82652]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:17.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:17 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/4068363731' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:01:17.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:17 smithi064 ceph-mon[93757]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:17.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:17 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/4068363731' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:01:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:18 smithi192 ceph-mon[82652]: pgmap v80: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:01:18.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:18 smithi064 ceph-mon[93757]: pgmap v80: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:01:20.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:20 smithi192 ceph-mon[82652]: pgmap v81: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:20.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:20 smithi064 ceph-mon[93757]: pgmap v81: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:22.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:22 smithi192 ceph-mon[82652]: pgmap v82: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:22.293 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:22 smithi064 ceph-mon[93757]: pgmap v82: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:24.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:23 smithi192 ceph-mon[82652]: pgmap v83: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:24.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:23 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:24.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:23 smithi064 ceph-mon[93757]: pgmap v83: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:24.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:23 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:26.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:25 smithi192 ceph-mon[82652]: pgmap v84: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:26.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:25 smithi064 ceph-mon[93757]: pgmap v84: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:26.970 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:26 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T23:01:26.971 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:26 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T23:01:26.971 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:26 smithi064 ceph-mon[93757]: Upgrade: osd.2 is safe to restart 2024-02-16T23:01:27.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:26 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T23:01:27.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:26 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T23:01:27.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:26 smithi192 ceph-mon[82652]: Upgrade: osd.2 is safe to restart 2024-02-16T23:01:28.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:27 smithi192 ceph-mon[82652]: pgmap v85: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:28.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:27 smithi064 ceph-mon[93757]: pgmap v85: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:28 smithi064 ceph-mon[93757]: Upgrade: Updating osd.2 2024-02-16T23:01:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T23:01:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:01:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:28 smithi064 ceph-mon[93757]: Deploying daemon osd.2 on smithi064 2024-02-16T23:01:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:28 smithi192 ceph-mon[82652]: Upgrade: Updating osd.2 2024-02-16T23:01:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T23:01:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:01:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:28 smithi192 ceph-mon[82652]: Deploying daemon osd.2 on smithi064 2024-02-16T23:01:30.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:29 smithi192 ceph-mon[82652]: pgmap v86: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:30.056 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:29 smithi064 ceph-mon[93757]: pgmap v86: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:31.328 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:31 smithi064 systemd[1]: Stopping Ceph osd.2 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:01:31.618 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:31 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[62667]: 2024-02-16T23:01:31.327+0000 7f94dafb8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:01:31.618 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:31 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[62667]: 2024-02-16T23:01:31.327+0000 7f94dafb8700 -1 osd.2 60 *** Got signal Terminated *** 2024-02-16T23:01:31.618 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:31 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[62667]: 2024-02-16T23:01:31.327+0000 7f94dafb8700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:01:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:31 smithi192 ceph-mon[82652]: pgmap v87: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:31 smithi192 ceph-mon[82652]: osd.2 marked itself down and dead 2024-02-16T23:01:32.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:31 smithi064 ceph-mon[93757]: pgmap v87: 65 pgs: 65 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:32.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:31 smithi064 ceph-mon[93757]: osd.2 marked itself down and dead 2024-02-16T23:01:32.564 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:32 smithi064 podman[113509]: 2024-02-16 23:01:32.287206945 +0000 UTC m=+1.057367855 container died a3b3c9bcf8687042e23379bb4646e2d19004069d9003172261957edd73ae23bb (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=quincy-b88fae6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-02-16T23:01:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:32 smithi064 ceph-mon[93757]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:01:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:32 smithi064 ceph-mon[93757]: osdmap e61: 6 total, 5 up, 6 in 2024-02-16T23:01:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:32 smithi064 ceph-mon[93757]: failed to encode map e61 with expected crc 2024-02-16T23:01:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:32 smithi064 ceph-mon[93757]: failed to encode map e61 with expected crc 2024-02-16T23:01:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:32 smithi064 ceph-mon[93757]: failed to encode map e61 with expected crc 2024-02-16T23:01:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:01:32.868 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:32 smithi064 podman[113509]: 2024-02-16 23:01:32.653418839 +0000 UTC m=+1.423579758 container cleanup a3b3c9bcf8687042e23379bb4646e2d19004069d9003172261957edd73ae23bb (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=quincy-b88fae6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-16T23:01:32.868 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:32 smithi064 bash[113509]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2 2024-02-16T23:01:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:32 smithi192 ceph-mon[82652]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:01:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:32 smithi192 ceph-mon[82652]: osdmap e61: 6 total, 5 up, 6 in 2024-02-16T23:01:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:32 smithi192 ceph-mon[82652]: failed to encode map e61 with expected crc 2024-02-16T23:01:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:32 smithi192 ceph-mon[82652]: failed to encode map e61 with expected crc 2024-02-16T23:01:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:32 smithi192 ceph-mon[82652]: failed to encode map e61 with expected crc 2024-02-16T23:01:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:01:33.287 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:32 smithi064 podman[113526]: 2024-02-16 23:01:32.929886008 +0000 UTC m=+0.642730482 container remove a3b3c9bcf8687042e23379bb4646e2d19004069d9003172261957edd73ae23bb (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=quincy-b88fae6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:01:33.618 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:33 smithi064 podman[113620]: 2024-02-16 23:01:33.190792938 +0000 UTC m=+0.018706421 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:01:33.618 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:33 smithi064 podman[113620]: 2024-02-16 23:01:33.29770868 +0000 UTC m=+0.125622164 container create f79a46e4de11cba417d9bde5377db398761d805ef8bbed242c22834007f43d33 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-16T23:01:33.618 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:33 smithi064 podman[113620]: 2024-02-16 23:01:33.525513816 +0000 UTC m=+0.353427303 container init f79a46e4de11cba417d9bde5377db398761d805ef8bbed242c22834007f43d33 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:01:33.618 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:33 smithi064 podman[113620]: 2024-02-16 23:01:33.528771254 +0000 UTC m=+0.356684739 container start f79a46e4de11cba417d9bde5377db398761d805ef8bbed242c22834007f43d33 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:01:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: pgmap v89: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: osdmap e62: 6 total, 5 up, 6 in 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:33 smithi192 ceph-mon[82652]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: pgmap v89: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: osdmap e62: 6 total, 5 up, 6 in 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:33 smithi064 ceph-mon[93757]: failed to encode map e62 with expected crc 2024-02-16T23:01:34.119 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:33 smithi064 podman[113620]: 2024-02-16 23:01:33.625441865 +0000 UTC m=+0.453355369 container attach f79a46e4de11cba417d9bde5377db398761d805ef8bbed242c22834007f43d33 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:01:34.119 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:33 smithi064 podman[113620]: 2024-02-16 23:01:33.832505682 +0000 UTC m=+0.660419165 container died f79a46e4de11cba417d9bde5377db398761d805ef8bbed242c22834007f43d33 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True) 2024-02-16T23:01:34.396 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:34 smithi064 podman[113646]: 2024-02-16 23:01:34.379462621 +0000 UTC m=+0.539843276 container remove f79a46e4de11cba417d9bde5377db398761d805ef8bbed242c22834007f43d33 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-16T23:01:34.807 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:34 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-16T23:01:34.808 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:34 smithi064 systemd[1]: Stopped Ceph osd.2 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:01:34.808 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:34 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.2.service: Consumed 46.137s CPU time. 2024-02-16T23:01:34.808 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:34 smithi064 systemd[1]: Starting Ceph osd.2 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:01:35.091 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:34 smithi064 ceph-mon[93757]: Health check failed: Degraded data redundancy: 11/261 objects degraded (4.215%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T23:01:35.091 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:34 smithi064 podman[113749]: 2024-02-16 23:01:34.737059411 +0000 UTC m=+0.017958202 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:01:35.091 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:34 smithi064 podman[113749]: 2024-02-16 23:01:34.847373893 +0000 UTC m=+0.128272677 container create 3d68c6012deb36b6b973a289c78be47dd953e21271784d23f73ede747a7fc17e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-16T23:01:35.091 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 podman[113749]: 2024-02-16 23:01:35.091608849 +0000 UTC m=+0.372507633 container init 3d68c6012deb36b6b973a289c78be47dd953e21271784d23f73ede747a7fc17e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-16T23:01:35.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:34 smithi192 ceph-mon[82652]: Health check failed: Degraded data redundancy: 11/261 objects degraded (4.215%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T23:01:35.368 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 podman[113749]: 2024-02-16 23:01:35.094966093 +0000 UTC m=+0.375864886 container start 3d68c6012deb36b6b973a289c78be47dd953e21271784d23f73ede747a7fc17e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:01:35.368 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 podman[113749]: 2024-02-16 23:01:35.200039319 +0000 UTC m=+0.480938112 container attach 3d68c6012deb36b6b973a289c78be47dd953e21271784d23f73ede747a7fc17e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:01:35.809 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate[113765]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T23:01:35.809 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate[113765]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T23:01:35.809 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 bash[113749]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T23:01:35.809 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 bash[113749]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T23:01:35.809 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 bash[113749]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T23:01:35.809 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 bash[113749]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T23:01:35.809 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 bash[113749]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-16T23:01:35.810 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 bash[113749]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T23:01:35.810 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 bash[113749]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-16T23:01:35.810 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate[113765]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T23:01:35.810 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate[113765]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T23:01:35.810 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate[113765]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-16T23:01:35.810 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate[113765]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T23:01:35.810 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate[113765]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-16T23:01:35.810 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:35 smithi064 podman[113749]: 2024-02-16 23:01:35.569726194 +0000 UTC m=+0.850624978 container died 3d68c6012deb36b6b973a289c78be47dd953e21271784d23f73ede747a7fc17e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T23:01:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:35 smithi064 ceph-mon[93757]: pgmap v91: 65 pgs: 5 active+undersized+degraded, 7 peering, 3 stale+active+clean, 7 active+undersized, 43 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 11/261 objects degraded (4.215%) 2024-02-16T23:01:36.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:35 smithi192 ceph-mon[82652]: pgmap v91: 65 pgs: 5 active+undersized+degraded, 7 peering, 3 stale+active+clean, 7 active+undersized, 43 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 11/261 objects degraded (4.215%) 2024-02-16T23:01:36.465 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:36 smithi064 podman[113894]: 2024-02-16 23:01:36.151916194 +0000 UTC m=+0.577567234 container remove 3d68c6012deb36b6b973a289c78be47dd953e21271784d23f73ede747a7fc17e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-activate, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:01:36.465 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:36 smithi064 podman[113960]: 2024-02-16 23:01:36.266929262 +0000 UTC m=+0.019298543 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:01:36.465 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:36 smithi064 podman[113960]: 2024-02-16 23:01:36.370844731 +0000 UTC m=+0.123214009 container create a73d025bae8ba90de081fa338e4d08e4abd5d73b46132828c84ef86936e8bb46 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-16T23:01:36.868 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:36 smithi064 podman[113960]: 2024-02-16 23:01:36.572920417 +0000 UTC m=+0.325289696 container init a73d025bae8ba90de081fa338e4d08e4abd5d73b46132828c84ef86936e8bb46 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-16T23:01:36.868 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:36 smithi064 podman[113960]: 2024-02-16 23:01:36.575826195 +0000 UTC m=+0.328195482 container start a73d025bae8ba90de081fa338e4d08e4abd5d73b46132828c84ef86936e8bb46 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T23:01:36.868 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:36 smithi064 ceph-osd[113979]: -- 172.21.15.64:0/2618519921 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56135e4e51e0 con 0x56135e4b6c00 2024-02-16T23:01:36.868 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:36 smithi064 bash[113960]: a73d025bae8ba90de081fa338e4d08e4abd5d73b46132828c84ef86936e8bb46 2024-02-16T23:01:36.868 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:36 smithi064 systemd[1]: Started Ceph osd.2 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:01:37.368 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:37 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[113975]: 2024-02-16T23:01:37.116+0000 7f77b6c507c0 -1 Falling back to public interface 2024-02-16T23:01:37.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:37 smithi064 ceph-mon[93757]: pgmap v92: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 20/261 objects degraded (7.663%) 2024-02-16T23:01:37.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:37 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:37.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:37 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:37.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:37 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:01:38.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:37 smithi192 ceph-mon[82652]: pgmap v92: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 20/261 objects degraded (7.663%) 2024-02-16T23:01:38.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:37 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:38.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:37 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:38.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:37 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:01:38.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:38 smithi064 ceph-mon[93757]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-02-16T23:01:39.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:38 smithi192 ceph-mon[82652]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-02-16T23:01:40.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:39 smithi192 ceph-mon[82652]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:40.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:39 smithi064 ceph-mon[93757]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:41.327 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:41 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[113975]: 2024-02-16T23:01:41.028+0000 7f77b6c507c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-02-16T23:01:41.327 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:41 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[113975]: 2024-02-16T23:01:41.326+0000 7f77b6c507c0 -1 osd.2 60 log_to_monitors true 2024-02-16T23:01:42.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:41 smithi192 ceph-mon[82652]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:42.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:41 smithi192 ceph-mon[82652]: from='osd.2 [v2:172.21.15.64:6818/530939834,v1:172.21.15.64:6819/530939834]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T23:01:42.117 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:01:41 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[113975]: 2024-02-16T23:01:41.815+0000 7f77a80a8700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T23:01:42.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:41 smithi064 ceph-mon[93757]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 251 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:42.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:41 smithi064 ceph-mon[93757]: from='osd.2 [v2:172.21.15.64:6818/530939834,v1:172.21.15.64:6819/530939834]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T23:01:43.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:42 smithi064 ceph-mon[93757]: from='osd.2 [v2:172.21.15.64:6818/530939834,v1:172.21.15.64:6819/530939834]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T23:01:43.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:42 smithi064 ceph-mon[93757]: osdmap e63: 6 total, 5 up, 6 in 2024-02-16T23:01:43.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:42 smithi064 ceph-mon[93757]: from='osd.2 [v2:172.21.15.64:6818/530939834,v1:172.21.15.64:6819/530939834]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T23:01:43.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:42 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:43.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:42 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:43.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:42 smithi192 ceph-mon[82652]: from='osd.2 [v2:172.21.15.64:6818/530939834,v1:172.21.15.64:6819/530939834]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T23:01:43.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:42 smithi192 ceph-mon[82652]: osdmap e63: 6 total, 5 up, 6 in 2024-02-16T23:01:43.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:42 smithi192 ceph-mon[82652]: from='osd.2 [v2:172.21.15.64:6818/530939834,v1:172.21.15.64:6819/530939834]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-02-16T23:01:43.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:42 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:43.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:42 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: osd.2 [v2:172.21.15.64:6818/530939834,v1:172.21.15.64:6819/530939834] boot 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: osdmap e64: 6 total, 6 up, 6 in 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: failed to encode map e63 with expected crc 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: failed to encode map e63 with expected crc 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: failed to encode map e63 with expected crc 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:43 smithi064 ceph-mon[93757]: osdmap e65: 6 total, 6 up, 6 in 2024-02-16T23:01:44.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 34/261 objects degraded (13.027%) 2024-02-16T23:01:44.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: osd.2 [v2:172.21.15.64:6818/530939834,v1:172.21.15.64:6819/530939834] boot 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: osdmap e64: 6 total, 6 up, 6 in 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: failed to encode map e63 with expected crc 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: failed to encode map e63 with expected crc 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: failed to encode map e63 with expected crc 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:44.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:43 smithi192 ceph-mon[82652]: osdmap e65: 6 total, 6 up, 6 in 2024-02-16T23:01:45.102 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:44 smithi064 ceph-mon[93757]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.102 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:44 smithi064 ceph-mon[93757]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.102 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:44 smithi064 ceph-mon[93757]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.102 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:44 smithi064 ceph-mon[93757]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.102 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:44 smithi064 ceph-mon[93757]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.102 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:44 smithi064 ceph-mon[93757]: Health check update: Degraded data redundancy: 34/261 objects degraded (13.027%), 12 pgs degraded (PG_DEGRADED) 2024-02-16T23:01:45.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:44 smithi192 ceph-mon[82652]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:44 smithi192 ceph-mon[82652]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:44 smithi192 ceph-mon[82652]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:44 smithi192 ceph-mon[82652]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:44 smithi192 ceph-mon[82652]: failed to encode map e65 with expected crc 2024-02-16T23:01:45.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:44 smithi192 ceph-mon[82652]: Health check update: Degraded data redundancy: 34/261 objects degraded (13.027%), 12 pgs degraded (PG_DEGRADED) 2024-02-16T23:01:46.085 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:45 smithi064 ceph-mon[93757]: pgmap v99: 65 pgs: 10 peering, 8 active+undersized+degraded, 9 active+undersized, 38 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 24/261 objects degraded (9.195%) 2024-02-16T23:01:46.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.254+0000 7fc0ad808700 1 -- 172.21.15.64:0/1421845511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a80fda80 msgr2=0x7fc0a80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.254+0000 7fc0ad808700 1 --2- 172.21.15.64:0/1421845511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a80fda80 0x7fc0a80ffe60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc098004940 tx=0x7fc098009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:46.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.254+0000 7fc0ad808700 1 -- 172.21.15.64:0/1421845511 shutdown_connections 2024-02-16T23:01:46.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.254+0000 7fc0ad808700 1 --2- 172.21.15.64:0/1421845511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0a81003a0 0x7fc0a8102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.254+0000 7fc0ad808700 1 --2- 172.21.15.64:0/1421845511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a80fda80 0x7fc0a80ffe60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.254+0000 7fc0ad808700 1 -- 172.21.15.64:0/1421845511 >> 172.21.15.64:0/1421845511 conn(0x7fc0a80f7500 msgr2=0x7fc0a80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:46.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.254+0000 7fc0ad808700 1 -- 172.21.15.64:0/1421845511 shutdown_connections 2024-02-16T23:01:46.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.254+0000 7fc0ad808700 1 -- 172.21.15.64:0/1421845511 wait complete. 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0ad808700 1 Processor -- start 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0ad808700 1 -- start start 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0ad808700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0a80fda80 0x7fc0a80fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0ad808700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a81003a0 0x7fc0a80fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0ad808700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80fbf90 con 0x7fc0a81003a0 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0ad808700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80fc100 con 0x7fc0a80fda80 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0a67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a81003a0 0x7fc0a80fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0a67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a81003a0 0x7fc0a80fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49150/0 (socket says 172.21.15.64:49150) 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.255+0000 7fc0a67fc700 1 -- 172.21.15.64:0/3706042754 learned_addr learned my addr 172.21.15.64:0/3706042754 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc0a6ffd700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0a80fda80 0x7fc0a80fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc0a67fc700 1 -- 172.21.15.64:0/3706042754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0a80fda80 msgr2=0x7fc0a80fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc0a67fc700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0a80fda80 0x7fc0a80fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc0a67fc700 1 -- 172.21.15.64:0/3706042754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098005040 con 0x7fc0a81003a0 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc0a6ffd700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0a80fda80 0x7fc0a80fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:01:46.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc0a67fc700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a81003a0 0x7fc0a80fb990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc09c002a80 tx=0x7fc09c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc08f7fe700 1 -- 172.21.15.64:0/3706042754 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc09c007cc0 con 0x7fc0a81003a0 2024-02-16T23:01:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc08f7fe700 1 -- 172.21.15.64:0/3706042754 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc09c00f040 con 0x7fc0a81003a0 2024-02-16T23:01:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a80fc3e0 con 0x7fc0a81003a0 2024-02-16T23:01:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc08f7fe700 1 -- 172.21.15.64:0/3706042754 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc09c0096f0 con 0x7fc0a81003a0 2024-02-16T23:01:46.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.256+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a81158c0 con 0x7fc0a81003a0 2024-02-16T23:01:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.257+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a8107a50 con 0x7fc0a81003a0 2024-02-16T23:01:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.258+0000 7fc08f7fe700 1 -- 172.21.15.64:0/3706042754 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc09c00d690 con 0x7fc0a81003a0 2024-02-16T23:01:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.258+0000 7fc08f7fe700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc09006d7e0 0x7fc09006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.259+0000 7fc08f7fe700 1 -- 172.21.15.64:0/3706042754 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc09c08c680 con 0x7fc0a81003a0 2024-02-16T23:01:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.259+0000 7fc0a6ffd700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc09006d7e0 0x7fc09006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.259+0000 7fc0a6ffd700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc09006d7e0 0x7fc09006fc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc098009fd0 tx=0x7fc098009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:46.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.263+0000 7fc08f7fe700 1 -- 172.21.15.64:0/3706042754 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc09c059a50 con 0x7fc0a81003a0 2024-02-16T23:01:46.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:45 smithi192 ceph-mon[82652]: pgmap v99: 65 pgs: 10 peering, 8 active+undersized+degraded, 9 active+undersized, 38 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 24/261 objects degraded (9.195%) 2024-02-16T23:01:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.495+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0a805e3d0 con 0x7fc09006d7e0 2024-02-16T23:01:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.496+0000 7fc08f7fe700 1 -- 172.21.15.64:0/3706042754 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc0a805e3d0 con 0x7fc09006d7e0 2024-02-16T23:01:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc09006d7e0 msgr2=0x7fc09006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc09006d7e0 0x7fc09006fc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc098009fd0 tx=0x7fc098009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a81003a0 msgr2=0x7fc0a80fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a81003a0 0x7fc0a80fb990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc09c002a80 tx=0x7fc09c0046b0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 shutdown_connections 2024-02-16T23:01:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc09006d7e0 0x7fc09006fc90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0a80fda80 0x7fc0a80fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 --2- 172.21.15.64:0/3706042754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a81003a0 0x7fc0a80fb990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 >> 172.21.15.64:0/3706042754 conn(0x7fc0a80f7500 msgr2=0x7fc0a80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 shutdown_connections 2024-02-16T23:01:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.499+0000 7fc0ad808700 1 -- 172.21.15.64:0/3706042754 wait complete. 2024-02-16T23:01:46.512 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:01:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.607+0000 7f47054dd700 1 -- 172.21.15.64:0/3351757032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 msgr2=0x7f47000fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.607+0000 7f47054dd700 1 --2- 172.21.15.64:0/3351757032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 0x7f47000fb630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46f0004940 tx=0x7f46f0009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.607+0000 7f47054dd700 1 -- 172.21.15.64:0/3351757032 shutdown_connections 2024-02-16T23:01:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.607+0000 7f47054dd700 1 --2- 172.21.15.64:0/3351757032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47000fe2b0 0x7f47000fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.607+0000 7f47054dd700 1 --2- 172.21.15.64:0/3351757032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 0x7f47000fb630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.607+0000 7f47054dd700 1 -- 172.21.15.64:0/3351757032 >> 172.21.15.64:0/3351757032 conn(0x7f47000f6ca0 msgr2=0x7f47000f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.607+0000 7f47054dd700 1 -- 172.21.15.64:0/3351757032 shutdown_connections 2024-02-16T23:01:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.607+0000 7f47054dd700 1 -- 172.21.15.64:0/3351757032 wait complete. 2024-02-16T23:01:46.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f47054dd700 1 Processor -- start 2024-02-16T23:01:46.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f47054dd700 1 -- start start 2024-02-16T23:01:46.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f47054dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 0x7f470010fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f47054dd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47000fe2b0 0x7f4700110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f47054dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4700110800 con 0x7f47000fb220 2024-02-16T23:01:46.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f47054dd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4700110f90 con 0x7f47000fe2b0 2024-02-16T23:01:46.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f46feffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 0x7f470010fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f46feffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 0x7f470010fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49160/0 (socket says 172.21.15.64:49160) 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.608+0000 7f46feffd700 1 -- 172.21.15.64:0/1610072567 learned_addr learned my addr 172.21.15.64:0/1610072567 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f46fe7fc700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47000fe2b0 0x7f4700110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f46feffd700 1 -- 172.21.15.64:0/1610072567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47000fe2b0 msgr2=0x7f4700110200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f46feffd700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47000fe2b0 0x7f4700110200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f46feffd700 1 -- 172.21.15.64:0/1610072567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0005040 con 0x7f47000fb220 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f46feffd700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 0x7f470010fcc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009150 tx=0x7f46f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f46e77fe700 1 -- 172.21.15.64:0/1610072567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f000e040 con 0x7f47000fb220 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f46e77fe700 1 -- 172.21.15.64:0/1610072567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46f000bbf0 con 0x7f47000fb220 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4700111210 con 0x7f47000fb220 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f46e77fe700 1 -- 172.21.15.64:0/1610072567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f001c630 con 0x7f47000fb220 2024-02-16T23:01:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.609+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4700111750 con 0x7f47000fb220 2024-02-16T23:01:46.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.610+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47001024d0 con 0x7f47000fb220 2024-02-16T23:01:46.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.611+0000 7f46e77fe700 1 -- 172.21.15.64:0/1610072567 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f46f000bd60 con 0x7f47000fb220 2024-02-16T23:01:46.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.611+0000 7f46e77fe700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f46e806d7e0 0x7f46e806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.611+0000 7f46e77fe700 1 -- 172.21.15.64:0/1610072567 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f46f008e210 con 0x7f47000fb220 2024-02-16T23:01:46.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.611+0000 7f46fe7fc700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f46e806d7e0 0x7f46e806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.612+0000 7f46fe7fc700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f46e806d7e0 0x7f46e806fc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46f4004990 tx=0x7f46f4009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:46.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.620+0000 7f46e77fe700 1 -- 172.21.15.64:0/1610072567 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f46f005b660 con 0x7f47000fb220 2024-02-16T23:01:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.852+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f470002b3b0 con 0x7f46e806d7e0 2024-02-16T23:01:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.854+0000 7f46e77fe700 1 -- 172.21.15.64:0/1610072567 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f470002b3b0 con 0x7f46e806d7e0 2024-02-16T23:01:46.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f46e806d7e0 msgr2=0x7f46e806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f46e806d7e0 0x7f46e806fc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46f4004990 tx=0x7f46f4009ee0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 msgr2=0x7f470010fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 0x7f470010fcc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009150 tx=0x7f46f0009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:46.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 shutdown_connections 2024-02-16T23:01:46.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f46e806d7e0 0x7f46e806fc90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47000fb220 0x7f470010fcc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 --2- 172.21.15.64:0/1610072567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47000fe2b0 0x7f4700110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 >> 172.21.15.64:0/1610072567 conn(0x7f47000f6ca0 msgr2=0x7f47000f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 shutdown_connections 2024-02-16T23:01:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.856+0000 7f47054dd700 1 -- 172.21.15.64:0/1610072567 wait complete. 2024-02-16T23:01:46.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:46 smithi064 ceph-mon[93757]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:46.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:46.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:46.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:01:46.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:01:46.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 -- 172.21.15.64:0/3428992423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fbe40 msgr2=0x7fcf240fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 --2- 172.21.15.64:0/3428992423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fbe40 0x7fcf240fc250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf18004940 tx=0x7fcf18009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 -- 172.21.15.64:0/3428992423 shutdown_connections 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 --2- 172.21.15.64:0/3428992423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf240fcfb0 0x7fcf240fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 --2- 172.21.15.64:0/3428992423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fbe40 0x7fcf240fc250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 -- 172.21.15.64:0/3428992423 >> 172.21.15.64:0/3428992423 conn(0x7fcf240f7500 msgr2=0x7fcf240f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 -- 172.21.15.64:0/3428992423 shutdown_connections 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 -- 172.21.15.64:0/3428992423 wait complete. 2024-02-16T23:01:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 Processor -- start 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.962+0000 7fcf2b21b700 1 -- start start 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf2b21b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf240fbe40 0x7fcf24112a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf2b21b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fcfb0 0x7fcf24114fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf2b21b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf24066be0 con 0x7fcf240fcfb0 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf2b21b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf24066d20 con 0x7fcf240fbe40 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf23fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fcfb0 0x7fcf24114fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf23fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fcfb0 0x7fcf24114fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49176/0 (socket says 172.21.15.64:49176) 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf23fff700 1 -- 172.21.15.64:0/2258316855 learned_addr learned my addr 172.21.15.64:0/2258316855 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf28fb7700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf240fbe40 0x7fcf24112a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf23fff700 1 -- 172.21.15.64:0/2258316855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf240fbe40 msgr2=0x7fcf24112a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf23fff700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf240fbe40 0x7fcf24112a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf23fff700 1 -- 172.21.15.64:0/2258316855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf18005040 con 0x7fcf240fcfb0 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf28fb7700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf240fbe40 0x7fcf24112a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf23fff700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fcfb0 0x7fcf24114fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000b790 tx=0x7fcf10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf217fa700 1 -- 172.21.15.64:0/2258316855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf1000f770 con 0x7fcf240fcfb0 2024-02-16T23:01:46.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf24115500 con 0x7fcf240fcfb0 2024-02-16T23:01:46.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf217fa700 1 -- 172.21.15.64:0/2258316855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf1000fda0 con 0x7fcf240fcfb0 2024-02-16T23:01:46.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.963+0000 7fcf217fa700 1 -- 172.21.15.64:0/2258316855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf1000d310 con 0x7fcf240fcfb0 2024-02-16T23:01:46.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.964+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf24115990 con 0x7fcf240fcfb0 2024-02-16T23:01:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.964+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2404bed0 con 0x7fcf240fcfb0 2024-02-16T23:01:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.966+0000 7fcf217fa700 1 -- 172.21.15.64:0/2258316855 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcf10010420 con 0x7fcf240fcfb0 2024-02-16T23:01:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.966+0000 7fcf217fa700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf1406d7e0 0x7fcf1406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.966+0000 7fcf217fa700 1 -- 172.21.15.64:0/2258316855 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcf1008cda0 con 0x7fcf240fcfb0 2024-02-16T23:01:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.966+0000 7fcf28fb7700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf1406d7e0 0x7fcf1406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.966+0000 7fcf28fb7700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf1406d7e0 0x7fcf1406fc90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800b3b0 tx=0x7fcf18009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:46.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:46.975+0000 7fcf217fa700 1 -- 172.21.15.64:0/2258316855 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf10056760 con 0x7fcf240fcfb0 2024-02-16T23:01:47.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.187+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf24101990 con 0x7fcf1406d7e0 2024-02-16T23:01:47.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.198+0000 7fcf217fa700 1 -- 172.21.15.64:0/2258316855 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fcf24101990 con 0x7fcf1406d7e0 2024-02-16T23:01:47.199 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:01:47.199 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (4m) 5s ago 13m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:01:47.199 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (109s) 5s ago 13m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:01:47.199 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (103s) 100s ago 12m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (4m) 5s ago 13m 84.4M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (7m) 5s ago 7m 17.9M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (8m) 5s ago 7m 153M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (7m) 100s ago 7m 16.5M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (7m) 100s ago 7m 16.2M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (3m) 5s ago 14m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (3m) 100s ago 12m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (2m) 5s ago 14m 45.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (2m) 100s ago 12m 35.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (5m) 5s ago 13m 15.6M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (4m) 100s ago 12m 15.8M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (82s) 5s ago 11m 96.7M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (46s) 5s ago 11m 83.8M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:01:47.200 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (10s) 5s ago 10m 11.8M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:01:47.201 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (10m) 100s ago 10m 234M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 c8536a8d1d6d 2024-02-16T23:01:47.201 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (9m) 100s ago 9m 210M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T23:01:47.201 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (9m) 100s ago 9m 186M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T23:01:47.201 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (3m) 5s ago 12m 59.3M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:01:47.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf1406d7e0 msgr2=0x7fcf1406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf1406d7e0 0x7fcf1406fc90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800b3b0 tx=0x7fcf18009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:47.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fcfb0 msgr2=0x7fcf24114fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fcfb0 0x7fcf24114fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000b790 tx=0x7fcf10009250 comp rx=0 tx=0).stop 2024-02-16T23:01:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 shutdown_connections 2024-02-16T23:01:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf1406d7e0 0x7fcf1406fc90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf240fbe40 0x7fcf24112a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 --2- 172.21.15.64:0/2258316855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf240fcfb0 0x7fcf24114fc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 >> 172.21.15.64:0/2258316855 conn(0x7fcf240f7500 msgr2=0x7fcf24100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 shutdown_connections 2024-02-16T23:01:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.200+0000 7fcf2b21b700 1 -- 172.21.15.64:0/2258316855 wait complete. 2024-02-16T23:01:47.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:46 smithi192 ceph-mon[82652]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:47.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:47.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:47.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:01:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:01:47.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.302+0000 7fa091254700 1 -- 172.21.15.64:0/1709602502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 msgr2=0x7fa08c0fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.302+0000 7fa091254700 1 --2- 172.21.15.64:0/1709602502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 0x7fa08c0fe700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa080004940 tx=0x7fa080009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.302+0000 7fa091254700 1 -- 172.21.15.64:0/1709602502 shutdown_connections 2024-02-16T23:01:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.302+0000 7fa091254700 1 --2- 172.21.15.64:0/1709602502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 0x7fa08c0fe700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.302+0000 7fa091254700 1 --2- 172.21.15.64:0/1709602502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa08c0fb220 0x7fa08c0fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.302+0000 7fa091254700 1 -- 172.21.15.64:0/1709602502 >> 172.21.15.64:0/1709602502 conn(0x7fa08c0f6ca0 msgr2=0x7fa08c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.302+0000 7fa091254700 1 -- 172.21.15.64:0/1709602502 shutdown_connections 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.302+0000 7fa091254700 1 -- 172.21.15.64:0/1709602502 wait complete. 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa091254700 1 Processor -- start 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa091254700 1 -- start start 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa091254700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa08c0fb220 0x7fa08c0fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa091254700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 0x7fa08c0fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa091254700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c0fc7c0 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa091254700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c0fc930 con 0x7fa08c0fb220 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa08a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 0x7fa08c0fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa08a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 0x7fa08c0fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49198/0 (socket says 172.21.15.64:49198) 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa08a59c700 1 -- 172.21.15.64:0/1751300854 learned_addr learned my addr 172.21.15.64:0/1751300854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.303+0000 7fa08ad9d700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa08c0fb220 0x7fa08c0fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa08a59c700 1 -- 172.21.15.64:0/1751300854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa08c0fb220 msgr2=0x7fa08c0fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa08a59c700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa08c0fb220 0x7fa08c0fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa08a59c700 1 -- 172.21.15.64:0/1751300854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa080005040 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa08a59c700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 0x7fa08c0fc1c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa080009b30 tx=0x7fa080009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:47.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa0737fe700 1 -- 172.21.15.64:0/1751300854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa080021070 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c0fcbb0 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa0737fe700 1 -- 172.21.15.64:0/1751300854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa08001c400 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa0737fe700 1 -- 172.21.15.64:0/1751300854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08000e070 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.304+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c1159d0 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.305+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08c04bed0 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.306+0000 7fa0737fe700 1 -- 172.21.15.64:0/1751300854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa08001ca00 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.306+0000 7fa0737fe700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa07406d7e0 0x7fa07406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:47.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.306+0000 7fa0737fe700 1 -- 172.21.15.64:0/1751300854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa08008e680 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.306+0000 7fa08ad9d700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa07406d7e0 0x7fa07406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:47.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.307+0000 7fa08ad9d700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa07406d7e0 0x7fa07406fc90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0fdad0 tx=0x7fa07c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:47.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.310+0000 7fa0737fe700 1 -- 172.21.15.64:0/1751300854 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa08005bb70 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.600+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa08c105a60 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.600+0000 7fa0737fe700 1 -- 172.21.15.64:0/1751300854 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fa080004490 con 0x7fa08c0fe2b0 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 3, 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:01:47.602 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:01:47.603 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T23:01:47.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:01:47.603 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:01:47.603 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 7, 2024-02-16T23:01:47.603 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 7 2024-02-16T23:01:47.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:01:47.603 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa07406d7e0 msgr2=0x7fa07406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa07406d7e0 0x7fa07406fc90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0fdad0 tx=0x7fa07c00b040 comp rx=0 tx=0).stop 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 msgr2=0x7fa08c0fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 0x7fa08c0fc1c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa080009b30 tx=0x7fa080009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 shutdown_connections 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa07406d7e0 0x7fa07406fc90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa08c0fb220 0x7fa08c0fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 --2- 172.21.15.64:0/1751300854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0fe2b0 0x7fa08c0fc1c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 >> 172.21.15.64:0/1751300854 conn(0x7fa08c0f6ca0 msgr2=0x7fa08c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.602+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 shutdown_connections 2024-02-16T23:01:47.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.603+0000 7fa091254700 1 -- 172.21.15.64:0/1751300854 wait complete. 2024-02-16T23:01:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.708+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/1971478153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fcfb0 msgr2=0x7f4f080fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.708+0000 7f4f0d4ff700 1 --2- 172.21.15.64:0/1971478153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fcfb0 0x7f4f080fd420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4efc004940 tx=0x7f4efc009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.708+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/1971478153 shutdown_connections 2024-02-16T23:01:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.708+0000 7f4f0d4ff700 1 --2- 172.21.15.64:0/1971478153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fcfb0 0x7f4f080fd420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.708+0000 7f4f0d4ff700 1 --2- 172.21.15.64:0/1971478153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f080fbe40 0x7f4f080fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.708+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/1971478153 >> 172.21.15.64:0/1971478153 conn(0x7f4f080f7520 msgr2=0x7f4f080f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.708+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/1971478153 shutdown_connections 2024-02-16T23:01:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.708+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/1971478153 wait complete. 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f0d4ff700 1 Processor -- start 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f0d4ff700 1 -- start start 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f0d4ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fbe40 0x7f4f081105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f0d4ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f080fcfb0 0x7f4f08112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f0d4ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08066cf0 con 0x7f4f080fbe40 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f0d4ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08066e30 con 0x7f4f080fcfb0 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f06ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fbe40 0x7f4f081105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f06ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fbe40 0x7f4f081105b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49210/0 (socket says 172.21.15.64:49210) 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.709+0000 7f4f06ffd700 1 -- 172.21.15.64:0/3573314405 learned_addr learned my addr 172.21.15.64:0/3573314405 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4f067fc700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f080fcfb0 0x7f4f08112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4f06ffd700 1 -- 172.21.15.64:0/3573314405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f080fcfb0 msgr2=0x7f4f08112b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4f06ffd700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f080fcfb0 0x7f4f08112b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4f06ffd700 1 -- 172.21.15.64:0/3573314405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc005040 con 0x7f4f080fbe40 2024-02-16T23:01:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4f067fc700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f080fcfb0 0x7f4f08112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:01:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4f06ffd700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fbe40 0x7f4f081105b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800eb40 tx=0x7f4ef800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4eef7fe700 1 -- 172.21.15.64:0/3573314405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ef8007cc0 con 0x7f4f080fbe40 2024-02-16T23:01:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4eef7fe700 1 -- 172.21.15.64:0/3573314405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ef800f040 con 0x7f4f080fbe40 2024-02-16T23:01:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f08113100 con 0x7f4f080fbe40 2024-02-16T23:01:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4eef7fe700 1 -- 172.21.15.64:0/3573314405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ef8009620 con 0x7f4f080fbe40 2024-02-16T23:01:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.710+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f08113640 con 0x7f4f080fbe40 2024-02-16T23:01:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.711+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0804bed0 con 0x7f4f080fbe40 2024-02-16T23:01:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.712+0000 7f4eef7fe700 1 -- 172.21.15.64:0/3573314405 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4ef8009780 con 0x7f4f080fbe40 2024-02-16T23:01:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.712+0000 7f4eef7fe700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4ef006d7f0 0x7f4ef006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.712+0000 7f4eef7fe700 1 -- 172.21.15.64:0/3573314405 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ef808ca60 con 0x7f4f080fbe40 2024-02-16T23:01:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.712+0000 7f4f067fc700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4ef006d7f0 0x7f4ef006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.713+0000 7f4f067fc700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4ef006d7f0 0x7f4ef006fca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00b0d0 tx=0x7f4efc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.717+0000 7f4eef7fe700 1 -- 172.21.15.64:0/3573314405 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ef8056430 con 0x7f4f080fbe40 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: pgmap v100: 65 pgs: 10 peering, 6 active+undersized+degraded, 6 active+undersized, 43 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 21/261 objects degraded (8.046%) 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1751300854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:47.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:47.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:01:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.968+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4f08104930 con 0x7f4f080fbe40 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.968+0000 7f4eef7fe700 1 -- 172.21.15.64:0/3573314405 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f4ef8019750 con 0x7f4f080fbe40 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:01:47.970 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:01:47.971 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:01:47.972 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:01:47.973 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:01:47.973 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:01:47.973 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:01:47.973 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:01:47.973 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:01:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4ef006d7f0 msgr2=0x7f4ef006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4ef006d7f0 0x7f4ef006fca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00b0d0 tx=0x7f4efc009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fbe40 msgr2=0x7f4f081105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fbe40 0x7f4f081105b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800eb40 tx=0x7f4ef800b040 comp rx=0 tx=0).stop 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 shutdown_connections 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4ef006d7f0 0x7f4ef006fca0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f080fbe40 0x7f4f081105b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 --2- 172.21.15.64:0/3573314405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f080fcfb0 0x7f4f08112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.970+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 >> 172.21.15.64:0/3573314405 conn(0x7f4f080f7520 msgr2=0x7f4f08100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.971+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 shutdown_connections 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:47.971+0000 7f4f0d4ff700 1 -- 172.21.15.64:0/3573314405 wait complete. 2024-02-16T23:01:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T23:01:48.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 -- 172.21.15.64:0/4212022290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 msgr2=0x7f865c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:48.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 --2- 172.21.15.64:0/4212022290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 0x7f865c0fe550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8644004940 tx=0x7f8644009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:48.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 -- 172.21.15.64:0/4212022290 shutdown_connections 2024-02-16T23:01:48.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 --2- 172.21.15.64:0/4212022290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 0x7f865c0fe550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 --2- 172.21.15.64:0/4212022290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865c0fcef0 0x7f865c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 -- 172.21.15.64:0/4212022290 >> 172.21.15.64:0/4212022290 conn(0x7f865c0f8620 msgr2=0x7f865c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 -- 172.21.15.64:0/4212022290 shutdown_connections 2024-02-16T23:01:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 -- 172.21.15.64:0/4212022290 wait complete. 2024-02-16T23:01:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 Processor -- start 2024-02-16T23:01:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.074+0000 7f8661748700 1 -- start start 2024-02-16T23:01:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f8661748700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865c0fcef0 0x7f865c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f8661748700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 0x7f865c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f8661748700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c070970 con 0x7f865c0fe0e0 2024-02-16T23:01:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f8661748700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c070ae0 con 0x7f865c0fcef0 2024-02-16T23:01:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f865a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 0x7f865c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f865a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 0x7f865c0702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49224/0 (socket says 172.21.15.64:49224) 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f865a7fc700 1 -- 172.21.15.64:0/561128281 learned_addr learned my addr 172.21.15.64:0/561128281 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f865affd700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865c0fcef0 0x7f865c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f865a7fc700 1 -- 172.21.15.64:0/561128281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865c0fcef0 msgr2=0x7f865c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f865a7fc700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865c0fcef0 0x7f865c071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f865a7fc700 1 -- 172.21.15.64:0/561128281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8644005040 con 0x7f865c0fe0e0 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f865a7fc700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 0x7f865c0702e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86440093b0 tx=0x7f8644009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f86537fe700 1 -- 172.21.15.64:0/561128281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864400e040 con 0x7f865c0fe0e0 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f86537fe700 1 -- 172.21.15.64:0/561128281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f864400bcc0 con 0x7f865c0fe0e0 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c070d60 con 0x7f865c0fe0e0 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f86537fe700 1 -- 172.21.15.64:0/561128281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864401c630 con 0x7f865c0fe0e0 2024-02-16T23:01:48.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.075+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c1159e0 con 0x7f865c0fe0e0 2024-02-16T23:01:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.076+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c04bed0 con 0x7f865c0fe0e0 2024-02-16T23:01:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.077+0000 7f86537fe700 1 -- 172.21.15.64:0/561128281 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f86440139f0 con 0x7f865c0fe0e0 2024-02-16T23:01:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.077+0000 7f86537fe700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f864806d7e0 0x7f864806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.077+0000 7f86537fe700 1 -- 172.21.15.64:0/561128281 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f864408e2e0 con 0x7f865c0fe0e0 2024-02-16T23:01:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.077+0000 7f865affd700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f864806d7e0 0x7f864806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.078+0000 7f865affd700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f864806d7e0 0x7f864806fc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f865c071bf0 tx=0x7f864c009620 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:48.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.082+0000 7f86537fe700 1 -- 172.21.15.64:0/561128281 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f864405b730 con 0x7f865c0fe0e0 2024-02-16T23:01:48.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: pgmap v100: 65 pgs: 10 peering, 6 active+undersized+degraded, 6 active+undersized, 43 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 21/261 objects degraded (8.046%) 2024-02-16T23:01:48.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1751300854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:01:48.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.304+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f865c102ac0 con 0x7f864806d7e0 2024-02-16T23:01:48.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.305+0000 7f86537fe700 1 -- 172.21.15.64:0/561128281 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f865c102ac0 con 0x7f864806d7e0 2024-02-16T23:01:48.306 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "9/21 daemons upgraded", 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:01:48.307 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f864806d7e0 msgr2=0x7f864806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f864806d7e0 0x7f864806fc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f865c071bf0 tx=0x7f864c009620 comp rx=0 tx=0).stop 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 msgr2=0x7f865c0702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 0x7f865c0702e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86440093b0 tx=0x7f8644009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 shutdown_connections 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f864806d7e0 0x7f864806fc90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865c0fcef0 0x7f865c071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 --2- 172.21.15.64:0/561128281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c0fe0e0 0x7f865c0702e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 >> 172.21.15.64:0/561128281 conn(0x7f865c0f8620 msgr2=0x7f865c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 shutdown_connections 2024-02-16T23:01:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.307+0000 7f8661748700 1 -- 172.21.15.64:0/561128281 wait complete. 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.410+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1669387258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fcef0 msgr2=0x7f1e200fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.410+0000 7f1e24fe9700 1 --2- 172.21.15.64:0/1669387258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fcef0 0x7f1e200fd320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e14004940 tx=0x7f1e14009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.410+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1669387258 shutdown_connections 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.410+0000 7f1e24fe9700 1 --2- 172.21.15.64:0/1669387258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fe0e0 0x7f1e200fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.410+0000 7f1e24fe9700 1 --2- 172.21.15.64:0/1669387258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fcef0 0x7f1e200fd320 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.410+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1669387258 >> 172.21.15.64:0/1669387258 conn(0x7f1e200f8620 msgr2=0x7f1e200faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.410+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1669387258 shutdown_connections 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.410+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1669387258 wait complete. 2024-02-16T23:01:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e24fe9700 1 Processor -- start 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e24fe9700 1 -- start start 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e24fe9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fcef0 0x7f1e20071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e24fe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fe0e0 0x7f1e200702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e24fe9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200708e0 con 0x7f1e200fe0e0 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e24fe9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20070a50 con 0x7f1e200fcef0 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e1dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fe0e0 0x7f1e200702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e1dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fe0e0 0x7f1e200702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49236/0 (socket says 172.21.15.64:49236) 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e1dd9b700 1 -- 172.21.15.64:0/1369481502 learned_addr learned my addr 172.21.15.64:0/1369481502 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e1e59c700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fcef0 0x7f1e20071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e1dd9b700 1 -- 172.21.15.64:0/1369481502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fcef0 msgr2=0x7f1e20071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e1dd9b700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fcef0 0x7f1e20071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e1dd9b700 1 -- 172.21.15.64:0/1369481502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14005040 con 0x7f1e200fe0e0 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.411+0000 7f1e1e59c700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fcef0 0x7f1e20071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.412+0000 7f1e1dd9b700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fe0e0 0x7f1e200702e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c830 tx=0x7f1e10004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.412+0000 7f1e0effd700 1 -- 172.21.15.64:0/1369481502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1000a760 con 0x7f1e200fe0e0 2024-02-16T23:01:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.412+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e20070d30 con 0x7f1e200fe0e0 2024-02-16T23:01:48.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.412+0000 7f1e0effd700 1 -- 172.21.15.64:0/1369481502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e1000ad90 con 0x7f1e200fe0e0 2024-02-16T23:01:48.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.412+0000 7f1e0effd700 1 -- 172.21.15.64:0/1369481502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e100113a0 con 0x7f1e200fe0e0 2024-02-16T23:01:48.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.412+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201159a0 con 0x7f1e200fe0e0 2024-02-16T23:01:48.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.413+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2004bed0 con 0x7f1e200fe0e0 2024-02-16T23:01:48.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.413+0000 7f1e0effd700 1 -- 172.21.15.64:0/1369481502 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1e10012400 con 0x7f1e200fe0e0 2024-02-16T23:01:48.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.414+0000 7f1e0effd700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1e0806d7f0 0x7f1e0806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:01:48.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.414+0000 7f1e0effd700 1 -- 172.21.15.64:0/1369481502 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1e1008ced0 con 0x7f1e200fe0e0 2024-02-16T23:01:48.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.414+0000 7f1e1e59c700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1e0806d7f0 0x7f1e0806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:01:48.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.414+0000 7f1e1e59c700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1e0806d7f0 0x7f1e0806fca0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009fd0 tx=0x7f1e14009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:01:48.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.418+0000 7f1e0effd700 1 -- 172.21.15.64:0/1369481502 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e1005a320 con 0x7f1e200fe0e0 2024-02-16T23:01:48.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.710+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1e20115e30 con 0x7f1e200fe0e0 2024-02-16T23:01:48.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.710+0000 7f1e0effd700 1 -- 172.21.15.64:0/1369481502 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+703 (secure 0 0 0) 0x7f1e10059eb0 con 0x7f1e200fe0e0 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 21/261 objects degraded (8.046%), 6 pgs degraded 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 21/261 objects degraded (8.046%), 6 pgs degraded 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-02-16T23:01:48.712 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-02-16T23:01:48.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1e0806d7f0 msgr2=0x7f1e0806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:48.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1e0806d7f0 0x7f1e0806fca0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009fd0 tx=0x7f1e14009e30 comp rx=0 tx=0).stop 2024-02-16T23:01:48.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fe0e0 msgr2=0x7f1e200702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:01:48.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fe0e0 0x7f1e200702e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c830 tx=0x7f1e10004470 comp rx=0 tx=0).stop 2024-02-16T23:01:48.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 shutdown_connections 2024-02-16T23:01:48.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1e0806d7f0 0x7f1e0806fca0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fcef0 0x7f1e20071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 --2- 172.21.15.64:0/1369481502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200fe0e0 0x7f1e200702e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:01:48.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 >> 172.21.15.64:0/1369481502 conn(0x7f1e200f8620 msgr2=0x7f1e201013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:01:48.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 shutdown_connections 2024-02-16T23:01:48.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:01:48.712+0000 7f1e24fe9700 1 -- 172.21.15.64:0/1369481502 wait complete. 2024-02-16T23:01:49.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:48 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3573314405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:01:49.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:48 smithi064 ceph-mon[93757]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:49.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:48 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1369481502' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:01:49.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:48 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3573314405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:01:49.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:48 smithi192 ceph-mon[82652]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:01:49.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:48 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1369481502' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:01:50.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:49 smithi064 ceph-mon[93757]: pgmap v101: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:50.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:49 smithi064 ceph-mon[93757]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/261 objects degraded (8.046%), 6 pgs degraded) 2024-02-16T23:01:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:49 smithi192 ceph-mon[82652]: pgmap v101: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:49 smithi192 ceph-mon[82652]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/261 objects degraded (8.046%), 6 pgs degraded) 2024-02-16T23:01:52.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:51 smithi064 ceph-mon[93757]: pgmap v102: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:52.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:51 smithi192 ceph-mon[82652]: pgmap v102: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:54.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:53 smithi064 ceph-mon[93757]: pgmap v103: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:54.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:53 smithi192 ceph-mon[82652]: pgmap v103: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:56.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:55 smithi064 ceph-mon[93757]: pgmap v104: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:55 smithi192 ceph-mon[82652]: pgmap v104: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:58.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:57 smithi064 ceph-mon[93757]: pgmap v105: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:58.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:57 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:01:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:57 smithi192 ceph-mon[82652]: pgmap v105: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:01:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:57 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:01:59 smithi064 ceph-mon[93757]: pgmap v106: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:01:59 smithi192 ceph-mon[82652]: pgmap v106: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:02.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:01 smithi064 ceph-mon[93757]: pgmap v107: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:02.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:01 smithi192 ceph-mon[82652]: pgmap v107: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:02 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T23:02:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:02 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T23:02:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:02 smithi192 ceph-mon[82652]: Upgrade: osd.3 is safe to restart 2024-02-16T23:02:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:02 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:03.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:02 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:02:03.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:02 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T23:02:03.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:02 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T23:02:03.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:02 smithi064 ceph-mon[93757]: Upgrade: osd.3 is safe to restart 2024-02-16T23:02:03.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:02 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:03.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:02 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:02:04.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:03 smithi064 ceph-mon[93757]: pgmap v108: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:04.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:04.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T23:02:04.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:02:04.167 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:03 smithi192 ceph-mon[82652]: pgmap v108: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:04.167 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:04.167 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T23:02:04.168 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:02:05.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:04 smithi064 ceph-mon[93757]: Upgrade: Updating osd.3 2024-02-16T23:02:05.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:04 smithi064 ceph-mon[93757]: Deploying daemon osd.3 on smithi192 2024-02-16T23:02:05.244 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:04 smithi192 ceph-mon[82652]: Upgrade: Updating osd.3 2024-02-16T23:02:05.244 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:04 smithi192 ceph-mon[82652]: Deploying daemon osd.3 on smithi192 2024-02-16T23:02:06.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:05 smithi064 ceph-mon[93757]: pgmap v109: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:06.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:05 smithi192 ceph-mon[82652]: pgmap v109: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:06.992 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:06 smithi192 systemd[1]: Stopping Ceph osd.3 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:02:07.284 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:06 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[41814]: 2024-02-16T23:02:06.991+0000 7f16d0f6a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:02:07.284 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:06 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[41814]: 2024-02-16T23:02:06.991+0000 7f16d0f6a700 -1 osd.3 65 *** Got signal Terminated *** 2024-02-16T23:02:07.284 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:06 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[41814]: 2024-02-16T23:02:06.991+0000 7f16d0f6a700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:02:08.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:07 smithi064 ceph-mon[93757]: pgmap v110: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:08.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:07 smithi064 ceph-mon[93757]: osd.3 marked itself down and dead 2024-02-16T23:02:08.186 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:07 smithi192 ceph-mon[82652]: pgmap v110: 65 pgs: 65 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:08.186 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:07 smithi192 ceph-mon[82652]: osd.3 marked itself down and dead 2024-02-16T23:02:08.534 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:08 smithi192 podman[92049]: 2024-02-16 23:02:08.326050799 +0000 UTC m=+1.429849371 container died c8536a8d1d6d2fa066260ac14ac01a91ee0ec268352a013efe4166ddf28b270e (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-b88fae6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-16T23:02:08.864 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:08 smithi192 podman[92049]: 2024-02-16 23:02:08.666694503 +0000 UTC m=+1.770493077 container cleanup c8536a8d1d6d2fa066260ac14ac01a91ee0ec268352a013efe4166ddf28b270e (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-b88fae6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-16T23:02:08.864 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:08 smithi192 bash[92049]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3 2024-02-16T23:02:08.864 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:08 smithi192 podman[92077]: 2024-02-16 23:02:08.834803848 +0000 UTC m=+0.510428238 container remove c8536a8d1d6d2fa066260ac14ac01a91ee0ec268352a013efe4166ddf28b270e (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-b88fae6, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-16T23:02:09.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:08 smithi064 ceph-mon[93757]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:02:09.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:08 smithi064 ceph-mon[93757]: failed to encode map e66 with expected crc 2024-02-16T23:02:09.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:08 smithi064 ceph-mon[93757]: osdmap e66: 6 total, 5 up, 6 in 2024-02-16T23:02:09.275 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:08 smithi192 ceph-mon[82652]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:02:09.275 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:08 smithi192 ceph-mon[82652]: failed to encode map e66 with expected crc 2024-02-16T23:02:09.275 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:08 smithi192 ceph-mon[82652]: osdmap e66: 6 total, 5 up, 6 in 2024-02-16T23:02:09.534 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:09 smithi192 podman[92171]: 2024-02-16 23:02:09.178230974 +0000 UTC m=+0.017485406 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:02:09.534 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:09 smithi192 podman[92171]: 2024-02-16 23:02:09.285684101 +0000 UTC m=+0.124938532 container create 2254b377c8aa420f99fbbb10a287748fc6a212fcc58896b0d400cbf79ea6a8f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-16T23:02:09.534 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:09 smithi192 podman[92171]: 2024-02-16 23:02:09.497401266 +0000 UTC m=+0.336655702 container init 2254b377c8aa420f99fbbb10a287748fc6a212fcc58896b0d400cbf79ea6a8f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-16T23:02:09.535 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:09 smithi192 podman[92171]: 2024-02-16 23:02:09.50064101 +0000 UTC m=+0.339895446 container start 2254b377c8aa420f99fbbb10a287748fc6a212fcc58896b0d400cbf79ea6a8f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-16T23:02:09.865 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:09 smithi192 podman[92171]: 2024-02-16 23:02:09.580564182 +0000 UTC m=+0.419818623 container attach 2254b377c8aa420f99fbbb10a287748fc6a212fcc58896b0d400cbf79ea6a8f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T23:02:09.865 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:09 smithi192 podman[92171]: 2024-02-16 23:02:09.787337677 +0000 UTC m=+0.626592106 container died 2254b377c8aa420f99fbbb10a287748fc6a212fcc58896b0d400cbf79ea6a8f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:02:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:09 smithi064 ceph-mon[93757]: pgmap v112: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:09 smithi064 ceph-mon[93757]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-02-16T23:02:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:09 smithi064 ceph-mon[93757]: osdmap e67: 6 total, 5 up, 6 in 2024-02-16T23:02:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:09 smithi064 ceph-mon[93757]: failed to encode map e67 with expected crc 2024-02-16T23:02:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:09 smithi064 ceph-mon[93757]: failed to encode map e67 with expected crc 2024-02-16T23:02:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:09 smithi064 ceph-mon[93757]: failed to encode map e67 with expected crc 2024-02-16T23:02:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:09 smithi064 ceph-mon[93757]: failed to encode map e67 with expected crc 2024-02-16T23:02:10.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:09 smithi192 ceph-mon[82652]: pgmap v112: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:10.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:09 smithi192 ceph-mon[82652]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-02-16T23:02:10.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:09 smithi192 ceph-mon[82652]: osdmap e67: 6 total, 5 up, 6 in 2024-02-16T23:02:10.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:09 smithi192 ceph-mon[82652]: failed to encode map e67 with expected crc 2024-02-16T23:02:10.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:09 smithi192 ceph-mon[82652]: failed to encode map e67 with expected crc 2024-02-16T23:02:10.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:09 smithi192 ceph-mon[82652]: failed to encode map e67 with expected crc 2024-02-16T23:02:10.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:09 smithi192 ceph-mon[82652]: failed to encode map e67 with expected crc 2024-02-16T23:02:10.751 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 podman[92197]: 2024-02-16 23:02:10.334598563 +0000 UTC m=+0.540341063 container remove 2254b377c8aa420f99fbbb10a287748fc6a212fcc58896b0d400cbf79ea6a8f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-16T23:02:10.752 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-16T23:02:10.752 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 systemd[1]: Stopped Ceph osd.3 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:02:10.752 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.3.service: Consumed 1min 368ms CPU time. 2024-02-16T23:02:10.752 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 systemd[1]: Starting Ceph osd.3 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:02:11.034 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 podman[92300]: 2024-02-16 23:02:10.752146059 +0000 UTC m=+0.085327769 container create 1cbbc7c82690778cd5202b73e4bdb26326cafc9fdc44e135d954376d4621ae0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-16T23:02:11.034 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 podman[92300]: 2024-02-16 23:02:10.684659292 +0000 UTC m=+0.017840960 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:02:11.034 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 podman[92300]: 2024-02-16 23:02:10.979651553 +0000 UTC m=+0.312833222 container init 1cbbc7c82690778cd5202b73e4bdb26326cafc9fdc44e135d954376d4621ae0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-16T23:02:11.034 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:10 smithi192 podman[92300]: 2024-02-16 23:02:10.983026314 +0000 UTC m=+0.316207982 container start 1cbbc7c82690778cd5202b73e4bdb26326cafc9fdc44e135d954376d4621ae0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:02:11.427 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 podman[92300]: 2024-02-16 23:02:11.050153414 +0000 UTC m=+0.383335093 container attach 1cbbc7c82690778cd5202b73e4bdb26326cafc9fdc44e135d954376d4621ae0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-16T23:02:11.732 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate[92315]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T23:02:11.732 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate[92315]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T23:02:11.732 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 bash[92300]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T23:02:11.732 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 bash[92300]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T23:02:11.732 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 bash[92300]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T23:02:11.732 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 bash[92300]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T23:02:11.732 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 bash[92300]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-16T23:02:11.733 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 bash[92300]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T23:02:11.733 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 bash[92300]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-16T23:02:11.733 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate[92315]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T23:02:11.733 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate[92315]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T23:02:11.733 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate[92315]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-16T23:02:11.733 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate[92315]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T23:02:11.733 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate[92315]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-16T23:02:11.733 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:11 smithi192 podman[92300]: 2024-02-16 23:02:11.451462749 +0000 UTC m=+0.784644412 container died 1cbbc7c82690778cd5202b73e4bdb26326cafc9fdc44e135d954376d4621ae0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True) 2024-02-16T23:02:12.008 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:11 smithi192 ceph-mon[82652]: pgmap v114: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:12.260 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 podman[92450]: 2024-02-16 23:02:12.008825143 +0000 UTC m=+0.552549517 container remove 1cbbc7c82690778cd5202b73e4bdb26326cafc9fdc44e135d954376d4621ae0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-activate, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:02:12.260 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 podman[92514]: 2024-02-16 23:02:12.122976698 +0000 UTC m=+0.018617052 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:02:12.260 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 podman[92514]: 2024-02-16 23:02:12.260023702 +0000 UTC m=+0.155664043 container create 3b60e36c880384ee3dd9f06789c55c8f66d9a3e4a64f2fcc21d93a5da041170a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T23:02:12.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:11 smithi064 ceph-mon[93757]: pgmap v114: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:12.520 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 podman[92514]: 2024-02-16 23:02:12.437816601 +0000 UTC m=+0.333456941 container init 3b60e36c880384ee3dd9f06789c55c8f66d9a3e4a64f2fcc21d93a5da041170a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-16T23:02:12.520 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 podman[92514]: 2024-02-16 23:02:12.440831869 +0000 UTC m=+0.336472217 container start 3b60e36c880384ee3dd9f06789c55c8f66d9a3e4a64f2fcc21d93a5da041170a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-16T23:02:12.520 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 ceph-osd[92533]: -- 172.21.15.192:0/40051647 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653779d91e0 con 0x5653779ab000 2024-02-16T23:02:12.520 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 bash[92514]: 3b60e36c880384ee3dd9f06789c55c8f66d9a3e4a64f2fcc21d93a5da041170a 2024-02-16T23:02:12.784 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 systemd[1]: Started Ceph osd.3 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:02:13.284 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:12 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[92529]: 2024-02-16T23:02:12.990+0000 7fde88b2c7c0 -1 Falling back to public interface 2024-02-16T23:02:13.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:13.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:13.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:02:13.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:13.982 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:13.982 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:02:14.846 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:14 smithi192 ceph-mon[82652]: pgmap v115: 65 pgs: 10 active+undersized+degraded, 18 peering, 10 active+undersized, 27 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 28/261 objects degraded (10.728%) 2024-02-16T23:02:14.846 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:14 smithi192 ceph-mon[82652]: Health check failed: Degraded data redundancy: 28/261 objects degraded (10.728%), 10 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:14.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:14 smithi064 ceph-mon[93757]: pgmap v115: 65 pgs: 10 active+undersized+degraded, 18 peering, 10 active+undersized, 27 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 28/261 objects degraded (10.728%) 2024-02-16T23:02:14.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:14 smithi064 ceph-mon[93757]: Health check failed: Degraded data redundancy: 28/261 objects degraded (10.728%), 10 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:16 smithi192 ceph-mon[82652]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/261 objects degraded (22.605%) 2024-02-16T23:02:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:16.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:16 smithi192 ceph-mon[82652]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-02-16T23:02:16.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:16 smithi064 ceph-mon[93757]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/261 objects degraded (22.605%) 2024-02-16T23:02:16.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:16.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:16.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:16 smithi064 ceph-mon[93757]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-02-16T23:02:17.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:17.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:17.784 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:17 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[92529]: 2024-02-16T23:02:17.531+0000 7fde88b2c7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-02-16T23:02:17.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:17.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:18.228 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:17 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[92529]: 2024-02-16T23:02:17.901+0000 7fde88b2c7c0 -1 osd.3 65 log_to_monitors true 2024-02-16T23:02:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:18 smithi192 ceph-mon[82652]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/261 objects degraded (22.605%) 2024-02-16T23:02:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:02:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:18 smithi192 ceph-mon[82652]: from='osd.3 [v2:172.21.15.192:6800/2367052696,v1:172.21.15.192:6801/2367052696]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T23:02:18.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:18 smithi192 ceph-mon[82652]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T23:02:18.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.827+0000 7f5fa5807700 1 -- 172.21.15.64:0/832238804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 msgr2=0x7f5fa00ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:18.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.827+0000 7f5fa5807700 1 --2- 172.21.15.64:0/832238804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 0x7f5fa00ffb80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f90004940 tx=0x7f5f90009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:18.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.828+0000 7f5fa5807700 1 -- 172.21.15.64:0/832238804 shutdown_connections 2024-02-16T23:02:18.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.828+0000 7f5fa5807700 1 --2- 172.21.15.64:0/832238804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa01000c0 0x7f5fa01024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:18.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.828+0000 7f5fa5807700 1 --2- 172.21.15.64:0/832238804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 0x7f5fa00ffb80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:18.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.828+0000 7f5fa5807700 1 -- 172.21.15.64:0/832238804 >> 172.21.15.64:0/832238804 conn(0x7f5fa00f7590 msgr2=0x7f5fa00f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:18.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.828+0000 7f5fa5807700 1 -- 172.21.15.64:0/832238804 shutdown_connections 2024-02-16T23:02:18.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.828+0000 7f5fa5807700 1 -- 172.21.15.64:0/832238804 wait complete. 2024-02-16T23:02:18.829 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:18 smithi064 ceph-mon[93757]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/261 objects degraded (22.605%) 2024-02-16T23:02:18.829 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:18.829 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:02:18.829 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:18 smithi064 ceph-mon[93757]: from='osd.3 [v2:172.21.15.192:6800/2367052696,v1:172.21.15.192:6801/2367052696]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T23:02:18.829 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:18 smithi064 ceph-mon[93757]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T23:02:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.828+0000 7f5fa5807700 1 Processor -- start 2024-02-16T23:02:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.828+0000 7f5fa5807700 1 -- start start 2024-02-16T23:02:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5fa5807700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 0x7f5fa010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5fa5807700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa01000c0 0x7f5fa010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5fa5807700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa010e4e0 con 0x7f5fa0066c30 2024-02-16T23:02:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5fa5807700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa010ed20 con 0x7f5fa01000c0 2024-02-16T23:02:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f9effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 0x7f5fa010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f9effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 0x7f5fa010d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40530/0 (socket says 172.21.15.64:40530) 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f9effd700 1 -- 172.21.15.64:0/2423260207 learned_addr learned my addr 172.21.15.64:0/2423260207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f9e7fc700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa01000c0 0x7f5fa010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f9effd700 1 -- 172.21.15.64:0/2423260207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa01000c0 msgr2=0x7f5fa010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f9effd700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa01000c0 0x7f5fa010de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f9effd700 1 -- 172.21.15.64:0/2423260207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90005040 con 0x7f5fa0066c30 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f9effd700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 0x7f5fa010d910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000b3b0 tx=0x7f5f90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f877fe700 1 -- 172.21.15.64:0/2423260207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9000e040 con 0x7f5fa0066c30 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5f877fe700 1 -- 172.21.15.64:0/2423260207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f9000bcc0 con 0x7f5fa0066c30 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.829+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa010efa0 con 0x7f5fa0066c30 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.830+0000 7f5f877fe700 1 -- 172.21.15.64:0/2423260207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9001c630 con 0x7f5fa0066c30 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.830+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa010f4e0 con 0x7f5fa0066c30 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.830+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa004bed0 con 0x7f5fa0066c30 2024-02-16T23:02:18.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.831+0000 7f5f877fe700 1 -- 172.21.15.64:0/2423260207 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f90013980 con 0x7f5fa0066c30 2024-02-16T23:02:18.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.831+0000 7f5f877fe700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5f8806d7e0 0x7f5f8806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:18.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.832+0000 7f5f877fe700 1 -- 172.21.15.64:0/2423260207 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5f9008e300 con 0x7f5fa0066c30 2024-02-16T23:02:18.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.832+0000 7f5f9e7fc700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5f8806d7e0 0x7f5f8806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:18.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.832+0000 7f5f9e7fc700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5f8806d7e0 0x7f5f8806fc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009bf0 tx=0x7f5f940091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:18.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:18.836+0000 7f5f877fe700 1 -- 172.21.15.64:0/2423260207 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f9005b750 con 0x7f5fa0066c30 2024-02-16T23:02:19.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.058+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fa010f940 con 0x7f5f8806d7e0 2024-02-16T23:02:19.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.060+0000 7f5f877fe700 1 -- 172.21.15.64:0/2423260207 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5fa010f940 con 0x7f5f8806d7e0 2024-02-16T23:02:19.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5f8806d7e0 msgr2=0x7f5f8806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5f8806d7e0 0x7f5f8806fc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009bf0 tx=0x7f5f940091a0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 msgr2=0x7f5fa010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 0x7f5fa010d910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000b3b0 tx=0x7f5f90009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:19.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 shutdown_connections 2024-02-16T23:02:19.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5f8806d7e0 0x7f5f8806fc90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fa0066c30 0x7f5fa010d910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 --2- 172.21.15.64:0/2423260207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa01000c0 0x7f5fa010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 >> 172.21.15.64:0/2423260207 conn(0x7f5fa00f7590 msgr2=0x7f5fa00f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:19.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 shutdown_connections 2024-02-16T23:02:19.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.062+0000 7f5fa5807700 1 -- 172.21.15.64:0/2423260207 wait complete. 2024-02-16T23:02:19.074 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.165+0000 7ff0bf13e700 1 -- 172.21.15.64:0/2413016376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 msgr2=0x7ff0b80fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.165+0000 7ff0bf13e700 1 --2- 172.21.15.64:0/2413016376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 0x7ff0b80fe700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4004940 tx=0x7ff0a4009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.165+0000 7ff0bf13e700 1 -- 172.21.15.64:0/2413016376 shutdown_connections 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.165+0000 7ff0bf13e700 1 --2- 172.21.15.64:0/2413016376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 0x7ff0b80fe700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.165+0000 7ff0bf13e700 1 --2- 172.21.15.64:0/2413016376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b80fb220 0x7ff0b80fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.165+0000 7ff0bf13e700 1 -- 172.21.15.64:0/2413016376 >> 172.21.15.64:0/2413016376 conn(0x7ff0b80f6ca0 msgr2=0x7ff0b80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.166+0000 7ff0bf13e700 1 -- 172.21.15.64:0/2413016376 shutdown_connections 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.166+0000 7ff0bf13e700 1 -- 172.21.15.64:0/2413016376 wait complete. 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.166+0000 7ff0bf13e700 1 Processor -- start 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.166+0000 7ff0bf13e700 1 -- start start 2024-02-16T23:02:19.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.166+0000 7ff0bf13e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b80fb220 0x7ff0b810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.166+0000 7ff0bf13e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 0x7ff0b810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.166+0000 7ff0bf13e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b810e550 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.166+0000 7ff0bf13e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b810edf0 con 0x7ff0b80fb220 2024-02-16T23:02:19.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 0x7ff0b810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 0x7ff0b810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40548/0 (socket says 172.21.15.64:40548) 2024-02-16T23:02:19.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b7fff700 1 -- 172.21.15.64:0/591075051 learned_addr learned my addr 172.21.15.64:0/591075051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:19.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0bceda700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b80fb220 0x7ff0b810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b7fff700 1 -- 172.21.15.64:0/591075051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b80fb220 msgr2=0x7ff0b810da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b7fff700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b80fb220 0x7ff0b810da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b7fff700 1 -- 172.21.15.64:0/591075051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4005040 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b7fff700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 0x7ff0b810df50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400b3b0 tx=0x7ff0a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b57fa700 1 -- 172.21.15.64:0/591075051 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a400e040 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b57fa700 1 -- 172.21.15.64:0/591075051 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0a400bbf0 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b810f070 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0b57fa700 1 -- 172.21.15.64:0/591075051 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a401c630 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.167+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b810f5b0 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.168+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b8107a80 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.169+0000 7ff0b57fa700 1 -- 172.21.15.64:0/591075051 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff0a400bd60 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.169+0000 7ff0b57fa700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff0a806d7e0 0x7ff0a806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.169+0000 7ff0b57fa700 1 -- 172.21.15.64:0/591075051 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff0a408e120 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.169+0000 7ff0bceda700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff0a806d7e0 0x7ff0a806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.170+0000 7ff0bceda700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff0a806d7e0 0x7ff0a806fc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8065cb0 tx=0x7ff0ac00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:19.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.174+0000 7ff0b57fa700 1 -- 172.21.15.64:0/591075051 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0a405b6c0 con 0x7ff0b80fe2b0 2024-02-16T23:02:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.399+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0b810f950 con 0x7ff0a806d7e0 2024-02-16T23:02:19.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.400+0000 7ff0b57fa700 1 -- 172.21.15.64:0/591075051 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff0b810f950 con 0x7ff0a806d7e0 2024-02-16T23:02:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff0a806d7e0 msgr2=0x7ff0a806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff0a806d7e0 0x7ff0a806fc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8065cb0 tx=0x7ff0ac00b040 comp rx=0 tx=0).stop 2024-02-16T23:02:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 msgr2=0x7ff0b810df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 0x7ff0b810df50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400b3b0 tx=0x7ff0a4009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 shutdown_connections 2024-02-16T23:02:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff0a806d7e0 0x7ff0a806fc90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b80fb220 0x7ff0b810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 --2- 172.21.15.64:0/591075051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0b80fe2b0 0x7ff0b810df50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 >> 172.21.15.64:0/591075051 conn(0x7ff0b80f6ca0 msgr2=0x7ff0b80f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 shutdown_connections 2024-02-16T23:02:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.402+0000 7ff0bf13e700 1 -- 172.21.15.64:0/591075051 wait complete. 2024-02-16T23:02:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.505+0000 7f68f5f92700 1 -- 172.21.15.64:0/3362054342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 msgr2=0x7f68f00fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.505+0000 7f68f5f92700 1 --2- 172.21.15.64:0/3362054342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 0x7f68f00fe550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68e0004940 tx=0x7f68e0009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.505+0000 7f68f5f92700 1 -- 172.21.15.64:0/3362054342 shutdown_connections 2024-02-16T23:02:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.505+0000 7f68f5f92700 1 --2- 172.21.15.64:0/3362054342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 0x7f68f00fe550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.505+0000 7f68f5f92700 1 --2- 172.21.15.64:0/3362054342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68f00fcef0 0x7f68f00fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.505+0000 7f68f5f92700 1 -- 172.21.15.64:0/3362054342 >> 172.21.15.64:0/3362054342 conn(0x7f68f00f8620 msgr2=0x7f68f00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.505+0000 7f68f5f92700 1 -- 172.21.15.64:0/3362054342 shutdown_connections 2024-02-16T23:02:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.505+0000 7f68f5f92700 1 -- 172.21.15.64:0/3362054342 wait complete. 2024-02-16T23:02:19.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68f5f92700 1 Processor -- start 2024-02-16T23:02:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68f5f92700 1 -- start start 2024-02-16T23:02:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68f5f92700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68f00fcef0 0x7f68f010fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68f5f92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 0x7f68f01100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68f5f92700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f01106a0 con 0x7f68f00fe0e0 2024-02-16T23:02:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68f5f92700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f0110f40 con 0x7f68f00fcef0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68eeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 0x7f68f01100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68eeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 0x7f68f01100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40566/0 (socket says 172.21.15.64:40566) 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68eeffd700 1 -- 172.21.15.64:0/1601262426 learned_addr learned my addr 172.21.15.64:0/1601262426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68ef7fe700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68f00fcef0 0x7f68f010fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68eeffd700 1 -- 172.21.15.64:0/1601262426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68f00fcef0 msgr2=0x7f68f010fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68eeffd700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68f00fcef0 0x7f68f010fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.506+0000 7f68eeffd700 1 -- 172.21.15.64:0/1601262426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e0005040 con 0x7f68f00fe0e0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.507+0000 7f68eeffd700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 0x7f68f01100a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f68e00093b0 tx=0x7f68e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.507+0000 7f68d7fff700 1 -- 172.21.15.64:0/1601262426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e000e040 con 0x7f68f00fe0e0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.507+0000 7f68d7fff700 1 -- 172.21.15.64:0/1601262426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68e000bcc0 con 0x7f68f00fe0e0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.507+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f01111c0 con 0x7f68f00fe0e0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.507+0000 7f68d7fff700 1 -- 172.21.15.64:0/1601262426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e001c630 con 0x7f68f00fe0e0 2024-02-16T23:02:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.507+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f0111700 con 0x7f68f00fe0e0 2024-02-16T23:02:19.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.508+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f0102210 con 0x7f68f00fe0e0 2024-02-16T23:02:19.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.508+0000 7f68d7fff700 1 -- 172.21.15.64:0/1601262426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f68e0013420 con 0x7f68f00fe0e0 2024-02-16T23:02:19.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.509+0000 7f68d7fff700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f68d806d7e0 0x7f68d806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.509+0000 7f68d7fff700 1 -- 172.21.15.64:0/1601262426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f68e008e100 con 0x7f68f00fe0e0 2024-02-16T23:02:19.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.509+0000 7f68ef7fe700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f68d806d7e0 0x7f68d806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.509+0000 7f68ef7fe700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f68d806d7e0 0x7f68d806fc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f68f00fcc30 tx=0x7f68e400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:19.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.513+0000 7f68d7fff700 1 -- 172.21.15.64:0/1601262426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f68e005b5f0 con 0x7f68f00fe0e0 2024-02-16T23:02:19.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.730+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f68f002b310 con 0x7f68d806d7e0 2024-02-16T23:02:19.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.740+0000 7f68d7fff700 1 -- 172.21.15.64:0/1601262426 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f68f002b310 con 0x7f68d806d7e0 2024-02-16T23:02:19.741 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:02:19.741 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (5m) 37s ago 13m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (2m) 37s ago 14m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (2m) 4s ago 13m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (5m) 37s ago 13m 84.4M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (8m) 37s ago 8m 17.9M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (8m) 37s ago 8m 153M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (8m) 4s ago 8m 17.1M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (8m) 4s ago 8m 16.7M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (4m) 37s ago 15m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (3m) 4s ago 12m 450M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (3m) 37s ago 15m 45.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (3m) 4s ago 12m 39.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (5m) 37s ago 14m 15.6M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (4m) 4s ago 13m 15.7M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (115s) 37s ago 12m 96.7M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:02:19.742 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (79s) 37s ago 11m 83.8M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:02:19.743 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (43s) 37s ago 11m 11.8M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:02:19.743 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (7s) 4s ago 10m 14.9M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:02:19.743 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (10m) 4s ago 10m 212M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 e242755195ca 2024-02-16T23:02:19.743 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (10m) 4s ago 10m 189M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T23:02:19.743 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (3m) 37s ago 13m 59.3M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:02:19.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f68d806d7e0 msgr2=0x7f68d806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f68d806d7e0 0x7f68d806fc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f68f00fcc30 tx=0x7f68e400b040 comp rx=0 tx=0).stop 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 msgr2=0x7f68f01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 0x7f68f01100a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f68e00093b0 tx=0x7f68e0009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 shutdown_connections 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f68d806d7e0 0x7f68d806fc90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68f00fcef0 0x7f68f010fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 --2- 172.21.15.64:0/1601262426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68f00fe0e0 0x7f68f01100a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 >> 172.21.15.64:0/1601262426 conn(0x7f68f00f8620 msgr2=0x7f68f00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 shutdown_connections 2024-02-16T23:02:19.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.742+0000 7f68f5f92700 1 -- 172.21.15.64:0/1601262426 wait complete. 2024-02-16T23:02:19.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.844+0000 7fcb64632700 1 -- 172.21.15.64:0/2502047023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 msgr2=0x7fcb5c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.844+0000 7fcb64632700 1 --2- 172.21.15.64:0/2502047023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 0x7fcb5c102780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb50004940 tx=0x7fcb50009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:19.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.844+0000 7fcb64632700 1 -- 172.21.15.64:0/2502047023 shutdown_connections 2024-02-16T23:02:19.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.844+0000 7fcb64632700 1 --2- 172.21.15.64:0/2502047023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 0x7fcb5c102780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.844+0000 7fcb64632700 1 --2- 172.21.15.64:0/2502047023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0fda80 0x7fcb5c0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.844+0000 7fcb64632700 1 -- 172.21.15.64:0/2502047023 >> 172.21.15.64:0/2502047023 conn(0x7fcb5c0f74e0 msgr2=0x7fcb5c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:19.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.844+0000 7fcb64632700 1 -- 172.21.15.64:0/2502047023 shutdown_connections 2024-02-16T23:02:19.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.844+0000 7fcb64632700 1 -- 172.21.15.64:0/2502047023 wait complete. 2024-02-16T23:02:19.846 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/261 objects degraded (22.605%) 2024-02-16T23:02:19.846 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T23:02:19.846 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: from='osd.3 [v2:172.21.15.192:6800/2367052696,v1:172.21.15.192:6801/2367052696]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:02:19.846 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: osdmap e68: 6 total, 5 up, 6 in 2024-02-16T23:02:19.846 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:02:19.846 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:19.847 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: Health check update: Degraded data redundancy: 59/261 objects degraded (22.605%), 18 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:19.847 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: failed to encode map e68 with expected crc 2024-02-16T23:02:19.847 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:19.847 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:19 smithi064 ceph-mon[93757]: from='osd.3 ' entity='osd.3' 2024-02-16T23:02:19.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb64632700 1 Processor -- start 2024-02-16T23:02:19.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb64632700 1 -- start start 2024-02-16T23:02:19.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb64632700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0fda80 0x7fcb5c10d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb64632700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 0x7fcb5c10de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb64632700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c10e420 con 0x7fcb5c1003a0 2024-02-16T23:02:19.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb64632700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c10ecc0 con 0x7fcb5c0fda80 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb61bcd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 0x7fcb5c10de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb61bcd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 0x7fcb5c10de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40580/0 (socket says 172.21.15.64:40580) 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb61bcd700 1 -- 172.21.15.64:0/2015665572 learned_addr learned my addr 172.21.15.64:0/2015665572 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb623ce700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0fda80 0x7fcb5c10d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb61bcd700 1 -- 172.21.15.64:0/2015665572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0fda80 msgr2=0x7fcb5c10d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb61bcd700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0fda80 0x7fcb5c10d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.845+0000 7fcb61bcd700 1 -- 172.21.15.64:0/2015665572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50005040 con 0x7fcb5c1003a0 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.846+0000 7fcb61bcd700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 0x7fcb5c10de20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.846+0000 7fcb5affd700 1 -- 172.21.15.64:0/2015665572 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb50021070 con 0x7fcb5c1003a0 2024-02-16T23:02:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.846+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c10ef40 con 0x7fcb5c1003a0 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.846+0000 7fcb5affd700 1 -- 172.21.15.64:0/2015665572 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb5001c400 con 0x7fcb5c1003a0 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.846+0000 7fcb5affd700 1 -- 172.21.15.64:0/2015665572 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb5000e070 con 0x7fcb5c1003a0 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.846+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c10f420 con 0x7fcb5c1003a0 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.847+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c107a50 con 0x7fcb5c1003a0 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.847+0000 7fcb5affd700 1 -- 172.21.15.64:0/2015665572 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcb5001ca50 con 0x7fcb5c1003a0 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.848+0000 7fcb5affd700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcb4406d7e0 0x7fcb4406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.848+0000 7fcb5affd700 1 -- 172.21.15.64:0/2015665572 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcb5008df80 con 0x7fcb5c1003a0 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.848+0000 7fcb623ce700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcb4406d7e0 0x7fcb4406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.848+0000 7fcb623ce700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcb4406d7e0 0x7fcb4406fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb54001470 tx=0x7fcb5400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:19.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:19.853+0000 7fcb5affd700 1 -- 172.21.15.64:0/2015665572 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb5005b470 con 0x7fcb5c1003a0 2024-02-16T23:02:19.931 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[92529]: 2024-02-16T23:02:19.626+0000 7fde75f7c700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T23:02:19.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 251 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/261 objects degraded (22.605%) 2024-02-16T23:02:19.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T23:02:19.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: from='osd.3 [v2:172.21.15.192:6800/2367052696,v1:172.21.15.192:6801/2367052696]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:02:19.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: osdmap e68: 6 total, 5 up, 6 in 2024-02-16T23:02:19.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:02:19.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:19.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: Health check update: Degraded data redundancy: 59/261 objects degraded (22.605%), 18 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:19.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: failed to encode map e68 with expected crc 2024-02-16T23:02:19.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:19.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:19 smithi192 ceph-mon[82652]: from='osd.3 ' entity='osd.3' 2024-02-16T23:02:20.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.139+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb5c04bed0 con 0x7fcb5c1003a0 2024-02-16T23:02:20.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.140+0000 7fcb5affd700 1 -- 172.21.15.64:0/2015665572 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fcb5005b000 con 0x7fcb5c1003a0 2024-02-16T23:02:20.140 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:02:20.140 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:02:20.140 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2, 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6, 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 7 2024-02-16T23:02:20.141 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:02:20.142 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcb4406d7e0 msgr2=0x7fcb4406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcb4406d7e0 0x7fcb4406fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb54001470 tx=0x7fcb5400b040 comp rx=0 tx=0).stop 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 msgr2=0x7fcb5c10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 0x7fcb5c10de20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb50009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 shutdown_connections 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcb4406d7e0 0x7fcb4406fc90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0fda80 0x7fcb5c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 --2- 172.21.15.64:0/2015665572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c1003a0 0x7fcb5c10de20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 >> 172.21.15.64:0/2015665572 conn(0x7fcb5c0f74e0 msgr2=0x7fcb5c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 shutdown_connections 2024-02-16T23:02:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.142+0000 7fcb64632700 1 -- 172.21.15.64:0/2015665572 wait complete. 2024-02-16T23:02:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.244+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/3248810899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 msgr2=0x7f2d640fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.244+0000 7f2d6c4f8700 1 --2- 172.21.15.64:0/3248810899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 0x7f2d640fc260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d60004940 tx=0x7f2d60009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/3248810899 shutdown_connections 2024-02-16T23:02:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 --2- 172.21.15.64:0/3248810899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d640fcfc0 0x7f2d640fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 --2- 172.21.15.64:0/3248810899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 0x7f2d640fc260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/3248810899 >> 172.21.15.64:0/3248810899 conn(0x7f2d640f7510 msgr2=0x7f2d640f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/3248810899 shutdown_connections 2024-02-16T23:02:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/3248810899 wait complete. 2024-02-16T23:02:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 Processor -- start 2024-02-16T23:02:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 -- start start 2024-02-16T23:02:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 0x7f2d6410c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d640fcfc0 0x7f2d6410cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6410d320 con 0x7f2d640fbe50 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6c4f8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6410dbc0 con 0x7f2d640fcfc0 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6a294700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 0x7f2d6410c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.245+0000 7f2d6a294700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 0x7f2d6410c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40590/0 (socket says 172.21.15.64:40590) 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d6a294700 1 -- 172.21.15.64:0/1242147295 learned_addr learned my addr 172.21.15.64:0/1242147295 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d69a93700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d640fcfc0 0x7f2d6410cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d6a294700 1 -- 172.21.15.64:0/1242147295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d640fcfc0 msgr2=0x7f2d6410cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d6a294700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d640fcfc0 0x7f2d6410cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d6a294700 1 -- 172.21.15.64:0/1242147295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d60005040 con 0x7f2d640fbe50 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d6a294700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 0x7f2d6410c7e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d60000c00 tx=0x7f2d60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d5affd700 1 -- 172.21.15.64:0/1242147295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d6000e070 con 0x7f2d640fbe50 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d5affd700 1 -- 172.21.15.64:0/1242147295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d60013d00 con 0x7f2d640fbe50 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6410de40 con 0x7f2d640fbe50 2024-02-16T23:02:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d5affd700 1 -- 172.21.15.64:0/1242147295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d60023650 con 0x7f2d640fbe50 2024-02-16T23:02:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.246+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6410e320 con 0x7f2d640fbe50 2024-02-16T23:02:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.247+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6404bed0 con 0x7f2d640fbe50 2024-02-16T23:02:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.248+0000 7f2d5affd700 1 -- 172.21.15.64:0/1242147295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2d6001aa50 con 0x7f2d640fbe50 2024-02-16T23:02:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.248+0000 7f2d5affd700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2d5006d7e0 0x7f2d5006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.248+0000 7f2d5affd700 1 -- 172.21.15.64:0/1242147295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2d600906a0 con 0x7f2d640fbe50 2024-02-16T23:02:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.248+0000 7f2d69a93700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2d5006d7e0 0x7f2d5006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.249+0000 7f2d69a93700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2d5006d7e0 0x7f2d5006fc90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009b80 tx=0x7f2d5400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.253+0000 7f2d5affd700 1 -- 172.21.15.64:0/1242147295 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d6005db90 con 0x7f2d640fbe50 2024-02-16T23:02:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.503+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2d6410ea10 con 0x7f2d640fbe50 2024-02-16T23:02:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.503+0000 7f2d5affd700 1 -- 172.21.15.64:0/1242147295 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f2d6005d720 con 0x7f2d640fbe50 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:02:20.504 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:02:20.505 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:02:20.506 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:02:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2d5006d7e0 msgr2=0x7f2d5006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2d5006d7e0 0x7f2d5006fc90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009b80 tx=0x7f2d5400b040 comp rx=0 tx=0).stop 2024-02-16T23:02:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 msgr2=0x7f2d6410c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 0x7f2d6410c7e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d60000c00 tx=0x7f2d60009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 shutdown_connections 2024-02-16T23:02:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2d5006d7e0 0x7f2d5006fc90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d640fbe50 0x7f2d6410c7e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 --2- 172.21.15.64:0/1242147295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d640fcfc0 0x7f2d6410cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 >> 172.21.15.64:0/1242147295 conn(0x7f2d640f7510 msgr2=0x7f2d64100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.505+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 shutdown_connections 2024-02-16T23:02:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.506+0000 7f2d6c4f8700 1 -- 172.21.15.64:0/1242147295 wait complete. 2024-02-16T23:02:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T23:02:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.608+0000 7f864814e700 1 -- 172.21.15.64:0/2412588434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 msgr2=0x7f86400ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.608+0000 7f864814e700 1 --2- 172.21.15.64:0/2412588434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 0x7f86400ffe60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f863c004940 tx=0x7f863c009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.608+0000 7f864814e700 1 -- 172.21.15.64:0/2412588434 shutdown_connections 2024-02-16T23:02:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.608+0000 7f864814e700 1 --2- 172.21.15.64:0/2412588434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86401003a0 0x7f8640102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.608+0000 7f864814e700 1 --2- 172.21.15.64:0/2412588434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 0x7f86400ffe60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.608+0000 7f864814e700 1 -- 172.21.15.64:0/2412588434 >> 172.21.15.64:0/2412588434 conn(0x7f86400f7520 msgr2=0x7f86400f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:20.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f864814e700 1 -- 172.21.15.64:0/2412588434 shutdown_connections 2024-02-16T23:02:20.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f864814e700 1 -- 172.21.15.64:0/2412588434 wait complete. 2024-02-16T23:02:20.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f864814e700 1 Processor -- start 2024-02-16T23:02:20.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f864814e700 1 -- start start 2024-02-16T23:02:20.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f864814e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 0x7f86400fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f864814e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86401003a0 0x7f86400fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f864814e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400fc020 con 0x7f86400fda80 2024-02-16T23:02:20.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f864814e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400fc190 con 0x7f86401003a0 2024-02-16T23:02:20.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f8645eea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 0x7f86400fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f8645eea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 0x7f86400fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51282/0 (socket says 172.21.15.64:51282) 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f8645eea700 1 -- 172.21.15.64:0/1042324169 learned_addr learned my addr 172.21.15.64:0/1042324169 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.609+0000 7f86456e9700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86401003a0 0x7f86400fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f8645eea700 1 -- 172.21.15.64:0/1042324169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86401003a0 msgr2=0x7f86400fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f8645eea700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86401003a0 0x7f86400fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f8645eea700 1 -- 172.21.15.64:0/1042324169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c005040 con 0x7f86400fda80 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f8645eea700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 0x7f86400fd510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f863c009fd0 tx=0x7f863c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f86367fc700 1 -- 172.21.15.64:0/1042324169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f863c00e040 con 0x7f86400fda80 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f86367fc700 1 -- 172.21.15.64:0/1042324169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f863c00bcc0 con 0x7f86400fda80 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86400fc410 con 0x7f86400fda80 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f86367fc700 1 -- 172.21.15.64:0/1042324169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f863c01c630 con 0x7f86400fda80 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.610+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8640115960 con 0x7f86400fda80 2024-02-16T23:02:20.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.611+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8640107ad0 con 0x7f86400fda80 2024-02-16T23:02:20.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.611+0000 7f86367fc700 1 -- 172.21.15.64:0/1042324169 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f863c013980 con 0x7f86400fda80 2024-02-16T23:02:20.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.612+0000 7f86367fc700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f862c06d7f0 0x7f862c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.612+0000 7f86367fc700 1 -- 172.21.15.64:0/1042324169 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f863c08e330 con 0x7f86400fda80 2024-02-16T23:02:20.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.612+0000 7f86456e9700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f862c06d7f0 0x7f862c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.612+0000 7f86456e9700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f862c06d7f0 0x7f862c06fca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8630006fd0 tx=0x7f8630008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:20.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.617+0000 7f86367fc700 1 -- 172.21.15.64:0/1042324169 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f863c05b780 con 0x7f86400fda80 2024-02-16T23:02:20.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.848+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f864005e3d0 con 0x7f862c06d7f0 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.850+0000 7f86367fc700 1 -- 172.21.15.64:0/1042324169 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f864005e3d0 con 0x7f862c06d7f0 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:02:20.850 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "10/21 daemons upgraded", 2024-02-16T23:02:20.851 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T23:02:20.851 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:02:20.851 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.851+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f862c06d7f0 msgr2=0x7f862c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.851+0000 7f864814e700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f862c06d7f0 0x7f862c06fca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8630006fd0 tx=0x7f8630008040 comp rx=0 tx=0).stop 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.851+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 msgr2=0x7f86400fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.851+0000 7f864814e700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 0x7f86400fd510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f863c009fd0 tx=0x7f863c009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.852+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 shutdown_connections 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.852+0000 7f864814e700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f862c06d7f0 0x7f862c06fca0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.852+0000 7f864814e700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86400fda80 0x7f86400fd510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.852+0000 7f864814e700 1 --2- 172.21.15.64:0/1042324169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86401003a0 0x7f86400fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.852+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 >> 172.21.15.64:0/1042324169 conn(0x7f86400f7520 msgr2=0x7f86400f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.852+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 shutdown_connections 2024-02-16T23:02:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.852+0000 7f864814e700 1 -- 172.21.15.64:0/1042324169 wait complete. 2024-02-16T23:02:20.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 -- 172.21.15.64:0/638471761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fcfb0 msgr2=0x7fb1c00fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 --2- 172.21.15.64:0/638471761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fcfb0 0x7fb1c00fd420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc004940 tx=0x7fb1bc009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:20.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 -- 172.21.15.64:0/638471761 shutdown_connections 2024-02-16T23:02:20.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 --2- 172.21.15.64:0/638471761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fcfb0 0x7fb1c00fd420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 --2- 172.21.15.64:0/638471761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c00fbe40 0x7fb1c00fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 -- 172.21.15.64:0/638471761 >> 172.21.15.64:0/638471761 conn(0x7fb1c00f7520 msgr2=0x7fb1c00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:20.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 -- 172.21.15.64:0/638471761 shutdown_connections 2024-02-16T23:02:20.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 -- 172.21.15.64:0/638471761 wait complete. 2024-02-16T23:02:20.955 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:20.955 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2015665572' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:20.955 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:20.955 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1242147295' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: osd.3 [v2:172.21.15.192:6800/2367052696,v1:172.21.15.192:6801/2367052696] boot 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: osdmap e69: 6 total, 6 up, 6 in 2024-02-16T23:02:20.956 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T23:02:20.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 Processor -- start 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.954+0000 7fb1c761f700 1 -- start start 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c761f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fbe40 0x7fb1c0110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c761f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c00fcfb0 0x7fb1c0112b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c761f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0066d50 con 0x7fb1c00fbe40 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c761f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0066e90 con 0x7fb1c00fcfb0 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c53bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fbe40 0x7fb1c0110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c53bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fbe40 0x7fb1c0110600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51298/0 (socket says 172.21.15.64:51298) 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c53bb700 1 -- 172.21.15.64:0/2467895346 learned_addr learned my addr 172.21.15.64:0/2467895346 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c4bba700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c00fcfb0 0x7fb1c0112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c53bb700 1 -- 172.21.15.64:0/2467895346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c00fcfb0 msgr2=0x7fb1c0112b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c53bb700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c00fcfb0 0x7fb1c0112b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c53bb700 1 -- 172.21.15.64:0/2467895346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc005040 con 0x7fb1c00fbe40 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c4bba700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c00fcfb0 0x7fb1c0112b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:02:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c53bb700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fbe40 0x7fb1c0110600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0002a80 tx=0x7fb1b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1b5ffb700 1 -- 172.21.15.64:0/2467895346 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b0007cc0 con 0x7fb1c00fbe40 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1b5ffb700 1 -- 172.21.15.64:0/2467895346 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1b000f040 con 0x7fb1c00fbe40 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1b5ffb700 1 -- 172.21.15.64:0/2467895346 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b0009640 con 0x7fb1c00fbe40 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.955+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c01131b0 con 0x7fb1c00fbe40 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.956+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c01136f0 con 0x7fb1c00fbe40 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.957+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c004bed0 con 0x7fb1c00fbe40 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.957+0000 7fb1b5ffb700 1 -- 172.21.15.64:0/2467895346 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb1b00097a0 con 0x7fb1c00fbe40 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.958+0000 7fb1b5ffb700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb1ac06d7e0 0x7fb1ac06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.958+0000 7fb1b5ffb700 1 -- 172.21.15.64:0/2467895346 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb1b008cb80 con 0x7fb1c00fbe40 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.958+0000 7fb1c4bba700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb1ac06d7e0 0x7fb1ac06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.958+0000 7fb1c4bba700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb1ac06d7e0 0x7fb1ac06fc90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc004560 tx=0x7fb1bc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:20.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:20.963+0000 7fb1b5ffb700 1 -- 172.21.15.64:0/2467895346 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb1b00564a0 con 0x7fb1c00fbe40 2024-02-16T23:02:21.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:21.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2015665572' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:21.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1242147295' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: osd.3 [v2:172.21.15.192:6800/2367052696,v1:172.21.15.192:6801/2367052696] boot 2024-02-16T23:02:21.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: osdmap e69: 6 total, 6 up, 6 in 2024-02-16T23:02:21.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T23:02:21.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.250+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb1c0113bf0 con 0x7fb1c00fbe40 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.250+0000 7fb1b5ffb700 1 -- 172.21.15.64:0/2467895346 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1372 (secure 0 0 0) 0x7fb1b0059b60 con 0x7fb1c00fbe40 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 59/261 objects degraded (22.605%), 18 pgs degraded 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 59/261 objects degraded (22.605%), 18 pgs degraded 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-02-16T23:02:21.251 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-02-16T23:02:21.252 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-02-16T23:02:21.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb1ac06d7e0 msgr2=0x7fb1ac06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb1ac06d7e0 0x7fb1ac06fc90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc004560 tx=0x7fb1bc009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fbe40 msgr2=0x7fb1c0110600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fbe40 0x7fb1c0110600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0002a80 tx=0x7fb1b00046b0 comp rx=0 tx=0).stop 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 shutdown_connections 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb1ac06d7e0 0x7fb1ac06fc90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c00fbe40 0x7fb1c0110600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 --2- 172.21.15.64:0/2467895346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c00fcfb0 0x7fb1c0112b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 >> 172.21.15.64:0/2467895346 conn(0x7fb1c00f7520 msgr2=0x7fb1c0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 shutdown_connections 2024-02-16T23:02:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:21.253+0000 7fb1c761f700 1 -- 172.21.15.64:0/2467895346 wait complete. 2024-02-16T23:02:22.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:21 smithi192 ceph-mon[82652]: failed to encode map e69 with expected crc 2024-02-16T23:02:22.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:21 smithi192 ceph-mon[82652]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/261 objects degraded (22.605%) 2024-02-16T23:02:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:21 smithi192 ceph-mon[82652]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:21 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2467895346' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:02:22.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:21 smithi192 ceph-mon[82652]: osdmap e70: 6 total, 6 up, 6 in 2024-02-16T23:02:22.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:21 smithi064 ceph-mon[93757]: failed to encode map e69 with expected crc 2024-02-16T23:02:22.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:21 smithi064 ceph-mon[93757]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/261 objects degraded (22.605%) 2024-02-16T23:02:22.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:21 smithi064 ceph-mon[93757]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:22.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:21 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2467895346' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:02:22.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:21 smithi064 ceph-mon[93757]: osdmap e70: 6 total, 6 up, 6 in 2024-02-16T23:02:23.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:22 smithi192 ceph-mon[82652]: failed to encode map e70 with expected crc 2024-02-16T23:02:23.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:22 smithi192 ceph-mon[82652]: failed to encode map e70 with expected crc 2024-02-16T23:02:23.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:22 smithi192 ceph-mon[82652]: failed to encode map e70 with expected crc 2024-02-16T23:02:23.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:22 smithi064 ceph-mon[93757]: failed to encode map e70 with expected crc 2024-02-16T23:02:23.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:22 smithi064 ceph-mon[93757]: failed to encode map e70 with expected crc 2024-02-16T23:02:23.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:22 smithi064 ceph-mon[93757]: failed to encode map e70 with expected crc 2024-02-16T23:02:24.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:23 smithi192 ceph-mon[82652]: pgmap v123: 65 pgs: 4 peering, 11 active+undersized+degraded, 13 active+undersized, 37 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 38/261 objects degraded (14.559%) 2024-02-16T23:02:24.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:23 smithi064 ceph-mon[93757]: pgmap v123: 65 pgs: 4 peering, 11 active+undersized+degraded, 13 active+undersized, 37 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 38/261 objects degraded (14.559%) 2024-02-16T23:02:25.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:24 smithi192 ceph-mon[82652]: Health check update: Degraded data redundancy: 38/261 objects degraded (14.559%), 11 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:25.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:24 smithi064 ceph-mon[93757]: Health check update: Degraded data redundancy: 38/261 objects degraded (14.559%), 11 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:26.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:25 smithi192 ceph-mon[82652]: pgmap v124: 65 pgs: 4 peering, 61 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:26.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:25 smithi192 ceph-mon[82652]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 38/261 objects degraded (14.559%), 11 pgs degraded) 2024-02-16T23:02:26.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:25 smithi064 ceph-mon[93757]: pgmap v124: 65 pgs: 4 peering, 61 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:26.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:25 smithi064 ceph-mon[93757]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 38/261 objects degraded (14.559%), 11 pgs degraded) 2024-02-16T23:02:28.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:27 smithi192 ceph-mon[82652]: pgmap v125: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:28.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:27 smithi064 ceph-mon[93757]: pgmap v125: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:30.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:29 smithi192 ceph-mon[82652]: pgmap v126: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:30.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:29 smithi064 ceph-mon[93757]: pgmap v126: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:31 smithi192 ceph-mon[82652]: pgmap v127: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:32.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:31 smithi064 ceph-mon[93757]: pgmap v127: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:33 smithi192 ceph-mon[82652]: pgmap v128: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:02:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:33 smithi064 ceph-mon[93757]: pgmap v128: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:02:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:36.015 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:35 smithi192 ceph-mon[82652]: pgmap v129: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:36.015 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T23:02:36.015 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:35 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T23:02:36.015 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:35 smithi192 ceph-mon[82652]: Upgrade: osd.4 is safe to restart 2024-02-16T23:02:36.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:35 smithi064 ceph-mon[93757]: pgmap v129: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:36.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T23:02:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:35 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T23:02:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:35 smithi064 ceph-mon[93757]: Upgrade: osd.4 is safe to restart 2024-02-16T23:02:38.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:38 smithi192 ceph-mon[82652]: pgmap v130: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:38.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:38 smithi192 ceph-mon[82652]: Upgrade: Updating osd.4 2024-02-16T23:02:38.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:38 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:38.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:38 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T23:02:38.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:38 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:02:38.254 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:38 smithi192 ceph-mon[82652]: Deploying daemon osd.4 on smithi192 2024-02-16T23:02:38.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:38 smithi064 ceph-mon[93757]: pgmap v130: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:38 smithi064 ceph-mon[93757]: Upgrade: Updating osd.4 2024-02-16T23:02:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:38 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:38 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T23:02:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:38 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:02:38.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:38 smithi064 ceph-mon[93757]: Deploying daemon osd.4 on smithi192 2024-02-16T23:02:40.273 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:40 smithi192 ceph-mon[82652]: pgmap v131: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:40.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:40 smithi064 ceph-mon[93757]: pgmap v131: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:40.524 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:40 smithi192 systemd[1]: Stopping Ceph osd.4 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:02:40.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:40 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[48675]: 2024-02-16T23:02:40.523+0000 7f5ae55b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:02:40.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:40 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[48675]: 2024-02-16T23:02:40.523+0000 7f5ae55b7700 -1 osd.4 70 *** Got signal Terminated *** 2024-02-16T23:02:40.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:40 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[48675]: 2024-02-16T23:02:40.523+0000 7f5ae55b7700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:02:41.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:41 smithi192 ceph-mon[82652]: osd.4 marked itself down and dead 2024-02-16T23:02:41.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:41 smithi064 ceph-mon[93757]: osd.4 marked itself down and dead 2024-02-16T23:02:41.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:41 smithi192 podman[96945]: 2024-02-16 23:02:41.454088737 +0000 UTC m=+1.034440204 container died e242755195ca16bb380226360c516bbafd4ea69451c09c6e5f332aa03b3ac894 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=quincy-b88fae6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, GIT_CLEAN=True) 2024-02-16T23:02:42.192 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:41 smithi192 podman[96945]: 2024-02-16 23:02:41.899596972 +0000 UTC m=+1.479948437 container cleanup e242755195ca16bb380226360c516bbafd4ea69451c09c6e5f332aa03b3ac894 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=quincy-b88fae6) 2024-02-16T23:02:42.192 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:41 smithi192 bash[96945]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4 2024-02-16T23:02:42.192 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:42 smithi192 ceph-mon[82652]: pgmap v132: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:42.193 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:42 smithi192 ceph-mon[82652]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:02:42.193 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:42 smithi192 ceph-mon[82652]: failed to encode map e71 with expected crc 2024-02-16T23:02:42.193 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:42 smithi192 ceph-mon[82652]: osdmap e71: 6 total, 5 up, 6 in 2024-02-16T23:02:42.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:42 smithi064 ceph-mon[93757]: pgmap v132: 65 pgs: 65 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:02:42.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:42 smithi064 ceph-mon[93757]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:02:42.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:42 smithi064 ceph-mon[93757]: failed to encode map e71 with expected crc 2024-02-16T23:02:42.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:42 smithi064 ceph-mon[93757]: osdmap e71: 6 total, 5 up, 6 in 2024-02-16T23:02:42.517 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:42 smithi192 podman[96964]: 2024-02-16 23:02:42.19304774 +0000 UTC m=+0.742204096 container remove e242755195ca16bb380226360c516bbafd4ea69451c09c6e5f332aa03b3ac894 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=quincy-b88fae6) 2024-02-16T23:02:42.779 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:42 smithi192 podman[97059]: 2024-02-16 23:02:42.421318758 +0000 UTC m=+0.018124135 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:02:42.779 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:42 smithi192 podman[97059]: 2024-02-16 23:02:42.519121397 +0000 UTC m=+0.115926803 container create f2394ee94b7e13aec5a75cf636296cde9dfaa2f36b41f71678245e6f67674aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:02:42.779 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:42 smithi192 podman[97059]: 2024-02-16 23:02:42.779974863 +0000 UTC m=+0.376780227 container init f2394ee94b7e13aec5a75cf636296cde9dfaa2f36b41f71678245e6f67674aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:02:43.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:43 smithi192 ceph-mon[82652]: failed to encode map e72 with expected crc 2024-02-16T23:02:43.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:43 smithi192 ceph-mon[82652]: osdmap e72: 6 total, 5 up, 6 in 2024-02-16T23:02:43.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:43 smithi192 ceph-mon[82652]: failed to encode map e72 with expected crc 2024-02-16T23:02:43.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:43 smithi192 ceph-mon[82652]: failed to encode map e72 with expected crc 2024-02-16T23:02:43.035 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:42 smithi192 podman[97059]: 2024-02-16 23:02:42.783222826 +0000 UTC m=+0.380028201 container start f2394ee94b7e13aec5a75cf636296cde9dfaa2f36b41f71678245e6f67674aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-16T23:02:43.035 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:42 smithi192 podman[97059]: 2024-02-16 23:02:42.855009889 +0000 UTC m=+0.451815273 container attach f2394ee94b7e13aec5a75cf636296cde9dfaa2f36b41f71678245e6f67674aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True) 2024-02-16T23:02:43.358 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:43 smithi192 podman[97059]: 2024-02-16 23:02:43.072220004 +0000 UTC m=+0.669025359 container died f2394ee94b7e13aec5a75cf636296cde9dfaa2f36b41f71678245e6f67674aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-16T23:02:43.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:43 smithi064 ceph-mon[93757]: failed to encode map e72 with expected crc 2024-02-16T23:02:43.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:43 smithi064 ceph-mon[93757]: osdmap e72: 6 total, 5 up, 6 in 2024-02-16T23:02:43.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:43 smithi064 ceph-mon[93757]: failed to encode map e72 with expected crc 2024-02-16T23:02:43.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:43 smithi064 ceph-mon[93757]: failed to encode map e72 with expected crc 2024-02-16T23:02:44.025 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:43 smithi192 podman[97085]: 2024-02-16 23:02:43.65927225 +0000 UTC m=+0.577304627 container remove f2394ee94b7e13aec5a75cf636296cde9dfaa2f36b41f71678245e6f67674aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-16T23:02:44.026 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:43 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.4.service: Deactivated successfully. 2024-02-16T23:02:44.026 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:43 smithi192 systemd[1]: Stopped Ceph osd.4 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:02:44.026 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:43 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.4.service: Consumed 50.894s CPU time. 2024-02-16T23:02:44.026 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:43 smithi192 systemd[1]: Starting Ceph osd.4 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:02:44.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-mon[82652]: pgmap v135: 65 pgs: 4 active+undersized+degraded, 10 peering, 2 stale+active+clean, 4 active+undersized, 45 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 7/261 objects degraded (2.682%) 2024-02-16T23:02:44.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-mon[82652]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-16T23:02:44.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-mon[82652]: Health check failed: Degraded data redundancy: 7/261 objects degraded (2.682%), 4 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:44.284 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 podman[97191]: 2024-02-16 23:02:44.013421016 +0000 UTC m=+0.019046118 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:02:44.285 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 podman[97191]: 2024-02-16 23:02:44.144173393 +0000 UTC m=+0.149798491 container create d56e43ce037ddbe8f921d8a0bba9849ba4850b3f457facfec8d48f4de66f1c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-16T23:02:44.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:44 smithi064 ceph-mon[93757]: pgmap v135: 65 pgs: 4 active+undersized+degraded, 10 peering, 2 stale+active+clean, 4 active+undersized, 45 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 7/261 objects degraded (2.682%) 2024-02-16T23:02:44.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:44 smithi064 ceph-mon[93757]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-16T23:02:44.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:44 smithi064 ceph-mon[93757]: Health check failed: Degraded data redundancy: 7/261 objects degraded (2.682%), 4 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:44.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 podman[97191]: 2024-02-16 23:02:44.405091652 +0000 UTC m=+0.410716751 container init d56e43ce037ddbe8f921d8a0bba9849ba4850b3f457facfec8d48f4de66f1c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-16T23:02:44.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 podman[97191]: 2024-02-16 23:02:44.408239466 +0000 UTC m=+0.413864564 container start d56e43ce037ddbe8f921d8a0bba9849ba4850b3f457facfec8d48f4de66f1c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-16T23:02:44.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 podman[97191]: 2024-02-16 23:02:44.505054601 +0000 UTC m=+0.510679714 container attach d56e43ce037ddbe8f921d8a0bba9849ba4850b3f457facfec8d48f4de66f1c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-16T23:02:45.174 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate[97207]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T23:02:45.174 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate[97207]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 bash[97191]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 bash[97191]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 bash[97191]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 bash[97191]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 bash[97191]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 bash[97191]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 bash[97191]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate[97207]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate[97207]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate[97207]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate[97207]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate[97207]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-16T23:02:45.175 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:44 smithi192 podman[97191]: 2024-02-16 23:02:44.873690493 +0000 UTC m=+0.879315592 container died d56e43ce037ddbe8f921d8a0bba9849ba4850b3f457facfec8d48f4de66f1c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-16T23:02:45.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:45 smithi192 podman[97326]: 2024-02-16 23:02:45.467416825 +0000 UTC m=+0.588178940 container remove d56e43ce037ddbe8f921d8a0bba9849ba4850b3f457facfec8d48f4de66f1c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-16T23:02:45.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:45 smithi192 podman[97388]: 2024-02-16 23:02:45.583086039 +0000 UTC m=+0.018237279 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:02:45.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:45 smithi192 podman[97388]: 2024-02-16 23:02:45.702129552 +0000 UTC m=+0.137280814 container create f93a7734ffc482c2df81816227752ae0cb88ab4e063cd4ac0694a7a4ad7ed9b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS) 2024-02-16T23:02:46.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:46 smithi192 ceph-mon[82652]: pgmap v136: 65 pgs: 11 active+undersized+degraded, 10 peering, 13 active+undersized, 31 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36/261 objects degraded (13.793%) 2024-02-16T23:02:46.284 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:45 smithi192 podman[97388]: 2024-02-16 23:02:45.946514542 +0000 UTC m=+0.381665769 container init f93a7734ffc482c2df81816227752ae0cb88ab4e063cd4ac0694a7a4ad7ed9b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-16T23:02:46.284 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:45 smithi192 podman[97388]: 2024-02-16 23:02:45.949562483 +0000 UTC m=+0.384713704 container start f93a7734ffc482c2df81816227752ae0cb88ab4e063cd4ac0694a7a4ad7ed9b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-16T23:02:46.284 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:45 smithi192 ceph-osd[97408]: -- 172.21.15.192:0/3530767496 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d5d2f591e0 con 0x55d5d2f2ac00 2024-02-16T23:02:46.285 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:46 smithi192 bash[97388]: f93a7734ffc482c2df81816227752ae0cb88ab4e063cd4ac0694a7a4ad7ed9b9 2024-02-16T23:02:46.285 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:46 smithi192 systemd[1]: Started Ceph osd.4 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:02:46.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:46 smithi064 ceph-mon[93757]: pgmap v136: 65 pgs: 11 active+undersized+degraded, 10 peering, 13 active+undersized, 31 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36/261 objects degraded (13.793%) 2024-02-16T23:02:46.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:46 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:02:46.493+0000 7f84d5a197c0 -1 Falling back to public interface 2024-02-16T23:02:47.281 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:47.282 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:47.282 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:02:47.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:47.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:47.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:02:48.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:48 smithi064 ceph-mon[93757]: pgmap v137: 65 pgs: 15 active+undersized+degraded, 2 peering, 17 active+undersized, 31 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:48.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:48.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:02:48.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:48 smithi192 ceph-mon[82652]: pgmap v137: 65 pgs: 15 active+undersized+degraded, 2 peering, 17 active+undersized, 31 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:48.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:48.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:02:49.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:49 smithi192 ceph-mon[82652]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-16T23:02:49.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:49 smithi192 ceph-mon[82652]: Health check update: Degraded data redundancy: 45/261 objects degraded (17.241%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:49.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:49 smithi064 ceph-mon[93757]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-16T23:02:49.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:49 smithi064 ceph-mon[93757]: Health check update: Degraded data redundancy: 45/261 objects degraded (17.241%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:50 smithi192 ceph-mon[82652]: pgmap v138: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:50 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:50 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:50.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:50 smithi064 ceph-mon[93757]: pgmap v138: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:50.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:50 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:50.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:50 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:51.211 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:50 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:02:50.997+0000 7f84d5a197c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-02-16T23:02:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.369+0000 7fac42054700 1 -- 172.21.15.64:0/654002152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 msgr2=0x7fac3c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.369+0000 7fac42054700 1 --2- 172.21.15.64:0/654002152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 0x7fac3c101f10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac2c004940 tx=0x7fac2c009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.369+0000 7fac42054700 1 -- 172.21.15.64:0/654002152 shutdown_connections 2024-02-16T23:02:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.369+0000 7fac42054700 1 --2- 172.21.15.64:0/654002152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac3c102450 0x7fac3c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.369+0000 7fac42054700 1 --2- 172.21.15.64:0/654002152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 0x7fac3c101f10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.369+0000 7fac42054700 1 -- 172.21.15.64:0/654002152 >> 172.21.15.64:0/654002152 conn(0x7fac3c0f74e0 msgr2=0x7fac3c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.370+0000 7fac42054700 1 -- 172.21.15.64:0/654002152 shutdown_connections 2024-02-16T23:02:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.370+0000 7fac42054700 1 -- 172.21.15.64:0/654002152 wait complete. 2024-02-16T23:02:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.370+0000 7fac42054700 1 Processor -- start 2024-02-16T23:02:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.370+0000 7fac42054700 1 -- start start 2024-02-16T23:02:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac42054700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 0x7fac3c10fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac42054700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac3c102450 0x7fac3c110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac42054700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c110670 con 0x7fac3c0663c0 2024-02-16T23:02:51.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac42054700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c110f10 con 0x7fac3c102450 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 0x7fac3c10fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 0x7fac3c10fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53898/0 (socket says 172.21.15.64:53898) 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac3b7fe700 1 -- 172.21.15.64:0/4199051894 learned_addr learned my addr 172.21.15.64:0/4199051894 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac3affd700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac3c102450 0x7fac3c110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac3b7fe700 1 -- 172.21.15.64:0/4199051894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac3c102450 msgr2=0x7fac3c110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac3b7fe700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac3c102450 0x7fac3c110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac3b7fe700 1 -- 172.21.15.64:0/4199051894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c005040 con 0x7fac3c0663c0 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac3b7fe700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 0x7fac3c10fb30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac2c000c00 tx=0x7fac2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.371+0000 7fac23fff700 1 -- 172.21.15.64:0/4199051894 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac2c021070 con 0x7fac3c0663c0 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.372+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3c111190 con 0x7fac3c0663c0 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.372+0000 7fac23fff700 1 -- 172.21.15.64:0/4199051894 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac2c01c400 con 0x7fac3c0663c0 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.372+0000 7fac23fff700 1 -- 172.21.15.64:0/4199051894 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac2c00e070 con 0x7fac3c0663c0 2024-02-16T23:02:51.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.372+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c111670 con 0x7fac3c0663c0 2024-02-16T23:02:51.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.373+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3c109ca0 con 0x7fac3c0663c0 2024-02-16T23:02:51.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.373+0000 7fac23fff700 1 -- 172.21.15.64:0/4199051894 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fac2c01ca50 con 0x7fac3c0663c0 2024-02-16T23:02:51.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.373+0000 7fac23fff700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac2406d7e0 0x7fac2406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:51.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.374+0000 7fac23fff700 1 -- 172.21.15.64:0/4199051894 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fac2c08e0e0 con 0x7fac3c0663c0 2024-02-16T23:02:51.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.374+0000 7fac3affd700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac2406d7e0 0x7fac2406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:51.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.374+0000 7fac3affd700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac2406d7e0 0x7fac2406fc90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac30009b80 tx=0x7fac3000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:51.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.378+0000 7fac23fff700 1 -- 172.21.15.64:0/4199051894 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac2c05b530 con 0x7fac3c0663c0 2024-02-16T23:02:51.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:51 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:51.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:51 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:51.534 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:51 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:02:51.348+0000 7f84d5a197c0 -1 osd.4 70 log_to_monitors true 2024-02-16T23:02:51.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.603+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac3c1119a0 con 0x7fac2406d7e0 2024-02-16T23:02:51.604 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:51 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:51.604 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:51 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:51.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.605+0000 7fac23fff700 1 -- 172.21.15.64:0/4199051894 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fac3c1119a0 con 0x7fac2406d7e0 2024-02-16T23:02:51.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac2406d7e0 msgr2=0x7fac2406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:51.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac2406d7e0 0x7fac2406fc90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac30009b80 tx=0x7fac3000b040 comp rx=0 tx=0).stop 2024-02-16T23:02:51.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 msgr2=0x7fac3c10fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:51.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 0x7fac3c10fb30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac2c000c00 tx=0x7fac2c009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:51.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 shutdown_connections 2024-02-16T23:02:51.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac2406d7e0 0x7fac2406fc90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c0663c0 0x7fac3c10fb30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 --2- 172.21.15.64:0/4199051894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac3c102450 0x7fac3c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 >> 172.21.15.64:0/4199051894 conn(0x7fac3c0f74e0 msgr2=0x7fac3c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:51.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 shutdown_connections 2024-02-16T23:02:51.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.607+0000 7fac42054700 1 -- 172.21.15.64:0/4199051894 wait complete. 2024-02-16T23:02:51.618 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:02:51.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.710+0000 7f86cd28b700 1 -- 172.21.15.64:0/2281496025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fb220 msgr2=0x7f86c80fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:51.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.710+0000 7f86cd28b700 1 --2- 172.21.15.64:0/2281496025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fb220 0x7f86c80fb630 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86b8004940 tx=0x7f86b8009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:51.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.710+0000 7f86cd28b700 1 -- 172.21.15.64:0/2281496025 shutdown_connections 2024-02-16T23:02:51.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.710+0000 7f86cd28b700 1 --2- 172.21.15.64:0/2281496025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80fe2b0 0x7f86c80fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.710+0000 7f86cd28b700 1 --2- 172.21.15.64:0/2281496025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fb220 0x7f86c80fb630 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.710+0000 7f86cd28b700 1 -- 172.21.15.64:0/2281496025 >> 172.21.15.64:0/2281496025 conn(0x7f86c80f6ca0 msgr2=0x7f86c80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:51.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.710+0000 7f86cd28b700 1 -- 172.21.15.64:0/2281496025 shutdown_connections 2024-02-16T23:02:51.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.710+0000 7f86cd28b700 1 -- 172.21.15.64:0/2281496025 wait complete. 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86cd28b700 1 Processor -- start 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86cd28b700 1 -- start start 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86cd28b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80fb220 0x7f86c80fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86cd28b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fe2b0 0x7f86c80fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86cd28b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80fc850 con 0x7f86c80fe2b0 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86cd28b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80fc9c0 con 0x7f86c80fb220 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86c67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fe2b0 0x7f86c80fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86c67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fe2b0 0x7f86c80fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53912/0 (socket says 172.21.15.64:53912) 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86c67fc700 1 -- 172.21.15.64:0/3271005543 learned_addr learned my addr 172.21.15.64:0/3271005543 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.711+0000 7f86c6ffd700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80fb220 0x7f86c80fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86c67fc700 1 -- 172.21.15.64:0/3271005543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80fb220 msgr2=0x7f86c80fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86c67fc700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80fb220 0x7f86c80fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86c67fc700 1 -- 172.21.15.64:0/3271005543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b8005040 con 0x7f86c80fe2b0 2024-02-16T23:02:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86c67fc700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fe2b0 0x7f86c80fc1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00b790 tx=0x7f86bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86af7fe700 1 -- 172.21.15.64:0/3271005543 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86bc00f6f0 con 0x7f86c80fe2b0 2024-02-16T23:02:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86af7fe700 1 -- 172.21.15.64:0/3271005543 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86bc00fd20 con 0x7f86c80fe2b0 2024-02-16T23:02:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86af7fe700 1 -- 172.21.15.64:0/3271005543 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86bc0103a0 con 0x7f86c80fe2b0 2024-02-16T23:02:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c80fcca0 con 0x7f86c80fe2b0 2024-02-16T23:02:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.712+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c80fd020 con 0x7f86c80fe2b0 2024-02-16T23:02:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.713+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c804bed0 con 0x7f86c80fe2b0 2024-02-16T23:02:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.713+0000 7f86af7fe700 1 -- 172.21.15.64:0/3271005543 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f86bc011400 con 0x7f86c80fe2b0 2024-02-16T23:02:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.714+0000 7f86af7fe700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f86b006d7e0 0x7f86b006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.714+0000 7f86af7fe700 1 -- 172.21.15.64:0/3271005543 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f86bc08cc70 con 0x7f86c80fe2b0 2024-02-16T23:02:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.714+0000 7f86c6ffd700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f86b006d7e0 0x7f86b006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.714+0000 7f86c6ffd700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f86b006d7e0 0x7f86b006fc90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86b8004890 tx=0x7f86b8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:51.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.719+0000 7f86af7fe700 1 -- 172.21.15.64:0/3271005543 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86bc05a0c0 con 0x7f86c80fe2b0 2024-02-16T23:02:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.946+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86c8102c00 con 0x7f86b006d7e0 2024-02-16T23:02:51.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.947+0000 7f86af7fe700 1 -- 172.21.15.64:0/3271005543 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f86c8102c00 con 0x7f86b006d7e0 2024-02-16T23:02:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f86b006d7e0 msgr2=0x7f86b006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f86b006d7e0 0x7f86b006fc90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86b8004890 tx=0x7f86b8009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fe2b0 msgr2=0x7f86c80fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fe2b0 0x7f86c80fc1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00b790 tx=0x7f86bc009250 comp rx=0 tx=0).stop 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 shutdown_connections 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f86b006d7e0 0x7f86b006fc90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80fb220 0x7f86c80fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 --2- 172.21.15.64:0/3271005543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c80fe2b0 0x7f86c80fc1c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 >> 172.21.15.64:0/3271005543 conn(0x7f86c80f6ca0 msgr2=0x7f86c81014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 shutdown_connections 2024-02-16T23:02:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:51.949+0000 7f86cd28b700 1 -- 172.21.15.64:0/3271005543 wait complete. 2024-02-16T23:02:52.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 -- 172.21.15.64:0/946873053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fe0e0 msgr2=0x7f4a3c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 --2- 172.21.15.64:0/946873053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fe0e0 0x7f4a3c0fe550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a30004940 tx=0x7f4a30009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:52.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 -- 172.21.15.64:0/946873053 shutdown_connections 2024-02-16T23:02:52.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 --2- 172.21.15.64:0/946873053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fe0e0 0x7f4a3c0fe550 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 --2- 172.21.15.64:0/946873053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0fcef0 0x7f4a3c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 -- 172.21.15.64:0/946873053 >> 172.21.15.64:0/946873053 conn(0x7f4a3c0f8620 msgr2=0x7f4a3c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:52.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 -- 172.21.15.64:0/946873053 shutdown_connections 2024-02-16T23:02:52.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 -- 172.21.15.64:0/946873053 wait complete. 2024-02-16T23:02:52.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.052+0000 7f4a428b9700 1 Processor -- start 2024-02-16T23:02:52.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a428b9700 1 -- start start 2024-02-16T23:02:52.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a428b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fcef0 0x7f4a3c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a428b9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0fe0e0 0x7f4a3c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a428b9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0708e0 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a428b9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c070a50 con 0x7f4a3c0fe0e0 2024-02-16T23:02:52.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a3bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fcef0 0x7f4a3c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a3bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fcef0 0x7f4a3c071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53936/0 (socket says 172.21.15.64:53936) 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a3bfff700 1 -- 172.21.15.64:0/1527688233 learned_addr learned my addr 172.21.15.64:0/1527688233 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a3b7fe700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0fe0e0 0x7f4a3c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a3bfff700 1 -- 172.21.15.64:0/1527688233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0fe0e0 msgr2=0x7f4a3c0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a3bfff700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0fe0e0 0x7f4a3c0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a3bfff700 1 -- 172.21.15.64:0/1527688233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30005040 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a3bfff700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fcef0 0x7f4a3c071e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00c8e0 tx=0x7f4a2c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.053+0000 7f4a38ff9700 1 -- 172.21.15.64:0/1527688233 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a2c00a940 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.054+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c070d30 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.054+0000 7f4a38ff9700 1 -- 172.21.15.64:0/1527688233 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a2c00ce80 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.054+0000 7f4a38ff9700 1 -- 172.21.15.64:0/1527688233 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a2c0113a0 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.054+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c1159a0 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.055+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c04bed0 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.055+0000 7f4a38ff9700 1 -- 172.21.15.64:0/1527688233 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4a2c012400 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.055+0000 7f4a38ff9700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4a2406d7e0 0x7f4a2406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.055+0000 7f4a38ff9700 1 -- 172.21.15.64:0/1527688233 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4a2c08ce50 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.056+0000 7f4a3b7fe700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4a2406d7e0 0x7f4a2406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.056+0000 7f4a3b7fe700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4a2406d7e0 0x7f4a2406fc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a30004f90 tx=0x7f4a30009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:52.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.060+0000 7f4a38ff9700 1 -- 172.21.15.64:0/1527688233 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a2c05a340 con 0x7f4a3c0fcef0 2024-02-16T23:02:52.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.273+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a3c102ac0 con 0x7f4a2406d7e0 2024-02-16T23:02:52.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.283+0000 7f4a38ff9700 1 -- 172.21.15.64:0/1527688233 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4a3c102ac0 con 0x7f4a2406d7e0 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (6m) 70s ago 14m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (2m) 70s ago 14m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (2m) 3s ago 13m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (5m) 70s ago 14m 84.4M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (8m) 70s ago 8m 17.9M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (9m) 70s ago 9m 153M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (8m) 3s ago 8m 17.2M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (9m) 3s ago 9m 16.9M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (4m) 70s ago 15m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (4m) 3s ago 13m 450M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (3m) 70s ago 15m 45.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:02:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (3m) 3s ago 13m 38.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (6m) 70s ago 14m 15.6M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 3s ago 13m 15.9M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (2m) 70s ago 12m 96.7M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (111s) 70s ago 12m 83.8M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (76s) 70s ago 11m 11.8M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (40s) 3s ago 11m 105M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (6s) 3s ago 11m 12.7M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (10m) 3s ago 10m 189M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 8ecb480f948a 2024-02-16T23:02:52.285 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (4m) 70s ago 14m 59.3M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4a2406d7e0 msgr2=0x7f4a2406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4a2406d7e0 0x7f4a2406fc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a30004f90 tx=0x7f4a30009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fcef0 msgr2=0x7f4a3c071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fcef0 0x7f4a3c071e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00c8e0 tx=0x7f4a2c0044c0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 shutdown_connections 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4a2406d7e0 0x7f4a2406fc90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a3c0fcef0 0x7f4a3c071e60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 --2- 172.21.15.64:0/1527688233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0fe0e0 0x7f4a3c0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 >> 172.21.15.64:0/1527688233 conn(0x7f4a3c0f8620 msgr2=0x7f4a3c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:52.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 shutdown_connections 2024-02-16T23:02:52.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.285+0000 7f4a428b9700 1 -- 172.21.15.64:0/1527688233 wait complete. 2024-02-16T23:02:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 -- 172.21.15.64:0/2212440210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 msgr2=0x7f06c40fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 --2- 172.21.15.64:0/2212440210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 0x7f06c40fc260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f06b4004940 tx=0x7f06b4009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 -- 172.21.15.64:0/2212440210 shutdown_connections 2024-02-16T23:02:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 --2- 172.21.15.64:0/2212440210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40fcfc0 0x7f06c40fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 --2- 172.21.15.64:0/2212440210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 0x7f06c40fc260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 -- 172.21.15.64:0/2212440210 >> 172.21.15.64:0/2212440210 conn(0x7f06c40f7510 msgr2=0x7f06c40f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 -- 172.21.15.64:0/2212440210 shutdown_connections 2024-02-16T23:02:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 -- 172.21.15.64:0/2212440210 wait complete. 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.387+0000 7f06ca7f1700 1 Processor -- start 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06ca7f1700 1 -- start start 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06ca7f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 0x7f06c40fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06ca7f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40fcfc0 0x7f06c40fe390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06ca7f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40fe990 con 0x7f06c40fbe50 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06ca7f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40feb00 con 0x7f06c40fcfc0 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 0x7f06c40fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 0x7f06c40fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53948/0 (socket says 172.21.15.64:53948) 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06c3fff700 1 -- 172.21.15.64:0/3108441795 learned_addr learned my addr 172.21.15.64:0/3108441795 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06c37fe700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40fcfc0 0x7f06c40fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06c3fff700 1 -- 172.21.15.64:0/3108441795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40fcfc0 msgr2=0x7f06c40fe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06c3fff700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40fcfc0 0x7f06c40fe390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06c3fff700 1 -- 172.21.15.64:0/3108441795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b4005040 con 0x7f06c40fbe50 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.388+0000 7f06c3fff700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 0x7f06c40fff10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06b4000c00 tx=0x7f06b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:52.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.389+0000 7f06c0ff9700 1 -- 172.21.15.64:0/3108441795 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b4021070 con 0x7f06c40fbe50 2024-02-16T23:02:52.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.389+0000 7f06c0ff9700 1 -- 172.21.15.64:0/3108441795 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06b401c400 con 0x7f06c40fbe50 2024-02-16T23:02:52.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.389+0000 7f06c0ff9700 1 -- 172.21.15.64:0/3108441795 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b400e070 con 0x7f06c40fbe50 2024-02-16T23:02:52.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.389+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c40fed80 con 0x7f06c40fbe50 2024-02-16T23:02:52.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.389+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c41158e0 con 0x7f06c40fbe50 2024-02-16T23:02:52.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.390+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c404bed0 con 0x7f06c40fbe50 2024-02-16T23:02:52.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.390+0000 7f06c0ff9700 1 -- 172.21.15.64:0/3108441795 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f06b401ca50 con 0x7f06c40fbe50 2024-02-16T23:02:52.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.390+0000 7f06c0ff9700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f06b006d7e0 0x7f06b006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.391+0000 7f06c0ff9700 1 -- 172.21.15.64:0/3108441795 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f06b408e000 con 0x7f06c40fbe50 2024-02-16T23:02:52.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.391+0000 7f06c37fe700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f06b006d7e0 0x7f06b006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.391+0000 7f06c37fe700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f06b006d7e0 0x7f06b006fc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009b80 tx=0x7f06ac00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:52.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.395+0000 7f06c0ff9700 1 -- 172.21.15.64:0/3108441795 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06b405b450 con 0x7f06c40fbe50 2024-02-16T23:02:52.461 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:52 smithi192 ceph-mon[82652]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:52.461 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:52 smithi192 ceph-mon[82652]: from='osd.4 [v2:172.21.15.192:6808/2408603569,v1:172.21.15.192:6809/2408603569]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T23:02:52.461 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:52 smithi192 ceph-mon[82652]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T23:02:52.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:52 smithi064 ceph-mon[93757]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 251 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:52.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:52 smithi064 ceph-mon[93757]: from='osd.4 [v2:172.21.15.192:6808/2408603569,v1:172.21.15.192:6809/2408603569]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T23:02:52.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:52 smithi064 ceph-mon[93757]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T23:02:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.683+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f06c4115ad0 con 0x7f06c40fbe50 2024-02-16T23:02:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.684+0000 7f06c0ff9700 1 -- 172.21.15.64:0/3108441795 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f06b405afe0 con 0x7f06c40fbe50 2024-02-16T23:02:52.684 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:02:52.684 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:02:52.684 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:02:52.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:02:52.684 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 5, 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 8 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:02:52.685 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:02:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f06b006d7e0 msgr2=0x7f06b006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f06b006d7e0 0x7f06b006fc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009b80 tx=0x7f06ac00b040 comp rx=0 tx=0).stop 2024-02-16T23:02:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 msgr2=0x7f06c40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 0x7f06c40fff10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06b4000c00 tx=0x7f06b4009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 shutdown_connections 2024-02-16T23:02:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f06b006d7e0 0x7f06b006fc90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06c40fbe50 0x7f06c40fff10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 --2- 172.21.15.64:0/3108441795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40fcfc0 0x7f06c40fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 >> 172.21.15.64:0/3108441795 conn(0x7f06c40f7510 msgr2=0x7f06c4065f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 shutdown_connections 2024-02-16T23:02:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.686+0000 7f06ca7f1700 1 -- 172.21.15.64:0/3108441795 wait complete. 2024-02-16T23:02:52.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 -- 172.21.15.64:0/1590254041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fbe80 msgr2=0x7fb5600fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 --2- 172.21.15.64:0/1590254041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fbe80 0x7fb5600fc290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb550004940 tx=0x7fb550009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 -- 172.21.15.64:0/1590254041 shutdown_connections 2024-02-16T23:02:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 --2- 172.21.15.64:0/1590254041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5600fd080 0x7fb5600fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 --2- 172.21.15.64:0/1590254041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fbe80 0x7fb5600fc290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 -- 172.21.15.64:0/1590254041 >> 172.21.15.64:0/1590254041 conn(0x7fb5600f7590 msgr2=0x7fb5600f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 -- 172.21.15.64:0/1590254041 shutdown_connections 2024-02-16T23:02:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 -- 172.21.15.64:0/1590254041 wait complete. 2024-02-16T23:02:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.788+0000 7fb565e26700 1 Processor -- start 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb565e26700 1 -- start start 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb565e26700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5600fbe80 0x7fb56010fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb565e26700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fd080 0x7fb560110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb565e26700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560110670 con 0x7fb5600fd080 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb565e26700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560110f10 con 0x7fb5600fbe80 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb55effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fd080 0x7fb560110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb55effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fd080 0x7fb560110070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53970/0 (socket says 172.21.15.64:53970) 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb55effd700 1 -- 172.21.15.64:0/3195727654 learned_addr learned my addr 172.21.15.64:0/3195727654 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb55f7fe700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5600fbe80 0x7fb56010fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb55effd700 1 -- 172.21.15.64:0/3195727654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5600fbe80 msgr2=0x7fb56010fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb55effd700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5600fbe80 0x7fb56010fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb55effd700 1 -- 172.21.15.64:0/3195727654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550005040 con 0x7fb5600fd080 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.789+0000 7fb55effd700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fd080 0x7fb560110070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb554002a50 tx=0x7fb5540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.790+0000 7fb547fff700 1 -- 172.21.15.64:0/3195727654 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb554007cc0 con 0x7fb5600fd080 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.790+0000 7fb547fff700 1 -- 172.21.15.64:0/3195727654 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb55400f040 con 0x7fb5600fd080 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.790+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5601111f0 con 0x7fb5600fd080 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.790+0000 7fb547fff700 1 -- 172.21.15.64:0/3195727654 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb55400da70 con 0x7fb5600fd080 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.790+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb560111730 con 0x7fb5600fd080 2024-02-16T23:02:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.791+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56004bed0 con 0x7fb5600fd080 2024-02-16T23:02:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.791+0000 7fb547fff700 1 -- 172.21.15.64:0/3195727654 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb55400dc10 con 0x7fb5600fd080 2024-02-16T23:02:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.792+0000 7fb547fff700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb54806d7e0 0x7fb54806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.792+0000 7fb547fff700 1 -- 172.21.15.64:0/3195727654 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb55408c4e0 con 0x7fb5600fd080 2024-02-16T23:02:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.792+0000 7fb55f7fe700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb54806d7e0 0x7fb54806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.792+0000 7fb55f7fe700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb54806d7e0 0x7fb54806fc90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb550004890 tx=0x7fb550009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:52.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:52.796+0000 7fb547fff700 1 -- 172.21.15.64:0/3195727654 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5540598b0 con 0x7fb5600fd080 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.044+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb560104930 con 0x7fb5600fd080 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.044+0000 7fb547fff700 1 -- 172.21.15.64:0/3195727654 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7fb554059440 con 0x7fb5600fd080 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:02:53.045 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:02:53.046 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:02:53.047 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:02:53.048 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:02:53.048 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:02:53.048 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:02:53.048 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:02:53.048 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:02:53.048 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb54806d7e0 msgr2=0x7fb54806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb54806d7e0 0x7fb54806fc90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb550004890 tx=0x7fb550009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fd080 msgr2=0x7fb560110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fd080 0x7fb560110070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb554002a50 tx=0x7fb5540046b0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 shutdown_connections 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb54806d7e0 0x7fb54806fc90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5600fbe80 0x7fb56010fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 --2- 172.21.15.64:0/3195727654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5600fd080 0x7fb560110070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.047+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 >> 172.21.15.64:0/3195727654 conn(0x7fb5600f7590 msgr2=0x7fb560065fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.048+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 shutdown_connections 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.048+0000 7fb565e26700 1 -- 172.21.15.64:0/3195727654 wait complete. 2024-02-16T23:02:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T23:02:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/3776363795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 msgr2=0x7fe3b40ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 --2- 172.21.15.64:0/3776363795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 0x7fe3b40ffe60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4004940 tx=0x7fe3a4009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/3776363795 shutdown_connections 2024-02-16T23:02:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 --2- 172.21.15.64:0/3776363795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3b41003a0 0x7fe3b4102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 --2- 172.21.15.64:0/3776363795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 0x7fe3b40ffe60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/3776363795 >> 172.21.15.64:0/3776363795 conn(0x7fe3b40f7520 msgr2=0x7fe3b40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/3776363795 shutdown_connections 2024-02-16T23:02:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/3776363795 wait complete. 2024-02-16T23:02:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 Processor -- start 2024-02-16T23:02:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.152+0000 7fe3b9ee5700 1 -- start start 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b9ee5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 0x7fe3b410d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b9ee5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3b41003a0 0x7fe3b410dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b9ee5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b410e3a0 con 0x7fe3b40fda80 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b9ee5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b410ec40 con 0x7fe3b41003a0 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 0x7fe3b410d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 0x7fe3b410d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53990/0 (socket says 172.21.15.64:53990) 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b37fe700 1 -- 172.21.15.64:0/2151329437 learned_addr learned my addr 172.21.15.64:0/2151329437 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b2ffd700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3b41003a0 0x7fe3b410dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b37fe700 1 -- 172.21.15.64:0/2151329437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3b41003a0 msgr2=0x7fe3b410dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b37fe700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3b41003a0 0x7fe3b410dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b37fe700 1 -- 172.21.15.64:0/2151329437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a4005040 con 0x7fe3b40fda80 2024-02-16T23:02:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b37fe700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 0x7fe3b410d860 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400b3b0 tx=0x7fe3a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe39bfff700 1 -- 172.21.15.64:0/2151329437 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3a400e040 con 0x7fe3b40fda80 2024-02-16T23:02:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe39bfff700 1 -- 172.21.15.64:0/2151329437 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3a400bd90 con 0x7fe3b40fda80 2024-02-16T23:02:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b410eec0 con 0x7fe3b40fda80 2024-02-16T23:02:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe39bfff700 1 -- 172.21.15.64:0/2151329437 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3a401c630 con 0x7fe3b40fda80 2024-02-16T23:02:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.153+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b410f400 con 0x7fe3b40fda80 2024-02-16T23:02:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.154+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b4107a50 con 0x7fe3b40fda80 2024-02-16T23:02:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.155+0000 7fe39bfff700 1 -- 172.21.15.64:0/2151329437 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe3a4013980 con 0x7fe3b40fda80 2024-02-16T23:02:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.155+0000 7fe39bfff700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe39c06d7f0 0x7fe39c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.156+0000 7fe39bfff700 1 -- 172.21.15.64:0/2151329437 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe3a408e310 con 0x7fe3b40fda80 2024-02-16T23:02:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.156+0000 7fe3b2ffd700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe39c06d7f0 0x7fe39c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.156+0000 7fe3b2ffd700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe39c06d7f0 0x7fe39c06fca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009fd0 tx=0x7fe3a800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:53.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.160+0000 7fe39bfff700 1 -- 172.21.15.64:0/2151329437 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3a405b760 con 0x7fe3b40fda80 2024-02-16T23:02:53.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: failed to encode map e73 with expected crc 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: osdmap e73: 6 total, 5 up, 6 in 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='osd.4 [v2:172.21.15.192:6808/2408603569,v1:172.21.15.192:6809/2408603569]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3108441795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3195727654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:02:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:53 smithi064 ceph-mon[93757]: from='osd.4 ' entity='osd.4' 2024-02-16T23:02:53.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.385+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3b402b3b0 con 0x7fe39c06d7f0 2024-02-16T23:02:53.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.386+0000 7fe39bfff700 1 -- 172.21.15.64:0/2151329437 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe3b402b3b0 con 0x7fe39c06d7f0 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "11/21 daemons upgraded", 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:02:53.387 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe39c06d7f0 msgr2=0x7fe39c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe39c06d7f0 0x7fe39c06fca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009fd0 tx=0x7fe3a800b040 comp rx=0 tx=0).stop 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 msgr2=0x7fe3b410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 0x7fe3b410d860 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400b3b0 tx=0x7fe3a4009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 shutdown_connections 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe39c06d7f0 0x7fe39c06fca0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b40fda80 0x7fe3b410d860 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 --2- 172.21.15.64:0/2151329437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3b41003a0 0x7fe3b410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.388+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 >> 172.21.15.64:0/2151329437 conn(0x7fe3b40f7520 msgr2=0x7fe3b40f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:53.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.389+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 shutdown_connections 2024-02-16T23:02:53.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.389+0000 7fe3b9ee5700 1 -- 172.21.15.64:0/2151329437 wait complete. 2024-02-16T23:02:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 -- 172.21.15.64:0/3075816215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fe2b0 msgr2=0x7f874c0fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 --2- 172.21.15.64:0/3075816215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fe2b0 0x7f874c0fe700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8740004940 tx=0x7f8740009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 -- 172.21.15.64:0/3075816215 shutdown_connections 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 --2- 172.21.15.64:0/3075816215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fe2b0 0x7f874c0fe700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 --2- 172.21.15.64:0/3075816215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874c0fb220 0x7f874c0fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 -- 172.21.15.64:0/3075816215 >> 172.21.15.64:0/3075816215 conn(0x7f874c0f6ca0 msgr2=0x7f874c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 -- 172.21.15.64:0/3075816215 shutdown_connections 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 -- 172.21.15.64:0/3075816215 wait complete. 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 Processor -- start 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.493+0000 7f875412d700 1 -- start start 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f875412d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fb220 0x7f874c0fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f875412d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874c0fe2b0 0x7f874c0fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f875412d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0fc7c0 con 0x7f874c0fb220 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f875412d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0fc930 con 0x7f874c0fe2b0 2024-02-16T23:02:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f8751ec9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fb220 0x7f874c0fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f8751ec9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fb220 0x7f874c0fdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54010/0 (socket says 172.21.15.64:54010) 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f8751ec9700 1 -- 172.21.15.64:0/1702235376 learned_addr learned my addr 172.21.15.64:0/1702235376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f87516c8700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874c0fe2b0 0x7f874c0fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f8751ec9700 1 -- 172.21.15.64:0/1702235376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874c0fe2b0 msgr2=0x7f874c0fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f8751ec9700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874c0fe2b0 0x7f874c0fc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f8751ec9700 1 -- 172.21.15.64:0/1702235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740005040 con 0x7f874c0fb220 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f8751ec9700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fb220 0x7f874c0fdd40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8744001ce0 tx=0x7f8744009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f874a7fc700 1 -- 172.21.15.64:0/1702235376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8744007b30 con 0x7f874c0fb220 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f874a7fc700 1 -- 172.21.15.64:0/1702235376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f874400f040 con 0x7f874c0fb220 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c0fcc10 con 0x7f874c0fb220 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f874a7fc700 1 -- 172.21.15.64:0/1702235376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87440096d0 con 0x7f874c0fb220 2024-02-16T23:02:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.494+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c115a50 con 0x7f874c0fb220 2024-02-16T23:02:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.495+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c04bed0 con 0x7f874c0fb220 2024-02-16T23:02:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.496+0000 7f874a7fc700 1 -- 172.21.15.64:0/1702235376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8744007c90 con 0x7f874c0fb220 2024-02-16T23:02:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.496+0000 7f874a7fc700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f873406d7e0 0x7f873406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:02:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.496+0000 7f874a7fc700 1 -- 172.21.15.64:0/1702235376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f874408c820 con 0x7f874c0fb220 2024-02-16T23:02:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.496+0000 7f87516c8700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f873406d7e0 0x7f873406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:02:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.497+0000 7f87516c8700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f873406d7e0 0x7f873406fc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8740004f90 tx=0x7f8740009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:02:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.501+0000 7f874a7fc700 1 -- 172.21.15.64:0/1702235376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8744059ca0 con 0x7f874c0fb220 2024-02-16T23:02:53.534 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:02:53.152+0000 7f84c2e69700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T23:02:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T23:02:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: failed to encode map e73 with expected crc 2024-02-16T23:02:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: osdmap e73: 6 total, 5 up, 6 in 2024-02-16T23:02:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='osd.4 [v2:172.21.15.192:6808/2408603569,v1:172.21.15.192:6809/2408603569]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:02:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:02:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3108441795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3195727654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:02:53.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:53 smithi192 ceph-mon[82652]: from='osd.4 ' entity='osd.4' 2024-02-16T23:02:53.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.793+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f874c115e30 con 0x7f874c0fb220 2024-02-16T23:02:53.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.794+0000 7f874a7fc700 1 -- 172.21.15.64:0/1702235376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1296 (secure 0 0 0) 0x7f8744059830 con 0x7f874c0fb220 2024-02-16T23:02:53.794 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 45/261 objects degraded (17.241%), 15 pgs degraded 2024-02-16T23:02:53.794 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:02:53.794 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: osd.4 (root=default,host=smithi192) is down 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 45/261 objects degraded (17.241%), 15 pgs degraded 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-02-16T23:02:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-02-16T23:02:53.796 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-02-16T23:02:53.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f873406d7e0 msgr2=0x7f873406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f873406d7e0 0x7f873406fc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8740004f90 tx=0x7f8740009e30 comp rx=0 tx=0).stop 2024-02-16T23:02:53.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fb220 msgr2=0x7f874c0fdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:02:53.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fb220 0x7f874c0fdd40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8744001ce0 tx=0x7f8744009e60 comp rx=0 tx=0).stop 2024-02-16T23:02:53.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 shutdown_connections 2024-02-16T23:02:53.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f873406d7e0 0x7f873406fc90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f874c0fb220 0x7f874c0fdd40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 --2- 172.21.15.64:0/1702235376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874c0fe2b0 0x7f874c0fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:02:53.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 >> 172.21.15.64:0/1702235376 conn(0x7f874c0f6ca0 msgr2=0x7f874c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:02:53.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 shutdown_connections 2024-02-16T23:02:53.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:02:53.797+0000 7f875412d700 1 -- 172.21.15.64:0/1702235376 wait complete. 2024-02-16T23:02:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1702235376' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:02:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:02:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:02:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: osd.4 [v2:172.21.15.192:6808/2408603569,v1:172.21.15.192:6809/2408603569] boot 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: osdmap e74: 6 total, 6 up, 6 in 2024-02-16T23:02:54.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:54 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T23:02:54.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:54.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:02:54.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1702235376' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:02:54.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:54.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: osd.4 [v2:172.21.15.192:6808/2408603569,v1:172.21.15.192:6809/2408603569] boot 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: osdmap e74: 6 total, 6 up, 6 in 2024-02-16T23:02:54.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:54 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T23:02:55.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:55 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T23:02:55.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:55 smithi064 ceph-mon[93757]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T23:02:55.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:55 smithi064 ceph-mon[93757]: failed to encode map e74 with expected crc 2024-02-16T23:02:55.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:55 smithi064 ceph-mon[93757]: osdmap e75: 6 total, 6 up, 6 in 2024-02-16T23:02:55.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:55 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T23:02:55.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:55 smithi192 ceph-mon[82652]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T23:02:55.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:55 smithi192 ceph-mon[82652]: failed to encode map e74 with expected crc 2024-02-16T23:02:55.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:55 smithi192 ceph-mon[82652]: osdmap e75: 6 total, 6 up, 6 in 2024-02-16T23:02:56.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:56 smithi064 ceph-mon[93757]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:56.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:56 smithi064 ceph-mon[93757]: failed to encode map e75 with expected crc 2024-02-16T23:02:56.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:56 smithi192 ceph-mon[82652]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/261 objects degraded (17.241%) 2024-02-16T23:02:56.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:56 smithi192 ceph-mon[82652]: failed to encode map e75 with expected crc 2024-02-16T23:02:57.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:57 smithi064 ceph-mon[93757]: Health check update: Degraded data redundancy: 39/261 objects degraded (14.943%), 12 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:57.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:57 smithi192 ceph-mon[82652]: Health check update: Degraded data redundancy: 39/261 objects degraded (14.943%), 12 pgs degraded (PG_DEGRADED) 2024-02-16T23:02:58.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:02:58 smithi064 ceph-mon[93757]: pgmap v145: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 39/261 objects degraded (14.943%) 2024-02-16T23:02:58.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:02:58 smithi192 ceph-mon[82652]: pgmap v145: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 39/261 objects degraded (14.943%) 2024-02-16T23:03:00.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:00 smithi064 ceph-mon[93757]: pgmap v146: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23/261 objects degraded (8.812%) 2024-02-16T23:03:00.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:00 smithi192 ceph-mon[82652]: pgmap v146: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23/261 objects degraded (8.812%) 2024-02-16T23:03:01.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:01 smithi192 ceph-mon[82652]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/261 objects degraded (8.812%), 5 pgs degraded) 2024-02-16T23:03:01.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:01 smithi064 ceph-mon[93757]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/261 objects degraded (8.812%), 5 pgs degraded) 2024-02-16T23:03:02.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:02 smithi064 ceph-mon[93757]: pgmap v147: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:02.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:02 smithi192 ceph-mon[82652]: pgmap v147: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:03 smithi192 ceph-mon[82652]: pgmap v148: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:03:04.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:03 smithi064 ceph-mon[93757]: pgmap v148: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:04.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:04.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:03:06.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:05 smithi192 ceph-mon[82652]: pgmap v149: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:06.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:05 smithi064 ceph-mon[93757]: pgmap v149: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:08.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:07 smithi192 ceph-mon[82652]: pgmap v150: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:08.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:07 smithi064 ceph-mon[93757]: pgmap v150: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:09.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:08 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:09.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:09.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:09 smithi192 ceph-mon[82652]: pgmap v151: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:09.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T23:03:09.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:09 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T23:03:09.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:09 smithi192 ceph-mon[82652]: Upgrade: osd.5 is safe to restart 2024-02-16T23:03:10.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:09 smithi064 ceph-mon[93757]: pgmap v151: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:10.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T23:03:10.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:09 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T23:03:10.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:09 smithi064 ceph-mon[93757]: Upgrade: osd.5 is safe to restart 2024-02-16T23:03:11.737 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:11 smithi192 ceph-mon[82652]: Upgrade: Updating osd.5 2024-02-16T23:03:11.737 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:11.737 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T23:03:11.737 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:11.737 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:11 smithi192 ceph-mon[82652]: Deploying daemon osd.5 on smithi192 2024-02-16T23:03:11.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:11 smithi064 ceph-mon[93757]: Upgrade: Updating osd.5 2024-02-16T23:03:11.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:11.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T23:03:11.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:11.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:11 smithi064 ceph-mon[93757]: Deploying daemon osd.5 on smithi192 2024-02-16T23:03:12.701 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:12 smithi192 ceph-mon[82652]: pgmap v152: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:12.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:12 smithi064 ceph-mon[93757]: pgmap v152: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:13.784 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:13 smithi192 systemd[1]: Stopping Ceph osd.5 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:03:14.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:13 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[55165]: 2024-02-16T23:03:13.873+0000 7fe98cf67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:03:14.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:13 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[55165]: 2024-02-16T23:03:13.873+0000 7fe98cf67700 -1 osd.5 75 *** Got signal Terminated *** 2024-02-16T23:03:14.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:13 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[55165]: 2024-02-16T23:03:13.873+0000 7fe98cf67700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:03:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:14 smithi064 ceph-mon[93757]: pgmap v153: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:14 smithi064 ceph-mon[93757]: osd.5 marked itself down and dead 2024-02-16T23:03:14.998 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:14 smithi192 ceph-mon[82652]: pgmap v153: 65 pgs: 65 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:14.999 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:14 smithi192 ceph-mon[82652]: osd.5 marked itself down and dead 2024-02-16T23:03:15.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:14 smithi192 podman[102240]: 2024-02-16 23:03:14.999285318 +0000 UTC m=+1.252011153 container died 8ecb480f948a1ba925be67e7ff7eaa5ada23c764be11bd0d65185f38ddad76c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=quincy-b88fae6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:03:15.560 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:15 smithi192 podman[102240]: 2024-02-16 23:03:15.366603934 +0000 UTC m=+1.619329766 container cleanup 8ecb480f948a1ba925be67e7ff7eaa5ada23c764be11bd0d65185f38ddad76c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=quincy-b88fae6, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-16T23:03:15.563 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:15 smithi192 bash[102240]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5 2024-02-16T23:03:15.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:15 smithi064 ceph-mon[93757]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:03:15.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:15 smithi064 ceph-mon[93757]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-16T23:03:15.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:15 smithi064 ceph-mon[93757]: osdmap e76: 6 total, 5 up, 6 in 2024-02-16T23:03:15.974 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:15 smithi192 ceph-mon[82652]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T23:03:15.974 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:15 smithi192 ceph-mon[82652]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-16T23:03:15.974 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:15 smithi192 ceph-mon[82652]: osdmap e76: 6 total, 5 up, 6 in 2024-02-16T23:03:15.974 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:15 smithi192 podman[102257]: 2024-02-16 23:03:15.634832507 +0000 UTC m=+0.632813574 container remove 8ecb480f948a1ba925be67e7ff7eaa5ada23c764be11bd0d65185f38ddad76c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=quincy-b88fae6, org.label-schema.build-date=20240214) 2024-02-16T23:03:16.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:15 smithi192 podman[102346]: 2024-02-16 23:03:15.877546436 +0000 UTC m=+0.017677132 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:03:16.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:15 smithi192 podman[102346]: 2024-02-16 23:03:15.977308513 +0000 UTC m=+0.117439248 container create aab5f55ccb0cea609a0de52e9d255d0d2db35f2a50d8c38feafa2c513b74f356 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-16T23:03:16.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:16 smithi192 podman[102346]: 2024-02-16 23:03:16.208553716 +0000 UTC m=+0.348684408 container init aab5f55ccb0cea609a0de52e9d255d0d2db35f2a50d8c38feafa2c513b74f356 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-16T23:03:16.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:16 smithi192 podman[102346]: 2024-02-16 23:03:16.211716604 +0000 UTC m=+0.351847305 container start aab5f55ccb0cea609a0de52e9d255d0d2db35f2a50d8c38feafa2c513b74f356 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:03:16.574 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:16 smithi192 podman[102346]: 2024-02-16 23:03:16.300212143 +0000 UTC m=+0.440342840 container attach aab5f55ccb0cea609a0de52e9d255d0d2db35f2a50d8c38feafa2c513b74f356 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-16T23:03:16.575 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:16 smithi192 podman[102346]: 2024-02-16 23:03:16.506294889 +0000 UTC m=+0.646425581 container died aab5f55ccb0cea609a0de52e9d255d0d2db35f2a50d8c38feafa2c513b74f356 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-16T23:03:16.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:16 smithi064 ceph-mon[93757]: pgmap v155: 65 pgs: 13 stale+active+clean, 52 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:16.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:16 smithi064 ceph-mon[93757]: osdmap e77: 6 total, 5 up, 6 in 2024-02-16T23:03:17.016 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:16 smithi192 ceph-mon[82652]: pgmap v155: 65 pgs: 13 stale+active+clean, 52 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:17.016 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:16 smithi192 ceph-mon[82652]: osdmap e77: 6 total, 5 up, 6 in 2024-02-16T23:03:17.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 podman[102371]: 2024-02-16 23:03:17.017100366 +0000 UTC m=+0.501561972 container remove aab5f55ccb0cea609a0de52e9d255d0d2db35f2a50d8c38feafa2c513b74f356 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-16T23:03:17.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.5.service: Deactivated successfully. 2024-02-16T23:03:17.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 systemd[1]: Stopped Ceph osd.5 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:03:17.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.5.service: Consumed 49.433s CPU time. 2024-02-16T23:03:17.284 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 systemd[1]: Starting Ceph osd.5 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:03:17.729 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 podman[102472]: 2024-02-16 23:03:17.355138356 +0000 UTC m=+0.017253170 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:03:17.729 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 podman[102472]: 2024-02-16 23:03:17.476828793 +0000 UTC m=+0.138943658 container create 41a5b3c4da54b0f04a3819a7934c57513493ab41fba6b5e038f8824117a215f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:03:17.730 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 podman[102472]: 2024-02-16 23:03:17.730289425 +0000 UTC m=+0.392404241 container init 41a5b3c4da54b0f04a3819a7934c57513493ab41fba6b5e038f8824117a215f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD) 2024-02-16T23:03:18.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 podman[102472]: 2024-02-16 23:03:17.733992026 +0000 UTC m=+0.396106836 container start 41a5b3c4da54b0f04a3819a7934c57513493ab41fba6b5e038f8824117a215f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:03:18.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:17 smithi192 podman[102472]: 2024-02-16 23:03:17.796060634 +0000 UTC m=+0.458175462 container attach 41a5b3c4da54b0f04a3819a7934c57513493ab41fba6b5e038f8824117a215f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214) 2024-02-16T23:03:18.490 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate[102486]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T23:03:18.490 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate[102486]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T23:03:18.490 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 bash[102472]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T23:03:18.490 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 bash[102472]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T23:03:18.490 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 bash[102472]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 bash[102472]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 bash[102472]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 bash[102472]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 bash[102472]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate[102486]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate[102486]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate[102486]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate[102486]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate[102486]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-16T23:03:18.491 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 podman[102472]: 2024-02-16 23:03:18.204944157 +0000 UTC m=+0.867058958 container died 41a5b3c4da54b0f04a3819a7934c57513493ab41fba6b5e038f8824117a215f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-16T23:03:18.759 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-mon[82652]: pgmap v157: 65 pgs: 11 stale+active+clean, 3 active+undersized, 51 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:18.759 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:18.760 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:03:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:18 smithi064 ceph-mon[93757]: pgmap v157: 65 pgs: 11 stale+active+clean, 3 active+undersized, 51 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:18.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:03:19.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 podman[102607]: 2024-02-16 23:03:18.760418482 +0000 UTC m=+0.550424654 container remove 41a5b3c4da54b0f04a3819a7934c57513493ab41fba6b5e038f8824117a215f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-16T23:03:19.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:18 smithi192 podman[102672]: 2024-02-16 23:03:18.874957965 +0000 UTC m=+0.018196746 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:03:19.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:19 smithi192 podman[102672]: 2024-02-16 23:03:19.003054142 +0000 UTC m=+0.146292916 container create 7bb5b265fc665280365fb5136240ab3ca9137ad452f41944fc0e9d04136a7c21 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-16T23:03:19.534 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:19 smithi192 podman[102672]: 2024-02-16 23:03:19.205785483 +0000 UTC m=+0.349024255 container init 7bb5b265fc665280365fb5136240ab3ca9137ad452f41944fc0e9d04136a7c21 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-16T23:03:19.534 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:19 smithi192 podman[102672]: 2024-02-16 23:03:19.209165502 +0000 UTC m=+0.352404278 container start 7bb5b265fc665280365fb5136240ab3ca9137ad452f41944fc0e9d04136a7c21 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-16T23:03:19.534 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:19 smithi192 ceph-osd[102691]: -- 172.21.15.192:0/642841199 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c3564ef1e0 con 0x55c3564c1000 2024-02-16T23:03:19.534 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:19 smithi192 bash[102672]: 7bb5b265fc665280365fb5136240ab3ca9137ad452f41944fc0e9d04136a7c21 2024-02-16T23:03:19.535 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:19 smithi192 systemd[1]: Started Ceph osd.5 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:03:20.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:19 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:03:19.747+0000 7f174c0dc7c0 -1 Falling back to public interface 2024-02-16T23:03:20.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:20 smithi192 ceph-mon[82652]: pgmap v158: 65 pgs: 7 active+undersized+degraded, 3 stale+active+clean, 11 active+undersized, 44 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 16/261 objects degraded (6.130%) 2024-02-16T23:03:20.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:20.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:20.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:03:20.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:20 smithi192 ceph-mon[82652]: Health check failed: Degraded data redundancy: 16/261 objects degraded (6.130%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T23:03:20.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:20 smithi064 ceph-mon[93757]: pgmap v158: 65 pgs: 7 active+undersized+degraded, 3 stale+active+clean, 11 active+undersized, 44 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 16/261 objects degraded (6.130%) 2024-02-16T23:03:20.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:03:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:20 smithi064 ceph-mon[93757]: Health check failed: Degraded data redundancy: 16/261 objects degraded (6.130%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T23:03:22.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:22 smithi064 ceph-mon[93757]: pgmap v159: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 40/261 objects degraded (15.326%) 2024-02-16T23:03:22.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:22 smithi192 ceph-mon[82652]: pgmap v159: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 40/261 objects degraded (15.326%) 2024-02-16T23:03:23.670 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:23 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:23.670 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:23 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:23.670 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:23 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:03:23.661+0000 7f174c0dc7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-02-16T23:03:23.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:23 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:23.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:23 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:23.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.916+0000 7f9546c16700 1 -- 172.21.15.64:0/2629162162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fcef0 msgr2=0x7f95400fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:23.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.916+0000 7f9546c16700 1 --2- 172.21.15.64:0/2629162162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fcef0 0x7f95400fd320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9538009ce0 tx=0x7f9538004d10 comp rx=0 tx=0).stop 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.916+0000 7f9546c16700 1 -- 172.21.15.64:0/2629162162 shutdown_connections 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.916+0000 7f9546c16700 1 --2- 172.21.15.64:0/2629162162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95400fe0e0 0x7f95400fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.916+0000 7f9546c16700 1 --2- 172.21.15.64:0/2629162162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fcef0 0x7f95400fd320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.916+0000 7f9546c16700 1 -- 172.21.15.64:0/2629162162 >> 172.21.15.64:0/2629162162 conn(0x7f95400f8620 msgr2=0x7f95400faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.916+0000 7f9546c16700 1 -- 172.21.15.64:0/2629162162 shutdown_connections 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.916+0000 7f9546c16700 1 -- 172.21.15.64:0/2629162162 wait complete. 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f9546c16700 1 Processor -- start 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f9546c16700 1 -- start start 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f9546c16700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95400fcef0 0x7f954010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f9546c16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fe0e0 0x7f954010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f9546c16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954010e4b0 con 0x7f95400fe0e0 2024-02-16T23:03:23.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f9546c16700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954010ed50 con 0x7f95400fcef0 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f953ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fe0e0 0x7f954010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f953ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fe0e0 0x7f954010deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38448/0 (socket says 172.21.15.64:38448) 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f953ffff700 1 -- 172.21.15.64:0/3073725538 learned_addr learned my addr 172.21.15.64:0/3073725538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f95449b2700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95400fcef0 0x7f954010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f953ffff700 1 -- 172.21.15.64:0/3073725538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95400fcef0 msgr2=0x7f954010d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f953ffff700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95400fcef0 0x7f954010d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f953ffff700 1 -- 172.21.15.64:0/3073725538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9538009940 con 0x7f95400fe0e0 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.917+0000 7f95449b2700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95400fcef0 0x7f954010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.918+0000 7f953ffff700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fe0e0 0x7f954010deb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f953000eb40 tx=0x7f953000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.918+0000 7f953d7fa700 1 -- 172.21.15.64:0/3073725538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9530007cc0 con 0x7f95400fe0e0 2024-02-16T23:03:23.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.918+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954010efd0 con 0x7f95400fe0e0 2024-02-16T23:03:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.918+0000 7f953d7fa700 1 -- 172.21.15.64:0/3073725538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f953000f040 con 0x7f95400fe0e0 2024-02-16T23:03:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.918+0000 7f953d7fa700 1 -- 172.21.15.64:0/3073725538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953000ca10 con 0x7f95400fe0e0 2024-02-16T23:03:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.918+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954010f510 con 0x7f95400fe0e0 2024-02-16T23:03:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.919+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954004bed0 con 0x7f95400fe0e0 2024-02-16T23:03:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.919+0000 7f953d7fa700 1 -- 172.21.15.64:0/3073725538 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f953000cc80 con 0x7f95400fe0e0 2024-02-16T23:03:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.920+0000 7f953d7fa700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f952c06d7e0 0x7f952c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.920+0000 7f953d7fa700 1 -- 172.21.15.64:0/3073725538 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f953008ca20 con 0x7f95400fe0e0 2024-02-16T23:03:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.920+0000 7f95449b2700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f952c06d7e0 0x7f952c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.921+0000 7f95449b2700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f952c06d7e0 0x7f952c06fc90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9538009460 tx=0x7f9538004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:23.925+0000 7f953d7fa700 1 -- 172.21.15.64:0/3073725538 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9530059ea0 con 0x7f95400fe0e0 2024-02-16T23:03:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.149+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f954010f960 con 0x7f952c06d7e0 2024-02-16T23:03:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.150+0000 7f953d7fa700 1 -- 172.21.15.64:0/3073725538 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f954010f960 con 0x7f952c06d7e0 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f952c06d7e0 msgr2=0x7f952c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f952c06d7e0 0x7f952c06fc90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9538009460 tx=0x7f9538004d10 comp rx=0 tx=0).stop 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fe0e0 msgr2=0x7f954010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fe0e0 0x7f954010deb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f953000eb40 tx=0x7f953000b040 comp rx=0 tx=0).stop 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 shutdown_connections 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f952c06d7e0 0x7f952c06fc90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95400fcef0 0x7f954010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 --2- 172.21.15.64:0/3073725538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95400fe0e0 0x7f954010deb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 >> 172.21.15.64:0/3073725538 conn(0x7f95400f8620 msgr2=0x7f95401013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 shutdown_connections 2024-02-16T23:03:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.152+0000 7f9546c16700 1 -- 172.21.15.64:0/3073725538 wait complete. 2024-02-16T23:03:24.163 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:03:24.246 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:23 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:03:23.941+0000 7f174c0dc7c0 -1 osd.5 75 log_to_monitors true 2024-02-16T23:03:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 -- 172.21.15.64:0/3855699492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fbe40 msgr2=0x7f2cc40fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 --2- 172.21.15.64:0/3855699492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fbe40 0x7f2cc40fc250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4004940 tx=0x7f2cb4009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 -- 172.21.15.64:0/3855699492 shutdown_connections 2024-02-16T23:03:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 --2- 172.21.15.64:0/3855699492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40fcfb0 0x7f2cc40fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 --2- 172.21.15.64:0/3855699492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fbe40 0x7f2cc40fc250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 -- 172.21.15.64:0/3855699492 >> 172.21.15.64:0/3855699492 conn(0x7f2cc40f74e0 msgr2=0x7f2cc40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 -- 172.21.15.64:0/3855699492 shutdown_connections 2024-02-16T23:03:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 -- 172.21.15.64:0/3855699492 wait complete. 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.256+0000 7f2cca272700 1 Processor -- start 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cca272700 1 -- start start 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cca272700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40fbe40 0x7f2cc41105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cca272700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fcfb0 0x7f2cc4112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cca272700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4066480 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cca272700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc40665c0 con 0x7f2cc40fbe40 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cc37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fcfb0 0x7f2cc4112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cc37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fcfb0 0x7f2cc4112b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38460/0 (socket says 172.21.15.64:38460) 2024-02-16T23:03:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cc37fe700 1 -- 172.21.15.64:0/4069574644 learned_addr learned my addr 172.21.15.64:0/4069574644 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cc3fff700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40fbe40 0x7f2cc41105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cc37fe700 1 -- 172.21.15.64:0/4069574644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40fbe40 msgr2=0x7f2cc41105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cc37fe700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40fbe40 0x7f2cc41105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.257+0000 7f2cc37fe700 1 -- 172.21.15.64:0/4069574644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4005040 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.258+0000 7f2cc3fff700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40fbe40 0x7f2cc41105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.258+0000 7f2cc37fe700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fcfb0 0x7f2cc4112b20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00eb40 tx=0x7f2cac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.258+0000 7f2cc0ff9700 1 -- 172.21.15.64:0/4069574644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cac007cc0 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.258+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc4113120 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.258+0000 7f2cc0ff9700 1 -- 172.21.15.64:0/4069574644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cac00f040 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.258+0000 7f2cc0ff9700 1 -- 172.21.15.64:0/4069574644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cac00ca10 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.258+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc4113610 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.259+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc404bed0 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.260+0000 7f2cc0ff9700 1 -- 172.21.15.64:0/4069574644 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2cac00cc80 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.260+0000 7f2cc0ff9700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2cb006d7f0 0x7f2cb006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.260+0000 7f2cc0ff9700 1 -- 172.21.15.64:0/4069574644 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2cac08ca20 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.260+0000 7f2cc3fff700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2cb006d7f0 0x7f2cb006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.261+0000 7f2cc3fff700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2cb006d7f0 0x7f2cb006fca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4013740 tx=0x7f2cb4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:24.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.265+0000 7f2cc0ff9700 1 -- 172.21.15.64:0/4069574644 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cac056370 con 0x7f2cc40fcfb0 2024-02-16T23:03:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.488+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cc4101990 con 0x7f2cb006d7f0 2024-02-16T23:03:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.489+0000 7f2cc0ff9700 1 -- 172.21.15.64:0/4069574644 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2cc4101990 con 0x7f2cb006d7f0 2024-02-16T23:03:24.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2cb006d7f0 msgr2=0x7f2cb006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2cb006d7f0 0x7f2cb006fca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4013740 tx=0x7f2cb4009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:24.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fcfb0 msgr2=0x7f2cc4112b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fcfb0 0x7f2cc4112b20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00eb40 tx=0x7f2cac00b040 comp rx=0 tx=0).stop 2024-02-16T23:03:24.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 shutdown_connections 2024-02-16T23:03:24.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2cb006d7f0 0x7f2cb006fca0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40fbe40 0x7f2cc41105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 --2- 172.21.15.64:0/4069574644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cc40fcfb0 0x7f2cc4112b20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 >> 172.21.15.64:0/4069574644 conn(0x7f2cc40f74e0 msgr2=0x7f2cc4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:24.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 shutdown_connections 2024-02-16T23:03:24.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.492+0000 7f2cca272700 1 -- 172.21.15.64:0/4069574644 wait complete. 2024-02-16T23:03:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.597+0000 7fa36417a700 1 -- 172.21.15.64:0/2021244166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fe0e0 msgr2=0x7fa35c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.597+0000 7fa36417a700 1 --2- 172.21.15.64:0/2021244166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fe0e0 0x7fa35c0fe550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa34c004940 tx=0x7fa34c009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.597+0000 7fa36417a700 1 -- 172.21.15.64:0/2021244166 shutdown_connections 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.597+0000 7fa36417a700 1 --2- 172.21.15.64:0/2021244166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fe0e0 0x7fa35c0fe550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.597+0000 7fa36417a700 1 --2- 172.21.15.64:0/2021244166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c0fcef0 0x7fa35c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.597+0000 7fa36417a700 1 -- 172.21.15.64:0/2021244166 >> 172.21.15.64:0/2021244166 conn(0x7fa35c0f8620 msgr2=0x7fa35c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.597+0000 7fa36417a700 1 -- 172.21.15.64:0/2021244166 shutdown_connections 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.597+0000 7fa36417a700 1 -- 172.21.15.64:0/2021244166 wait complete. 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa36417a700 1 Processor -- start 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa36417a700 1 -- start start 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa36417a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fcef0 0x7fa35c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa36417a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c0fe0e0 0x7fa35c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa36417a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c070970 con 0x7fa35c0fcef0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa36417a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c070ae0 con 0x7fa35c0fe0e0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa361f16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fcef0 0x7fa35c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa361f16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fcef0 0x7fa35c071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38480/0 (socket says 172.21.15.64:38480) 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa361f16700 1 -- 172.21.15.64:0/1565962264 learned_addr learned my addr 172.21.15.64:0/1565962264 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa361715700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c0fe0e0 0x7fa35c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa361f16700 1 -- 172.21.15.64:0/1565962264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c0fe0e0 msgr2=0x7fa35c0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa361f16700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c0fe0e0 0x7fa35c0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.598+0000 7fa361f16700 1 -- 172.21.15.64:0/1565962264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c005040 con 0x7fa35c0fcef0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.599+0000 7fa361f16700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fcef0 0x7fa35c071e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa35800b790 tx=0x7fa358009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.599+0000 7fa3527fc700 1 -- 172.21.15.64:0/1565962264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35800f6f0 con 0x7fa35c0fcef0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.599+0000 7fa3527fc700 1 -- 172.21.15.64:0/1565962264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35800fd20 con 0x7fa35c0fcef0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.599+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c070dc0 con 0x7fa35c0fcef0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.599+0000 7fa3527fc700 1 -- 172.21.15.64:0/1565962264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3580103a0 con 0x7fa35c0fcef0 2024-02-16T23:03:24.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.599+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c071140 con 0x7fa35c0fcef0 2024-02-16T23:03:24.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.600+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c04bed0 con 0x7fa35c0fcef0 2024-02-16T23:03:24.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.600+0000 7fa3527fc700 1 -- 172.21.15.64:0/1565962264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa358011400 con 0x7fa35c0fcef0 2024-02-16T23:03:24.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.601+0000 7fa3527fc700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa34806d7f0 0x7fa34806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.601+0000 7fa3527fc700 1 -- 172.21.15.64:0/1565962264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa35808cc60 con 0x7fa35c0fcef0 2024-02-16T23:03:24.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.601+0000 7fa361715700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa34806d7f0 0x7fa34806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.601+0000 7fa361715700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa34806d7f0 0x7fa34806fca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00b7a0 tx=0x7fa34c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:24.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.605+0000 7fa3527fc700 1 -- 172.21.15.64:0/1565962264 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa35805a0b0 con 0x7fa35c0fcef0 2024-02-16T23:03:24.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:24 smithi192 ceph-mon[82652]: pgmap v160: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 40/261 objects degraded (15.326%) 2024-02-16T23:03:24.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:24 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:24.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:24 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:24.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:24 smithi192 ceph-mon[82652]: from='osd.5 [v2:172.21.15.192:6816/2198637799,v1:172.21.15.192:6817/2198637799]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T23:03:24.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:24 smithi192 ceph-mon[82652]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T23:03:24.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.819+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa35c102ac0 con 0x7fa34806d7f0 2024-02-16T23:03:24.820 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:24 smithi064 ceph-mon[93757]: pgmap v160: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 251 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 40/261 objects degraded (15.326%) 2024-02-16T23:03:24.820 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:24 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:24.820 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:24 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:24.820 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:24 smithi064 ceph-mon[93757]: from='osd.5 [v2:172.21.15.192:6816/2198637799,v1:172.21.15.192:6817/2198637799]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T23:03:24.820 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:24 smithi064 ceph-mon[93757]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T23:03:24.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.830+0000 7fa3527fc700 1 -- 172.21.15.64:0/1565962264 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa35c102ac0 con 0x7fa34806d7f0 2024-02-16T23:03:24.830 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (6m) 102s ago 14m 26.6M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (3m) 102s ago 15m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (3m) 2s ago 14m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (6m) 102s ago 14m 84.4M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (9m) 102s ago 9m 17.9M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (9m) 102s ago 9m 153M - 17.2.7-513-gb88fae66 870a0944ccb8 436d256fc113 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (9m) 2s ago 9m 17.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (9m) 2s ago 9m 17.0M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (5m) 102s ago 16m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (4m) 2s ago 14m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (4m) 102s ago 16m 45.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:03:24.831 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (4m) 2s ago 13m 39.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (6m) 102s ago 15m 15.6M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 2s ago 14m 15.2M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (3m) 102s ago 13m 96.7M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (2m) 102s ago 12m 83.8M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (108s) 102s ago 12m 11.8M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (72s) 2s ago 11m 107M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (39s) 2s ago 11m 99.3M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (5s) 2s ago 11m 13.5M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 7bb5b265fc66 2024-02-16T23:03:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (4m) 102s ago 14m 59.3M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:03:24.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa34806d7f0 msgr2=0x7fa34806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa34806d7f0 0x7fa34806fca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00b7a0 tx=0x7fa34c009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:24.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fcef0 msgr2=0x7fa35c071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fcef0 0x7fa35c071e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa35800b790 tx=0x7fa358009220 comp rx=0 tx=0).stop 2024-02-16T23:03:24.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 shutdown_connections 2024-02-16T23:03:24.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa34806d7f0 0x7fa34806fca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa35c0fcef0 0x7fa35c071e60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 --2- 172.21.15.64:0/1565962264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c0fe0e0 0x7fa35c0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 >> 172.21.15.64:0/1565962264 conn(0x7fa35c0f8620 msgr2=0x7fa35c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:24.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 shutdown_connections 2024-02-16T23:03:24.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.832+0000 7fa36417a700 1 -- 172.21.15.64:0/1565962264 wait complete. 2024-02-16T23:03:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.933+0000 7fb096417700 1 -- 172.21.15.64:0/2573216956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fe0e0 msgr2=0x7fb0900fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.933+0000 7fb096417700 1 --2- 172.21.15.64:0/2573216956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fe0e0 0x7fb0900fe550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb084004940 tx=0x7fb084009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.933+0000 7fb096417700 1 -- 172.21.15.64:0/2573216956 shutdown_connections 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.933+0000 7fb096417700 1 --2- 172.21.15.64:0/2573216956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fe0e0 0x7fb0900fe550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.933+0000 7fb096417700 1 --2- 172.21.15.64:0/2573216956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0900fcef0 0x7fb0900fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.933+0000 7fb096417700 1 -- 172.21.15.64:0/2573216956 >> 172.21.15.64:0/2573216956 conn(0x7fb0900f8620 msgr2=0x7fb0900faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.933+0000 7fb096417700 1 -- 172.21.15.64:0/2573216956 shutdown_connections 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.933+0000 7fb096417700 1 -- 172.21.15.64:0/2573216956 wait complete. 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb096417700 1 Processor -- start 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb096417700 1 -- start start 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb096417700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fcef0 0x7fb09010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb096417700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0900fe0e0 0x7fb09010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb096417700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09010e4a0 con 0x7fb0900fcef0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb096417700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09010ed40 con 0x7fb0900fe0e0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb08ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fcef0 0x7fb09010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb08ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fcef0 0x7fb09010d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38498/0 (socket says 172.21.15.64:38498) 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb08ffff700 1 -- 172.21.15.64:0/1775679901 learned_addr learned my addr 172.21.15.64:0/1775679901 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.934+0000 7fb08f7fe700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0900fe0e0 0x7fb09010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb08ffff700 1 -- 172.21.15.64:0/1775679901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0900fe0e0 msgr2=0x7fb09010dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb08ffff700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0900fe0e0 0x7fb09010dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb08ffff700 1 -- 172.21.15.64:0/1775679901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb084005040 con 0x7fb0900fcef0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb08ffff700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fcef0 0x7fb09010d960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb080002a80 tx=0x7fb080004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb08cff9700 1 -- 172.21.15.64:0/1775679901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb080007cc0 con 0x7fb0900fcef0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb08cff9700 1 -- 172.21.15.64:0/1775679901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb08000f040 con 0x7fb0900fcef0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09010f020 con 0x7fb0900fcef0 2024-02-16T23:03:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb08cff9700 1 -- 172.21.15.64:0/1775679901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb08000da30 con 0x7fb0900fcef0 2024-02-16T23:03:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.935+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09010f560 con 0x7fb0900fcef0 2024-02-16T23:03:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.936+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09004bed0 con 0x7fb0900fcef0 2024-02-16T23:03:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.936+0000 7fb08cff9700 1 -- 172.21.15.64:0/1775679901 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb08000db90 con 0x7fb0900fcef0 2024-02-16T23:03:24.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.937+0000 7fb08cff9700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb07806d7f0 0x7fb07806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:24.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.937+0000 7fb08cff9700 1 -- 172.21.15.64:0/1775679901 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb08008cc00 con 0x7fb0900fcef0 2024-02-16T23:03:24.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.937+0000 7fb08f7fe700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb07806d7f0 0x7fb07806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:24.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.937+0000 7fb08f7fe700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb07806d7f0 0x7fb07806fca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb08400b7a0 tx=0x7fb084009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:24.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:24.942+0000 7fb08cff9700 1 -- 172.21.15.64:0/1775679901 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb080092020 con 0x7fb0900fcef0 2024-02-16T23:03:25.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.233+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb09010f940 con 0x7fb0900fcef0 2024-02-16T23:03:25.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.233+0000 7fb08cff9700 1 -- 172.21.15.64:0/1775679901 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+732 (secure 0 0 0) 0x7fb080059fd0 con 0x7fb0900fcef0 2024-02-16T23:03:25.234 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:03:25.234 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:03:25.234 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:03:25.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:03:25.234 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:03:25.234 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:03:25.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4, 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 9 2024-02-16T23:03:25.235 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:03:25.236 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:03:25.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb07806d7f0 msgr2=0x7fb07806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb07806d7f0 0x7fb07806fca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb08400b7a0 tx=0x7fb084009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:25.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fcef0 msgr2=0x7fb09010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fcef0 0x7fb09010d960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb080002a80 tx=0x7fb080004600 comp rx=0 tx=0).stop 2024-02-16T23:03:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 shutdown_connections 2024-02-16T23:03:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb07806d7f0 0x7fb07806fca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0900fcef0 0x7fb09010d960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 --2- 172.21.15.64:0/1775679901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0900fe0e0 0x7fb09010dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.235+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 >> 172.21.15.64:0/1775679901 conn(0x7fb0900f8620 msgr2=0x7fb0901013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.236+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 shutdown_connections 2024-02-16T23:03:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.236+0000 7fb096417700 1 -- 172.21.15.64:0/1775679901 wait complete. 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.338+0000 7fce747e1700 1 -- 172.21.15.64:0/3037635904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 msgr2=0x7fce6c0fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.338+0000 7fce747e1700 1 --2- 172.21.15.64:0/3037635904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 0x7fce6c0fe700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce68004940 tx=0x7fce68009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.338+0000 7fce747e1700 1 -- 172.21.15.64:0/3037635904 shutdown_connections 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.338+0000 7fce747e1700 1 --2- 172.21.15.64:0/3037635904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 0x7fce6c0fe700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.338+0000 7fce747e1700 1 --2- 172.21.15.64:0/3037635904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c0fb220 0x7fce6c0fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.338+0000 7fce747e1700 1 -- 172.21.15.64:0/3037635904 >> 172.21.15.64:0/3037635904 conn(0x7fce6c0f6ca0 msgr2=0x7fce6c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.338+0000 7fce747e1700 1 -- 172.21.15.64:0/3037635904 shutdown_connections 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.338+0000 7fce747e1700 1 -- 172.21.15.64:0/3037635904 wait complete. 2024-02-16T23:03:25.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce747e1700 1 Processor -- start 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce747e1700 1 -- start start 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce747e1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c0fb220 0x7fce6c0fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce747e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 0x7fce6c0fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce747e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0fc7c0 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce747e1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0fc930 con 0x7fce6c0fb220 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce71d7c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 0x7fce6c0fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce71d7c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 0x7fce6c0fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38508/0 (socket says 172.21.15.64:38508) 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce71d7c700 1 -- 172.21.15.64:0/3573163186 learned_addr learned my addr 172.21.15.64:0/3573163186 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce7257d700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c0fb220 0x7fce6c0fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce71d7c700 1 -- 172.21.15.64:0/3573163186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c0fb220 msgr2=0x7fce6c0fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce71d7c700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c0fb220 0x7fce6c0fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce71d7c700 1 -- 172.21.15.64:0/3573163186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68005040 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce71d7c700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 0x7fce6c0fc1c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fce68009180 tx=0x7fce68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.339+0000 7fce62ffd700 1 -- 172.21.15.64:0/3573163186 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6800e040 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.340+0000 7fce62ffd700 1 -- 172.21.15.64:0/3573163186 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce68009410 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.340+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c0fcbb0 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.340+0000 7fce62ffd700 1 -- 172.21.15.64:0/3573163186 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6801c630 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.340+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c115a40 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.341+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c107a80 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.341+0000 7fce62ffd700 1 -- 172.21.15.64:0/3573163186 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fce6800bbf0 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.341+0000 7fce62ffd700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fce5806d7e0 0x7fce5806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:25.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.342+0000 7fce62ffd700 1 -- 172.21.15.64:0/3573163186 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce6808e0c0 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.342+0000 7fce7257d700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fce5806d7e0 0x7fce5806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:25.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.342+0000 7fce7257d700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fce5806d7e0 0x7fce5806fc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0fdad0 tx=0x7fce5c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:25.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.346+0000 7fce62ffd700 1 -- 172.21.15.64:0/3573163186 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce6805b5b0 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.506 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:03:25.388+0000 7f173952c700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T23:03:25.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.597+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce6c04bed0 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.597+0000 7fce62ffd700 1 -- 172.21.15.64:0/3573163186 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1846 (secure 0 0 0) 0x7fce6805b140 con 0x7fce6c0fe2b0 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:e13 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 10 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T22:53:56.166293+0000 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:03:25.598 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14474} 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:03:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:14474} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.64:6826/3220035256,v1:172.21.15.64:6827/3220035256] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:03:25.600 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:03:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.599+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fce5806d7e0 msgr2=0x7fce5806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.599+0000 7fce747e1700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fce5806d7e0 0x7fce5806fc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0fdad0 tx=0x7fce5c00b040 comp rx=0 tx=0).stop 2024-02-16T23:03:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.599+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 msgr2=0x7fce6c0fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.599+0000 7fce747e1700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 0x7fce6c0fc1c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fce68009180 tx=0x7fce68009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.600+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 shutdown_connections 2024-02-16T23:03:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.600+0000 7fce747e1700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fce5806d7e0 0x7fce5806fc90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.600+0000 7fce747e1700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c0fb220 0x7fce6c0fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.600+0000 7fce747e1700 1 --2- 172.21.15.64:0/3573163186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce6c0fe2b0 0x7fce6c0fc1c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.600+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 >> 172.21.15.64:0/3573163186 conn(0x7fce6c0f6ca0 msgr2=0x7fce6c0f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.600+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 shutdown_connections 2024-02-16T23:03:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.600+0000 7fce747e1700 1 -- 172.21.15.64:0/3573163186 wait complete. 2024-02-16T23:03:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T23:03:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.704+0000 7fe1e3c70700 1 -- 172.21.15.64:0/4105324089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 msgr2=0x7fe1dc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.704+0000 7fe1e3c70700 1 --2- 172.21.15.64:0/4105324089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 0x7fe1dc0fe550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8004940 tx=0x7fe1d8009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.704+0000 7fe1e3c70700 1 -- 172.21.15.64:0/4105324089 shutdown_connections 2024-02-16T23:03:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.704+0000 7fe1e3c70700 1 --2- 172.21.15.64:0/4105324089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 0x7fe1dc0fe550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.704+0000 7fe1e3c70700 1 --2- 172.21.15.64:0/4105324089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc0fcef0 0x7fe1dc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.704+0000 7fe1e3c70700 1 -- 172.21.15.64:0/4105324089 >> 172.21.15.64:0/4105324089 conn(0x7fe1dc0f8620 msgr2=0x7fe1dc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.704+0000 7fe1e3c70700 1 -- 172.21.15.64:0/4105324089 shutdown_connections 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.704+0000 7fe1e3c70700 1 -- 172.21.15.64:0/4105324089 wait complete. 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e3c70700 1 Processor -- start 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e3c70700 1 -- start start 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e3c70700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc0fcef0 0x7fe1dc10fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e3c70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 0x7fe1dc110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e3c70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc110690 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e3c70700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc110f30 con 0x7fe1dc0fcef0 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e120b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 0x7fe1dc110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e120b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 0x7fe1dc110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38512/0 (socket says 172.21.15.64:38512) 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e120b700 1 -- 172.21.15.64:0/966892352 learned_addr learned my addr 172.21.15.64:0/966892352 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e1a0c700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc0fcef0 0x7fe1dc10fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e120b700 1 -- 172.21.15.64:0/966892352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc0fcef0 msgr2=0x7fe1dc10fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e120b700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc0fcef0 0x7fe1dc10fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e120b700 1 -- 172.21.15.64:0/966892352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8005040 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.705+0000 7fe1e120b700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 0x7fe1dc110090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80093b0 tx=0x7fe1d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.706+0000 7fe1d27fc700 1 -- 172.21.15.64:0/966892352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d800e040 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.706+0000 7fe1d27fc700 1 -- 172.21.15.64:0/966892352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1d800bcc0 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.706+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc1111b0 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.706+0000 7fe1d27fc700 1 -- 172.21.15.64:0/966892352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d801c630 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.706+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc1116f0 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.706+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc102210 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.707+0000 7fe1d27fc700 1 -- 172.21.15.64:0/966892352 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe1d8013980 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.707+0000 7fe1d27fc700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe1c806d7e0 0x7fe1c806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.708+0000 7fe1d27fc700 1 -- 172.21.15.64:0/966892352 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe1d808e220 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.708+0000 7fe1e1a0c700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe1c806d7e0 0x7fe1c806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.708+0000 7fe1e1a0c700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe1c806d7e0 0x7fe1c806fc90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0fcc30 tx=0x7fe1cc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:25.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.712+0000 7fe1d27fc700 1 -- 172.21.15.64:0/966892352 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1d805b670 con 0x7fe1dc0fe0e0 2024-02-16T23:03:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-mon[82652]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-mon[82652]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-mon[82652]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T23:03:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-mon[82652]: osdmap e78: 6 total, 5 up, 6 in 2024-02-16T23:03:25.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-mon[82652]: from='osd.5 [v2:172.21.15.192:6816/2198637799,v1:172.21.15.192:6817/2198637799]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:03:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-mon[82652]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:03:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1775679901' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:25.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:25 smithi192 ceph-mon[82652]: from='osd.5 ' entity='osd.5' 2024-02-16T23:03:25.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:25 smithi064 ceph-mon[93757]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:25.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:25 smithi064 ceph-mon[93757]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:25.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:25 smithi064 ceph-mon[93757]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T23:03:25.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:25 smithi064 ceph-mon[93757]: osdmap e78: 6 total, 5 up, 6 in 2024-02-16T23:03:25.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:25 smithi064 ceph-mon[93757]: from='osd.5 [v2:172.21.15.192:6816/2198637799,v1:172.21.15.192:6817/2198637799]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:03:25.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:25 smithi064 ceph-mon[93757]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-02-16T23:03:25.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:25 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1775679901' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:25.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:25 smithi064 ceph-mon[93757]: from='osd.5 ' entity='osd.5' 2024-02-16T23:03:25.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.935+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1dc111ae0 con 0x7fe1c806d7e0 2024-02-16T23:03:25.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.936+0000 7fe1d27fc700 1 -- 172.21.15.64:0/966892352 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe1dc111ae0 con 0x7fe1c806d7e0 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "osd", 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "12/21 daemons upgraded", 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:03:25.937 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe1c806d7e0 msgr2=0x7fe1c806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe1c806d7e0 0x7fe1c806fc90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0fcc30 tx=0x7fe1cc00b040 comp rx=0 tx=0).stop 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 msgr2=0x7fe1dc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 0x7fe1dc110090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80093b0 tx=0x7fe1d8009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 shutdown_connections 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe1c806d7e0 0x7fe1c806fc90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc0fcef0 0x7fe1dc10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 --2- 172.21.15.64:0/966892352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0fe0e0 0x7fe1dc110090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.938+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 >> 172.21.15.64:0/966892352 conn(0x7fe1dc0f8620 msgr2=0x7fe1dc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.939+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 shutdown_connections 2024-02-16T23:03:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:25.939+0000 7fe1e3c70700 1 -- 172.21.15.64:0/966892352 wait complete. 2024-02-16T23:03:26.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.040+0000 7f4c41097700 1 -- 172.21.15.64:0/2148697949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 msgr2=0x7f4c3c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:26.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.040+0000 7f4c41097700 1 --2- 172.21.15.64:0/2148697949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 0x7f4c3c0fd320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c004940 tx=0x7f4c2c009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.040+0000 7f4c41097700 1 -- 172.21.15.64:0/2148697949 shutdown_connections 2024-02-16T23:03:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.040+0000 7f4c41097700 1 --2- 172.21.15.64:0/2148697949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0fe0e0 0x7f4c3c0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.040+0000 7f4c41097700 1 --2- 172.21.15.64:0/2148697949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 0x7f4c3c0fd320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.040+0000 7f4c41097700 1 -- 172.21.15.64:0/2148697949 >> 172.21.15.64:0/2148697949 conn(0x7f4c3c0f8620 msgr2=0x7f4c3c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.040+0000 7f4c41097700 1 -- 172.21.15.64:0/2148697949 shutdown_connections 2024-02-16T23:03:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.040+0000 7f4c41097700 1 -- 172.21.15.64:0/2148697949 wait complete. 2024-02-16T23:03:26.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c41097700 1 Processor -- start 2024-02-16T23:03:26.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c41097700 1 -- start start 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c41097700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 0x7f4c3c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c41097700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0fe0e0 0x7f4c3c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c41097700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c10e4b0 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c41097700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c10ed50 con 0x7f4c3c0fe0e0 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c3ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 0x7f4c3c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c3ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 0x7f4c3c10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38532/0 (socket says 172.21.15.64:38532) 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c3ad9d700 1 -- 172.21.15.64:0/3741546877 learned_addr learned my addr 172.21.15.64:0/3741546877 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c3a59c700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0fe0e0 0x7f4c3c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c3ad9d700 1 -- 172.21.15.64:0/3741546877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0fe0e0 msgr2=0x7f4c3c10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c3ad9d700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0fe0e0 0x7f4c3c10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.041+0000 7f4c3ad9d700 1 -- 172.21.15.64:0/3741546877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c005040 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.042+0000 7f4c3ad9d700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 0x7f4c3c10d970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c000c00 tx=0x7f4c2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.042+0000 7f4c237fe700 1 -- 172.21.15.64:0/3741546877 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2c00e040 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.042+0000 7f4c237fe700 1 -- 172.21.15.64:0/3741546877 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c2c00bbf0 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.042+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c10efd0 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.042+0000 7f4c237fe700 1 -- 172.21.15.64:0/3741546877 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2c01c630 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.042+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c10f510 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.043+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c04bed0 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.043+0000 7f4c237fe700 1 -- 172.21.15.64:0/3741546877 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4c2c00bd60 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.044+0000 7f4c237fe700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4c2406d7f0 0x7f4c2406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.044+0000 7f4c237fe700 1 -- 172.21.15.64:0/3741546877 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4c2c08e120 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.044+0000 7f4c3a59c700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4c2406d7f0 0x7f4c2406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:26.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.044+0000 7f4c3a59c700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4c2406d7f0 0x7f4c2406fca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4c30007fd0 tx=0x7f4c30009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.048+0000 7f4c237fe700 1 -- 172.21.15.64:0/3741546877 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c2c05b570 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.334+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4c3c105a30 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.335+0000 7f4c237fe700 1 -- 172.21.15.64:0/3741546877 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1424 (secure 0 0 0) 0x7f4c2c013e60 con 0x7f4c3c0fcef0 2024-02-16T23:03:26.335 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 40/261 objects degraded (15.326%), 13 pgs degraded 2024-02-16T23:03:26.335 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:03:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:03:26.335 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-16T23:03:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: osd.5 (root=default,host=smithi192) is down 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 40/261 objects degraded (15.326%), 13 pgs degraded 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-02-16T23:03:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-02-16T23:03:26.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4c2406d7f0 msgr2=0x7f4c2406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:26.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4c2406d7f0 0x7f4c2406fca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4c30007fd0 tx=0x7f4c30009040 comp rx=0 tx=0).stop 2024-02-16T23:03:26.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 msgr2=0x7f4c3c10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:26.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 0x7f4c3c10d970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c000c00 tx=0x7f4c2c009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:26.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 shutdown_connections 2024-02-16T23:03:26.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4c2406d7f0 0x7f4c2406fca0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:26.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c3c0fcef0 0x7f4c3c10d970 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:26.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 --2- 172.21.15.64:0/3741546877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0fe0e0 0x7f4c3c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:26.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 >> 172.21.15.64:0/3741546877 conn(0x7f4c3c0f8620 msgr2=0x7f4c3c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:26.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 shutdown_connections 2024-02-16T23:03:26.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:26.337+0000 7f4c41097700 1 -- 172.21.15.64:0/3741546877 wait complete. 2024-02-16T23:03:26.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:26 smithi064 ceph-mon[93757]: pgmap v162: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 251 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 40/261 objects degraded (15.326%) 2024-02-16T23:03:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:26 smithi064 ceph-mon[93757]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:26 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3573163186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:03:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:26 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3741546877' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:03:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:26 smithi064 ceph-mon[93757]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:03:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:26 smithi064 ceph-mon[93757]: osd.5 [v2:172.21.15.192:6816/2198637799,v1:172.21.15.192:6817/2198637799] boot 2024-02-16T23:03:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:26 smithi064 ceph-mon[93757]: osdmap e79: 6 total, 6 up, 6 in 2024-02-16T23:03:26.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:26 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T23:03:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:26 smithi192 ceph-mon[82652]: pgmap v162: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 251 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 40/261 objects degraded (15.326%) 2024-02-16T23:03:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:26 smithi192 ceph-mon[82652]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:26 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3573163186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:03:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:26 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3741546877' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:03:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:26 smithi192 ceph-mon[82652]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T23:03:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:26 smithi192 ceph-mon[82652]: osd.5 [v2:172.21.15.192:6816/2198637799,v1:172.21.15.192:6817/2198637799] boot 2024-02-16T23:03:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:26 smithi192 ceph-mon[82652]: osdmap e79: 6 total, 6 up, 6 in 2024-02-16T23:03:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:26 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T23:03:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:03:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:27.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-16T23:03:27.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-16T23:03:27.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-16T23:03:27.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-16T23:03:27.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: osdmap e80: 6 total, 6 up, 6 in 2024-02-16T23:03:27.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-16T23:03:27.870 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:27 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-16T23:03:28.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:28.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:28.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:28.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:28.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:03:28.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-16T23:03:28.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-16T23:03:28.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-16T23:03:28.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-16T23:03:28.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: osdmap e80: 6 total, 6 up, 6 in 2024-02-16T23:03:28.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-16T23:03:28.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:27 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-16T23:03:28.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:28 smithi064 ceph-mon[93757]: pgmap v164: 65 pgs: 1 peering, 13 active+undersized+degraded, 14 active+undersized, 37 active+clean; 251 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 40/261 objects degraded (15.326%) 2024-02-16T23:03:28.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:28 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all osd 2024-02-16T23:03:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:28 smithi064 ceph-mon[93757]: Upgrade: Setting require_osd_release to 19 squid 2024-02-16T23:03:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:28 smithi064 ceph-mon[93757]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-16T23:03:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-16T23:03:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:28 smithi064 ceph-mon[93757]: osdmap e81: 6 total, 6 up, 6 in 2024-02-16T23:03:28.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:28 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi064.tqesxx"]}]: dispatch 2024-02-16T23:03:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:28 smithi192 ceph-mon[82652]: pgmap v164: 65 pgs: 1 peering, 13 active+undersized+degraded, 14 active+undersized, 37 active+clean; 251 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 40/261 objects degraded (15.326%) 2024-02-16T23:03:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:28 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all osd 2024-02-16T23:03:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:28 smithi192 ceph-mon[82652]: Upgrade: Setting require_osd_release to 19 squid 2024-02-16T23:03:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:28 smithi192 ceph-mon[82652]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-16T23:03:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-16T23:03:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:28 smithi192 ceph-mon[82652]: osdmap e81: 6 total, 6 up, 6 in 2024-02-16T23:03:29.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:28 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi064.tqesxx"]}]: dispatch 2024-02-16T23:03:29.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:29 smithi064 ceph-mon[93757]: Upgrade: It appears safe to stop mds.cephfs.smithi064.tqesxx 2024-02-16T23:03:29.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:29 smithi064 ceph-mon[93757]: Health check update: Degraded data redundancy: 40/261 objects degraded (15.326%), 13 pgs degraded (PG_DEGRADED) 2024-02-16T23:03:30.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:29 smithi192 ceph-mon[82652]: Upgrade: It appears safe to stop mds.cephfs.smithi064.tqesxx 2024-02-16T23:03:30.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:29 smithi192 ceph-mon[82652]: Health check update: Degraded data redundancy: 40/261 objects degraded (15.326%), 13 pgs degraded (PG_DEGRADED) 2024-02-16T23:03:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:30 smithi064 ceph-mon[93757]: pgmap v167: 65 pgs: 8 peering, 9 active+undersized+degraded, 11 active+undersized, 37 active+clean; 251 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 31/261 objects degraded (11.877%) 2024-02-16T23:03:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.tqesxx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T23:03:30.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:30 smithi192 ceph-mon[82652]: pgmap v167: 65 pgs: 8 peering, 9 active+undersized+degraded, 11 active+undersized, 37 active+clean; 251 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 31/261 objects degraded (11.877%) 2024-02-16T23:03:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.tqesxx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T23:03:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:31.781 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:31 smithi064 ceph-mon[93757]: Upgrade: Updating mds.cephfs.smithi064.tqesxx 2024-02-16T23:03:31.781 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:31 smithi064 ceph-mon[93757]: Deploying daemon mds.cephfs.smithi064.tqesxx on smithi064 2024-02-16T23:03:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:31 smithi192 ceph-mon[82652]: Upgrade: Updating mds.cephfs.smithi064.tqesxx 2024-02-16T23:03:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:31 smithi192 ceph-mon[82652]: Deploying daemon mds.cephfs.smithi064.tqesxx on smithi064 2024-02-16T23:03:32.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[93753]: 2024-02-16T23:03:32.043+0000 7fef0c04c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: pgmap v168: 65 pgs: 8 peering, 57 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/261 objects degraded (11.877%), 9 pgs degraded) 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: osdmap e82: 6 total, 6 up, 6 in 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: Standby daemon mds.cephfs.smithi064.bmcyuf assigned to filesystem cephfs as rank 0 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T23:03:32.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:32 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi064.bmcyuf=up:replay} 2 up:standby 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: pgmap v168: 65 pgs: 8 peering, 57 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/261 objects degraded (11.877%), 9 pgs degraded) 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: osdmap e82: 6 total, 6 up, 6 in 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: Standby daemon mds.cephfs.smithi064.bmcyuf assigned to filesystem cephfs as rank 0 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T23:03:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:32 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi064.bmcyuf=up:replay} 2 up:standby 2024-02-16T23:03:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:33 smithi192 ceph-mon[82652]: pgmap v170: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 1 op/s 2024-02-16T23:03:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:03:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T23:03:34.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:33 smithi064 ceph-mon[93757]: pgmap v170: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 1 op/s 2024-02-16T23:03:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:03:34.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T23:03:36.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:35 smithi192 ceph-mon[82652]: pgmap v171: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-02-16T23:03:36.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:35 smithi064 ceph-mon[93757]: pgmap v171: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-02-16T23:03:38.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:37 smithi192 ceph-mon[82652]: pgmap v172: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-02-16T23:03:38.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:37 smithi064 ceph-mon[93757]: pgmap v172: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-02-16T23:03:39.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:38 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:39.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:38 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:40.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:39 smithi192 ceph-mon[82652]: pgmap v173: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-16T23:03:40.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:39 smithi064 ceph-mon[93757]: pgmap v173: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-16T23:03:42.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:41 smithi192 ceph-mon[82652]: pgmap v174: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-02-16T23:03:42.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:41 smithi064 ceph-mon[93757]: pgmap v174: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-02-16T23:03:44.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:43 smithi192 ceph-mon[82652]: pgmap v175: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-02-16T23:03:44.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:43 smithi192 ceph-mon[82652]: reconnect by client.14528 192.168.0.1:0/3011212362 after 0.000999982 2024-02-16T23:03:44.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:43 smithi192 ceph-mon[82652]: reconnect by client.24327 192.168.0.1:0/1356869742 after 0.000999982 2024-02-16T23:03:44.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:43 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:reconnect 2024-02-16T23:03:44.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:43 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi064.bmcyuf=up:reconnect} 2 up:standby 2024-02-16T23:03:44.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:43 smithi064 ceph-mon[93757]: pgmap v175: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-02-16T23:03:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:43 smithi064 ceph-mon[93757]: reconnect by client.14528 192.168.0.1:0/3011212362 after 0.000999982 2024-02-16T23:03:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:43 smithi064 ceph-mon[93757]: reconnect by client.24327 192.168.0.1:0/1356869742 after 0.000999982 2024-02-16T23:03:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:43 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:reconnect 2024-02-16T23:03:44.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:43 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi064.bmcyuf=up:reconnect} 2 up:standby 2024-02-16T23:03:45.032 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:44 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:rejoin 2024-02-16T23:03:45.032 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:44 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi064.bmcyuf=up:rejoin} 2 up:standby 2024-02-16T23:03:45.032 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:44 smithi064 ceph-mon[93757]: daemon mds.cephfs.smithi064.bmcyuf is now active in filesystem cephfs as rank 0 2024-02-16T23:03:45.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:44 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:rejoin 2024-02-16T23:03:45.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:44 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi064.bmcyuf=up:rejoin} 2 up:standby 2024-02-16T23:03:45.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:44 smithi192 ceph-mon[82652]: daemon mds.cephfs.smithi064.bmcyuf is now active in filesystem cephfs as rank 0 2024-02-16T23:03:46.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:45 smithi064 ceph-mon[93757]: pgmap v176: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-02-16T23:03:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:45 smithi064 ceph-mon[93757]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T23:03:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:45 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:active 2024-02-16T23:03:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:45 smithi064 ceph-mon[93757]: fsmap cephfs:1 {0=cephfs.smithi064.bmcyuf=up:active} 2 up:standby 2024-02-16T23:03:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:45 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:45 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:46.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:45 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:03:46.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:45 smithi192 ceph-mon[82652]: pgmap v176: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-02-16T23:03:46.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:45 smithi192 ceph-mon[82652]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T23:03:46.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:45 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] up:active 2024-02-16T23:03:46.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:45 smithi192 ceph-mon[82652]: fsmap cephfs:1 {0=cephfs.smithi064.bmcyuf=up:active} 2 up:standby 2024-02-16T23:03:46.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:45 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:46.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:45 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:46.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:45 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:03:46.912 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:46 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] up:boot 2024-02-16T23:03:46.912 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:46 smithi064 ceph-mon[93757]: fsmap cephfs:1 {0=cephfs.smithi064.bmcyuf=up:active} 3 up:standby 2024-02-16T23:03:46.912 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T23:03:47.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:46 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] up:boot 2024-02-16T23:03:47.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:46 smithi192 ceph-mon[82652]: fsmap cephfs:1 {0=cephfs.smithi064.bmcyuf=up:active} 3 up:standby 2024-02-16T23:03:47.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T23:03:48.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:47 smithi064 ceph-mon[93757]: pgmap v177: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-16T23:03:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:48.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:03:48.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:47 smithi192 ceph-mon[82652]: pgmap v177: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-16T23:03:48.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:48.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:03:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:49 smithi192 ceph-mon[82652]: pgmap v178: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 255 B/s wr, 6 op/s 2024-02-16T23:03:50.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:49 smithi064 ceph-mon[93757]: pgmap v178: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 255 B/s wr, 6 op/s 2024-02-16T23:03:51.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:51 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:51.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:51 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:51.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:51 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:51.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:51 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:52.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:52 smithi192 ceph-mon[82652]: pgmap v179: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 9.0 MiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-02-16T23:03:52.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:52 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:52.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:52 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:52.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:52 smithi064 ceph-mon[93757]: pgmap v179: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 9.0 MiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-02-16T23:03:52.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:52 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:52.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:52 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:54.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:54 smithi064 ceph-mon[93757]: pgmap v180: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-02-16T23:03:54.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:54 smithi192 ceph-mon[82652]: pgmap v180: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-02-16T23:03:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: pgmap v181: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 308 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-02-16T23:03:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: Detected new or changed devices on smithi064 2024-02-16T23:03:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi064.bmcyuf"]}]: dispatch 2024-02-16T23:03:56.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:56 smithi192 ceph-mon[82652]: Upgrade: It appears safe to stop mds.cephfs.smithi064.bmcyuf 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: pgmap v181: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 308 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: Detected new or changed devices on smithi064 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:03:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi064.bmcyuf"]}]: dispatch 2024-02-16T23:03:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:56 smithi064 ceph-mon[93757]: Upgrade: It appears safe to stop mds.cephfs.smithi064.bmcyuf 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.444+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/144264841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 msgr2=0x7f8fdc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.444+0000 7f8fe3e8b700 1 --2- 172.21.15.64:0/144264841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 0x7f8fdc0fd320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8004940 tx=0x7f8fd8009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.444+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/144264841 shutdown_connections 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.444+0000 7f8fe3e8b700 1 --2- 172.21.15.64:0/144264841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fdc0fe0e0 0x7f8fdc0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.444+0000 7f8fe3e8b700 1 --2- 172.21.15.64:0/144264841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 0x7f8fdc0fd320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.444+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/144264841 >> 172.21.15.64:0/144264841 conn(0x7f8fdc0f8620 msgr2=0x7f8fdc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/144264841 shutdown_connections 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/144264841 wait complete. 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe3e8b700 1 Processor -- start 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe3e8b700 1 -- start start 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe3e8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 0x7f8fdc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe3e8b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fdc0fe0e0 0x7f8fdc0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe3e8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc0708e0 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe3e8b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc070a50 con 0x7f8fdc0fe0e0 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe1c27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 0x7f8fdc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe1c27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 0x7f8fdc071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42434/0 (socket says 172.21.15.64:42434) 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.445+0000 7f8fe1c27700 1 -- 172.21.15.64:0/282583738 learned_addr learned my addr 172.21.15.64:0/282583738 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fe1426700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fdc0fe0e0 0x7f8fdc0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fe1c27700 1 -- 172.21.15.64:0/282583738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fdc0fe0e0 msgr2=0x7f8fdc0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fe1c27700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fdc0fe0e0 0x7f8fdc0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fe1c27700 1 -- 172.21.15.64:0/282583738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8005040 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fe1c27700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 0x7f8fdc071e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800b3b0 tx=0x7f8fd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fd27fc700 1 -- 172.21.15.64:0/282583738 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fd800e040 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fd27fc700 1 -- 172.21.15.64:0/282583738 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fd800bcc0 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc070cd0 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fd27fc700 1 -- 172.21.15.64:0/282583738 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fd801c630 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.446+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc1159a0 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.447+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc04bed0 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.448+0000 7f8fd27fc700 1 -- 172.21.15.64:0/282583738 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8fd8013980 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.448+0000 7f8fd27fc700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc806d7e0 0x7f8fc806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.448+0000 7f8fd27fc700 1 -- 172.21.15.64:0/282583738 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8fd808e380 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.448+0000 7f8fe1426700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc806d7e0 0x7f8fc806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.449+0000 7f8fe1426700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc806d7e0 0x7f8fc806fc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc004f40 tx=0x7f8fcc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.453+0000 7f8fd27fc700 1 -- 172.21.15.64:0/282583738 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fd805b780 con 0x7f8fdc0fcef0 2024-02-16T23:03:56.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.676+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fdc102ac0 con 0x7f8fc806d7e0 2024-02-16T23:03:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.677+0000 7f8fd27fc700 1 -- 172.21.15.64:0/282583738 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8fdc102ac0 con 0x7f8fc806d7e0 2024-02-16T23:03:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc806d7e0 msgr2=0x7f8fc806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc806d7e0 0x7f8fc806fc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc004f40 tx=0x7f8fcc00b040 comp rx=0 tx=0).stop 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 msgr2=0x7f8fdc071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 0x7f8fdc071e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800b3b0 tx=0x7f8fd8009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 shutdown_connections 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc806d7e0 0x7f8fc806fc90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc0fcef0 0x7f8fdc071e60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 --2- 172.21.15.64:0/282583738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fdc0fe0e0 0x7f8fdc0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 >> 172.21.15.64:0/282583738 conn(0x7f8fdc0f8620 msgr2=0x7f8fdc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 shutdown_connections 2024-02-16T23:03:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.679+0000 7f8fe3e8b700 1 -- 172.21.15.64:0/282583738 wait complete. 2024-02-16T23:03:56.690 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.787+0000 7fd7bfab2700 1 -- 172.21.15.64:0/1246489225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fd430 msgr2=0x7fd7b80fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.787+0000 7fd7bfab2700 1 --2- 172.21.15.64:0/1246489225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fd430 0x7fd7b80fd840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4004940 tx=0x7fd7b4009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.787+0000 7fd7bfab2700 1 -- 172.21.15.64:0/1246489225 shutdown_connections 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.787+0000 7fd7bfab2700 1 --2- 172.21.15.64:0/1246489225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b80fc0a0 0x7fd7b80fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.787+0000 7fd7bfab2700 1 --2- 172.21.15.64:0/1246489225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fd430 0x7fd7b80fd840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.787+0000 7fd7bfab2700 1 -- 172.21.15.64:0/1246489225 >> 172.21.15.64:0/1246489225 conn(0x7fd7b80f6ca0 msgr2=0x7fd7b80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bfab2700 1 -- 172.21.15.64:0/1246489225 shutdown_connections 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bfab2700 1 -- 172.21.15.64:0/1246489225 wait complete. 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bfab2700 1 Processor -- start 2024-02-16T23:03:56.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bfab2700 1 -- start start 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bfab2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fc0a0 0x7fd7b80fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bfab2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b80fd430 0x7fd7b80fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bfab2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80fe9f0 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bfab2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80feb60 con 0x7fd7b80fd430 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bd84e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fc0a0 0x7fd7b80fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bd84e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fc0a0 0x7fd7b80fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42466/0 (socket says 172.21.15.64:42466) 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.788+0000 7fd7bd84e700 1 -- 172.21.15.64:0/2302919981 learned_addr learned my addr 172.21.15.64:0/2302919981 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.789+0000 7fd7bd84e700 1 -- 172.21.15.64:0/2302919981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b80fd430 msgr2=0x7fd7b80fe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.789+0000 7fd7bd84e700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b80fd430 0x7fd7b80fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.789+0000 7fd7bd84e700 1 -- 172.21.15.64:0/2302919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4005040 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.789+0000 7fd7bd84e700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fc0a0 0x7fd7b80fff70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009150 tx=0x7fd7b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.790+0000 7fd7ae7fc700 1 -- 172.21.15.64:0/2302919981 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7b400e040 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.790+0000 7fd7ae7fc700 1 -- 172.21.15.64:0/2302919981 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7b400bbf0 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.790+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b80fede0 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.790+0000 7fd7ae7fc700 1 -- 172.21.15.64:0/2302919981 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7b401c630 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.791+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b8115a70 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.792+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b804bed0 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.792+0000 7fd7ae7fc700 1 -- 172.21.15.64:0/2302919981 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd7b400bd60 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.792+0000 7fd7ae7fc700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fd7a406d7e0 0x7fd7a406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:56.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.793+0000 7fd7ae7fc700 1 -- 172.21.15.64:0/2302919981 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd7b408e140 con 0x7fd7b80fc0a0 2024-02-16T23:03:56.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.793+0000 7fd7bd04d700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fd7a406d7e0 0x7fd7a406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:56.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.793+0000 7fd7bd04d700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fd7a406d7e0 0x7fd7a406fc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80fd170 tx=0x7fd7a8009ce0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:56.797+0000 7fd7ae7fc700 1 -- 172.21.15.64:0/2302919981 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7b405b5e0 con 0x7fd7b80fc0a0 2024-02-16T23:03:57.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.036+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7b80fb6a0 con 0x7fd7a406d7e0 2024-02-16T23:03:57.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.038+0000 7fd7ae7fc700 1 -- 172.21.15.64:0/2302919981 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd7b80fb6a0 con 0x7fd7a406d7e0 2024-02-16T23:03:57.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fd7a406d7e0 msgr2=0x7fd7a406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fd7a406d7e0 0x7fd7a406fc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80fd170 tx=0x7fd7a8009ce0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fc0a0 msgr2=0x7fd7b80fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fc0a0 0x7fd7b80fff70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009150 tx=0x7fd7b4009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:57.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 shutdown_connections 2024-02-16T23:03:57.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fd7a406d7e0 0x7fd7a406fc90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b80fc0a0 0x7fd7b80fff70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 --2- 172.21.15.64:0/2302919981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b80fd430 0x7fd7b80fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 >> 172.21.15.64:0/2302919981 conn(0x7fd7b80f6ca0 msgr2=0x7fd7b80faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:57.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 shutdown_connections 2024-02-16T23:03:57.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.040+0000 7fd7bfab2700 1 -- 172.21.15.64:0/2302919981 wait complete. 2024-02-16T23:03:57.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.149+0000 7f04ee419700 1 -- 172.21.15.64:0/3734593004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fc7f0 msgr2=0x7f04e80fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.149+0000 7f04ee419700 1 --2- 172.21.15.64:0/3734593004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fc7f0 0x7f04e80fcc40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04dc004940 tx=0x7f04dc009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:57.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.149+0000 7f04ee419700 1 -- 172.21.15.64:0/3734593004 shutdown_connections 2024-02-16T23:03:57.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.149+0000 7f04ee419700 1 --2- 172.21.15.64:0/3734593004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fc7f0 0x7f04e80fcc40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.149+0000 7f04ee419700 1 --2- 172.21.15.64:0/3734593004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e80fb5e0 0x7f04e80fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.149+0000 7f04ee419700 1 -- 172.21.15.64:0/3734593004 >> 172.21.15.64:0/3734593004 conn(0x7f04e80f6ca0 msgr2=0x7f04e80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.149+0000 7f04ee419700 1 -- 172.21.15.64:0/3734593004 shutdown_connections 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.149+0000 7f04ee419700 1 -- 172.21.15.64:0/3734593004 wait complete. 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04ee419700 1 Processor -- start 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04ee419700 1 -- start start 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04ee419700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fb5e0 0x7f04e81007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04ee419700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e80fc7f0 0x7f04e80fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04ee419700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e80ff2d0 con 0x7f04e80fb5e0 2024-02-16T23:03:57.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04ee419700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e80ff440 con 0x7f04e80fc7f0 2024-02-16T23:03:57.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04e7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fb5e0 0x7f04e81007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:57.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04e7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fb5e0 0x7f04e81007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42480/0 (socket says 172.21.15.64:42480) 2024-02-16T23:03:57.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.150+0000 7f04e7fff700 1 -- 172.21.15.64:0/4268066098 learned_addr learned my addr 172.21.15.64:0/4268066098 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04e7fff700 1 -- 172.21.15.64:0/4268066098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e80fc7f0 msgr2=0x7f04e80fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04e7fff700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e80fc7f0 0x7f04e80fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04e7fff700 1 -- 172.21.15.64:0/4268066098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc005040 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04e7fff700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fb5e0 0x7f04e81007c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f04d800b790 tx=0x7f04d8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04e4ff9700 1 -- 172.21.15.64:0/4268066098 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04d800f6f0 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e80ff720 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04e4ff9700 1 -- 172.21.15.64:0/4268066098 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04d800fd20 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04e4ff9700 1 -- 172.21.15.64:0/4268066098 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04d80103a0 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.151+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e80ffaa0 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.152+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e804bed0 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.153+0000 7f04e4ff9700 1 -- 172.21.15.64:0/4268066098 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f04d8011400 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.153+0000 7f04e4ff9700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f04d006d7e0 0x7f04d006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.153+0000 7f04e4ff9700 1 -- 172.21.15.64:0/4268066098 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f04d808cc60 con 0x7f04e80fb5e0 2024-02-16T23:03:57.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.153+0000 7f04e77fe700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f04d006d7e0 0x7f04d006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:57.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.153+0000 7f04e77fe700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f04d006d7e0 0x7f04d006fc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04dc004f90 tx=0x7f04dc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:57.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.162+0000 7f04e4ff9700 1 -- 172.21.15.64:0/4268066098 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04d805a060 con 0x7f04e80fb5e0 2024-02-16T23:03:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.383+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f04e8115d40 con 0x7f04d006d7e0 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.394+0000 7f04e4ff9700 1 -- 172.21.15.64:0/4268066098 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f04e8115d40 con 0x7f04d006d7e0 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (7m) 7s ago 15m 26.0M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (3m) 7s ago 15m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (3m) 34s ago 14m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (6m) 7s ago 15m 84.5M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (10m) 7s ago 10m 176M - 17.2.7-513-gb88fae66 870a0944ccb8 06812e00895f 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (12s) 7s ago 10m 13.7M - 19.0.0-1450-gf78a58c0 f081f847070a bd35cb99fa04 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (10m) 34s ago 10m 17.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (10m) 34s ago 10m 17.0M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (6m) 7s ago 16m 507M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:03:57.394 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (5m) 34s ago 14m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (5m) 7s ago 17m 48.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (4m) 34s ago 14m 39.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (7m) 7s ago 15m 15.1M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 34s ago 14m 15.2M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (3m) 7s ago 13m 105M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (2m) 7s ago 13m 96.3M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (2m) 7s ago 12m 83.9M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (105s) 34s ago 12m 107M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (71s) 34s ago 12m 99.3M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (38s) 34s ago 11m 13.5M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 7bb5b265fc66 2024-02-16T23:03:57.395 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (5m) 7s ago 15m 61.5M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f04d006d7e0 msgr2=0x7f04d006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f04d006d7e0 0x7f04d006fc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04dc004f90 tx=0x7f04dc009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fb5e0 msgr2=0x7f04e81007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fb5e0 0x7f04e81007c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f04d800b790 tx=0x7f04d8009220 comp rx=0 tx=0).stop 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 shutdown_connections 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f04d006d7e0 0x7f04d006fc90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e80fb5e0 0x7f04e81007c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 --2- 172.21.15.64:0/4268066098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e80fc7f0 0x7f04e80fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 >> 172.21.15.64:0/4268066098 conn(0x7f04e80f6ca0 msgr2=0x7f04e80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 shutdown_connections 2024-02-16T23:03:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.396+0000 7f04ee419700 1 -- 172.21.15.64:0/4268066098 wait complete. 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.498+0000 7f9122e58700 1 -- 172.21.15.64:0/725638007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fe0e0 msgr2=0x7f911c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.498+0000 7f9122e58700 1 --2- 172.21.15.64:0/725638007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fe0e0 0x7f911c0fe550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9108004940 tx=0x7f9108009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.498+0000 7f9122e58700 1 -- 172.21.15.64:0/725638007 shutdown_connections 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.498+0000 7f9122e58700 1 --2- 172.21.15.64:0/725638007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fe0e0 0x7f911c0fe550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.498+0000 7f9122e58700 1 --2- 172.21.15.64:0/725638007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f911c0fcef0 0x7f911c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.498+0000 7f9122e58700 1 -- 172.21.15.64:0/725638007 >> 172.21.15.64:0/725638007 conn(0x7f911c0f8620 msgr2=0x7f911c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.498+0000 7f9122e58700 1 -- 172.21.15.64:0/725638007 shutdown_connections 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.498+0000 7f9122e58700 1 -- 172.21.15.64:0/725638007 wait complete. 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9122e58700 1 Processor -- start 2024-02-16T23:03:57.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9122e58700 1 -- start start 2024-02-16T23:03:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9122e58700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fcef0 0x7f911c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9122e58700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f911c0fe0e0 0x7f911c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9122e58700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c10e4b0 con 0x7f911c0fcef0 2024-02-16T23:03:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9122e58700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c10ed50 con 0x7f911c0fe0e0 2024-02-16T23:03:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9120bf4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fcef0 0x7f911c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9120bf4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fcef0 0x7f911c10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42498/0 (socket says 172.21.15.64:42498) 2024-02-16T23:03:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.499+0000 7f9120bf4700 1 -- 172.21.15.64:0/1641833362 learned_addr learned my addr 172.21.15.64:0/1641833362 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f911bfff700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f911c0fe0e0 0x7f911c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f9120bf4700 1 -- 172.21.15.64:0/1641833362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f911c0fe0e0 msgr2=0x7f911c10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f9120bf4700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f911c0fe0e0 0x7f911c10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f9120bf4700 1 -- 172.21.15.64:0/1641833362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9108005040 con 0x7f911c0fcef0 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f911bfff700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f911c0fe0e0 0x7f911c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f9120bf4700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fcef0 0x7f911c10d970 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9110002a80 tx=0x7f9110004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f91197fa700 1 -- 172.21.15.64:0/1641833362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9110007cc0 con 0x7f911c0fcef0 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f91197fa700 1 -- 172.21.15.64:0/1641833362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f911000f040 con 0x7f911c0fcef0 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c10f030 con 0x7f911c0fcef0 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f91197fa700 1 -- 172.21.15.64:0/1641833362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f911000dad0 con 0x7f911c0fcef0 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.500+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c10f570 con 0x7f911c0fcef0 2024-02-16T23:03:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.501+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c04bed0 con 0x7f911c0fcef0 2024-02-16T23:03:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.501+0000 7f91197fa700 1 -- 172.21.15.64:0/1641833362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f911000dc30 con 0x7f911c0fcef0 2024-02-16T23:03:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.502+0000 7f91197fa700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f910c06d7e0 0x7f910c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.502+0000 7f91197fa700 1 -- 172.21.15.64:0/1641833362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f911008cc80 con 0x7f911c0fcef0 2024-02-16T23:03:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.502+0000 7f911bfff700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f910c06d7e0 0x7f910c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.502+0000 7f911bfff700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f910c06d7e0 0x7f910c06fc90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9108000c00 tx=0x7f9108009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.506+0000 7f91197fa700 1 -- 172.21.15.64:0/1641833362 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f911005a000 con 0x7f911c0fcef0 2024-02-16T23:03:57.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.810+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f911c10fb70 con 0x7f911c0fcef0 2024-02-16T23:03:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.811+0000 7f91197fa700 1 -- 172.21.15.64:0/1641833362 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f9110059b90 con 0x7f911c0fcef0 2024-02-16T23:03:57.811 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:03:57.811 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:03:57.811 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:03:57.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:03:57.811 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:03:57.811 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 3, 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 3, 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 11 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:03:57.812 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:03:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f910c06d7e0 msgr2=0x7f910c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f910c06d7e0 0x7f910c06fc90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9108000c00 tx=0x7f9108009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fcef0 msgr2=0x7f911c10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fcef0 0x7f911c10d970 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9110002a80 tx=0x7f9110004600 comp rx=0 tx=0).stop 2024-02-16T23:03:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 shutdown_connections 2024-02-16T23:03:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f910c06d7e0 0x7f910c06fc90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f911c0fcef0 0x7f911c10d970 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 --2- 172.21.15.64:0/1641833362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f911c0fe0e0 0x7f911c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 >> 172.21.15.64:0/1641833362 conn(0x7f911c0f8620 msgr2=0x7f911c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 shutdown_connections 2024-02-16T23:03:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.813+0000 7f9122e58700 1 -- 172.21.15.64:0/1641833362 wait complete. 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 -- 172.21.15.64:0/1408099909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 msgr2=0x7fc6500fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 --2- 172.21.15.64:0/1408099909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 0x7fc6500fd320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc640004940 tx=0x7fc640009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 -- 172.21.15.64:0/1408099909 shutdown_connections 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 --2- 172.21.15.64:0/1408099909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6500fe0e0 0x7fc6500fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 --2- 172.21.15.64:0/1408099909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 0x7fc6500fd320 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 -- 172.21.15.64:0/1408099909 >> 172.21.15.64:0/1408099909 conn(0x7fc6500f8620 msgr2=0x7fc6500faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 -- 172.21.15.64:0/1408099909 shutdown_connections 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 -- 172.21.15.64:0/1408099909 wait complete. 2024-02-16T23:03:57.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.916+0000 7fc657ae6700 1 Processor -- start 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc657ae6700 1 -- start start 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc657ae6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 0x7fc65010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc657ae6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6500fe0e0 0x7fc65010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc657ae6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65010e4d0 con 0x7fc6500fcef0 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc657ae6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65010ed70 con 0x7fc6500fe0e0 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc655882700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 0x7fc65010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc655882700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 0x7fc65010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42514/0 (socket says 172.21.15.64:42514) 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc655882700 1 -- 172.21.15.64:0/690745540 learned_addr learned my addr 172.21.15.64:0/690745540 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc655081700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6500fe0e0 0x7fc65010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc655882700 1 -- 172.21.15.64:0/690745540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6500fe0e0 msgr2=0x7fc65010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc655882700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6500fe0e0 0x7fc65010ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc655882700 1 -- 172.21.15.64:0/690745540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640005040 con 0x7fc6500fcef0 2024-02-16T23:03:57.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc655882700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 0x7fc65010d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc64000b3b0 tx=0x7fc640009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:57.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.917+0000 7fc6467fc700 1 -- 172.21.15.64:0/690745540 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc64000e040 con 0x7fc6500fcef0 2024-02-16T23:03:57.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.918+0000 7fc6467fc700 1 -- 172.21.15.64:0/690745540 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc64000bcc0 con 0x7fc6500fcef0 2024-02-16T23:03:57.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.918+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65010eff0 con 0x7fc6500fcef0 2024-02-16T23:03:57.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.918+0000 7fc6467fc700 1 -- 172.21.15.64:0/690745540 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc64001c630 con 0x7fc6500fcef0 2024-02-16T23:03:57.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.918+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65010f530 con 0x7fc6500fcef0 2024-02-16T23:03:57.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.919+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65004bed0 con 0x7fc6500fcef0 2024-02-16T23:03:57.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.919+0000 7fc6467fc700 1 -- 172.21.15.64:0/690745540 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc640013980 con 0x7fc6500fcef0 2024-02-16T23:03:57.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.920+0000 7fc6467fc700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc63c06d7e0 0x7fc63c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:57.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.920+0000 7fc6467fc700 1 -- 172.21.15.64:0/690745540 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc64008e380 con 0x7fc6500fcef0 2024-02-16T23:03:57.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.920+0000 7fc655081700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc63c06d7e0 0x7fc63c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:57.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.920+0000 7fc655081700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc63c06d7e0 0x7fc63c06fc90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc64c004f60 tx=0x7fc64c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:57.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:57.924+0000 7fc6467fc700 1 -- 172.21.15.64:0/690745540 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc64005b780 con 0x7fc6500fcef0 2024-02-16T23:03:58.072 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:58.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: pgmap v182: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-02-16T23:03:58.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: Upgrade: Updating mds.cephfs.smithi064.bmcyuf 2024-02-16T23:03:58.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:58.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.bmcyuf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T23:03:58.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:58.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: Deploying daemon mds.cephfs.smithi064.bmcyuf on smithi064 2024-02-16T23:03:58.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:58.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:57 smithi064 ceph-mon[93757]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:58.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.184+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc650105a50 con 0x7fc6500fcef0 2024-02-16T23:03:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.185+0000 7fc6467fc700 1 -- 172.21.15.64:0/690745540 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1849 (secure 0 0 0) 0x7fc64005b310 con 0x7fc6500fcef0 2024-02-16T23:03:58.185 INFO:teuthology.orchestra.run.smithi064.stdout:e19 2024-02-16T23:03:58.185 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:03:58.185 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:03:58.185 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:03:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:03:58.185 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:03:58.185 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 18 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T23:03:43.908124+0000 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 82 2024-02-16T23:03:58.186 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=14484} 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{0:14484} state up:active seq 151 join_fscid=1 addr [v2:172.21.15.64:6828/168979118,v1:172.21.15.64:6829/168979118] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:03:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:03:58.188 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:03:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:03:58.188 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:24281} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:03:58.188 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:24297} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:03:58.188 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc63c06d7e0 msgr2=0x7fc63c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc63c06d7e0 0x7fc63c06fc90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc64c004f60 tx=0x7fc64c00b040 comp rx=0 tx=0).stop 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 msgr2=0x7fc65010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 0x7fc65010d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc64000b3b0 tx=0x7fc640009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 shutdown_connections 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc63c06d7e0 0x7fc63c06fc90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6500fcef0 0x7fc65010d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 --2- 172.21.15.64:0/690745540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6500fe0e0 0x7fc65010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 >> 172.21.15.64:0/690745540 conn(0x7fc6500f8620 msgr2=0x7fc6501013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 shutdown_connections 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.187+0000 7fc657ae6700 1 -- 172.21.15.64:0/690745540 wait complete. 2024-02-16T23:03:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 19 2024-02-16T23:03:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: pgmap v182: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-02-16T23:03:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: Upgrade: Updating mds.cephfs.smithi064.bmcyuf 2024-02-16T23:03:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:03:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi064.bmcyuf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T23:03:58.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:03:58.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: Deploying daemon mds.cephfs.smithi064.bmcyuf on smithi064 2024-02-16T23:03:58.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:58.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:57 smithi192 ceph-mon[82652]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 -- 172.21.15.64:0/3973429813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fcef0 msgr2=0x7fe4e40fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 --2- 172.21.15.64:0/3973429813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fcef0 0x7fe4e40fd320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009ce0 tx=0x7fe4dc004d10 comp rx=0 tx=0).stop 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 -- 172.21.15.64:0/3973429813 shutdown_connections 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 --2- 172.21.15.64:0/3973429813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e40fe0e0 0x7fe4e40fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 --2- 172.21.15.64:0/3973429813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fcef0 0x7fe4e40fd320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 -- 172.21.15.64:0/3973429813 >> 172.21.15.64:0/3973429813 conn(0x7fe4e40f8620 msgr2=0x7fe4e40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 -- 172.21.15.64:0/3973429813 shutdown_connections 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 -- 172.21.15.64:0/3973429813 wait complete. 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 Processor -- start 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.291+0000 7fe4ec443700 1 -- start start 2024-02-16T23:03:58.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4ec443700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e40fcef0 0x7fe4e410dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4ec443700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fe0e0 0x7fe4e410e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4ec443700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e410e610 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4ec443700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e410eeb0 con 0x7fe4e40fcef0 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e99de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fe0e0 0x7fe4e410e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e99de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fe0e0 0x7fe4e410e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42536/0 (socket says 172.21.15.64:42536) 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e99de700 1 -- 172.21.15.64:0/842013738 learned_addr learned my addr 172.21.15.64:0/842013738 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4ea1df700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e40fcef0 0x7fe4e410dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e99de700 1 -- 172.21.15.64:0/842013738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e40fcef0 msgr2=0x7fe4e410dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e99de700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e40fcef0 0x7fe4e410dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e99de700 1 -- 172.21.15.64:0/842013738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc009940 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e99de700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fe0e0 0x7fe4e410e010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8002a80 tx=0x7fe4d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e2ffd700 1 -- 172.21.15.64:0/842013738 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4d8007cc0 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e2ffd700 1 -- 172.21.15.64:0/842013738 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4d800f040 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e410f190 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.292+0000 7fe4e2ffd700 1 -- 172.21.15.64:0/842013738 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4d80096f0 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.293+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e410f6d0 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.293+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e404bed0 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.294+0000 7fe4e2ffd700 1 -- 172.21.15.64:0/842013738 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe4d800d690 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.294+0000 7fe4e2ffd700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe4cc06d7f0 0x7fe4cc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:58.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.294+0000 7fe4e2ffd700 1 -- 172.21.15.64:0/842013738 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe4d808c680 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.294+0000 7fe4ea1df700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe4cc06d7f0 0x7fe4cc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:58.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.295+0000 7fe4ea1df700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe4cc06d7f0 0x7fe4cc06fca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b740 tx=0x7fe4dc004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:58.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.299+0000 7fe4e2ffd700 1 -- 172.21.15.64:0/842013738 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4d8059a00 con 0x7fe4e40fe0e0 2024-02-16T23:03:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.549+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4e410fb20 con 0x7fe4cc06d7f0 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.550+0000 7fe4e2ffd700 1 -- 172.21.15.64:0/842013738 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe4e410fb20 con 0x7fe4cc06d7f0 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "osd", 2024-02-16T23:03:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:03:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:03:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "13/21 daemons upgraded", 2024-02-16T23:03:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T23:03:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:03:58.552 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.552+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe4cc06d7f0 msgr2=0x7fe4cc06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.552+0000 7fe4ec443700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe4cc06d7f0 0x7fe4cc06fca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b740 tx=0x7fe4dc004d10 comp rx=0 tx=0).stop 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.552+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fe0e0 msgr2=0x7fe4e410e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.552+0000 7fe4ec443700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fe0e0 0x7fe4e410e010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8002a80 tx=0x7fe4d80046b0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.553+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 shutdown_connections 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.553+0000 7fe4ec443700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fe4cc06d7f0 0x7fe4cc06fca0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.553+0000 7fe4ec443700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e40fcef0 0x7fe4e410dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.553+0000 7fe4ec443700 1 --2- 172.21.15.64:0/842013738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e40fe0e0 0x7fe4e410e010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.553+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 >> 172.21.15.64:0/842013738 conn(0x7fe4e40f8620 msgr2=0x7fe4e41013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:58.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.553+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 shutdown_connections 2024-02-16T23:03:58.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.553+0000 7fe4ec443700 1 -- 172.21.15.64:0/842013738 wait complete. 2024-02-16T23:03:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.656+0000 7fabfad3f700 1 -- 172.21.15.64:0/3731688269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf40fbca0 msgr2=0x7fabf40fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.656+0000 7fabfad3f700 1 --2- 172.21.15.64:0/3731688269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf40fbca0 0x7fabf40fe550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fabe0004940 tx=0x7fabe0009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.657+0000 7fabfad3f700 1 -- 172.21.15.64:0/3731688269 shutdown_connections 2024-02-16T23:03:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.657+0000 7fabfad3f700 1 --2- 172.21.15.64:0/3731688269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf40fbca0 0x7fabf40fe550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.657+0000 7fabfad3f700 1 --2- 172.21.15.64:0/3731688269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabf40fb2c0 0x7fabf40fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.657+0000 7fabfad3f700 1 -- 172.21.15.64:0/3731688269 >> 172.21.15.64:0/3731688269 conn(0x7fabf40f6ca0 msgr2=0x7fabf40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.657+0000 7fabfad3f700 1 -- 172.21.15.64:0/3731688269 shutdown_connections 2024-02-16T23:03:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.657+0000 7fabfad3f700 1 -- 172.21.15.64:0/3731688269 wait complete. 2024-02-16T23:03:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.657+0000 7fabfad3f700 1 Processor -- start 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.657+0000 7fabfad3f700 1 -- start start 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabfad3f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabf40fb2c0 0x7fabf410dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabfad3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf410e210 0x7fabf410ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabfad3f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf410e6d0 con 0x7fabf410e210 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabfad3f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf410e810 con 0x7fabf40fb2c0 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabf3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf410e210 0x7fabf410ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabf3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf410e210 0x7fabf410ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42554/0 (socket says 172.21.15.64:42554) 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabf3fff700 1 -- 172.21.15.64:0/1111349161 learned_addr learned my addr 172.21.15.64:0/1111349161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabf8adb700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabf40fb2c0 0x7fabf410dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabf3fff700 1 -- 172.21.15.64:0/1111349161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabf40fb2c0 msgr2=0x7fabf410dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabf3fff700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabf40fb2c0 0x7fabf410dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.658+0000 7fabf3fff700 1 -- 172.21.15.64:0/1111349161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe0005040 con 0x7fabf410e210 2024-02-16T23:03:58.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.659+0000 7fabf3fff700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf410e210 0x7fabf410ef70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fabe000b3b0 tx=0x7fabe0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:58.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.659+0000 7fabf17fa700 1 -- 172.21.15.64:0/1111349161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe000e040 con 0x7fabf410e210 2024-02-16T23:03:58.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.659+0000 7fabf17fa700 1 -- 172.21.15.64:0/1111349161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabe000bbf0 con 0x7fabf410e210 2024-02-16T23:03:58.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.659+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf410f5f0 con 0x7fabf410e210 2024-02-16T23:03:58.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.659+0000 7fabf17fa700 1 -- 172.21.15.64:0/1111349161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe001c630 con 0x7fabf410e210 2024-02-16T23:03:58.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.659+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf410fb30 con 0x7fabf410e210 2024-02-16T23:03:58.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.660+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf404bed0 con 0x7fabf410e210 2024-02-16T23:03:58.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.661+0000 7fabf17fa700 1 -- 172.21.15.64:0/1111349161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fabe000bd60 con 0x7fabf410e210 2024-02-16T23:03:58.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.661+0000 7fabf17fa700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fabe406d7e0 0x7fabe406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:03:58.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.661+0000 7fabf17fa700 1 -- 172.21.15.64:0/1111349161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fabe008e280 con 0x7fabf410e210 2024-02-16T23:03:58.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.661+0000 7fabf8adb700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fabe406d7e0 0x7fabe406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:03:58.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.662+0000 7fabf8adb700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fabe406d7e0 0x7fabe406fc90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabe8004f40 tx=0x7fabe800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:03:58.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.668+0000 7fabf17fa700 1 -- 172.21.15.64:0/1111349161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fabe005b680 con 0x7fabf410e210 2024-02-16T23:03:58.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:58 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1641833362' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:58.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:58 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/690745540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:03:58.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:58 smithi064 ceph-mon[93757]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.983+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fabf410fd50 con 0x7fabf410e210 2024-02-16T23:03:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.984+0000 7fabf17fa700 1 -- 172.21.15.64:0/1111349161 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fabe005b210 con 0x7fabf410e210 2024-02-16T23:03:58.984 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:03:58.985 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:03:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:03:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fabe406d7e0 msgr2=0x7fabe406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fabe406d7e0 0x7fabe406fc90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabe8004f40 tx=0x7fabe800b040 comp rx=0 tx=0).stop 2024-02-16T23:03:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf410e210 msgr2=0x7fabf410ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:03:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf410e210 0x7fabf410ef70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fabe000b3b0 tx=0x7fabe0009e30 comp rx=0 tx=0).stop 2024-02-16T23:03:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 shutdown_connections 2024-02-16T23:03:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fabe406d7e0 0x7fabe406fc90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabf40fb2c0 0x7fabf410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 --2- 172.21.15.64:0/1111349161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf410e210 0x7fabf410ef70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:03:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 >> 172.21.15.64:0/1111349161 conn(0x7fabf40f6ca0 msgr2=0x7fabf4101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:03:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 shutdown_connections 2024-02-16T23:03:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:03:58.986+0000 7fabfad3f700 1 -- 172.21.15.64:0/1111349161 wait complete. 2024-02-16T23:03:59.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:58 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1641833362' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:03:59.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:58 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/690745540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:03:59.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:58 smithi192 ceph-mon[82652]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:03:59.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[93753]: 2024-02-16T23:03:59.012+0000 7fef0c04c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: pgmap v183: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1111349161' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: osdmap e83: 6 total, 6 up, 6 in 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: Standby daemon mds.cephfs.smithi192.tnppti assigned to filesystem cephfs as rank 0 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T23:04:00.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:03:59 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi192.tnppti=up:replay} 2 up:standby 2024-02-16T23:04:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: pgmap v183: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-02-16T23:04:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1111349161' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:04:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T23:04:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: osdmap e83: 6 total, 6 up, 6 in 2024-02-16T23:04:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: Standby daemon mds.cephfs.smithi192.tnppti assigned to filesystem cephfs as rank 0 2024-02-16T23:04:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T23:04:00.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T23:04:00.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:03:59 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi192.tnppti=up:replay} 2 up:standby 2024-02-16T23:04:02.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:01 smithi064 ceph-mon[93757]: pgmap v185: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 0 op/s 2024-02-16T23:04:02.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:01 smithi192 ceph-mon[82652]: pgmap v185: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 0 op/s 2024-02-16T23:04:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:03 smithi192 ceph-mon[82652]: pgmap v186: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 2 op/s 2024-02-16T23:04:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:04.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:04:04.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:03 smithi064 ceph-mon[93757]: pgmap v186: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 2 op/s 2024-02-16T23:04:04.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:04.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:04:06.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:05 smithi192 ceph-mon[82652]: pgmap v187: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-02-16T23:04:06.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:05 smithi064 ceph-mon[93757]: pgmap v187: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-02-16T23:04:08.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:07 smithi192 ceph-mon[82652]: pgmap v188: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-02-16T23:04:08.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:07 smithi064 ceph-mon[93757]: pgmap v188: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-02-16T23:04:10.004 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:09 smithi064 ceph-mon[93757]: pgmap v189: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T23:04:10.004 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:09 smithi064 ceph-mon[93757]: reconnect by client.24327 192.168.0.1:0/1356869742 after 0 2024-02-16T23:04:10.005 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:09 smithi064 ceph-mon[93757]: reconnect by client.14528 192.168.0.1:0/3011212362 after 0.00099998 2024-02-16T23:04:10.005 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:09 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:reconnect 2024-02-16T23:04:10.005 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:09 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi192.tnppti=up:reconnect} 2 up:standby 2024-02-16T23:04:10.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:09 smithi192 ceph-mon[82652]: pgmap v189: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T23:04:10.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:09 smithi192 ceph-mon[82652]: reconnect by client.24327 192.168.0.1:0/1356869742 after 0 2024-02-16T23:04:10.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:09 smithi192 ceph-mon[82652]: reconnect by client.14528 192.168.0.1:0/3011212362 after 0.00099998 2024-02-16T23:04:10.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:09 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:reconnect 2024-02-16T23:04:10.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:09 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi192.tnppti=up:reconnect} 2 up:standby 2024-02-16T23:04:11.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:10 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:rejoin 2024-02-16T23:04:11.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:10 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi192.tnppti=up:rejoin} 2 up:standby 2024-02-16T23:04:11.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:10 smithi192 ceph-mon[82652]: daemon mds.cephfs.smithi192.tnppti is now active in filesystem cephfs as rank 0 2024-02-16T23:04:11.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:10 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:11.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:10 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:11.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:10 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:11.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:10 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:rejoin 2024-02-16T23:04:11.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:10 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi192.tnppti=up:rejoin} 2 up:standby 2024-02-16T23:04:11.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:10 smithi064 ceph-mon[93757]: daemon mds.cephfs.smithi192.tnppti is now active in filesystem cephfs as rank 0 2024-02-16T23:04:11.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:10 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:11.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:10 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:11.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:10 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:12.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:11 smithi192 ceph-mon[82652]: pgmap v190: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-02-16T23:04:12.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:11 smithi192 ceph-mon[82652]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T23:04:12.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:11 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:active 2024-02-16T23:04:12.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:11 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.64:6828/2551190588,v1:172.21.15.64:6829/2551190588] up:boot 2024-02-16T23:04:12.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:11 smithi192 ceph-mon[82652]: fsmap cephfs:1 {0=cephfs.smithi192.tnppti=up:active} 3 up:standby 2024-02-16T23:04:12.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:11 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T23:04:12.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:11 smithi064 ceph-mon[93757]: pgmap v190: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-02-16T23:04:12.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:11 smithi064 ceph-mon[93757]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T23:04:12.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:11 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.192:6824/1462436420,v1:172.21.15.192:6825/1462436420] up:active 2024-02-16T23:04:12.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:11 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.64:6828/2551190588,v1:172.21.15.64:6829/2551190588] up:boot 2024-02-16T23:04:12.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:11 smithi064 ceph-mon[93757]: fsmap cephfs:1 {0=cephfs.smithi192.tnppti=up:active} 3 up:standby 2024-02-16T23:04:12.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:11 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T23:04:13.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:13 smithi064 ceph-mon[93757]: pgmap v191: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 8 op/s 2024-02-16T23:04:14.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:13 smithi192 ceph-mon[82652]: pgmap v191: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 8 op/s 2024-02-16T23:04:16.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:16 smithi192 ceph-mon[82652]: pgmap v192: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 7 op/s 2024-02-16T23:04:16.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:16.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:16.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:16 smithi064 ceph-mon[93757]: pgmap v192: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 7 op/s 2024-02-16T23:04:16.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:16.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:17.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:17.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:17.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:17.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:18.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:18 smithi064 ceph-mon[93757]: pgmap v193: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 170 B/s wr, 4 op/s 2024-02-16T23:04:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:18.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:04:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:18 smithi192 ceph-mon[82652]: pgmap v193: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 170 B/s wr, 4 op/s 2024-02-16T23:04:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: pgmap v194: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 170 B/s wr, 4 op/s 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:20.869 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi192.tnppti"]}]: dispatch 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: pgmap v194: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 170 B/s wr, 4 op/s 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:21.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:21.010 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:21.010 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:21.010 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi192.tnppti"]}]: dispatch 2024-02-16T23:04:21.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:21 smithi192 ceph-mon[82652]: Upgrade: It appears safe to stop mds.cephfs.smithi192.tnppti 2024-02-16T23:04:21.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:21 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:21.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:21 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.tnppti", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T23:04:21.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:21 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:04:21.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:21 smithi064 ceph-mon[93757]: Upgrade: It appears safe to stop mds.cephfs.smithi192.tnppti 2024-02-16T23:04:21.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:21 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:21.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:21 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.tnppti", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T23:04:21.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:21 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:04:22.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:22 smithi192 ceph-mon[82652]: pgmap v195: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:04:22.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:22 smithi192 ceph-mon[82652]: Upgrade: Updating mds.cephfs.smithi192.tnppti 2024-02-16T23:04:22.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:22 smithi192 ceph-mon[82652]: Deploying daemon mds.cephfs.smithi192.tnppti on smithi192 2024-02-16T23:04:22.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:22 smithi064 ceph-mon[93757]: pgmap v195: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:04:22.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:22 smithi064 ceph-mon[93757]: Upgrade: Updating mds.cephfs.smithi192.tnppti 2024-02-16T23:04:22.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:22 smithi064 ceph-mon[93757]: Deploying daemon mds.cephfs.smithi192.tnppti on smithi192 2024-02-16T23:04:23.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:23 smithi192 ceph-mon[82652]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T23:04:23.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:23 smithi192 ceph-mon[82652]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:23.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:23 smithi192 ceph-mon[82652]: osdmap e84: 6 total, 6 up, 6 in 2024-02-16T23:04:23.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:23 smithi192 ceph-mon[82652]: Standby daemon mds.cephfs.smithi192.luiybj assigned to filesystem cephfs as rank 0 2024-02-16T23:04:23.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:23 smithi192 ceph-mon[82652]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T23:04:23.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:23 smithi192 ceph-mon[82652]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T23:04:23.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:23 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi192.luiybj=up:replay} 2 up:standby 2024-02-16T23:04:23.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[93753]: 2024-02-16T23:04:23.398+0000 7fef0c04c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:23.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:23 smithi064 ceph-mon[93757]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T23:04:23.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:23 smithi064 ceph-mon[93757]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:23.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:23 smithi064 ceph-mon[93757]: osdmap e84: 6 total, 6 up, 6 in 2024-02-16T23:04:23.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:23 smithi064 ceph-mon[93757]: Standby daemon mds.cephfs.smithi192.luiybj assigned to filesystem cephfs as rank 0 2024-02-16T23:04:23.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:23 smithi064 ceph-mon[93757]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T23:04:23.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:23 smithi064 ceph-mon[93757]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T23:04:23.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:23 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi192.luiybj=up:replay} 2 up:standby 2024-02-16T23:04:24.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:24 smithi064 ceph-mon[93757]: pgmap v196: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:04:24.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:24 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T23:04:25.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:24 smithi192 ceph-mon[82652]: pgmap v196: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:04:25.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:24 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T23:04:26.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:26 smithi064 ceph-mon[93757]: pgmap v198: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-02-16T23:04:27.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:26 smithi192 ceph-mon[82652]: pgmap v198: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-02-16T23:04:28.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:28 smithi064 ceph-mon[93757]: pgmap v199: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 4.8 KiB/s wr, 4 op/s 2024-02-16T23:04:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:28 smithi192 ceph-mon[82652]: pgmap v199: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 4.8 KiB/s wr, 4 op/s 2024-02-16T23:04:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/3441406842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 msgr2=0x7f1cf00fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 --2- 172.21.15.64:0/3441406842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 0x7f1cf00fd430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4004940 tx=0x7f1ce4009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/3441406842 shutdown_connections 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 --2- 172.21.15.64:0/3441406842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 0x7f1cf00fd430 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 --2- 172.21.15.64:0/3441406842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf00fbe50 0x7f1cf00fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/3441406842 >> 172.21.15.64:0/3441406842 conn(0x7f1cf00f7510 msgr2=0x7f1cf00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/3441406842 shutdown_connections 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/3441406842 wait complete. 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 Processor -- start 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.105+0000 7f1cf4ed0700 1 -- start start 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cf4ed0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf00fbe50 0x7f1cf01105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cf4ed0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 0x7f1cf0112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cf4ed0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0066cf0 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cf4ed0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0066e30 con 0x7f1cf00fbe50 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cedd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 0x7f1cf0112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cee59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf00fbe50 0x7f1cf01105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cedd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 0x7f1cf0112b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37716/0 (socket says 172.21.15.64:37716) 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cee59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf00fbe50 0x7f1cf01105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:40280/0 (socket says 172.21.15.64:40280) 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cedd9b700 1 -- 172.21.15.64:0/734317577 learned_addr learned my addr 172.21.15.64:0/734317577 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cedd9b700 1 -- 172.21.15.64:0/734317577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf00fbe50 msgr2=0x7f1cf01105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cedd9b700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf00fbe50 0x7f1cf01105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cedd9b700 1 -- 172.21.15.64:0/734317577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce4005040 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cee59c700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf00fbe50 0x7f1cf01105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cedd9b700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 0x7f1cf0112b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4000c00 tx=0x7f1ce4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cdeffd700 1 -- 172.21.15.64:0/734317577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ce4021070 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cdeffd700 1 -- 172.21.15.64:0/734317577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ce401c400 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.106+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf01130c0 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.107+0000 7f1cdeffd700 1 -- 172.21.15.64:0/734317577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ce400e070 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.107+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf0113600 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.107+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf004bed0 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.108+0000 7f1cdeffd700 1 -- 172.21.15.64:0/734317577 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1ce401ca50 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.108+0000 7f1cdeffd700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1cd806d7e0 0x7f1cd806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.109+0000 7f1cdeffd700 1 -- 172.21.15.64:0/734317577 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1ce408e100 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.109+0000 7f1cee59c700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1cd806d7e0 0x7f1cd806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.109+0000 7f1cee59c700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1cd806d7e0 0x7f1cd806fc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0066970 tx=0x7f1ce000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.113+0000 7f1cdeffd700 1 -- 172.21.15.64:0/734317577 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ce4057920 con 0x7f1cf00fcfc0 2024-02-16T23:04:29.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.338+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cf01019a0 con 0x7f1cd806d7e0 2024-02-16T23:04:29.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.339+0000 7f1cdeffd700 1 -- 172.21.15.64:0/734317577 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1cf01019a0 con 0x7f1cd806d7e0 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1cd806d7e0 msgr2=0x7f1cd806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1cd806d7e0 0x7f1cd806fc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0066970 tx=0x7f1ce000b040 comp rx=0 tx=0).stop 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 msgr2=0x7f1cf0112b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 0x7f1cf0112b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4000c00 tx=0x7f1ce4009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 shutdown_connections 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1cd806d7e0 0x7f1cd806fc90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf00fbe50 0x7f1cf01105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 --2- 172.21.15.64:0/734317577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf00fcfc0 0x7f1cf0112b20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 >> 172.21.15.64:0/734317577 conn(0x7f1cf00f7510 msgr2=0x7f1cf0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:29.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 shutdown_connections 2024-02-16T23:04:29.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.341+0000 7f1cf4ed0700 1 -- 172.21.15.64:0/734317577 wait complete. 2024-02-16T23:04:29.351 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:04:29.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.445+0000 7f8d887cc700 1 -- 172.21.15.64:0/1254235236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 msgr2=0x7f8d800fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.445+0000 7f8d887cc700 1 --2- 172.21.15.64:0/1254235236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 0x7f8d800fcce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d74004a50 tx=0x7f8d7400af40 comp rx=0 tx=0).stop 2024-02-16T23:04:29.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.445+0000 7f8d887cc700 1 -- 172.21.15.64:0/1254235236 shutdown_connections 2024-02-16T23:04:29.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.445+0000 7f8d887cc700 1 --2- 172.21.15.64:0/1254235236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 0x7f8d800fcce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.445+0000 7f8d887cc700 1 --2- 172.21.15.64:0/1254235236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d800fb5c0 0x7f8d800fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.445+0000 7f8d887cc700 1 -- 172.21.15.64:0/1254235236 >> 172.21.15.64:0/1254235236 conn(0x7f8d800f6d20 msgr2=0x7f8d800f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:29.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.445+0000 7f8d887cc700 1 -- 172.21.15.64:0/1254235236 shutdown_connections 2024-02-16T23:04:29.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.445+0000 7f8d887cc700 1 -- 172.21.15.64:0/1254235236 wait complete. 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d887cc700 1 Processor -- start 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d887cc700 1 -- start start 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d887cc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d800fb5c0 0x7f8d8010dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d887cc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 0x7f8d8010e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d887cc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8010e720 con 0x7f8d800fc870 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d887cc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8010efc0 con 0x7f8d800fb5c0 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d85d67700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 0x7f8d8010e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d85d67700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 0x7f8d8010e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37720/0 (socket says 172.21.15.64:37720) 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d85d67700 1 -- 172.21.15.64:0/2351350494 learned_addr learned my addr 172.21.15.64:0/2351350494 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d86568700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d800fb5c0 0x7f8d8010dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d85d67700 1 -- 172.21.15.64:0/2351350494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d800fb5c0 msgr2=0x7f8d8010dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d85d67700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d800fb5c0 0x7f8d8010dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.446+0000 7f8d85d67700 1 -- 172.21.15.64:0/2351350494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74005040 con 0x7f8d800fc870 2024-02-16T23:04:29.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.447+0000 7f8d85d67700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 0x7f8d8010e120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8d74004710 tx=0x7f8d7400af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:29.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.447+0000 7f8d7effd700 1 -- 172.21.15.64:0/2351350494 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7400f070 con 0x7f8d800fc870 2024-02-16T23:04:29.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.447+0000 7f8d7effd700 1 -- 172.21.15.64:0/2351350494 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d7401d950 con 0x7f8d800fc870 2024-02-16T23:04:29.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.447+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8010f240 con 0x7f8d800fc870 2024-02-16T23:04:29.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.447+0000 7f8d7effd700 1 -- 172.21.15.64:0/2351350494 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d74014840 con 0x7f8d800fc870 2024-02-16T23:04:29.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.447+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8010f780 con 0x7f8d800fc870 2024-02-16T23:04:29.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.448+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d80105860 con 0x7f8d800fc870 2024-02-16T23:04:29.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.448+0000 7f8d7effd700 1 -- 172.21.15.64:0/2351350494 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8d7401dac0 con 0x7f8d800fc870 2024-02-16T23:04:29.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.449+0000 7f8d7effd700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8d6806d7e0 0x7f8d6806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.449+0000 7f8d7effd700 1 -- 172.21.15.64:0/2351350494 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d7408e3b0 con 0x7f8d800fc870 2024-02-16T23:04:29.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.449+0000 7f8d86568700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8d6806d7e0 0x7f8d6806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.449+0000 7f8d86568700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8d6806d7e0 0x7f8d6806fc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d78007b20 tx=0x7f8d78009e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.453+0000 7f8d7effd700 1 -- 172.21.15.64:0/2351350494 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d7405b700 con 0x7f8d800fc870 2024-02-16T23:04:29.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.676+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d80103820 con 0x7f8d6806d7e0 2024-02-16T23:04:29.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.677+0000 7f8d7effd700 1 -- 172.21.15.64:0/2351350494 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8d80103820 con 0x7f8d6806d7e0 2024-02-16T23:04:29.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8d6806d7e0 msgr2=0x7f8d6806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8d6806d7e0 0x7f8d6806fc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d78007b20 tx=0x7f8d78009e20 comp rx=0 tx=0).stop 2024-02-16T23:04:29.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 msgr2=0x7f8d8010e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 0x7f8d8010e120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8d74004710 tx=0x7f8d7400af40 comp rx=0 tx=0).stop 2024-02-16T23:04:29.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 shutdown_connections 2024-02-16T23:04:29.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8d6806d7e0 0x7f8d6806fc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d800fb5c0 0x7f8d8010dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 --2- 172.21.15.64:0/2351350494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d800fc870 0x7f8d8010e120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 >> 172.21.15.64:0/2351350494 conn(0x7f8d800f6d20 msgr2=0x7f8d800ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:29.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 shutdown_connections 2024-02-16T23:04:29.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.680+0000 7f8d887cc700 1 -- 172.21.15.64:0/2351350494 wait complete. 2024-02-16T23:04:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.783+0000 7fac77370700 1 -- 172.21.15.64:0/940952705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 msgr2=0x7fac700fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.783+0000 7fac77370700 1 --2- 172.21.15.64:0/940952705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 0x7fac700fe550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac60004940 tx=0x7fac60009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.783+0000 7fac77370700 1 -- 172.21.15.64:0/940952705 shutdown_connections 2024-02-16T23:04:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.783+0000 7fac77370700 1 --2- 172.21.15.64:0/940952705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 0x7fac700fe550 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.783+0000 7fac77370700 1 --2- 172.21.15.64:0/940952705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700fcef0 0x7fac700fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.783+0000 7fac77370700 1 -- 172.21.15.64:0/940952705 >> 172.21.15.64:0/940952705 conn(0x7fac700f8620 msgr2=0x7fac700faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.783+0000 7fac77370700 1 -- 172.21.15.64:0/940952705 shutdown_connections 2024-02-16T23:04:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.783+0000 7fac77370700 1 -- 172.21.15.64:0/940952705 wait complete. 2024-02-16T23:04:29.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac77370700 1 Processor -- start 2024-02-16T23:04:29.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac77370700 1 -- start start 2024-02-16T23:04:29.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac77370700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700fcef0 0x7fac70071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac77370700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 0x7fac700702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac77370700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac700708e0 con 0x7fac700fe0e0 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac77370700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac70070a50 con 0x7fac700fcef0 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac7490b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 0x7fac700702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac7490b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 0x7fac700702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37732/0 (socket says 172.21.15.64:37732) 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac7490b700 1 -- 172.21.15.64:0/2787822728 learned_addr learned my addr 172.21.15.64:0/2787822728 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac7510c700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700fcef0 0x7fac70071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac7490b700 1 -- 172.21.15.64:0/2787822728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700fcef0 msgr2=0x7fac70071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac7490b700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700fcef0 0x7fac70071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.784+0000 7fac7490b700 1 -- 172.21.15.64:0/2787822728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac60005040 con 0x7fac700fe0e0 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.785+0000 7fac7490b700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 0x7fac700702e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fac600093b0 tx=0x7fac60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.785+0000 7fac65ffb700 1 -- 172.21.15.64:0/2787822728 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac6000e040 con 0x7fac700fe0e0 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.785+0000 7fac65ffb700 1 -- 172.21.15.64:0/2787822728 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac6000bcc0 con 0x7fac700fe0e0 2024-02-16T23:04:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.785+0000 7fac65ffb700 1 -- 172.21.15.64:0/2787822728 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac6001c630 con 0x7fac700fe0e0 2024-02-16T23:04:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.785+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac70070cd0 con 0x7fac700fe0e0 2024-02-16T23:04:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.785+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac70071150 con 0x7fac700fe0e0 2024-02-16T23:04:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.786+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7004bed0 con 0x7fac700fe0e0 2024-02-16T23:04:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.786+0000 7fac65ffb700 1 -- 172.21.15.64:0/2787822728 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fac60013980 con 0x7fac700fe0e0 2024-02-16T23:04:29.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.787+0000 7fac65ffb700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac5c06d7e0 0x7fac5c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:29.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.787+0000 7fac65ffb700 1 -- 172.21.15.64:0/2787822728 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fac6008e170 con 0x7fac700fe0e0 2024-02-16T23:04:29.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.787+0000 7fac7510c700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac5c06d7e0 0x7fac5c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:29.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.787+0000 7fac7510c700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac5c06d7e0 0x7fac5c06fc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac70071bf0 tx=0x7fac6c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:29.792+0000 7fac65ffb700 1 -- 172.21.15.64:0/2787822728 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac6005b4c0 con 0x7fac700fe0e0 2024-02-16T23:04:30.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.008+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fac70071150 con 0x7fac5c06d7e0 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.019+0000 7fac65ffb700 1 -- 172.21.15.64:0/2787822728 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fac70071150 con 0x7fac5c06d7e0 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (7m) 15s ago 16m 26.0M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (4m) 15s ago 16m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (4m) 67s ago 15m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (7m) 15s ago 15m 84.5M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (19s) 15s ago 10m 11.5M - 19.0.0-1450-gf78a58c0 f081f847070a d4dd49dc2ce8 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (45s) 15s ago 10m 15.1M - 19.0.0-1450-gf78a58c0 f081f847070a bd35cb99fa04 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (10m) 67s ago 10m 17.4M - 17.2.7-513-gb88fae66 870a0944ccb8 bf0fe082e119 2024-02-16T23:04:30.019 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (10m) 67s ago 10m 17.0M - 17.2.7-513-gb88fae66 870a0944ccb8 456161bcf989 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (6m) 15s ago 17m 507M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (5m) 67s ago 15m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (5m) 15s ago 17m 49.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (5m) 67s ago 15m 39.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (7m) 15s ago 16m 15.1M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 67s ago 15m 15.2M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (4m) 15s ago 14m 106M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (3m) 15s ago 13m 97.7M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (2m) 15s ago 13m 84.3M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (2m) 67s ago 13m 107M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (104s) 67s ago 12m 99.3M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (71s) 67s ago 12m 13.5M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 7bb5b265fc66 2024-02-16T23:04:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (5m) 15s ago 15m 61.5M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:04:30.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac5c06d7e0 msgr2=0x7fac5c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac5c06d7e0 0x7fac5c06fc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac70071bf0 tx=0x7fac6c00b040 comp rx=0 tx=0).stop 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 msgr2=0x7fac700702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 0x7fac700702e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fac600093b0 tx=0x7fac60009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 shutdown_connections 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fac5c06d7e0 0x7fac5c06fc90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700fcef0 0x7fac70071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 --2- 172.21.15.64:0/2787822728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac700fe0e0 0x7fac700702e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 >> 172.21.15.64:0/2787822728 conn(0x7fac700f8620 msgr2=0x7fac701013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 shutdown_connections 2024-02-16T23:04:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.021+0000 7fac77370700 1 -- 172.21.15.64:0/2787822728 wait complete. 2024-02-16T23:04:30.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.122+0000 7ff5c29e1700 1 -- 172.21.15.64:0/3341367794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 msgr2=0x7ff5bc0fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.122+0000 7ff5c29e1700 1 --2- 172.21.15.64:0/3341367794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 0x7ff5bc0fe700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0004940 tx=0x7ff5b0009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:30.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 -- 172.21.15.64:0/3341367794 shutdown_connections 2024-02-16T23:04:30.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 --2- 172.21.15.64:0/3341367794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 0x7ff5bc0fe700 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 --2- 172.21.15.64:0/3341367794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0fb220 0x7ff5bc0fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 -- 172.21.15.64:0/3341367794 >> 172.21.15.64:0/3341367794 conn(0x7ff5bc0f6ca0 msgr2=0x7ff5bc0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:30.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 -- 172.21.15.64:0/3341367794 shutdown_connections 2024-02-16T23:04:30.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 -- 172.21.15.64:0/3341367794 wait complete. 2024-02-16T23:04:30.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 Processor -- start 2024-02-16T23:04:30.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 -- start start 2024-02-16T23:04:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0fb220 0x7ff5bc10d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 0x7ff5bc10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc10e4c0 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.123+0000 7ff5c29e1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc10ed60 con 0x7ff5bc0fb220 2024-02-16T23:04:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 0x7ff5bc10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0fb220 0x7ff5bc10d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 0x7ff5bc10dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37748/0 (socket says 172.21.15.64:37748) 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0fb220 0x7ff5bc10d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:40318/0 (socket says 172.21.15.64:40318) 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bb7fe700 1 -- 172.21.15.64:0/1127936245 learned_addr learned my addr 172.21.15.64:0/1127936245 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bb7fe700 1 -- 172.21.15.64:0/1127936245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0fb220 msgr2=0x7ff5bc10d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bb7fe700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0fb220 0x7ff5bc10d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bb7fe700 1 -- 172.21.15.64:0/1127936245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0005040 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bbfff700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0fb220 0x7ff5bc10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5bb7fe700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 0x7ff5bc10dec0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009b30 tx=0x7ff5b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5b8ff9700 1 -- 172.21.15.64:0/1127936245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5b0021070 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5b8ff9700 1 -- 172.21.15.64:0/1127936245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5b001c400 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc10efe0 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5b8ff9700 1 -- 172.21.15.64:0/1127936245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5b000e070 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.124+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc10f4c0 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.125+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc04bed0 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.126+0000 7ff5b8ff9700 1 -- 172.21.15.64:0/1127936245 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff5b001ca00 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.126+0000 7ff5b8ff9700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff5a406d7e0 0x7ff5a406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.126+0000 7ff5b8ff9700 1 -- 172.21.15.64:0/1127936245 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff5b008dfc0 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.126+0000 7ff5bbfff700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff5a406d7e0 0x7ff5a406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.127+0000 7ff5bbfff700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff5a406d7e0 0x7ff5a406fc90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009b80 tx=0x7ff5ac00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:30.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.131+0000 7ff5b8ff9700 1 -- 172.21.15.64:0/1127936245 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff5b005b3b0 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.421+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff5bc105a40 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.422+0000 7ff5b8ff9700 1 -- 172.21.15.64:0/1127936245 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff5b0004490 con 0x7ff5bc0fe2b0 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T23:04:30.422 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:04:30.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:04:30.423 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:04:30.423 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T23:04:30.423 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 12 2024-02-16T23:04:30.423 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:04:30.423 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:04:30.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff5a406d7e0 msgr2=0x7ff5a406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff5a406d7e0 0x7ff5a406fc90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009b80 tx=0x7ff5ac00b040 comp rx=0 tx=0).stop 2024-02-16T23:04:30.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 msgr2=0x7ff5bc10dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 0x7ff5bc10dec0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009b30 tx=0x7ff5b0009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:30.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 shutdown_connections 2024-02-16T23:04:30.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff5a406d7e0 0x7ff5a406fc90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0fb220 0x7ff5bc10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 --2- 172.21.15.64:0/1127936245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0fe2b0 0x7ff5bc10dec0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 >> 172.21.15.64:0/1127936245 conn(0x7ff5bc0f6ca0 msgr2=0x7ff5bc1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:30.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 shutdown_connections 2024-02-16T23:04:30.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.424+0000 7ff5c29e1700 1 -- 172.21.15.64:0/1127936245 wait complete. 2024-02-16T23:04:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.527+0000 7fdf0528e700 1 -- 172.21.15.64:0/1295527352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 msgr2=0x7fdf000fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.527+0000 7fdf0528e700 1 --2- 172.21.15.64:0/1295527352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 0x7fdf000fe550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdef0004940 tx=0x7fdef0009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.527+0000 7fdf0528e700 1 -- 172.21.15.64:0/1295527352 shutdown_connections 2024-02-16T23:04:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.527+0000 7fdf0528e700 1 --2- 172.21.15.64:0/1295527352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 0x7fdf000fe550 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.527+0000 7fdf0528e700 1 --2- 172.21.15.64:0/1295527352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf000fcef0 0x7fdf000fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.527+0000 7fdf0528e700 1 -- 172.21.15.64:0/1295527352 >> 172.21.15.64:0/1295527352 conn(0x7fdf000f8620 msgr2=0x7fdf000faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:30.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.527+0000 7fdf0528e700 1 -- 172.21.15.64:0/1295527352 shutdown_connections 2024-02-16T23:04:30.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.527+0000 7fdf0528e700 1 -- 172.21.15.64:0/1295527352 wait complete. 2024-02-16T23:04:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdf0528e700 1 Processor -- start 2024-02-16T23:04:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdf0528e700 1 -- start start 2024-02-16T23:04:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdf0528e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf000fcef0 0x7fdf0010faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdf0528e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 0x7fdf00110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdf0528e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf00110630 con 0x7fdf000fe0e0 2024-02-16T23:04:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdf0528e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf00110ed0 con 0x7fdf000fcef0 2024-02-16T23:04:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdefe7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 0x7fdf00110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdefe7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 0x7fdf00110030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48618/0 (socket says 172.21.15.64:48618) 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdefe7fc700 1 -- 172.21.15.64:0/3097555356 learned_addr learned my addr 172.21.15.64:0/3097555356 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdefeffd700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf000fcef0 0x7fdf0010faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdefe7fc700 1 -- 172.21.15.64:0/3097555356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf000fcef0 msgr2=0x7fdf0010faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdefe7fc700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf000fcef0 0x7fdf0010faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdefe7fc700 1 -- 172.21.15.64:0/3097555356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0005040 con 0x7fdf000fe0e0 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.528+0000 7fdefe7fc700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 0x7fdf00110030 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdef00093b0 tx=0x7fdef0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.529+0000 7fdef77fe700 1 -- 172.21.15.64:0/3097555356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef000e040 con 0x7fdf000fe0e0 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.529+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf00111150 con 0x7fdf000fe0e0 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.529+0000 7fdef77fe700 1 -- 172.21.15.64:0/3097555356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdef000bcc0 con 0x7fdf000fe0e0 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.529+0000 7fdef77fe700 1 -- 172.21.15.64:0/3097555356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef001c630 con 0x7fdf000fe0e0 2024-02-16T23:04:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.529+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf00111690 con 0x7fdf000fe0e0 2024-02-16T23:04:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.530+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf00102210 con 0x7fdf000fe0e0 2024-02-16T23:04:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.530+0000 7fdef77fe700 1 -- 172.21.15.64:0/3097555356 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdef0013980 con 0x7fdf000fe0e0 2024-02-16T23:04:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.531+0000 7fdef77fe700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdeec06d7e0 0x7fdeec06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.531+0000 7fdefeffd700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdeec06d7e0 0x7fdeec06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.531+0000 7fdef77fe700 1 -- 172.21.15.64:0/3097555356 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdef008e050 con 0x7fdf000fe0e0 2024-02-16T23:04:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.531+0000 7fdefeffd700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdeec06d7e0 0x7fdeec06fc90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009fd0 tx=0x7fdee800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:30.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.536+0000 7fdef77fe700 1 -- 172.21.15.64:0/3097555356 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdef005b3a0 con 0x7fdf000fe0e0 2024-02-16T23:04:30.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.783+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdf0004bed0 con 0x7fdf000fe0e0 2024-02-16T23:04:30.783 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:30 smithi064 ceph-mon[93757]: pgmap v200: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 4.8 KiB/s wr, 5 op/s 2024-02-16T23:04:30.784 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:30 smithi064 ceph-mon[93757]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:04:30.784 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:30 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1127936245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:30.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.783+0000 7fdef77fe700 1 -- 172.21.15.64:0/3097555356 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1671 (secure 0 0 0) 0x7fdef005af30 con 0x7fdf000fe0e0 2024-02-16T23:04:30.784 INFO:teuthology.orchestra.run.smithi064.stdout:e26 2024-02-16T23:04:30.784 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:04:30.784 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:04:30.784 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:04:30.784 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 26 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T23:04:23.404386+0000 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:04:30.785 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 84 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=24297} 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{0:24297} state up:replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:04:30.786 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:04:30.787 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:04:30.787 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:04:30.787 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:04:30.787 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.64:6828/2551190588,v1:172.21.15.64:6829/2551190588] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:04:30.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdeec06d7e0 msgr2=0x7fdeec06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdeec06d7e0 0x7fdeec06fc90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009fd0 tx=0x7fdee800b040 comp rx=0 tx=0).stop 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 msgr2=0x7fdf00110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 0x7fdf00110030 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdef00093b0 tx=0x7fdef0009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 shutdown_connections 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdeec06d7e0 0x7fdeec06fc90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf000fcef0 0x7fdf0010faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 --2- 172.21.15.64:0/3097555356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf000fe0e0 0x7fdf00110030 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.786+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 >> 172.21.15.64:0/3097555356 conn(0x7fdf000f8620 msgr2=0x7fdf000ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.787+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 shutdown_connections 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.787+0000 7fdf0528e700 1 -- 172.21.15.64:0/3097555356 wait complete. 2024-02-16T23:04:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2024-02-16T23:04:30.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:30 smithi192 ceph-mon[82652]: pgmap v200: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 4.8 KiB/s wr, 5 op/s 2024-02-16T23:04:30.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:30 smithi192 ceph-mon[82652]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:04:30.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:30 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1127936245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.890+0000 7fed94bef700 1 -- 172.21.15.64:0/3660589985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fd4b0 msgr2=0x7fed900fd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.890+0000 7fed94bef700 1 --2- 172.21.15.64:0/3660589985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fd4b0 0x7fed900fd8c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fed80004940 tx=0x7fed80009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 -- 172.21.15.64:0/3660589985 shutdown_connections 2024-02-16T23:04:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 --2- 172.21.15.64:0/3660589985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed900fde00 0x7fed90104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 --2- 172.21.15.64:0/3660589985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fd4b0 0x7fed900fd8c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 -- 172.21.15.64:0/3660589985 >> 172.21.15.64:0/3660589985 conn(0x7fed900f6d20 msgr2=0x7fed900f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 -- 172.21.15.64:0/3660589985 shutdown_connections 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 -- 172.21.15.64:0/3660589985 wait complete. 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 Processor -- start 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 -- start start 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed900fd4b0 0x7fed9010feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fde00 0x7fed901103f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed901109f0 con 0x7fed900fde00 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed94bef700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed90111290 con 0x7fed900fd4b0 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed8dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fde00 0x7fed901103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed8dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fde00 0x7fed901103f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48630/0 (socket says 172.21.15.64:48630) 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed8dd9b700 1 -- 172.21.15.64:0/990768076 learned_addr learned my addr 172.21.15.64:0/990768076 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.891+0000 7fed8e59c700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed900fd4b0 0x7fed9010feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed8dd9b700 1 -- 172.21.15.64:0/990768076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed900fd4b0 msgr2=0x7fed9010feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed8dd9b700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed900fd4b0 0x7fed9010feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed8dd9b700 1 -- 172.21.15.64:0/990768076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80005040 con 0x7fed900fde00 2024-02-16T23:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed8dd9b700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fde00 0x7fed901103f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed84002a50 tx=0x7fed84004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:30.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed7effd700 1 -- 172.21.15.64:0/990768076 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed84007cc0 con 0x7fed900fde00 2024-02-16T23:04:30.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed7effd700 1 -- 172.21.15.64:0/990768076 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed8400f040 con 0x7fed900fde00 2024-02-16T23:04:30.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed90111570 con 0x7fed900fde00 2024-02-16T23:04:30.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed7effd700 1 -- 172.21.15.64:0/990768076 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed8400da30 con 0x7fed900fde00 2024-02-16T23:04:30.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.892+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed90111ab0 con 0x7fed900fde00 2024-02-16T23:04:30.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.893+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed90109c70 con 0x7fed900fde00 2024-02-16T23:04:30.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.894+0000 7fed7effd700 1 -- 172.21.15.64:0/990768076 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fed8400dca0 con 0x7fed900fde00 2024-02-16T23:04:30.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.894+0000 7fed7effd700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fed7806d7e0 0x7fed7806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:30.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.894+0000 7fed7effd700 1 -- 172.21.15.64:0/990768076 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fed8408cd10 con 0x7fed900fde00 2024-02-16T23:04:30.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.894+0000 7fed8e59c700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fed7806d7e0 0x7fed7806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:30.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.894+0000 7fed8e59c700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fed7806d7e0 0x7fed7806fc90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed8000b3b0 tx=0x7fed80009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:30.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:30.898+0000 7fed7effd700 1 -- 172.21.15.64:0/990768076 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed84059fe0 con 0x7fed900fde00 2024-02-16T23:04:31.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.123+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed90107ca0 con 0x7fed7806d7e0 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.125+0000 7fed7effd700 1 -- 172.21.15.64:0/990768076 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fed90107ca0 con 0x7fed7806d7e0 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:04:31.125 INFO:teuthology.orchestra.run.smithi064.stdout: "osd", 2024-02-16T23:04:31.126 INFO:teuthology.orchestra.run.smithi064.stdout: "crash" 2024-02-16T23:04:31.126 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:04:31.126 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "14/21 daemons upgraded", 2024-02-16T23:04:31.126 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T23:04:31.126 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:04:31.126 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:04:31.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fed7806d7e0 msgr2=0x7fed7806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:31.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fed7806d7e0 0x7fed7806fc90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed8000b3b0 tx=0x7fed80009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:31.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fde00 msgr2=0x7fed901103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fde00 0x7fed901103f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed84002a50 tx=0x7fed84004600 comp rx=0 tx=0).stop 2024-02-16T23:04:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 shutdown_connections 2024-02-16T23:04:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fed7806d7e0 0x7fed7806fc90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed900fd4b0 0x7fed9010feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 --2- 172.21.15.64:0/990768076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed900fde00 0x7fed901103f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 >> 172.21.15.64:0/990768076 conn(0x7fed900f6d20 msgr2=0x7fed900f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 shutdown_connections 2024-02-16T23:04:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.127+0000 7fed94bef700 1 -- 172.21.15.64:0/990768076 wait complete. 2024-02-16T23:04:31.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.231+0000 7f95d345e700 1 -- 172.21.15.64:0/857009757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 msgr2=0x7f95cc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:31.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.231+0000 7f95d345e700 1 --2- 172.21.15.64:0/857009757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 0x7f95cc0fe550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95bc004940 tx=0x7f95bc009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:31.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.231+0000 7f95d345e700 1 -- 172.21.15.64:0/857009757 shutdown_connections 2024-02-16T23:04:31.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.231+0000 7f95d345e700 1 --2- 172.21.15.64:0/857009757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 0x7f95cc0fe550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.231+0000 7f95d345e700 1 --2- 172.21.15.64:0/857009757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95cc0fcef0 0x7f95cc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.231+0000 7f95d345e700 1 -- 172.21.15.64:0/857009757 >> 172.21.15.64:0/857009757 conn(0x7f95cc0f8620 msgr2=0x7f95cc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:31.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d345e700 1 -- 172.21.15.64:0/857009757 shutdown_connections 2024-02-16T23:04:31.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d345e700 1 -- 172.21.15.64:0/857009757 wait complete. 2024-02-16T23:04:31.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d345e700 1 Processor -- start 2024-02-16T23:04:31.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d345e700 1 -- start start 2024-02-16T23:04:31.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d345e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95cc0fcef0 0x7f95cc10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:31.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d345e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 0x7f95cc10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:31.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d345e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc10e4a0 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d345e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc10ed40 con 0x7f95cc0fcef0 2024-02-16T23:04:31.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d09f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 0x7f95cc10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:31.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d09f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 0x7f95cc10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48652/0 (socket says 172.21.15.64:48652) 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d09f9700 1 -- 172.21.15.64:0/3783488733 learned_addr learned my addr 172.21.15.64:0/3783488733 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.232+0000 7f95d11fa700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95cc0fcef0 0x7f95cc10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95d09f9700 1 -- 172.21.15.64:0/3783488733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95cc0fcef0 msgr2=0x7f95cc10d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95d09f9700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95cc0fcef0 0x7f95cc10d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95d09f9700 1 -- 172.21.15.64:0/3783488733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc005040 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95d09f9700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 0x7f95cc10dea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0093b0 tx=0x7f95bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95c1ffb700 1 -- 172.21.15.64:0/3783488733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95bc00e040 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95c1ffb700 1 -- 172.21.15.64:0/3783488733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95bc00bcc0 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc10efc0 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95c1ffb700 1 -- 172.21.15.64:0/3783488733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95bc01c630 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.233+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc10f500 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.234+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc04bed0 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.235+0000 7f95c1ffb700 1 -- 172.21.15.64:0/3783488733 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f95bc013980 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.235+0000 7f95c1ffb700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f95b806d7e0 0x7f95b806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:04:31.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.235+0000 7f95c1ffb700 1 -- 172.21.15.64:0/3783488733 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f95bc08e220 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.235+0000 7f95d11fa700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f95b806d7e0 0x7f95b806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:04:31.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.236+0000 7f95d11fa700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f95b806d7e0 0x7f95b806fc90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0fcc30 tx=0x7f95c800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:04:31.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.240+0000 7f95c1ffb700 1 -- 172.21.15.64:0/3783488733 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95bc05b610 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.525+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f95cc10f9a0 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.525+0000 7f95c1ffb700 1 -- 172.21.15.64:0/3783488733 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f95bc05b1a0 con 0x7f95cc0fe0e0 2024-02-16T23:04:31.526 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-02-16T23:04:31.526 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-16T23:04:31.526 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs is degraded 2024-02-16T23:04:31.526 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:04:31.526 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:04:31.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.527+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f95b806d7e0 msgr2=0x7f95b806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:31.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.527+0000 7f95d345e700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f95b806d7e0 0x7f95b806fc90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0fcc30 tx=0x7f95c800b040 comp rx=0 tx=0).stop 2024-02-16T23:04:31.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.527+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 msgr2=0x7f95cc10dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:04:31.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.527+0000 7f95d345e700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 0x7f95cc10dea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0093b0 tx=0x7f95bc009e30 comp rx=0 tx=0).stop 2024-02-16T23:04:31.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.527+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 shutdown_connections 2024-02-16T23:04:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.528+0000 7f95d345e700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f95b806d7e0 0x7f95b806fc90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.528+0000 7f95d345e700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95cc0fcef0 0x7f95cc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.528+0000 7f95d345e700 1 --2- 172.21.15.64:0/3783488733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95cc0fe0e0 0x7f95cc10dea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:04:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.528+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 >> 172.21.15.64:0/3783488733 conn(0x7f95cc0f8620 msgr2=0x7f95cc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:04:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.528+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 shutdown_connections 2024-02-16T23:04:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:04:31.528+0000 7f95d345e700 1 -- 172.21.15.64:0/3783488733 wait complete. 2024-02-16T23:04:31.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:31 smithi064 ceph-mon[93757]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:04:31.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:31 smithi064 ceph-mon[93757]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:04:31.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:31 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3097555356' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:04:31.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:31 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3783488733' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:04:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:31 smithi192 ceph-mon[82652]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:04:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:31 smithi192 ceph-mon[82652]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:04:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:31 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3097555356' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:04:32.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:31 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3783488733' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:04:32.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:32 smithi064 ceph-mon[93757]: pgmap v201: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-02-16T23:04:32.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:32 smithi064 ceph-mon[93757]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:04:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:32 smithi192 ceph-mon[82652]: pgmap v201: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-02-16T23:04:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:32 smithi192 ceph-mon[82652]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:04:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:33 smithi192 ceph-mon[82652]: pgmap v202: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-02-16T23:04:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:04:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:33 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:reconnect 2024-02-16T23:04:34.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:33 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi192.luiybj=up:reconnect} 2 up:standby 2024-02-16T23:04:34.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:33 smithi064 ceph-mon[93757]: pgmap v202: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-02-16T23:04:34.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:34.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:04:34.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:33 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:reconnect 2024-02-16T23:04:34.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:33 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi192.luiybj=up:reconnect} 2 up:standby 2024-02-16T23:04:34.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:34 smithi192 ceph-mon[82652]: reconnect by client.24327 192.168.0.1:0/1356869742 after 0.00099998 2024-02-16T23:04:34.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:34 smithi192 ceph-mon[82652]: reconnect by client.14528 192.168.0.1:0/3011212362 after 0.00099998 2024-02-16T23:04:34.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:34 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:rejoin 2024-02-16T23:04:34.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:34 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi192.luiybj=up:rejoin} 2 up:standby 2024-02-16T23:04:34.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:34 smithi192 ceph-mon[82652]: daemon mds.cephfs.smithi192.luiybj is now active in filesystem cephfs as rank 0 2024-02-16T23:04:35.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:34 smithi064 ceph-mon[93757]: reconnect by client.24327 192.168.0.1:0/1356869742 after 0.00099998 2024-02-16T23:04:35.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:34 smithi064 ceph-mon[93757]: reconnect by client.14528 192.168.0.1:0/3011212362 after 0.00099998 2024-02-16T23:04:35.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:34 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:rejoin 2024-02-16T23:04:35.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:34 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi192.luiybj=up:rejoin} 2 up:standby 2024-02-16T23:04:35.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:34 smithi064 ceph-mon[93757]: daemon mds.cephfs.smithi192.luiybj is now active in filesystem cephfs as rank 0 2024-02-16T23:04:36.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:35 smithi192 ceph-mon[82652]: pgmap v203: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T23:04:36.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:35 smithi192 ceph-mon[82652]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T23:04:36.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:35 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:active 2024-02-16T23:04:36.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:35 smithi192 ceph-mon[82652]: fsmap cephfs:1 {0=cephfs.smithi192.luiybj=up:active} 2 up:standby 2024-02-16T23:04:36.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:35 smithi064 ceph-mon[93757]: pgmap v203: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T23:04:36.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:35 smithi064 ceph-mon[93757]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T23:04:36.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:35 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.192:6826/2853522025,v1:172.21.15.192:6827/2853522025] up:active 2024-02-16T23:04:36.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:35 smithi064 ceph-mon[93757]: fsmap cephfs:1 {0=cephfs.smithi192.luiybj=up:active} 2 up:standby 2024-02-16T23:04:36.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:36 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:36.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:36 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:36.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:36 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:36.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:36 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.192:6824/2028541376,v1:172.21.15.192:6825/2028541376] up:boot 2024-02-16T23:04:36.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:36 smithi192 ceph-mon[82652]: fsmap cephfs:1 {0=cephfs.smithi192.luiybj=up:active} 3 up:standby 2024-02-16T23:04:36.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:36 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T23:04:37.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:36 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:37.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:36 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:37.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:36 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:37.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:36 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.192:6824/2028541376,v1:172.21.15.192:6825/2028541376] up:boot 2024-02-16T23:04:37.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:36 smithi064 ceph-mon[93757]: fsmap cephfs:1 {0=cephfs.smithi192.luiybj=up:active} 3 up:standby 2024-02-16T23:04:37.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:36 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T23:04:38.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:38 smithi192 ceph-mon[82652]: pgmap v204: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-02-16T23:04:38.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:38 smithi064 ceph-mon[93757]: pgmap v204: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-02-16T23:04:40.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:39 smithi192 ceph-mon[82652]: pgmap v205: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 7 op/s 2024-02-16T23:04:40.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:40.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:40.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:39 smithi064 ceph-mon[93757]: pgmap v205: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 7 op/s 2024-02-16T23:04:40.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:40.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:41.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:40 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:41.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:40 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:41.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:40 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:41.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:40 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:42.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:41 smithi192 ceph-mon[82652]: pgmap v206: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T23:04:42.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:41 smithi064 ceph-mon[93757]: pgmap v206: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: pgmap v207: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 6.3 MiB/s rd, 170 B/s wr, 4 op/s 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: Detected new or changed devices on smithi192 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:44.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi192.luiybj"]}]: dispatch 2024-02-16T23:04:44.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:44 smithi192 ceph-mon[82652]: Upgrade: It appears safe to stop mds.cephfs.smithi192.luiybj 2024-02-16T23:04:44.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: pgmap v207: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 6.3 MiB/s rd, 170 B/s wr, 4 op/s 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: Detected new or changed devices on smithi192 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi192.luiybj"]}]: dispatch 2024-02-16T23:04:44.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:44 smithi064 ceph-mon[93757]: Upgrade: It appears safe to stop mds.cephfs.smithi192.luiybj 2024-02-16T23:04:46.244 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:46 smithi192 ceph-mon[82652]: pgmap v208: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:04:46.244 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:46 smithi192 ceph-mon[82652]: Upgrade: Updating mds.cephfs.smithi192.luiybj 2024-02-16T23:04:46.244 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:46.244 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.luiybj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T23:04:46.245 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:46 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:04:46.245 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:46 smithi192 ceph-mon[82652]: Deploying daemon mds.cephfs.smithi192.luiybj on smithi192 2024-02-16T23:04:46.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:46 smithi064 ceph-mon[93757]: pgmap v208: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:04:46.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:46 smithi064 ceph-mon[93757]: Upgrade: Updating mds.cephfs.smithi192.luiybj 2024-02-16T23:04:46.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:46.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.luiybj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T23:04:46.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:46 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:04:46.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:46 smithi064 ceph-mon[93757]: Deploying daemon mds.cephfs.smithi192.luiybj on smithi192 2024-02-16T23:04:47.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:47 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[93753]: 2024-02-16T23:04:47.133+0000 7fef0c04c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:48.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: pgmap v209: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:04:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T23:04:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: osdmap e85: 6 total, 6 up, 6 in 2024-02-16T23:04:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: Standby daemon mds.cephfs.smithi064.tqesxx assigned to filesystem cephfs as rank 0 2024-02-16T23:04:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T23:04:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T23:04:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi064.tqesxx=up:replay} 2 up:standby 2024-02-16T23:04:48.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:04:48.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: pgmap v209: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:04:48.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T23:04:48.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T23:04:48.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: osdmap e85: 6 total, 6 up, 6 in 2024-02-16T23:04:48.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: Standby daemon mds.cephfs.smithi064.tqesxx assigned to filesystem cephfs as rank 0 2024-02-16T23:04:48.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T23:04:48.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T23:04:48.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi064.tqesxx=up:replay} 2 up:standby 2024-02-16T23:04:48.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:04:49.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:49 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T23:04:49.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:49 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T23:04:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:50 smithi192 ceph-mon[82652]: pgmap v211: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-02-16T23:04:50.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:50 smithi064 ceph-mon[93757]: pgmap v211: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-02-16T23:04:52.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:52 smithi192 ceph-mon[82652]: pgmap v212: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-02-16T23:04:52.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:52 smithi064 ceph-mon[93757]: pgmap v212: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-02-16T23:04:54.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:54 smithi192 ceph-mon[82652]: pgmap v213: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-02-16T23:04:54.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:54 smithi064 ceph-mon[93757]: pgmap v213: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-02-16T23:04:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:56 smithi192 ceph-mon[82652]: pgmap v214: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T23:04:56.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:56 smithi064 ceph-mon[93757]: pgmap v214: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T23:04:58.329 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:58 smithi192 ceph-mon[82652]: pgmap v215: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T23:04:58.329 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:58 smithi192 ceph-mon[82652]: reconnect by client.14528 192.168.0.1:0/3011212362 after 0.000999981 2024-02-16T23:04:58.329 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:58 smithi192 ceph-mon[82652]: reconnect by client.24327 192.168.0.1:0/1356869742 after 0.000999981 2024-02-16T23:04:58.329 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:58 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] up:reconnect 2024-02-16T23:04:58.329 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:58 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi064.tqesxx=up:reconnect} 2 up:standby 2024-02-16T23:04:58.329 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:58 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:58.329 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:58 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:58.329 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:58 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:58.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:58 smithi064 ceph-mon[93757]: pgmap v215: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T23:04:58.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:58 smithi064 ceph-mon[93757]: reconnect by client.14528 192.168.0.1:0/3011212362 after 0.000999981 2024-02-16T23:04:58.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:58 smithi064 ceph-mon[93757]: reconnect by client.24327 192.168.0.1:0/1356869742 after 0.000999981 2024-02-16T23:04:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:58 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] up:reconnect 2024-02-16T23:04:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:58 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi064.tqesxx=up:reconnect} 2 up:standby 2024-02-16T23:04:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:58 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:58 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:04:58.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:58 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:04:59.110 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:59 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] up:rejoin 2024-02-16T23:04:59.110 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:59 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.192:6826/1878053024,v1:172.21.15.192:6827/1878053024] up:boot 2024-02-16T23:04:59.110 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:59 smithi192 ceph-mon[82652]: fsmap cephfs:1/1 {0=cephfs.smithi064.tqesxx=up:rejoin} 3 up:standby 2024-02-16T23:04:59.110 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:59 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T23:04:59.110 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:04:59 smithi192 ceph-mon[82652]: daemon mds.cephfs.smithi064.tqesxx is now active in filesystem cephfs as rank 0 2024-02-16T23:04:59.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:59 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] up:rejoin 2024-02-16T23:04:59.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:59 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.192:6826/1878053024,v1:172.21.15.192:6827/1878053024] up:boot 2024-02-16T23:04:59.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:59 smithi064 ceph-mon[93757]: fsmap cephfs:1/1 {0=cephfs.smithi064.tqesxx=up:rejoin} 3 up:standby 2024-02-16T23:04:59.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:59 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T23:04:59.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:04:59 smithi064 ceph-mon[93757]: daemon mds.cephfs.smithi064.tqesxx is now active in filesystem cephfs as rank 0 2024-02-16T23:05:00.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:00 smithi064 ceph-mon[93757]: pgmap v216: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T23:05:00.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:00 smithi064 ceph-mon[93757]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T23:05:00.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:00 smithi064 ceph-mon[93757]: mds.? [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] up:active 2024-02-16T23:05:00.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:00 smithi064 ceph-mon[93757]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T23:05:00.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:00 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:00.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:00 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:00.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:00 smithi192 ceph-mon[82652]: pgmap v216: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T23:05:00.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:00 smithi192 ceph-mon[82652]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T23:05:00.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:00 smithi192 ceph-mon[82652]: mds.? [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] up:active 2024-02-16T23:05:00.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:00 smithi192 ceph-mon[82652]: fsmap cephfs:1 {0=cephfs.smithi064.tqesxx=up:active} 3 up:standby 2024-02-16T23:05:00.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:00 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:00.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:00 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:01.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.644+0000 7f694ce86700 1 -- 172.21.15.64:0/3663835031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69480fbe40 msgr2=0x7f69480fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:01.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.644+0000 7f694ce86700 1 --2- 172.21.15.64:0/3663835031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69480fbe40 0x7f69480fc250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6938009ce0 tx=0x7f6938004d10 comp rx=0 tx=0).stop 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.644+0000 7f694ce86700 1 -- 172.21.15.64:0/3663835031 shutdown_connections 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.644+0000 7f694ce86700 1 --2- 172.21.15.64:0/3663835031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69480fcfb0 0x7f69480fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.644+0000 7f694ce86700 1 --2- 172.21.15.64:0/3663835031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69480fbe40 0x7f69480fc250 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.644+0000 7f694ce86700 1 -- 172.21.15.64:0/3663835031 >> 172.21.15.64:0/3663835031 conn(0x7f69480f7520 msgr2=0x7f69480f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.644+0000 7f694ce86700 1 -- 172.21.15.64:0/3663835031 shutdown_connections 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.644+0000 7f694ce86700 1 -- 172.21.15.64:0/3663835031 wait complete. 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694ce86700 1 Processor -- start 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694ce86700 1 -- start start 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694ce86700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69480fbe40 0x7f69480fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694ce86700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69480fcfb0 0x7f69480fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694ce86700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480fe980 con 0x7f69480fcfb0 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694ce86700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480feaf0 con 0x7f69480fbe40 2024-02-16T23:05:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69480fbe40 0x7f69480fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:01.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69480fbe40 0x7f69480fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:53738/0 (socket says 172.21.15.64:53738) 2024-02-16T23:05:01.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.645+0000 7f694659c700 1 -- 172.21.15.64:0/3199170912 learned_addr learned my addr 172.21.15.64:0/3199170912 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:01.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f6945d9b700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69480fcfb0 0x7f69480fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f694659c700 1 -- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69480fcfb0 msgr2=0x7f69480fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f694659c700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69480fcfb0 0x7f69480fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f694659c700 1 -- 172.21.15.64:0/3199170912 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6938009940 con 0x7f69480fbe40 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f694659c700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69480fbe40 0x7f69480fff00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6938015040 tx=0x7f6938004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f693effd700 1 -- 172.21.15.64:0/3199170912 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f693800e040 con 0x7f69480fbe40 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f693effd700 1 -- 172.21.15.64:0/3199170912 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69380128e0 con 0x7f69480fbe40 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69480fed70 con 0x7f69480fbe40 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.646+0000 7f693effd700 1 -- 172.21.15.64:0/3199170912 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f693801c940 con 0x7f69480fbe40 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.647+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6948115ad0 con 0x7f69480fbe40 2024-02-16T23:05:01.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.648+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694804bed0 con 0x7f69480fbe40 2024-02-16T23:05:01.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.648+0000 7f693effd700 1 -- 172.21.15.64:0/3199170912 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6938012410 con 0x7f69480fbe40 2024-02-16T23:05:01.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.648+0000 7f693effd700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f693406d7e0 0x7f693406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:01.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.649+0000 7f693effd700 1 -- 172.21.15.64:0/3199170912 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f693808ed40 con 0x7f69480fbe40 2024-02-16T23:05:01.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.649+0000 7f6945d9b700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f693406d7e0 0x7f693406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:01.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.649+0000 7f6945d9b700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f693406d7e0 0x7f693406fc90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6930007fd0 tx=0x7f6930009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:01.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.653+0000 7f693effd700 1 -- 172.21.15.64:0/3199170912 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6938092030 con 0x7f69480fbe40 2024-02-16T23:05:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.875+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6948066660 con 0x7f693406d7e0 2024-02-16T23:05:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.877+0000 7f693effd700 1 -- 172.21.15.64:0/3199170912 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6948066660 con 0x7f693406d7e0 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.878+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f693406d7e0 msgr2=0x7f693406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.878+0000 7f694ce86700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f693406d7e0 0x7f693406fc90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6930007fd0 tx=0x7f6930009040 comp rx=0 tx=0).stop 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.878+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69480fbe40 msgr2=0x7f69480fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.878+0000 7f694ce86700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69480fbe40 0x7f69480fff00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6938015040 tx=0x7f6938004d10 comp rx=0 tx=0).stop 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.879+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 shutdown_connections 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.879+0000 7f694ce86700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f693406d7e0 0x7f693406fc90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.879+0000 7f694ce86700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69480fbe40 0x7f69480fff00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.879+0000 7f694ce86700 1 --2- 172.21.15.64:0/3199170912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69480fcfb0 0x7f69480fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.879+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 >> 172.21.15.64:0/3199170912 conn(0x7f69480f7520 msgr2=0x7f6948065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.879+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 shutdown_connections 2024-02-16T23:05:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.879+0000 7f694ce86700 1 -- 172.21.15.64:0/3199170912 wait complete. 2024-02-16T23:05:01.889 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:05:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.982+0000 7f5e9a70c700 1 -- 172.21.15.64:0/1077377228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fb220 msgr2=0x7f5e940fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.982+0000 7f5e9a70c700 1 --2- 172.21.15.64:0/1077377228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fb220 0x7f5e940fb630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e84004940 tx=0x7f5e84009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.982+0000 7f5e9a70c700 1 -- 172.21.15.64:0/1077377228 shutdown_connections 2024-02-16T23:05:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.982+0000 7f5e9a70c700 1 --2- 172.21.15.64:0/1077377228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e940fe2b0 0x7f5e940fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.982+0000 7f5e9a70c700 1 --2- 172.21.15.64:0/1077377228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fb220 0x7f5e940fb630 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.982+0000 7f5e9a70c700 1 -- 172.21.15.64:0/1077377228 >> 172.21.15.64:0/1077377228 conn(0x7f5e940f6ca0 msgr2=0x7f5e940f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:01.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:01 smithi064 ceph-mon[93757]: pgmap v217: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T23:05:01.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:01 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:01.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:01 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:01.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.982+0000 7f5e9a70c700 1 -- 172.21.15.64:0/1077377228 shutdown_connections 2024-02-16T23:05:01.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.982+0000 7f5e9a70c700 1 -- 172.21.15.64:0/1077377228 wait complete. 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e9a70c700 1 Processor -- start 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e9a70c700 1 -- start start 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e9a70c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e940fb220 0x7f5e940fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e9a70c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fe2b0 0x7f5e940fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e9a70c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e940fc7c0 con 0x7f5e940fe2b0 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e9a70c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e940fc930 con 0x7f5e940fb220 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e937fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fe2b0 0x7f5e940fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e937fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fe2b0 0x7f5e940fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33884/0 (socket says 172.21.15.64:33884) 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.983+0000 7f5e937fe700 1 -- 172.21.15.64:0/2527316025 learned_addr learned my addr 172.21.15.64:0/2527316025 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e93fff700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e940fb220 0x7f5e940fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e937fe700 1 -- 172.21.15.64:0/2527316025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e940fb220 msgr2=0x7f5e940fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e937fe700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e940fb220 0x7f5e940fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e937fe700 1 -- 172.21.15.64:0/2527316025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e84005040 con 0x7f5e940fe2b0 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e937fe700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fe2b0 0x7f5e940fc1c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e88002a50 tx=0x7f5e880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e90ff9700 1 -- 172.21.15.64:0/2527316025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e88007cc0 con 0x7f5e940fe2b0 2024-02-16T23:05:01.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e940fcc10 con 0x7f5e940fe2b0 2024-02-16T23:05:01.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e90ff9700 1 -- 172.21.15.64:0/2527316025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e8800f040 con 0x7f5e940fe2b0 2024-02-16T23:05:01.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e90ff9700 1 -- 172.21.15.64:0/2527316025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e8800da30 con 0x7f5e940fe2b0 2024-02-16T23:05:01.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.984+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e94115ab0 con 0x7f5e940fe2b0 2024-02-16T23:05:01.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.985+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e94107a80 con 0x7f5e940fe2b0 2024-02-16T23:05:01.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.986+0000 7f5e90ff9700 1 -- 172.21.15.64:0/2527316025 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5e8800db90 con 0x7f5e940fe2b0 2024-02-16T23:05:01.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.986+0000 7f5e90ff9700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5e7c06d7f0 0x7f5e7c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:01.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.986+0000 7f5e90ff9700 1 -- 172.21.15.64:0/2527316025 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5e8808cc60 con 0x7f5e940fe2b0 2024-02-16T23:05:01.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.986+0000 7f5e93fff700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5e7c06d7f0 0x7f5e7c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:01.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.987+0000 7f5e93fff700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5e7c06d7f0 0x7f5e7c06fca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400b3b0 tx=0x7f5e84009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:01.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:01.991+0000 7f5e90ff9700 1 -- 172.21.15.64:0/2527316025 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e88059f60 con 0x7f5e940fe2b0 2024-02-16T23:05:02.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:01 smithi192 ceph-mon[82652]: pgmap v217: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T23:05:02.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:01 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:02.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:01 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:02.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.213+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e9402b340 con 0x7f5e7c06d7f0 2024-02-16T23:05:02.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.215+0000 7f5e90ff9700 1 -- 172.21.15.64:0/2527316025 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5e9402b340 con 0x7f5e7c06d7f0 2024-02-16T23:05:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5e7c06d7f0 msgr2=0x7f5e7c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5e7c06d7f0 0x7f5e7c06fca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400b3b0 tx=0x7f5e84009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fe2b0 msgr2=0x7f5e940fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fe2b0 0x7f5e940fc1c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e88002a50 tx=0x7f5e880046b0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 shutdown_connections 2024-02-16T23:05:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5e7c06d7f0 0x7f5e7c06fca0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e940fb220 0x7f5e940fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 --2- 172.21.15.64:0/2527316025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940fe2b0 0x7f5e940fc1c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 >> 172.21.15.64:0/2527316025 conn(0x7f5e940f6ca0 msgr2=0x7f5e940f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:02.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 shutdown_connections 2024-02-16T23:05:02.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.217+0000 7f5e9a70c700 1 -- 172.21.15.64:0/2527316025 wait complete. 2024-02-16T23:05:02.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/3491027178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fcfc0 msgr2=0x7f8bbc0fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 --2- 172.21.15.64:0/3491027178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fcfc0 0x7f8bbc0fd430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4004940 tx=0x7f8ba4009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/3491027178 shutdown_connections 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 --2- 172.21.15.64:0/3491027178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fcfc0 0x7f8bbc0fd430 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 --2- 172.21.15.64:0/3491027178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bbc0fbe50 0x7f8bbc0fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/3491027178 >> 172.21.15.64:0/3491027178 conn(0x7f8bbc0f7510 msgr2=0x7f8bbc0f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/3491027178 shutdown_connections 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/3491027178 wait complete. 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 Processor -- start 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.320+0000 7f8bc0b4b700 1 -- start start 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bc0b4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fbe50 0x7f8bbc1106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bc0b4b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bbc0fcfc0 0x7f8bbc112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bc0b4b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc066cf0 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bc0b4b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc066e30 con 0x7f8bbc0fcfc0 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bba59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fbe50 0x7f8bbc1106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bba59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fbe50 0x7f8bbc1106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33902/0 (socket says 172.21.15.64:33902) 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bba59c700 1 -- 172.21.15.64:0/2770743069 learned_addr learned my addr 172.21.15.64:0/2770743069 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bb9d9b700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bbc0fcfc0 0x7f8bbc112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bba59c700 1 -- 172.21.15.64:0/2770743069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bbc0fcfc0 msgr2=0x7f8bbc112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bba59c700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bbc0fcfc0 0x7f8bbc112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bba59c700 1 -- 172.21.15.64:0/2770743069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba4005040 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bba59c700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fbe50 0x7f8bbc1106c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00cb10 tx=0x7f8bac007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bb2ffd700 1 -- 172.21.15.64:0/2770743069 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bac004200 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bb2ffd700 1 -- 172.21.15.64:0/2770743069 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bac00ac70 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc113210 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.321+0000 7f8bb2ffd700 1 -- 172.21.15.64:0/2770743069 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bac003d90 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.322+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc113750 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.322+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc04bed0 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.323+0000 7f8bb2ffd700 1 -- 172.21.15.64:0/2770743069 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8bac003430 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.323+0000 7f8bb2ffd700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ba806d7e0 0x7f8ba806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.324+0000 7f8bb2ffd700 1 -- 172.21.15.64:0/2770743069 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8bac08d420 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.324+0000 7f8bb9d9b700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ba806d7e0 0x7f8ba806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.324+0000 7f8bb9d9b700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ba806d7e0 0x7f8ba806fc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b3b0 tx=0x7f8ba4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.328+0000 7f8bb2ffd700 1 -- 172.21.15.64:0/2770743069 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8bac056bf0 con 0x7f8bbc0fbe50 2024-02-16T23:05:02.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.539+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bbc1019a0 con 0x7f8ba806d7e0 2024-02-16T23:05:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.549+0000 7f8bb2ffd700 1 -- 172.21.15.64:0/2770743069 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8bbc1019a0 con 0x7f8ba806d7e0 2024-02-16T23:05:02.549 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (8m) 47s ago 16m 26.0M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (5m) 47s ago 16m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (4m) 2s ago 15m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (7m) 47s ago 16m 84.5M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (52s) 47s ago 11m 11.5M - 19.0.0-1450-gf78a58c0 f081f847070a d4dd49dc2ce8 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (77s) 47s ago 11m 15.1M - 19.0.0-1450-gf78a58c0 f081f847070a bd35cb99fa04 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (5s) 2s ago 11m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a 8d8b30783f08 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (27s) 2s ago 11m 12.4M - 19.0.0-1450-gf78a58c0 f081f847070a ca63fdf53f50 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (7m) 47s ago 17m 507M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (6m) 2s ago 15m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (6m) 47s ago 18m 49.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (5m) 2s ago 15m 42.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (8m) 47s ago 16m 15.1M - 1.5.0 0da6a335fe13 6953eac5e61a 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (7m) 2s ago 15m 15.1M - 1.5.0 0da6a335fe13 420afc763be8 2024-02-16T23:05:02.550 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (4m) 47s ago 14m 106M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:05:02.551 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (4m) 47s ago 14m 97.7M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:05:02.551 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (3m) 47s ago 13m 84.3M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:05:02.551 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (2m) 2s ago 13m 121M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:05:02.551 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (2m) 2s ago 13m 107M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:05:02.551 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (103s) 2s ago 12m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 7bb5b265fc66 2024-02-16T23:05:02.551 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (6m) 47s ago 16m 61.5M - 2.43.0 a07b618ecd1d 39cb20621d80 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ba806d7e0 msgr2=0x7f8ba806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ba806d7e0 0x7f8ba806fc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b3b0 tx=0x7f8ba4009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fbe50 msgr2=0x7f8bbc1106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fbe50 0x7f8bbc1106c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00cb10 tx=0x7f8bac007f70 comp rx=0 tx=0).stop 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 shutdown_connections 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8ba806d7e0 0x7f8ba806fc90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc0fbe50 0x7f8bbc1106c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 --2- 172.21.15.64:0/2770743069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bbc0fcfc0 0x7f8bbc112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.551+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 >> 172.21.15.64:0/2770743069 conn(0x7f8bbc0f7510 msgr2=0x7f8bbc100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.552+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 shutdown_connections 2024-02-16T23:05:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.552+0000 7f8bc0b4b700 1 -- 172.21.15.64:0/2770743069 wait complete. 2024-02-16T23:05:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 -- 172.21.15.64:0/2818244289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fbe40 msgr2=0x7f91080fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 --2- 172.21.15.64:0/2818244289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fbe40 0x7f91080fc250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90f0004940 tx=0x7f90f0009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 -- 172.21.15.64:0/2818244289 shutdown_connections 2024-02-16T23:05:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 --2- 172.21.15.64:0/2818244289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91080fcfb0 0x7f91080fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 --2- 172.21.15.64:0/2818244289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fbe40 0x7f91080fc250 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 -- 172.21.15.64:0/2818244289 >> 172.21.15.64:0/2818244289 conn(0x7f91080f7520 msgr2=0x7f91080f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 -- 172.21.15.64:0/2818244289 shutdown_connections 2024-02-16T23:05:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 -- 172.21.15.64:0/2818244289 wait complete. 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.653+0000 7f910ef6f700 1 Processor -- start 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f910ef6f700 1 -- start start 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f910ef6f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91080fbe40 0x7f910810d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f910ef6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fcfb0 0x7f910810dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f910ef6f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910810e3e0 con 0x7f91080fcfb0 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f910ef6f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910810ec80 con 0x7f91080fbe40 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f9107fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fcfb0 0x7f910810dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f910cd0b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91080fbe40 0x7f910810d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f9107fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fcfb0 0x7f910810dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33916/0 (socket says 172.21.15.64:33916) 2024-02-16T23:05:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f9107fff700 1 -- 172.21.15.64:0/3083105407 learned_addr learned my addr 172.21.15.64:0/3083105407 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f910cd0b700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91080fbe40 0x7f910810d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.64:53772/0 (socket says 172.21.15.64:53772) 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f9107fff700 1 -- 172.21.15.64:0/3083105407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91080fbe40 msgr2=0x7f910810d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f9107fff700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91080fbe40 0x7f910810d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f9107fff700 1 -- 172.21.15.64:0/3083105407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0005040 con 0x7f91080fcfb0 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.654+0000 7f9107fff700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fcfb0 0x7f910810dde0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90f8002a80 tx=0x7f90f80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.655+0000 7f91057fa700 1 -- 172.21.15.64:0/3083105407 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f8007cc0 con 0x7f91080fcfb0 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.655+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910810ef60 con 0x7f91080fcfb0 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.655+0000 7f91057fa700 1 -- 172.21.15.64:0/3083105407 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90f800f040 con 0x7f91080fcfb0 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.655+0000 7f91057fa700 1 -- 172.21.15.64:0/3083105407 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f80096f0 con 0x7f91080fcfb0 2024-02-16T23:05:02.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.655+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910810f4a0 con 0x7f91080fcfb0 2024-02-16T23:05:02.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.656+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910804bed0 con 0x7f91080fcfb0 2024-02-16T23:05:02.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.656+0000 7f91057fa700 1 -- 172.21.15.64:0/3083105407 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f90f800d690 con 0x7f91080fcfb0 2024-02-16T23:05:02.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.657+0000 7f91057fa700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f90f406d7f0 0x7f90f406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:02.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.657+0000 7f910cd0b700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f90f406d7f0 0x7f90f406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:02.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.657+0000 7f91057fa700 1 -- 172.21.15.64:0/3083105407 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f90f808c780 con 0x7f91080fcfb0 2024-02-16T23:05:02.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.657+0000 7f910cd0b700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f90f406d7f0 0x7f90f406fca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b020 tx=0x7f90f0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:02.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.661+0000 7f91057fa700 1 -- 172.21.15.64:0/3083105407 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f90f8059a00 con 0x7f91080fcfb0 2024-02-16T23:05:02.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:02 smithi064 ceph-mon[93757]: from='client.44355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:02.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:02 smithi064 ceph-mon[93757]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:02.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:02 smithi064 ceph-mon[93757]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:02.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:02 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:02.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:02 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:05:02.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.950+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9108104930 con 0x7f91080fcfb0 2024-02-16T23:05:02.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.950+0000 7f91057fa700 1 -- 172.21.15.64:0/3083105407 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f90f8059590 con 0x7f91080fcfb0 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T23:05:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:05:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:05:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T23:05:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:05:02.952 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.952+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f90f406d7f0 msgr2=0x7f90f406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.952+0000 7f910ef6f700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f90f406d7f0 0x7f90f406fca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b020 tx=0x7f90f0009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.952+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fcfb0 msgr2=0x7f910810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.952+0000 7f910ef6f700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fcfb0 0x7f910810dde0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90f8002a80 tx=0x7f90f80046b0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.953+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 shutdown_connections 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.953+0000 7f910ef6f700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f90f406d7f0 0x7f90f406fca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.953+0000 7f910ef6f700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91080fbe40 0x7f910810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.953+0000 7f910ef6f700 1 --2- 172.21.15.64:0/3083105407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91080fcfb0 0x7f910810dde0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.953+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 >> 172.21.15.64:0/3083105407 conn(0x7f91080f7520 msgr2=0x7f9108065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.953+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 shutdown_connections 2024-02-16T23:05:02.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:02.953+0000 7f910ef6f700 1 -- 172.21.15.64:0/3083105407 wait complete. 2024-02-16T23:05:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:02 smithi192 ceph-mon[82652]: from='client.44355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:02 smithi192 ceph-mon[82652]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:02 smithi192 ceph-mon[82652]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:02 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:02 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:05:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 -- 172.21.15.64:0/2296341519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a8100000 msgr2=0x7f86a8102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 --2- 172.21.15.64:0/2296341519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a8100000 0x7f86a8102420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8690004940 tx=0x7f8690009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 -- 172.21.15.64:0/2296341519 shutdown_connections 2024-02-16T23:05:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 --2- 172.21.15.64:0/2296341519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a8100000 0x7f86a8102420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 --2- 172.21.15.64:0/2296341519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a80663a0 0x7f86a80ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 -- 172.21.15.64:0/2296341519 >> 172.21.15.64:0/2296341519 conn(0x7f86a80f7530 msgr2=0x7f86a80f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 -- 172.21.15.64:0/2296341519 shutdown_connections 2024-02-16T23:05:03.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 -- 172.21.15.64:0/2296341519 wait complete. 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 Processor -- start 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.055+0000 7f86ada0f700 1 -- start start 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86ada0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a80663a0 0x7f86a810d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86ada0f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8100000 0x7f86a810ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86ada0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a810e3b0 con 0x7f86a80663a0 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86ada0f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a810ec50 con 0x7f86a8100000 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a80663a0 0x7f86a810d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a80663a0 0x7f86a810d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33934/0 (socket says 172.21.15.64:33934) 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86a6ffd700 1 -- 172.21.15.64:0/3914038854 learned_addr learned my addr 172.21.15.64:0/3914038854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86a67fc700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8100000 0x7f86a810ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86a6ffd700 1 -- 172.21.15.64:0/3914038854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8100000 msgr2=0x7f86a810ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86a6ffd700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8100000 0x7f86a810ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86a6ffd700 1 -- 172.21.15.64:0/3914038854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8690005040 con 0x7f86a80663a0 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86a6ffd700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a80663a0 0x7f86a810d870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f86a0001d00 tx=0x7f86a0009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f869f7fe700 1 -- 172.21.15.64:0/3914038854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a0007d10 con 0x7f86a80663a0 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f869f7fe700 1 -- 172.21.15.64:0/3914038854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86a0007e70 con 0x7f86a80663a0 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a810ef30 con 0x7f86a80663a0 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f869f7fe700 1 -- 172.21.15.64:0/3914038854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a0014400 con 0x7f86a80663a0 2024-02-16T23:05:03.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.056+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a810f470 con 0x7f86a80663a0 2024-02-16T23:05:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.057+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a8107a10 con 0x7f86a80663a0 2024-02-16T23:05:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.058+0000 7f869f7fe700 1 -- 172.21.15.64:0/3914038854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f86a000e900 con 0x7f86a80663a0 2024-02-16T23:05:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.058+0000 7f869f7fe700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f869406d7f0 0x7f869406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.058+0000 7f869f7fe700 1 -- 172.21.15.64:0/3914038854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f86a008ce40 con 0x7f86a80663a0 2024-02-16T23:05:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.058+0000 7f86a67fc700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f869406d7f0 0x7f869406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.059+0000 7f86a67fc700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f869406d7f0 0x7f869406fca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f869000b3b0 tx=0x7f8690009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.063+0000 7f869f7fe700 1 -- 172.21.15.64:0/3914038854 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86a005a170 con 0x7f86a80663a0 2024-02-16T23:05:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.314+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f86a804bed0 con 0x7f86a80663a0 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.315+0000 7f869f7fe700 1 -- 172.21.15.64:0/3914038854 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f86a0059d00 con 0x7f86a80663a0 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout:e35 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 35 2024-02-16T23:05:03.315 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T23:04:58.106175+0000 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 85 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:05:03.316 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=34404} 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:34404} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.64:6828/2551190588,v1:172.21.15.64:6829/2551190588] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:05:03.317 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/2028541376,v1:172.21.15.192:6825/2028541376] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:05:03.318 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/1878053024,v1:172.21.15.192:6827/1878053024] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:05:03.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f869406d7f0 msgr2=0x7f869406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f869406d7f0 0x7f869406fca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f869000b3b0 tx=0x7f8690009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a80663a0 msgr2=0x7f86a810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a80663a0 0x7f86a810d870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f86a0001d00 tx=0x7f86a0009e40 comp rx=0 tx=0).stop 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 shutdown_connections 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f869406d7f0 0x7f869406fca0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a80663a0 0x7f86a810d870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 --2- 172.21.15.64:0/3914038854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8100000 0x7f86a810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 >> 172.21.15.64:0/3914038854 conn(0x7f86a80f7530 msgr2=0x7f86a80f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 shutdown_connections 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.317+0000 7f86ada0f700 1 -- 172.21.15.64:0/3914038854 wait complete. 2024-02-16T23:05:03.319 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.422+0000 7fdb7a183700 1 -- 172.21.15.64:0/4160471254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 msgr2=0x7fdb74070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.422+0000 7fdb7a183700 1 --2- 172.21.15.64:0/4160471254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 0x7fdb74070300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdb64004940 tx=0x7fdb64009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.422+0000 7fdb7a183700 1 -- 172.21.15.64:0/4160471254 shutdown_connections 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.422+0000 7fdb7a183700 1 --2- 172.21.15.64:0/4160471254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb74070840 0x7fdb74070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.422+0000 7fdb7a183700 1 --2- 172.21.15.64:0/4160471254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 0x7fdb74070300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.422+0000 7fdb7a183700 1 -- 172.21.15.64:0/4160471254 >> 172.21.15.64:0/4160471254 conn(0x7fdb740f8680 msgr2=0x7fdb740faad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.422+0000 7fdb7a183700 1 -- 172.21.15.64:0/4160471254 shutdown_connections 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.422+0000 7fdb7a183700 1 -- 172.21.15.64:0/4160471254 wait complete. 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb7a183700 1 Processor -- start 2024-02-16T23:05:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb7a183700 1 -- start start 2024-02-16T23:05:03.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb7a183700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb74070840 0x7fdb7410fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb7a183700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 0x7fdb74110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb7a183700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb74110680 con 0x7fdb74071df0 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb7a183700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb74110f20 con 0x7fdb74070840 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb72ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 0x7fdb74110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb72ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 0x7fdb74110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33950/0 (socket says 172.21.15.64:33950) 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb72ffd700 1 -- 172.21.15.64:0/3147961458 learned_addr learned my addr 172.21.15.64:0/3147961458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.423+0000 7fdb737fe700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb74070840 0x7fdb7410fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb72ffd700 1 -- 172.21.15.64:0/3147961458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb74070840 msgr2=0x7fdb7410fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb72ffd700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb74070840 0x7fdb7410fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb72ffd700 1 -- 172.21.15.64:0/3147961458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb64005040 con 0x7fdb74071df0 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb72ffd700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 0x7fdb74110080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800eb40 tx=0x7fdb6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb5bfff700 1 -- 172.21.15.64:0/3147961458 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb68007cc0 con 0x7fdb74071df0 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb5bfff700 1 -- 172.21.15.64:0/3147961458 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb6800f040 con 0x7fdb74071df0 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb741111a0 con 0x7fdb74071df0 2024-02-16T23:05:03.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb5bfff700 1 -- 172.21.15.64:0/3147961458 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb6800ca10 con 0x7fdb74071df0 2024-02-16T23:05:03.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.424+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb741116e0 con 0x7fdb74071df0 2024-02-16T23:05:03.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.425+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7404bed0 con 0x7fdb74071df0 2024-02-16T23:05:03.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.426+0000 7fdb5bfff700 1 -- 172.21.15.64:0/3147961458 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdb6800cc80 con 0x7fdb74071df0 2024-02-16T23:05:03.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.426+0000 7fdb5bfff700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdb5c06d7f0 0x7fdb5c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.426+0000 7fdb737fe700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdb5c06d7f0 0x7fdb5c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.426+0000 7fdb5bfff700 1 -- 172.21.15.64:0/3147961458 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fdb6808cb70 con 0x7fdb74071df0 2024-02-16T23:05:03.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.427+0000 7fdb737fe700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdb5c06d7f0 0x7fdb5c06fca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400b0d0 tx=0x7fdb64009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:03.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.431+0000 7fdb5bfff700 1 -- 172.21.15.64:0/3147961458 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb68059ea0 con 0x7fdb74071df0 2024-02-16T23:05:03.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.652+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb74111b30 con 0x7fdb5c06d7f0 2024-02-16T23:05:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.653+0000 7fdb5bfff700 1 -- 172.21.15.64:0/3147961458 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdb74111b30 con 0x7fdb5c06d7f0 2024-02-16T23:05:03.653 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:05:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "mds", 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "osd" 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "16/21 daemons upgraded", 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:05:03.654 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdb5c06d7f0 msgr2=0x7fdb5c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdb5c06d7f0 0x7fdb5c06fca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400b0d0 tx=0x7fdb64009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 msgr2=0x7fdb74110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 0x7fdb74110080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800eb40 tx=0x7fdb6800b040 comp rx=0 tx=0).stop 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 shutdown_connections 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fdb5c06d7f0 0x7fdb5c06fca0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb74070840 0x7fdb7410fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 --2- 172.21.15.64:0/3147961458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb74071df0 0x7fdb74110080 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.655+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 >> 172.21.15.64:0/3147961458 conn(0x7fdb740f8680 msgr2=0x7fdb74102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.656+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 shutdown_connections 2024-02-16T23:05:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.656+0000 7fdb7a183700 1 -- 172.21.15.64:0/3147961458 wait complete. 2024-02-16T23:05:03.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.759+0000 7f30397f2700 1 -- 172.21.15.64:0/2369497310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 msgr2=0x7f30340fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.759+0000 7f30397f2700 1 --2- 172.21.15.64:0/2369497310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 0x7f30340fd420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f301c004940 tx=0x7f301c009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:03.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.759+0000 7f30397f2700 1 -- 172.21.15.64:0/2369497310 shutdown_connections 2024-02-16T23:05:03.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.759+0000 7f30397f2700 1 --2- 172.21.15.64:0/2369497310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 0x7f30340fd420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.759+0000 7f30397f2700 1 --2- 172.21.15.64:0/2369497310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30340fbe40 0x7f30340fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.759+0000 7f30397f2700 1 -- 172.21.15.64:0/2369497310 >> 172.21.15.64:0/2369497310 conn(0x7f30340f7520 msgr2=0x7f30340f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:03.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.760+0000 7f30397f2700 1 -- 172.21.15.64:0/2369497310 shutdown_connections 2024-02-16T23:05:03.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.760+0000 7f30397f2700 1 -- 172.21.15.64:0/2369497310 wait complete. 2024-02-16T23:05:03.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.760+0000 7f30397f2700 1 Processor -- start 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.760+0000 7f30397f2700 1 -- start start 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.760+0000 7f30397f2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30340fbe40 0x7f30340fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.760+0000 7f30397f2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 0x7f30340fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.760+0000 7f30397f2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340fe980 con 0x7f30340fcfb0 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.760+0000 7f30397f2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340feaf0 con 0x7f30340fbe40 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 0x7f30340fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 0x7f30340fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33958/0 (socket says 172.21.15.64:33958) 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30327fc700 1 -- 172.21.15.64:0/1856581252 learned_addr learned my addr 172.21.15.64:0/1856581252 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f3032ffd700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30340fbe40 0x7f30340fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30327fc700 1 -- 172.21.15.64:0/1856581252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30340fbe40 msgr2=0x7f30340fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30327fc700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30340fbe40 0x7f30340fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30327fc700 1 -- 172.21.15.64:0/1856581252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c005040 con 0x7f30340fcfb0 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30327fc700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 0x7f30340fe380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f301c009fd0 tx=0x7f301c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f302b7fe700 1 -- 172.21.15.64:0/1856581252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c00e040 con 0x7f30340fcfb0 2024-02-16T23:05:03.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f302b7fe700 1 -- 172.21.15.64:0/1856581252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f301c00bcc0 con 0x7f30340fcfb0 2024-02-16T23:05:03.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30340fed70 con 0x7f30340fcfb0 2024-02-16T23:05:03.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f302b7fe700 1 -- 172.21.15.64:0/1856581252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c01c630 con 0x7f30340fcfb0 2024-02-16T23:05:03.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.761+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3034115960 con 0x7f30340fcfb0 2024-02-16T23:05:03.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.762+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303404bed0 con 0x7f30340fcfb0 2024-02-16T23:05:03.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.763+0000 7f302b7fe700 1 -- 172.21.15.64:0/1856581252 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f301c013980 con 0x7f30340fcfb0 2024-02-16T23:05:03.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.763+0000 7f302b7fe700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f302006d7f0 0x7f302006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:03.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.763+0000 7f302b7fe700 1 -- 172.21.15.64:0/1856581252 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f301c08e460 con 0x7f30340fcfb0 2024-02-16T23:05:03.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.763+0000 7f3032ffd700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f302006d7f0 0x7f302006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:03.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.764+0000 7f3032ffd700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f302006d7f0 0x7f302006fca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f302c004550 tx=0x7f302c009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:03.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:03.768+0000 7f302b7fe700 1 -- 172.21.15.64:0/1856581252 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f301c05b760 con 0x7f30340fcfb0 2024-02-16T23:05:03.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:03 smithi192 ceph-mon[82652]: pgmap v218: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T23:05:03.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:03 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3083105407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:03.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:03 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3914038854' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:05:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.056+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3034115cf0 con 0x7f30340fcfb0 2024-02-16T23:05:04.057 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:03 smithi064 ceph-mon[93757]: pgmap v218: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T23:05:04.057 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:03 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3083105407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.057 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:03 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3914038854' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:05:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.057+0000 7f302b7fe700 1 -- 172.21.15.64:0/1856581252 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f301c05b2f0 con 0x7f30340fcfb0 2024-02-16T23:05:04.058 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:05:04.058 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:05:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:05:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f302006d7f0 msgr2=0x7f302006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f302006d7f0 0x7f302006fca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f302c004550 tx=0x7f302c009040 comp rx=0 tx=0).stop 2024-02-16T23:05:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 msgr2=0x7f30340fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 0x7f30340fe380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f301c009fd0 tx=0x7f301c009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 shutdown_connections 2024-02-16T23:05:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f302006d7f0 0x7f302006fca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30340fbe40 0x7f30340fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 --2- 172.21.15.64:0/1856581252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30340fcfb0 0x7f30340fe380 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 >> 172.21.15.64:0/1856581252 conn(0x7f30340f7520 msgr2=0x7f3034065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 shutdown_connections 2024-02-16T23:05:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:04.059+0000 7f30397f2700 1 -- 172.21.15.64:0/1856581252 wait complete. 2024-02-16T23:05:04.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:04.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1856581252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:05:04.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:05:04.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all mds 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi064.bmcyuf"}]': finished 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi064.tqesxx"}]': finished 2024-02-16T23:05:04.984 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.luiybj"}]': finished 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.tnppti"}]': finished 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all rgw 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all ceph-exporter 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all iscsi 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.985 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.986 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all nfs 2024-02-16T23:05:04.986 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:04.986 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:04.986 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: Upgrade: Setting container_image for all nvmeof 2024-02-16T23:05:04.986 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:04 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:05.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1856581252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:05:05.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all mds 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi064.bmcyuf"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi064.bmcyuf"}]': finished 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi064.tqesxx"}]: dispatch 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi064.tqesxx"}]': finished 2024-02-16T23:05:05.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.luiybj"}]: dispatch 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.luiybj"}]': finished 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.tnppti"}]: dispatch 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.tnppti"}]': finished 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all rgw 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all ceph-exporter 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all iscsi 2024-02-16T23:05:05.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all nfs 2024-02-16T23:05:05.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:05.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:05.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: Upgrade: Setting container_image for all nvmeof 2024-02-16T23:05:05.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:04 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:06.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:06 smithi192 ceph-mon[82652]: pgmap v219: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 8.6 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-02-16T23:05:06.551 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:06 smithi064 ceph-mon[93757]: pgmap v219: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 8.6 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-02-16T23:05:07.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:07 smithi064 ceph-mon[93757]: Upgrade: Updating node-exporter.smithi064 (1/2) 2024-02-16T23:05:07.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:07 smithi064 ceph-mon[93757]: Deploying daemon node-exporter.smithi064 on smithi064 2024-02-16T23:05:07.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:07 smithi192 ceph-mon[82652]: Upgrade: Updating node-exporter.smithi064 (1/2) 2024-02-16T23:05:07.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:07 smithi192 ceph-mon[82652]: Deploying daemon node-exporter.smithi064 on smithi064 2024-02-16T23:05:08.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:08 smithi064 ceph-mon[93757]: pgmap v220: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-02-16T23:05:08.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:08 smithi192 ceph-mon[82652]: pgmap v220: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-02-16T23:05:09.763 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:09.763 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:09.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:09.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:10.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:10 smithi192 ceph-mon[82652]: pgmap v221: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-02-16T23:05:10.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:10 smithi064 ceph-mon[93757]: pgmap v221: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-02-16T23:05:11.838 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:11 smithi192 ceph-mon[82652]: Upgrade: Updating node-exporter.smithi192 (2/2) 2024-02-16T23:05:11.838 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:11 smithi192 ceph-mon[82652]: Deploying daemon node-exporter.smithi192 on smithi192 2024-02-16T23:05:11.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:11 smithi064 ceph-mon[93757]: Upgrade: Updating node-exporter.smithi192 (2/2) 2024-02-16T23:05:11.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:11 smithi064 ceph-mon[93757]: Deploying daemon node-exporter.smithi192 on smithi192 2024-02-16T23:05:12.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:12 smithi064 ceph-mon[93757]: pgmap v222: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 318 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:05:13.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:12 smithi192 ceph-mon[82652]: pgmap v222: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 318 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T23:05:14.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:13 smithi192 ceph-mon[82652]: pgmap v223: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-02-16T23:05:14.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:14.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:14.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:14.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:13 smithi064 ceph-mon[93757]: pgmap v223: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-02-16T23:05:14.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:14.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:14.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:16.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:16 smithi064 ceph-mon[93757]: pgmap v224: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 4.1 KiB/s wr, 2 op/s 2024-02-16T23:05:16.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:16.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:16.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:16 smithi192 ceph-mon[82652]: pgmap v224: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 4.1 KiB/s wr, 2 op/s 2024-02-16T23:05:16.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:16.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:17.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:17.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:17.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:17.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:18 smithi192 ceph-mon[82652]: pgmap v225: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:05:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:18 smithi064 ceph-mon[93757]: pgmap v225: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:05:19.747 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:19 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:19.747 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:19 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:19.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:19 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:19.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:19 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:20.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:20 smithi064 ceph-mon[93757]: pgmap v226: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:20.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:20.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:20 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:21.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:20 smithi192 ceph-mon[82652]: pgmap v226: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:21.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:21.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:20 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:22.852 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: pgmap v227: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:22.852 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.853 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.854 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.854 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:22.854 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:22 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: pgmap v227: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:23.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:23.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:23.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:05:23.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:23.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:22 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:24.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:24 smithi064 ceph-mon[93757]: pgmap v228: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:25.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:24 smithi192 ceph-mon[82652]: pgmap v228: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:25.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:25 smithi064 ceph-mon[93757]: Upgrade: Updating prometheus.smithi064 2024-02-16T23:05:25.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:25 smithi064 ceph-mon[93757]: Deploying daemon prometheus.smithi064 on smithi064 2024-02-16T23:05:26.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:25 smithi192 ceph-mon[82652]: Upgrade: Updating prometheus.smithi064 2024-02-16T23:05:26.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:25 smithi192 ceph-mon[82652]: Deploying daemon prometheus.smithi064 on smithi064 2024-02-16T23:05:26.796 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:26 smithi064 ceph-mon[93757]: pgmap v229: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:27.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:26 smithi192 ceph-mon[82652]: pgmap v229: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:28.806 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:28 smithi064 ceph-mon[93757]: pgmap v230: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:28 smithi192 ceph-mon[82652]: pgmap v230: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:30.745 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:30 smithi064 ceph-mon[93757]: pgmap v231: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:30.745 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:30.745 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:30.745 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:30 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:30 smithi192 ceph-mon[82652]: pgmap v231: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:31.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:30 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:32.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:32 smithi064 ceph-mon[93757]: pgmap v232: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:33.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:32 smithi192 ceph-mon[82652]: pgmap v232: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:33.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:05:34.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:05:34.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 -- 172.21.15.64:0/1403432555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca34066c50 msgr2=0x7fca340ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 --2- 172.21.15.64:0/1403432555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca34066c50 0x7fca340ffb20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fca28004940 tx=0x7fca28009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:34.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 -- 172.21.15.64:0/1403432555 shutdown_connections 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 --2- 172.21.15.64:0/1403432555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca341000f0 0x7fca34102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 --2- 172.21.15.64:0/1403432555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca34066c50 0x7fca340ffb20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 -- 172.21.15.64:0/1403432555 >> 172.21.15.64:0/1403432555 conn(0x7fca340f7510 msgr2=0x7fca340f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 -- 172.21.15.64:0/1403432555 shutdown_connections 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 -- 172.21.15.64:0/1403432555 wait complete. 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.174+0000 7fca39737700 1 Processor -- start 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca39737700 1 -- start start 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca39737700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca34066c50 0x7fca340fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca39737700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca341000f0 0x7fca340fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca39737700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340fc020 con 0x7fca341000f0 2024-02-16T23:05:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca39737700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340fc190 con 0x7fca34066c50 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca341000f0 0x7fca340fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca341000f0 0x7fca340fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45634/0 (socket says 172.21.15.64:45634) 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca327fc700 1 -- 172.21.15.64:0/2837468384 learned_addr learned my addr 172.21.15.64:0/2837468384 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca32ffd700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca34066c50 0x7fca340fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca327fc700 1 -- 172.21.15.64:0/2837468384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca34066c50 msgr2=0x7fca340fd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca327fc700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca34066c50 0x7fca340fd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca327fc700 1 -- 172.21.15.64:0/2837468384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca28005040 con 0x7fca341000f0 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.175+0000 7fca327fc700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca341000f0 0x7fca340fba20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fca2400c800 tx=0x7fca24004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:34.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.176+0000 7fca1b7fe700 1 -- 172.21.15.64:0/2837468384 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca240106f0 con 0x7fca341000f0 2024-02-16T23:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.176+0000 7fca1b7fe700 1 -- 172.21.15.64:0/2837468384 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca24010d20 con 0x7fca341000f0 2024-02-16T23:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.176+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca340fc470 con 0x7fca341000f0 2024-02-16T23:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.176+0000 7fca1b7fe700 1 -- 172.21.15.64:0/2837468384 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca240113a0 con 0x7fca341000f0 2024-02-16T23:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.176+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca34115980 con 0x7fca341000f0 2024-02-16T23:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.177+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3404bed0 con 0x7fca341000f0 2024-02-16T23:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.177+0000 7fca1b7fe700 1 -- 172.21.15.64:0/2837468384 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fca24012400 con 0x7fca341000f0 2024-02-16T23:05:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.178+0000 7fca1b7fe700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fca1c06d7f0 0x7fca1c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.178+0000 7fca1b7fe700 1 -- 172.21.15.64:0/2837468384 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fca2408cef0 con 0x7fca341000f0 2024-02-16T23:05:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.178+0000 7fca32ffd700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fca1c06d7f0 0x7fca1c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.178+0000 7fca32ffd700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fca1c06d7f0 0x7fca1c06fca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca2800b3b0 tx=0x7fca28009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:34.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.183+0000 7fca1b7fe700 1 -- 172.21.15.64:0/2837468384 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca2405a1f0 con 0x7fca341000f0 2024-02-16T23:05:34.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.407+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca34100d50 con 0x7fca1c06d7f0 2024-02-16T23:05:34.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.409+0000 7fca1b7fe700 1 -- 172.21.15.64:0/2837468384 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fca34100d50 con 0x7fca1c06d7f0 2024-02-16T23:05:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fca1c06d7f0 msgr2=0x7fca1c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fca1c06d7f0 0x7fca1c06fca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca2800b3b0 tx=0x7fca28009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca341000f0 msgr2=0x7fca340fba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca341000f0 0x7fca340fba20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fca2400c800 tx=0x7fca24004470 comp rx=0 tx=0).stop 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 shutdown_connections 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fca1c06d7f0 0x7fca1c06fca0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca34066c50 0x7fca340fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 --2- 172.21.15.64:0/2837468384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca341000f0 0x7fca340fba20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 >> 172.21.15.64:0/2837468384 conn(0x7fca340f7510 msgr2=0x7fca340f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 shutdown_connections 2024-02-16T23:05:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.411+0000 7fca39737700 1 -- 172.21.15.64:0/2837468384 wait complete. 2024-02-16T23:05:34.421 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:05:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.516+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1869488370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 msgr2=0x7f34ac0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.516+0000 7f34b1c3b700 1 --2- 172.21.15.64:0/1869488370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 0x7f34ac0fc250 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f34a0004940 tx=0x7f34a0009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.517+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1869488370 shutdown_connections 2024-02-16T23:05:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.517+0000 7f34b1c3b700 1 --2- 172.21.15.64:0/1869488370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ac0fcfb0 0x7f34ac0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.517+0000 7f34b1c3b700 1 --2- 172.21.15.64:0/1869488370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 0x7f34ac0fc250 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.517+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1869488370 >> 172.21.15.64:0/1869488370 conn(0x7f34ac0f7500 msgr2=0x7f34ac0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.517+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1869488370 shutdown_connections 2024-02-16T23:05:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.517+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1869488370 wait complete. 2024-02-16T23:05:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.517+0000 7f34b1c3b700 1 Processor -- start 2024-02-16T23:05:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.517+0000 7f34b1c3b700 1 -- start start 2024-02-16T23:05:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34b1c3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 0x7f34ac110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34b1c3b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ac0fcfb0 0x7f34ac112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34b1c3b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac066d10 con 0x7f34ac0fbe40 2024-02-16T23:05:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34b1c3b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac066e50 con 0x7f34ac0fcfb0 2024-02-16T23:05:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34ab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 0x7f34ac110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34ab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 0x7f34ac110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45644/0 (socket says 172.21.15.64:45644) 2024-02-16T23:05:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34ab7fe700 1 -- 172.21.15.64:0/1283474255 learned_addr learned my addr 172.21.15.64:0/1283474255 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34aaffd700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ac0fcfb0 0x7f34ac112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34ab7fe700 1 -- 172.21.15.64:0/1283474255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ac0fcfb0 msgr2=0x7f34ac112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34ab7fe700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ac0fcfb0 0x7f34ac112b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.518+0000 7f34ab7fe700 1 -- 172.21.15.64:0/1283474255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a0005040 con 0x7f34ac0fbe40 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.519+0000 7f34ab7fe700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 0x7f34ac110640 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009fd0 tx=0x7f34a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.519+0000 7f3493fff700 1 -- 172.21.15.64:0/1283474255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34a000e040 con 0x7f34ac0fbe40 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.519+0000 7f3493fff700 1 -- 172.21.15.64:0/1283474255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34a000bcc0 con 0x7f34ac0fbe40 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.519+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac113190 con 0x7f34ac0fbe40 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.519+0000 7f3493fff700 1 -- 172.21.15.64:0/1283474255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34a001c630 con 0x7f34ac0fbe40 2024-02-16T23:05:34.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.519+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac113680 con 0x7f34ac0fbe40 2024-02-16T23:05:34.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.520+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac04bed0 con 0x7f34ac0fbe40 2024-02-16T23:05:34.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.521+0000 7f3493fff700 1 -- 172.21.15.64:0/1283474255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f34a0013420 con 0x7f34ac0fbe40 2024-02-16T23:05:34.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.521+0000 7f3493fff700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f349406d7f0 0x7f349406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.522+0000 7f3493fff700 1 -- 172.21.15.64:0/1283474255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f34a008e430 con 0x7f34ac0fbe40 2024-02-16T23:05:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.522+0000 7f34aaffd700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f349406d7f0 0x7f349406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.522+0000 7f34aaffd700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f349406d7f0 0x7f349406fca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f349c004f40 tx=0x7f349c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:34.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.527+0000 7f3493fff700 1 -- 172.21.15.64:0/1283474255 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f34a0057c00 con 0x7f34ac0fbe40 2024-02-16T23:05:34.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.758+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34ac101990 con 0x7f349406d7f0 2024-02-16T23:05:34.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.759+0000 7f3493fff700 1 -- 172.21.15.64:0/1283474255 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f34ac101990 con 0x7f349406d7f0 2024-02-16T23:05:34.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f349406d7f0 msgr2=0x7f349406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f349406d7f0 0x7f349406fca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f349c004f40 tx=0x7f349c00b040 comp rx=0 tx=0).stop 2024-02-16T23:05:34.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 msgr2=0x7f34ac110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 0x7f34ac110640 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009fd0 tx=0x7f34a0009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:34.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 shutdown_connections 2024-02-16T23:05:34.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f349406d7f0 0x7f349406fca0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34ac0fbe40 0x7f34ac110640 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 --2- 172.21.15.64:0/1283474255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ac0fcfb0 0x7f34ac112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 >> 172.21.15.64:0/1283474255 conn(0x7f34ac0f7500 msgr2=0x7f34ac100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:34.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 shutdown_connections 2024-02-16T23:05:34.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.761+0000 7f34b1c3b700 1 -- 172.21.15.64:0/1283474255 wait complete. 2024-02-16T23:05:34.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:34 smithi064 ceph-mon[93757]: pgmap v233: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:34.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:34 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:34.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:34 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.867+0000 7f94131a1700 1 -- 172.21.15.64:0/1461310130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0feed0 msgr2=0x7f940c0fb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.867+0000 7f94131a1700 1 --2- 172.21.15.64:0/1461310130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0feed0 0x7f940c0fb010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f93f8004940 tx=0x7f93f8009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.867+0000 7f94131a1700 1 -- 172.21.15.64:0/1461310130 shutdown_connections 2024-02-16T23:05:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.867+0000 7f94131a1700 1 --2- 172.21.15.64:0/1461310130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0feed0 0x7f940c0fb010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.867+0000 7f94131a1700 1 --2- 172.21.15.64:0/1461310130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f940c0fe580 0x7f940c0fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.867+0000 7f94131a1700 1 -- 172.21.15.64:0/1461310130 >> 172.21.15.64:0/1461310130 conn(0x7f940c0f6d20 msgr2=0x7f940c0f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.867+0000 7f94131a1700 1 -- 172.21.15.64:0/1461310130 shutdown_connections 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.867+0000 7f94131a1700 1 -- 172.21.15.64:0/1461310130 wait complete. 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f94131a1700 1 Processor -- start 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f94131a1700 1 -- start start 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f94131a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0fe580 0x7f940c10dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f94131a1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f940c0feed0 0x7f940c10e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f94131a1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c10e830 con 0x7f940c0fe580 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f94131a1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c10efc0 con 0x7f940c0feed0 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f9410f3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0fe580 0x7f940c10dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f9410f3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0fe580 0x7f940c10dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45658/0 (socket says 172.21.15.64:45658) 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f9410f3d700 1 -- 172.21.15.64:0/106220286 learned_addr learned my addr 172.21.15.64:0/106220286 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.868+0000 7f940bfff700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f940c0feed0 0x7f940c10e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f9410f3d700 1 -- 172.21.15.64:0/106220286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f940c0feed0 msgr2=0x7f940c10e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f9410f3d700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f940c0feed0 0x7f940c10e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f9410f3d700 1 -- 172.21.15.64:0/106220286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f8005040 con 0x7f940c0fe580 2024-02-16T23:05:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f9410f3d700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0fe580 0x7f940c10dcf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f940000c8c0 tx=0x7f94000044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f94097fa700 1 -- 172.21.15.64:0/106220286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f940000a760 con 0x7f940c0fe580 2024-02-16T23:05:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c10f2a0 con 0x7f940c0fe580 2024-02-16T23:05:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f94097fa700 1 -- 172.21.15.64:0/106220286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f940000ad90 con 0x7f940c0fe580 2024-02-16T23:05:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f94097fa700 1 -- 172.21.15.64:0/106220286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94000113a0 con 0x7f940c0fe580 2024-02-16T23:05:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.869+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c10f7e0 con 0x7f940c0fe580 2024-02-16T23:05:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.870+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c107a70 con 0x7f940c0fe580 2024-02-16T23:05:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.871+0000 7f94097fa700 1 -- 172.21.15.64:0/106220286 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9400012400 con 0x7f940c0fe580 2024-02-16T23:05:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.871+0000 7f94097fa700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f93fc06d7e0 0x7f93fc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.871+0000 7f94097fa700 1 -- 172.21.15.64:0/106220286 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f940008cf10 con 0x7f940c0fe580 2024-02-16T23:05:34.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.872+0000 7f940bfff700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f93fc06d7e0 0x7f93fc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:34.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.872+0000 7f940bfff700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f93fc06d7e0 0x7f93fc06fc90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f93f800b3b0 tx=0x7f93f8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:34.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:34.876+0000 7f94097fa700 1 -- 172.21.15.64:0/106220286 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f940005a2d0 con 0x7f940c0fe580 2024-02-16T23:05:35.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:34 smithi192 ceph-mon[82652]: pgmap v233: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:35.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:34 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:35.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:34 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.092+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f940c105ba0 con 0x7f93fc06d7e0 2024-02-16T23:05:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.103+0000 7f94097fa700 1 -- 172.21.15.64:0/106220286 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f940c105ba0 con 0x7f93fc06d7e0 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (8m) 1s ago 17m 26.1M - 0.25.0 c8568f914cd2 75c72d293b71 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (5m) 1s ago 17m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (5m) 19s ago 16m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (8m) 1s ago 16m 85.0M - 9.4.7 954c08fa6188 6a17180bdc47 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (84s) 1s ago 11m 12.0M - 19.0.0-1450-gf78a58c0 f081f847070a d4dd49dc2ce8 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (110s) 1s ago 11m 87.9M - 19.0.0-1450-gf78a58c0 f081f847070a bd35cb99fa04 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (38s) 19s ago 11m 14.0M - 19.0.0-1450-gf78a58c0 f081f847070a 8d8b30783f08 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (59s) 19s ago 11m 12.4M - 19.0.0-1450-gf78a58c0 f081f847070a ca63fdf53f50 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (7m) 1s ago 18m 510M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:05:35.103 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (6m) 19s ago 16m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (6m) 1s ago 18m 53.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (6m) 19s ago 16m 43.2M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (27s) 1s ago 17m 8468k - 1.5.0 0da6a335fe13 cbac3ca3c432 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (22s) 19s ago 16m 7906k - 1.5.0 0da6a335fe13 dd661245993a 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (5m) 1s ago 15m 107M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (4m) 1s ago 14m 100M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (3m) 1s ago 14m 86.5M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (3m) 19s ago 14m 121M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (2m) 19s ago 13m 108M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (2m) 19s ago 13m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 7bb5b265fc66 2024-02-16T23:05:35.104 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (5s) 1s ago 16m 40.0M - 2.43.0 a07b618ecd1d 51136d8386ab 2024-02-16T23:05:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f93fc06d7e0 msgr2=0x7f93fc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f93fc06d7e0 0x7f93fc06fc90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f93f800b3b0 tx=0x7f93f8009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0fe580 msgr2=0x7f940c10dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0fe580 0x7f940c10dcf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f940000c8c0 tx=0x7f94000044c0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 shutdown_connections 2024-02-16T23:05:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f93fc06d7e0 0x7f93fc06fc90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f940c0fe580 0x7f940c10dcf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 --2- 172.21.15.64:0/106220286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f940c0feed0 0x7f940c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 >> 172.21.15.64:0/106220286 conn(0x7f940c0f6d20 msgr2=0x7f940c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 shutdown_connections 2024-02-16T23:05:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.105+0000 7f94131a1700 1 -- 172.21.15.64:0/106220286 wait complete. 2024-02-16T23:05:35.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.211+0000 7f7f056d2700 1 -- 172.21.15.64:0/3854583673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f000fda80 msgr2=0x7f7f000ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.211+0000 7f7f056d2700 1 --2- 172.21.15.64:0/3854583673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f000fda80 0x7f7f000ffe60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0004940 tx=0x7f7ef0009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:35.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.211+0000 7f7f056d2700 1 -- 172.21.15.64:0/3854583673 shutdown_connections 2024-02-16T23:05:35.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.211+0000 7f7f056d2700 1 --2- 172.21.15.64:0/3854583673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f001003a0 0x7f7f00102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.211+0000 7f7f056d2700 1 --2- 172.21.15.64:0/3854583673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f000fda80 0x7f7f000ffe60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.211+0000 7f7f056d2700 1 -- 172.21.15.64:0/3854583673 >> 172.21.15.64:0/3854583673 conn(0x7f7f000f7500 msgr2=0x7f7f000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:35.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.212+0000 7f7f056d2700 1 -- 172.21.15.64:0/3854583673 shutdown_connections 2024-02-16T23:05:35.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.212+0000 7f7f056d2700 1 -- 172.21.15.64:0/3854583673 wait complete. 2024-02-16T23:05:35.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.212+0000 7f7f056d2700 1 Processor -- start 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.212+0000 7f7f056d2700 1 -- start start 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.212+0000 7f7f056d2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f000fda80 0x7f7f000fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7f056d2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f001003a0 0x7f7f000fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7f056d2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000fc020 con 0x7f7f001003a0 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7f056d2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000fc190 con 0x7f7f000fda80 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efe7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f001003a0 0x7f7f000fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efe7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f001003a0 0x7f7f000fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45670/0 (socket says 172.21.15.64:45670) 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efe7fc700 1 -- 172.21.15.64:0/3666169704 learned_addr learned my addr 172.21.15.64:0/3666169704 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efeffd700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f000fda80 0x7f7f000fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efe7fc700 1 -- 172.21.15.64:0/3666169704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f000fda80 msgr2=0x7f7f000fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efe7fc700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f000fda80 0x7f7f000fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efe7fc700 1 -- 172.21.15.64:0/3666169704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0005040 con 0x7f7f001003a0 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efeffd700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f000fda80 0x7f7f000fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7efe7fc700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f001003a0 0x7f7f000fb990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400b790 tx=0x7f7ef4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:35.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.213+0000 7f7ee77fe700 1 -- 172.21.15.64:0/3666169704 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef400f6f0 con 0x7f7f001003a0 2024-02-16T23:05:35.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.214+0000 7f7ee77fe700 1 -- 172.21.15.64:0/3666169704 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ef400fd20 con 0x7f7f001003a0 2024-02-16T23:05:35.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.214+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f000fc470 con 0x7f7f001003a0 2024-02-16T23:05:35.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.214+0000 7f7ee77fe700 1 -- 172.21.15.64:0/3666169704 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef40103a0 con 0x7f7f001003a0 2024-02-16T23:05:35.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.214+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f000fc7f0 con 0x7f7f001003a0 2024-02-16T23:05:35.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.215+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f00107ad0 con 0x7f7f001003a0 2024-02-16T23:05:35.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.215+0000 7f7ee77fe700 1 -- 172.21.15.64:0/3666169704 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7ef4011400 con 0x7f7f001003a0 2024-02-16T23:05:35.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.216+0000 7f7ee77fe700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7ee806d7f0 0x7f7ee806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:35.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.216+0000 7f7ee77fe700 1 -- 172.21.15.64:0/3666169704 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7ef408cca0 con 0x7f7f001003a0 2024-02-16T23:05:35.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.216+0000 7f7efeffd700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7ee806d7f0 0x7f7ee806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:35.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.216+0000 7f7efeffd700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7ee806d7f0 0x7f7ee806fca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000b3b0 tx=0x7f7ef0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:35.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.221+0000 7f7ee77fe700 1 -- 172.21.15.64:0/3666169704 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ef4091070 con 0x7f7f001003a0 2024-02-16T23:05:35.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.524+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7f0002b2a0 con 0x7f7f001003a0 2024-02-16T23:05:35.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.524+0000 7f7ee77fe700 1 -- 172.21.15.64:0/3666169704 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7ef405a060 con 0x7f7f001003a0 2024-02-16T23:05:35.524 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:05:35.524 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:05:35.524 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:05:35.525 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:05:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7ee806d7f0 msgr2=0x7f7ee806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7ee806d7f0 0x7f7ee806fca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000b3b0 tx=0x7f7ef0009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f001003a0 msgr2=0x7f7f000fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f001003a0 0x7f7f000fb990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400b790 tx=0x7f7ef4009250 comp rx=0 tx=0).stop 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 shutdown_connections 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f7ee806d7f0 0x7f7ee806fca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f000fda80 0x7f7f000fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 --2- 172.21.15.64:0/3666169704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f001003a0 0x7f7f000fb990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 >> 172.21.15.64:0/3666169704 conn(0x7f7f000f7500 msgr2=0x7f7f000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.526+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 shutdown_connections 2024-02-16T23:05:35.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.527+0000 7f7f056d2700 1 -- 172.21.15.64:0/3666169704 wait complete. 2024-02-16T23:05:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.632+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1374256622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fcef0 msgr2=0x7ff2180fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.632+0000 7ff21dfc8700 1 --2- 172.21.15.64:0/1374256622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fcef0 0x7ff2180fd320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff20c004940 tx=0x7ff20c009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.632+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1374256622 shutdown_connections 2024-02-16T23:05:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.632+0000 7ff21dfc8700 1 --2- 172.21.15.64:0/1374256622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2180fe0e0 0x7ff2180fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.632+0000 7ff21dfc8700 1 --2- 172.21.15.64:0/1374256622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fcef0 0x7ff2180fd320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.632+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1374256622 >> 172.21.15.64:0/1374256622 conn(0x7ff2180f8620 msgr2=0x7ff2180faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.632+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1374256622 shutdown_connections 2024-02-16T23:05:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.632+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1374256622 wait complete. 2024-02-16T23:05:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff21dfc8700 1 Processor -- start 2024-02-16T23:05:35.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff21dfc8700 1 -- start start 2024-02-16T23:05:35.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff21dfc8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2180fcef0 0x7ff21810fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff21dfc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fe0e0 0x7ff2181100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff21dfc8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181106c0 con 0x7ff2180fe0e0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff21dfc8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218110f60 con 0x7ff2180fcef0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff216ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fe0e0 0x7ff2181100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff216ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fe0e0 0x7ff2181100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45692/0 (socket says 172.21.15.64:45692) 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff216ffd700 1 -- 172.21.15.64:0/1813593094 learned_addr learned my addr 172.21.15.64:0/1813593094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff2177fe700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2180fcef0 0x7ff21810fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff216ffd700 1 -- 172.21.15.64:0/1813593094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2180fcef0 msgr2=0x7ff21810fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.633+0000 7ff216ffd700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2180fcef0 0x7ff21810fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.634+0000 7ff216ffd700 1 -- 172.21.15.64:0/1813593094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c005040 con 0x7ff2180fe0e0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.634+0000 7ff216ffd700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fe0e0 0x7ff2181100c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff208002a50 tx=0x7ff2080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.634+0000 7ff1fffff700 1 -- 172.21.15.64:0/1813593094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff208007cc0 con 0x7ff2180fe0e0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.634+0000 7ff1fffff700 1 -- 172.21.15.64:0/1813593094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff20800f040 con 0x7ff2180fe0e0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.634+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff218111240 con 0x7ff2180fe0e0 2024-02-16T23:05:35.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.634+0000 7ff1fffff700 1 -- 172.21.15.64:0/1813593094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20800da30 con 0x7ff2180fe0e0 2024-02-16T23:05:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.634+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff218111780 con 0x7ff2180fe0e0 2024-02-16T23:05:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.635+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff218102210 con 0x7ff2180fe0e0 2024-02-16T23:05:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.635+0000 7ff1fffff700 1 -- 172.21.15.64:0/1813593094 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff20800db90 con 0x7ff2180fe0e0 2024-02-16T23:05:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.636+0000 7ff1fffff700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff20006d7f0 0x7ff20006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.636+0000 7ff1fffff700 1 -- 172.21.15.64:0/1813593094 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff20808cd70 con 0x7ff2180fe0e0 2024-02-16T23:05:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.636+0000 7ff2177fe700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff20006d7f0 0x7ff20006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.637+0000 7ff2177fe700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff20006d7f0 0x7ff20006fca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00b3b0 tx=0x7ff20c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:35.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.641+0000 7ff1fffff700 1 -- 172.21.15.64:0/1813593094 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff208059ff0 con 0x7ff2180fe0e0 2024-02-16T23:05:35.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:35 smithi064 ceph-mon[93757]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:35.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:35.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:35 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:35.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:35 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3666169704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:35.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.902+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff21804bed0 con 0x7ff2180fe0e0 2024-02-16T23:05:35.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.902+0000 7ff1fffff700 1 -- 172.21.15.64:0/1813593094 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1872 (secure 0 0 0) 0x7ff208059b80 con 0x7ff2180fe0e0 2024-02-16T23:05:35.902 INFO:teuthology.orchestra.run.smithi064.stdout:e35 2024-02-16T23:05:35.902 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:05:35.902 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:05:35.902 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:05:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:05:35.902 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:05:35.902 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 35 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T23:04:58.106175+0000 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 85 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:05:35.903 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=34404} 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:34404} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:05:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:05:35.905 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.64:6828/2551190588,v1:172.21.15.64:6829/2551190588] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:05:35.905 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/2028541376,v1:172.21.15.192:6825/2028541376] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:05:35.905 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/1878053024,v1:172.21.15.192:6827/1878053024] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:05:35.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff20006d7f0 msgr2=0x7ff20006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff20006d7f0 0x7ff20006fca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00b3b0 tx=0x7ff20c009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:35.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fe0e0 msgr2=0x7ff2181100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fe0e0 0x7ff2181100c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff208002a50 tx=0x7ff2080046b0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 shutdown_connections 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff20006d7f0 0x7ff20006fca0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2180fcef0 0x7ff21810fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 --2- 172.21.15.64:0/1813593094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2180fe0e0 0x7ff2181100c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 >> 172.21.15.64:0/1813593094 conn(0x7ff2180f8620 msgr2=0x7ff2180ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 shutdown_connections 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:35.904+0000 7ff21dfc8700 1 -- 172.21.15.64:0/1813593094 wait complete. 2024-02-16T23:05:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2024-02-16T23:05:36.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.009+0000 7f206830a700 1 -- 172.21.15.64:0/3813171937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 msgr2=0x7f20600fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:36.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.009+0000 7f206830a700 1 --2- 172.21.15.64:0/3813171937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 0x7f20600fd420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2050004940 tx=0x7f2050009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.009+0000 7f206830a700 1 -- 172.21.15.64:0/3813171937 shutdown_connections 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.009+0000 7f206830a700 1 --2- 172.21.15.64:0/3813171937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 0x7f20600fd420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.009+0000 7f206830a700 1 --2- 172.21.15.64:0/3813171937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600fbe40 0x7f20600fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.009+0000 7f206830a700 1 -- 172.21.15.64:0/3813171937 >> 172.21.15.64:0/3813171937 conn(0x7f20600f7500 msgr2=0x7f20600f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f206830a700 1 -- 172.21.15.64:0/3813171937 shutdown_connections 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f206830a700 1 -- 172.21.15.64:0/3813171937 wait complete. 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f206830a700 1 Processor -- start 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f206830a700 1 -- start start 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f206830a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600fbe40 0x7f2060110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f206830a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 0x7f2060112b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f206830a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060066d50 con 0x7f20600fcfb0 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f206830a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060066e90 con 0x7f20600fbe40 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f20658a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 0x7f2060112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f20658a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 0x7f2060112b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45718/0 (socket says 172.21.15.64:45718) 2024-02-16T23:05:36.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f20658a5700 1 -- 172.21.15.64:0/995293029 learned_addr learned my addr 172.21.15.64:0/995293029 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:36.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.010+0000 7f20660a6700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600fbe40 0x7f2060110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:36.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f20658a5700 1 -- 172.21.15.64:0/995293029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600fbe40 msgr2=0x7f2060110600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:36.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f20658a5700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600fbe40 0x7f2060110600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f20658a5700 1 -- 172.21.15.64:0/995293029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050005040 con 0x7f20600fcfb0 2024-02-16T23:05:36.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f20658a5700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 0x7f2060112b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20500093e0 tx=0x7f2050009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f2056ffd700 1 -- 172.21.15.64:0/995293029 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f205000e040 con 0x7f20600fcfb0 2024-02-16T23:05:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f2056ffd700 1 -- 172.21.15.64:0/995293029 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f205000bcc0 con 0x7f20600fcfb0 2024-02-16T23:05:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2060113150 con 0x7f20600fcfb0 2024-02-16T23:05:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f2056ffd700 1 -- 172.21.15.64:0/995293029 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f205001c630 con 0x7f20600fcfb0 2024-02-16T23:05:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.011+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2060113690 con 0x7f20600fcfb0 2024-02-16T23:05:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.012+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206004bed0 con 0x7f20600fcfb0 2024-02-16T23:05:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.012+0000 7f2056ffd700 1 -- 172.21.15.64:0/995293029 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2050013980 con 0x7f20600fcfb0 2024-02-16T23:05:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.013+0000 7f2056ffd700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f204c06d7e0 0x7f204c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.013+0000 7f2056ffd700 1 -- 172.21.15.64:0/995293029 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f205008e370 con 0x7f20600fcfb0 2024-02-16T23:05:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.013+0000 7f20660a6700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f204c06d7e0 0x7f204c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:36.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.013+0000 7f20660a6700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f204c06d7e0 0x7f204c06fc90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f205c0041d0 tx=0x7f205c0043c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:36.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.018+0000 7f2056ffd700 1 -- 172.21.15.64:0/995293029 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2050057b40 con 0x7f20600fcfb0 2024-02-16T23:05:36.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:35 smithi192 ceph-mon[82652]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:36.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:36.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:35 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:36.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:35 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3666169704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.246+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2060101990 con 0x7f204c06d7e0 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.247+0000 7f2056ffd700 1 -- 172.21.15.64:0/995293029 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f2060101990 con 0x7f204c06d7e0 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "mds", 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:05:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "osd" 2024-02-16T23:05:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:05:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "16/21 daemons upgraded", 2024-02-16T23:05:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading prometheus daemons", 2024-02-16T23:05:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:05:36.248 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:05:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f204c06d7e0 msgr2=0x7f204c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f204c06d7e0 0x7f204c06fc90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f205c0041d0 tx=0x7f205c0043c0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 msgr2=0x7f2060112b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 0x7f2060112b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20500093e0 tx=0x7f2050009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 shutdown_connections 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f204c06d7e0 0x7f204c06fc90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600fbe40 0x7f2060110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 --2- 172.21.15.64:0/995293029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20600fcfb0 0x7f2060112b50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 >> 172.21.15.64:0/995293029 conn(0x7f20600f7500 msgr2=0x7f2060100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 shutdown_connections 2024-02-16T23:05:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.249+0000 7f206830a700 1 -- 172.21.15.64:0/995293029 wait complete. 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.352+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3234367214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 msgr2=0x7f51f80fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.352+0000 7f51ff8f1700 1 --2- 172.21.15.64:0/3234367214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 0x7f51f80fc250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f51f4004940 tx=0x7f51f4009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.352+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3234367214 shutdown_connections 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.352+0000 7f51ff8f1700 1 --2- 172.21.15.64:0/3234367214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f80fcfb0 0x7f51f80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.352+0000 7f51ff8f1700 1 --2- 172.21.15.64:0/3234367214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 0x7f51f80fc250 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.352+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3234367214 >> 172.21.15.64:0/3234367214 conn(0x7f51f80f7520 msgr2=0x7f51f80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3234367214 shutdown_connections 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3234367214 wait complete. 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51ff8f1700 1 Processor -- start 2024-02-16T23:05:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51ff8f1700 1 -- start start 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51ff8f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 0x7f51f810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51ff8f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f80fcfb0 0x7f51f810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51ff8f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f810e500 con 0x7f51f80fbe40 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51ff8f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f810ed40 con 0x7f51f80fcfb0 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51fd68d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 0x7f51f810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51fd68d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 0x7f51f810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45744/0 (socket says 172.21.15.64:45744) 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.353+0000 7f51fd68d700 1 -- 172.21.15.64:0/3215732546 learned_addr learned my addr 172.21.15.64:0/3215732546 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51fce8c700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f80fcfb0 0x7f51f810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51fd68d700 1 -- 172.21.15.64:0/3215732546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f80fcfb0 msgr2=0x7f51f810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51fd68d700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f80fcfb0 0x7f51f810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51fd68d700 1 -- 172.21.15.64:0/3215732546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4005040 con 0x7f51f80fbe40 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51fd68d700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 0x7f51f810d930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f51f40093b0 tx=0x7f51f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51edffb700 1 -- 172.21.15.64:0/3215732546 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f400e040 con 0x7f51f80fbe40 2024-02-16T23:05:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51edffb700 1 -- 172.21.15.64:0/3215732546 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51f400bcc0 con 0x7f51f80fbe40 2024-02-16T23:05:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f810efc0 con 0x7f51f80fbe40 2024-02-16T23:05:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51edffb700 1 -- 172.21.15.64:0/3215732546 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f401c630 con 0x7f51f80fbe40 2024-02-16T23:05:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.354+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f810f500 con 0x7f51f80fbe40 2024-02-16T23:05:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.355+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f804bed0 con 0x7f51f80fbe40 2024-02-16T23:05:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.356+0000 7f51edffb700 1 -- 172.21.15.64:0/3215732546 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f51f40139f0 con 0x7f51f80fbe40 2024-02-16T23:05:36.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.356+0000 7f51edffb700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f51e406d7e0 0x7f51e406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:05:36.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.356+0000 7f51edffb700 1 -- 172.21.15.64:0/3215732546 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f51f408e340 con 0x7f51f80fbe40 2024-02-16T23:05:36.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.356+0000 7f51fce8c700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f51e406d7e0 0x7f51e406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:05:36.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.357+0000 7f51fce8c700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f51e406d7e0 0x7f51e406fc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f51e8006fd0 tx=0x7f51e8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:05:36.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.361+0000 7f51edffb700 1 -- 172.21.15.64:0/3215732546 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51f405b700 con 0x7f51f80fbe40 2024-02-16T23:05:36.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.658+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f51f81049b0 con 0x7f51f80fbe40 2024-02-16T23:05:36.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.659+0000 7f51edffb700 1 -- 172.21.15.64:0/3215732546 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f51f405b290 con 0x7f51f80fbe40 2024-02-16T23:05:36.659 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:05:36.659 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:05:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:05:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f51e406d7e0 msgr2=0x7f51e406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f51e406d7e0 0x7f51e406fc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f51e8006fd0 tx=0x7f51e8008040 comp rx=0 tx=0).stop 2024-02-16T23:05:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 msgr2=0x7f51f810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:05:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 0x7f51f810d930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f51f40093b0 tx=0x7f51f4009e30 comp rx=0 tx=0).stop 2024-02-16T23:05:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 shutdown_connections 2024-02-16T23:05:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f51e406d7e0 0x7f51e406fc90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f80fbe40 0x7f51f810d930 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 --2- 172.21.15.64:0/3215732546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f80fcfb0 0x7f51f810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:05:36.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 >> 172.21.15.64:0/3215732546 conn(0x7f51f80f7520 msgr2=0x7f51f8065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:05:36.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 shutdown_connections 2024-02-16T23:05:36.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:05:36.661+0000 7f51ff8f1700 1 -- 172.21.15.64:0/3215732546 wait complete. 2024-02-16T23:05:36.972 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:36 smithi064 ceph-mon[93757]: pgmap v234: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:36.972 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:36 smithi064 ceph-mon[93757]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:36.972 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:36 smithi064 ceph-mon[93757]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:36.972 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:36 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1813593094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:05:36.972 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:36 smithi064 ceph-mon[93757]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:37.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:36 smithi192 ceph-mon[82652]: pgmap v234: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:37.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:36 smithi192 ceph-mon[82652]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:37.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:36 smithi192 ceph-mon[82652]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:37.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:36 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1813593094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:05:37.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:36 smithi192 ceph-mon[82652]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:05:38.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:37 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3215732546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:05:38.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:37 smithi192 ceph-mon[82652]: pgmap v235: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:38.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:37 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3215732546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:05:38.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:37 smithi064 ceph-mon[93757]: pgmap v235: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: pgmap v236: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.119 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:39 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: pgmap v236: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:40.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:40.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:40.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:05:40.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.285 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:40.286 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:39 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:41.172 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:40 smithi064 ceph-mon[93757]: Upgrade: Updating alertmanager.smithi064 2024-02-16T23:05:41.172 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:40 smithi064 ceph-mon[93757]: Deploying daemon alertmanager.smithi064 on smithi064 2024-02-16T23:05:41.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:40 smithi192 ceph-mon[82652]: Upgrade: Updating alertmanager.smithi064 2024-02-16T23:05:41.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:40 smithi192 ceph-mon[82652]: Deploying daemon alertmanager.smithi064 on smithi064 2024-02-16T23:05:42.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:41 smithi192 ceph-mon[82652]: pgmap v237: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:42.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:41 smithi064 ceph-mon[93757]: pgmap v237: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:44.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:43 smithi064 ceph-mon[93757]: pgmap v238: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:44.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:43 smithi192 ceph-mon[82652]: pgmap v238: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:46.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:46 smithi192 ceph-mon[82652]: pgmap v239: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:46.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:46 smithi064 ceph-mon[93757]: pgmap v239: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:47.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:47.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:47.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:47.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:47.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:47.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:48.349 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:48 smithi064 ceph-mon[93757]: pgmap v240: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:48.350 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:05:48.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:48 smithi192 ceph-mon[82652]: pgmap v240: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:48.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:05:50.494 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:50 smithi064 ceph-mon[93757]: pgmap v241: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:50.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:50 smithi192 ceph-mon[82652]: pgmap v241: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:52.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:51 smithi192 ceph-mon[82652]: pgmap v242: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:52.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:51 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:52.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:51 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:52.293 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:51 smithi064 ceph-mon[93757]: pgmap v242: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:52.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:51 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:52.294 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:51 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:53.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:53.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:53 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:53.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:53 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:54.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:54 smithi064 ceph-mon[93757]: pgmap v243: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:54.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:54 smithi192 ceph-mon[82652]: pgmap v243: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: pgmap v244: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.369 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:56 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: pgmap v244: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:05:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:05:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:05:56.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:56.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:56 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:05:57.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:57 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T23:05:57.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:57 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T23:05:58.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:58 smithi064 ceph-mon[93757]: pgmap v245: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:58.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:58 smithi192 ceph-mon[82652]: pgmap v245: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:05:59.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:59 smithi064 ceph-mon[93757]: Upgrade: Updating grafana.smithi064 2024-02-16T23:05:59.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:05:59 smithi064 ceph-mon[93757]: Deploying daemon grafana.smithi064 on smithi064 2024-02-16T23:05:59.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:59 smithi192 ceph-mon[82652]: Upgrade: Updating grafana.smithi064 2024-02-16T23:05:59.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:05:59 smithi192 ceph-mon[82652]: Deploying daemon grafana.smithi064 on smithi064 2024-02-16T23:06:00.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:00 smithi064 ceph-mon[93757]: pgmap v246: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:00.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:00 smithi192 ceph-mon[82652]: pgmap v246: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:02.293 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:02 smithi064 ceph-mon[93757]: pgmap v247: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:02.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:02 smithi192 ceph-mon[82652]: pgmap v247: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:03.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:06:03.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:03.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:03.368 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:06:03.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:06:03.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:03.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:03.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:06:04.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:04 smithi064 ceph-mon[93757]: pgmap v248: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:04.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:04 smithi192 ceph-mon[82652]: pgmap v248: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:06.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:06 smithi064 ceph-mon[93757]: pgmap v249: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:06.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:06 smithi192 ceph-mon[82652]: pgmap v249: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:06.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 -- 172.21.15.64:0/1391995322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 msgr2=0x7f68a80fd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:06.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 --2- 172.21.15.64:0/1391995322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 0x7f68a80fd800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f689c004940 tx=0x7f689c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:06.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 -- 172.21.15.64:0/1391995322 shutdown_connections 2024-02-16T23:06:06.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 --2- 172.21.15.64:0/1391995322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68a80fc140 0x7f68a80fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:06.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 --2- 172.21.15.64:0/1391995322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 0x7f68a80fd800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:06.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 -- 172.21.15.64:0/1391995322 >> 172.21.15.64:0/1391995322 conn(0x7f68a80f6ca0 msgr2=0x7f68a80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:06.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 -- 172.21.15.64:0/1391995322 shutdown_connections 2024-02-16T23:06:06.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 -- 172.21.15.64:0/1391995322 wait complete. 2024-02-16T23:06:06.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 Processor -- start 2024-02-16T23:06:06.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.768+0000 7f68aef4a700 1 -- start start 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68aef4a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68a80fc140 0x7f68a810c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68aef4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 0x7f68a810ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68aef4a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a810d450 con 0x7f68a80fd3f0 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68aef4a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a810dcf0 con 0x7f68a80fc140 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 0x7f68a810ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 0x7f68a810ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54322/0 (socket says 172.21.15.64:54322) 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a7fff700 1 -- 172.21.15.64:0/992417658 learned_addr learned my addr 172.21.15.64:0/992417658 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68acce6700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68a80fc140 0x7f68a810c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:06.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a7fff700 1 -- 172.21.15.64:0/992417658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68a80fc140 msgr2=0x7f68a810c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a7fff700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68a80fc140 0x7f68a810c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a7fff700 1 -- 172.21.15.64:0/992417658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c005040 con 0x7f68a80fd3f0 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68acce6700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68a80fc140 0x7f68a810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a7fff700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 0x7f68a810ce50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f689400c8c0 tx=0x7f68940044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a57fa700 1 -- 172.21.15.64:0/992417658 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f689400a980 con 0x7f68a80fd3f0 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a57fa700 1 -- 172.21.15.64:0/992417658 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f689400ce60 con 0x7f68a80fd3f0 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a810dfd0 con 0x7f68a80fd3f0 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68a57fa700 1 -- 172.21.15.64:0/992417658 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68940115e0 con 0x7f68a80fd3f0 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.769+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a810e510 con 0x7f68a80fd3f0 2024-02-16T23:06:06.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.770+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a804bed0 con 0x7f68a80fd3f0 2024-02-16T23:06:06.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.771+0000 7f68a57fa700 1 -- 172.21.15.64:0/992417658 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6894012400 con 0x7f68a80fd3f0 2024-02-16T23:06:06.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.771+0000 7f68a57fa700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f689806f9f0 0x7f6898071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:06.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.771+0000 7f68a57fa700 1 -- 172.21.15.64:0/992417658 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f689408d0b0 con 0x7f68a80fd3f0 2024-02-16T23:06:06.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.771+0000 7f68acce6700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f689806f9f0 0x7f6898071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:06.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.772+0000 7f68acce6700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f689806f9f0 0x7f6898071ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f689c004f90 tx=0x7f689c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:06.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:06.776+0000 7f68a57fa700 1 -- 172.21.15.64:0/992417658 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f689405a3b0 con 0x7f68a80fd3f0 2024-02-16T23:06:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.001+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68a810e970 con 0x7f689806f9f0 2024-02-16T23:06:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.003+0000 7f68a57fa700 1 -- 172.21.15.64:0/992417658 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f68a810e970 con 0x7f689806f9f0 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f689806f9f0 msgr2=0x7f6898071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f689806f9f0 0x7f6898071ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f689c004f90 tx=0x7f689c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 msgr2=0x7f68a810ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 0x7f68a810ce50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f689400c8c0 tx=0x7f68940044c0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 shutdown_connections 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f689806f9f0 0x7f6898071ea0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68a80fc140 0x7f68a810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 --2- 172.21.15.64:0/992417658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68a80fd3f0 0x7f68a810ce50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 >> 172.21.15.64:0/992417658 conn(0x7f68a80f6ca0 msgr2=0x7f68a8100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 shutdown_connections 2024-02-16T23:06:07.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.005+0000 7f68aef4a700 1 -- 172.21.15.64:0/992417658 wait complete. 2024-02-16T23:06:07.016 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:06:07.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.125+0000 7efc0aae1700 1 -- 172.21.15.64:0/1438865127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 msgr2=0x7efc040fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.125+0000 7efc0aae1700 1 --2- 172.21.15.64:0/1438865127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 0x7efc040fcc40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efbf00049c0 tx=0x7efbf0009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:07.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.125+0000 7efc0aae1700 1 -- 172.21.15.64:0/1438865127 shutdown_connections 2024-02-16T23:06:07.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.125+0000 7efc0aae1700 1 --2- 172.21.15.64:0/1438865127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 0x7efc040fcc40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.125+0000 7efc0aae1700 1 --2- 172.21.15.64:0/1438865127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc040fb5e0 0x7efc040fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.125+0000 7efc0aae1700 1 -- 172.21.15.64:0/1438865127 >> 172.21.15.64:0/1438865127 conn(0x7efc040f6ca0 msgr2=0x7efc040f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:07.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.126+0000 7efc0aae1700 1 -- 172.21.15.64:0/1438865127 shutdown_connections 2024-02-16T23:06:07.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.126+0000 7efc0aae1700 1 -- 172.21.15.64:0/1438865127 wait complete. 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.126+0000 7efc0aae1700 1 Processor -- start 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.126+0000 7efc0aae1700 1 -- start start 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc0aae1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc040fb5e0 0x7efc0410b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc0aae1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 0x7efc0410bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc0aae1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0410c380 con 0x7efc040fc7f0 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc0aae1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0410cc20 con 0x7efc040fb5e0 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc03fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 0x7efc0410bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc03fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 0x7efc0410bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54348/0 (socket says 172.21.15.64:54348) 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc03fff700 1 -- 172.21.15.64:0/796333674 learned_addr learned my addr 172.21.15.64:0/796333674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc0887d700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc040fb5e0 0x7efc0410b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc03fff700 1 -- 172.21.15.64:0/796333674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc040fb5e0 msgr2=0x7efc0410b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc03fff700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc040fb5e0 0x7efc0410b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc03fff700 1 -- 172.21.15.64:0/796333674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf000d040 con 0x7efc040fc7f0 2024-02-16T23:06:07.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.127+0000 7efc03fff700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 0x7efc0410bd80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efbf00099f0 tx=0x7efbf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:07.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.128+0000 7efc017fa700 1 -- 172.21.15.64:0/796333674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbf000c8b0 con 0x7efc040fc7f0 2024-02-16T23:06:07.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.128+0000 7efc017fa700 1 -- 172.21.15.64:0/796333674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbf0004290 con 0x7efc040fc7f0 2024-02-16T23:06:07.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.128+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0410cea0 con 0x7efc040fc7f0 2024-02-16T23:06:07.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.128+0000 7efc017fa700 1 -- 172.21.15.64:0/796333674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbf001d860 con 0x7efc040fc7f0 2024-02-16T23:06:07.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.128+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0410d3e0 con 0x7efc040fc7f0 2024-02-16T23:06:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.129+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc04105880 con 0x7efc040fc7f0 2024-02-16T23:06:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.129+0000 7efc017fa700 1 -- 172.21.15.64:0/796333674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efbf0012be0 con 0x7efc040fc7f0 2024-02-16T23:06:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.130+0000 7efc017fa700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7efbf406d7f0 0x7efbf406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.130+0000 7efc017fa700 1 -- 172.21.15.64:0/796333674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7efbf0090d90 con 0x7efc040fc7f0 2024-02-16T23:06:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.131+0000 7efc0887d700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7efbf406d7f0 0x7efbf406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.131+0000 7efc0887d700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7efbf406d7f0 0x7efbf406fca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc040fc650 tx=0x7efbf800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:07.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.135+0000 7efc017fa700 1 -- 172.21.15.64:0/796333674 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efbf005e090 con 0x7efc040fc7f0 2024-02-16T23:06:07.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.372+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc0410d650 con 0x7efbf406d7f0 2024-02-16T23:06:07.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.373+0000 7efc017fa700 1 -- 172.21.15.64:0/796333674 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7efc0410d650 con 0x7efbf406d7f0 2024-02-16T23:06:07.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7efbf406d7f0 msgr2=0x7efbf406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7efbf406d7f0 0x7efbf406fca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc040fc650 tx=0x7efbf800b040 comp rx=0 tx=0).stop 2024-02-16T23:06:07.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 msgr2=0x7efc0410bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 0x7efc0410bd80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efbf00099f0 tx=0x7efbf0009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:07.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 shutdown_connections 2024-02-16T23:06:07.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7efbf406d7f0 0x7efbf406fca0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc040fb5e0 0x7efc0410b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 --2- 172.21.15.64:0/796333674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040fc7f0 0x7efc0410bd80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 >> 172.21.15.64:0/796333674 conn(0x7efc040f6ca0 msgr2=0x7efc040f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 shutdown_connections 2024-02-16T23:06:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.376+0000 7efc0aae1700 1 -- 172.21.15.64:0/796333674 wait complete. 2024-02-16T23:06:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 -- 172.21.15.64:0/2417910919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fd440 msgr2=0x7f65040fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 --2- 172.21.15.64:0/2417910919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fd440 0x7f65040fd850 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f64f8004940 tx=0x7f64f8009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 -- 172.21.15.64:0/2417910919 shutdown_connections 2024-02-16T23:06:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 --2- 172.21.15.64:0/2417910919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65040fdd90 0x7f65040fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 --2- 172.21.15.64:0/2417910919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fd440 0x7f65040fd850 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 -- 172.21.15.64:0/2417910919 >> 172.21.15.64:0/2417910919 conn(0x7f65040f6d20 msgr2=0x7f65040f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 -- 172.21.15.64:0/2417910919 shutdown_connections 2024-02-16T23:06:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 -- 172.21.15.64:0/2417910919 wait complete. 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.482+0000 7f6509405700 1 Processor -- start 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f6509405700 1 -- start start 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f6509405700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65040fd440 0x7f65041109e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f6509405700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fdd90 0x7f6504112f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f6509405700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040fbe30 con 0x7f65040fdd90 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f6509405700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040fbf70 con 0x7f65040fd440 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f65027fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fdd90 0x7f6504112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f65027fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fdd90 0x7f6504112f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54372/0 (socket says 172.21.15.64:54372) 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f65027fc700 1 -- 172.21.15.64:0/1456477845 learned_addr learned my addr 172.21.15.64:0/1456477845 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f6502ffd700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65040fd440 0x7f65041109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f65027fc700 1 -- 172.21.15.64:0/1456477845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65040fd440 msgr2=0x7f65041109e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f65027fc700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65040fd440 0x7f65041109e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f65027fc700 1 -- 172.21.15.64:0/1456477845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f8005040 con 0x7f65040fdd90 2024-02-16T23:06:07.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.483+0000 7f65027fc700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fdd90 0x7f6504112f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64f4002a50 tx=0x7f64f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:07.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.484+0000 7f64eb7fe700 1 -- 172.21.15.64:0/1456477845 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64f4007cc0 con 0x7f65040fdd90 2024-02-16T23:06:07.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.484+0000 7f64eb7fe700 1 -- 172.21.15.64:0/1456477845 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64f400f040 con 0x7f65040fdd90 2024-02-16T23:06:07.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.484+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6504113530 con 0x7f65040fdd90 2024-02-16T23:06:07.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.484+0000 7f64eb7fe700 1 -- 172.21.15.64:0/1456477845 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64f400daf0 con 0x7f65040fdd90 2024-02-16T23:06:07.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.484+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6504113a70 con 0x7f65040fdd90 2024-02-16T23:06:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.485+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65041069a0 con 0x7f65040fdd90 2024-02-16T23:06:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.485+0000 7f64eb7fe700 1 -- 172.21.15.64:0/1456477845 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f64f4009810 con 0x7f65040fdd90 2024-02-16T23:06:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.486+0000 7f64eb7fe700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f64ec06d7f0 0x7f64ec06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.486+0000 7f64eb7fe700 1 -- 172.21.15.64:0/1456477845 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f64f408cca0 con 0x7f65040fdd90 2024-02-16T23:06:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.486+0000 7f6502ffd700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f64ec06d7f0 0x7f64ec06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.486+0000 7f6502ffd700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f64ec06d7f0 0x7f64ec06fca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f64f800b3b0 tx=0x7f64f8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.495+0000 7f64eb7fe700 1 -- 172.21.15.64:0/1456477845 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64f4056470 con 0x7f65040fdd90 2024-02-16T23:06:07.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.706+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6504104a60 con 0x7f64ec06d7f0 2024-02-16T23:06:07.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.718+0000 7f64eb7fe700 1 -- 172.21.15.64:0/1456477845 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6504104a60 con 0x7f64ec06d7f0 2024-02-16T23:06:07.718 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:06:07.718 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (22s) 0s ago 17m 16.3M - 0.25.0 c8568f914cd2 30b410320c0e 2024-02-16T23:06:07.718 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (6m) 0s ago 17m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:06:07.718 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (6m) 52s ago 16m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:06:07.718 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (5s) 0s ago 17m 55.2M - 9.4.12 a0ea5e3c46d9 302d36ce61ee 2024-02-16T23:06:07.718 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (117s) 0s ago 12m 12.2M - 19.0.0-1450-gf78a58c0 f081f847070a d4dd49dc2ce8 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (2m) 0s ago 12m 87.9M - 19.0.0-1450-gf78a58c0 f081f847070a bd35cb99fa04 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (70s) 52s ago 12m 14.0M - 19.0.0-1450-gf78a58c0 f081f847070a 8d8b30783f08 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (92s) 52s ago 12m 12.4M - 19.0.0-1450-gf78a58c0 f081f847070a ca63fdf53f50 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (8m) 0s ago 19m 511M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (7m) 52s ago 16m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (7m) 0s ago 19m 54.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (6m) 52s ago 16m 43.2M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (59s) 0s ago 17m 8220k - 1.5.0 0da6a335fe13 cbac3ca3c432 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (55s) 52s ago 16m 7906k - 1.5.0 0da6a335fe13 dd661245993a 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (5m) 0s ago 15m 107M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (5m) 0s ago 15m 100M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (4m) 0s ago 15m 86.6M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (3m) 52s ago 14m 121M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (3m) 52s ago 14m 108M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (2m) 52s ago 13m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 7bb5b265fc66 2024-02-16T23:06:07.719 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (38s) 0s ago 17m 44.0M - 2.43.0 a07b618ecd1d 51136d8386ab 2024-02-16T23:06:07.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f64ec06d7f0 msgr2=0x7f64ec06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f64ec06d7f0 0x7f64ec06fca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f64f800b3b0 tx=0x7f64f8009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:07.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fdd90 msgr2=0x7f6504112f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fdd90 0x7f6504112f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64f4002a50 tx=0x7f64f4004600 comp rx=0 tx=0).stop 2024-02-16T23:06:07.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 shutdown_connections 2024-02-16T23:06:07.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f64ec06d7f0 0x7f64ec06fca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65040fd440 0x7f65041109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 --2- 172.21.15.64:0/1456477845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040fdd90 0x7f6504112f30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 >> 172.21.15.64:0/1456477845 conn(0x7f65040f6d20 msgr2=0x7f65041005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:07.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 shutdown_connections 2024-02-16T23:06:07.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.720+0000 7f6509405700 1 -- 172.21.15.64:0/1456477845 wait complete. 2024-02-16T23:06:07.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.837+0000 7f1f61917700 1 -- 172.21.15.64:0/97554507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 msgr2=0x7f1f5c0fb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.837+0000 7f1f61917700 1 --2- 172.21.15.64:0/97554507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 0x7f1f5c0fb9d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c004940 tx=0x7f1f4c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:07.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.837+0000 7f1f61917700 1 -- 172.21.15.64:0/97554507 shutdown_connections 2024-02-16T23:06:07.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.837+0000 7f1f61917700 1 --2- 172.21.15.64:0/97554507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f5c0fc870 0x7f1f5c0fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.837+0000 7f1f61917700 1 --2- 172.21.15.64:0/97554507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 0x7f1f5c0fb9d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.837+0000 7f1f61917700 1 -- 172.21.15.64:0/97554507 >> 172.21.15.64:0/97554507 conn(0x7f1f5c0f6d20 msgr2=0x7f1f5c0f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.837+0000 7f1f61917700 1 -- 172.21.15.64:0/97554507 shutdown_connections 2024-02-16T23:06:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.837+0000 7f1f61917700 1 -- 172.21.15.64:0/97554507 wait complete. 2024-02-16T23:06:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f61917700 1 Processor -- start 2024-02-16T23:06:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f61917700 1 -- start start 2024-02-16T23:06:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f61917700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 0x7f1f5c10dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f61917700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f5c0fc870 0x7f1f5c10e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f61917700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c10efd0 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f61917700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c10f140 con 0x7f1f5c0fc870 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f5affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 0x7f1f5c10dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f5affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 0x7f1f5c10dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54378/0 (socket says 172.21.15.64:54378) 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f5affd700 1 -- 172.21.15.64:0/1917412409 learned_addr learned my addr 172.21.15.64:0/1917412409 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.838+0000 7f1f5a7fc700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f5c0fc870 0x7f1f5c10e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f5affd700 1 -- 172.21.15.64:0/1917412409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f5c0fc870 msgr2=0x7f1f5c10e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f5affd700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f5c0fc870 0x7f1f5c10e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f5affd700 1 -- 172.21.15.64:0/1917412409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c005040 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f5a7fc700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f5c0fc870 0x7f1f5c10e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f5affd700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 0x7f1f5c10dd30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0093b0 tx=0x7f1f4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f437fe700 1 -- 172.21.15.64:0/1917412409 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f4c00e040 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f437fe700 1 -- 172.21.15.64:0/1917412409 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f4c00bcc0 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f437fe700 1 -- 172.21.15.64:0/1917412409 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f4c01c630 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c10f3c0 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.839+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c10f900 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.840+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c04bed0 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.841+0000 7f1f437fe700 1 -- 172.21.15.64:0/1917412409 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1f4c0139f0 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.841+0000 7f1f437fe700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1f4406d7e0 0x7f1f4406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.841+0000 7f1f437fe700 1 -- 172.21.15.64:0/1917412409 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1f4c08edd0 con 0x7f1f5c0fb5c0 2024-02-16T23:06:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.841+0000 7f1f5a7fc700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1f4406d7e0 0x7f1f4406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:07.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.842+0000 7f1f5a7fc700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1f4406d7e0 0x7f1f4406fc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009b10 tx=0x7f1f50008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:07.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:07.845+0000 7f1f437fe700 1 -- 172.21.15.64:0/1917412409 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f4c05c190 con 0x7f1f5c0fb5c0 2024-02-16T23:06:08.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.136+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1f5c104130 con 0x7f1f5c0fb5c0 2024-02-16T23:06:08.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.136+0000 7f1f437fe700 1 -- 172.21.15.64:0/1917412409 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1f4c05bd20 con 0x7f1f5c0fb5c0 2024-02-16T23:06:08.137 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T23:06:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:06:08.139 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:06:08.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1f4406d7e0 msgr2=0x7f1f4406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1f4406d7e0 0x7f1f4406fc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009b10 tx=0x7f1f50008040 comp rx=0 tx=0).stop 2024-02-16T23:06:08.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 msgr2=0x7f1f5c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 0x7f1f5c10dd30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0093b0 tx=0x7f1f4c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:08.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 shutdown_connections 2024-02-16T23:06:08.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f1f4406d7e0 0x7f1f4406fc90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f5c0fb5c0 0x7f1f5c10dd30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 --2- 172.21.15.64:0/1917412409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f5c0fc870 0x7f1f5c10e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 >> 172.21.15.64:0/1917412409 conn(0x7f1f5c0f6d20 msgr2=0x7f1f5c0f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:08.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 shutdown_connections 2024-02-16T23:06:08.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.139+0000 7f1f61917700 1 -- 172.21.15.64:0/1917412409 wait complete. 2024-02-16T23:06:08.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 -- 172.21.15.64:0/4285150071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c1000c0 msgr2=0x7fc81c102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 --2- 172.21.15.64:0/4285150071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c1000c0 0x7fc81c102520 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc80c004940 tx=0x7fc80c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:08.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 -- 172.21.15.64:0/4285150071 shutdown_connections 2024-02-16T23:06:08.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 --2- 172.21.15.64:0/4285150071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c1000c0 0x7fc81c102520 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 --2- 172.21.15.64:0/4285150071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc81c0663e0 0x7fc81c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 -- 172.21.15.64:0/4285150071 >> 172.21.15.64:0/4285150071 conn(0x7fc81c0f74e0 msgr2=0x7fc81c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:08.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 -- 172.21.15.64:0/4285150071 shutdown_connections 2024-02-16T23:06:08.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 -- 172.21.15.64:0/4285150071 wait complete. 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 Processor -- start 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.245+0000 7fc820cc1700 1 -- start start 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc820cc1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0663e0 0x7fc81c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc820cc1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc81c1000c0 0x7fc81c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc820cc1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c10e4b0 con 0x7fc81c0663e0 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc820cc1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c10ed50 con 0x7fc81c1000c0 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc81a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0663e0 0x7fc81c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc81a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0663e0 0x7fc81c10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54410/0 (socket says 172.21.15.64:54410) 2024-02-16T23:06:08.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc81a59c700 1 -- 172.21.15.64:0/4262241036 learned_addr learned my addr 172.21.15.64:0/4262241036 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc819d9b700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc81c1000c0 0x7fc81c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc81a59c700 1 -- 172.21.15.64:0/4262241036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc81c1000c0 msgr2=0x7fc81c10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc81a59c700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc81c1000c0 0x7fc81c10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc81a59c700 1 -- 172.21.15.64:0/4262241036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c005040 con 0x7fc81c0663e0 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc81a59c700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0663e0 0x7fc81c10d970 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc81000c8c0 tx=0x7fc8100044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc80affd700 1 -- 172.21.15.64:0/4262241036 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc81000a760 con 0x7fc81c0663e0 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc80affd700 1 -- 172.21.15.64:0/4262241036 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc81000ad90 con 0x7fc81c0663e0 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc80affd700 1 -- 172.21.15.64:0/4262241036 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8100113a0 con 0x7fc81c0663e0 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c10f030 con 0x7fc81c0663e0 2024-02-16T23:06:08.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.246+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c10f520 con 0x7fc81c0663e0 2024-02-16T23:06:08.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.247+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c04bed0 con 0x7fc81c0663e0 2024-02-16T23:06:08.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.248+0000 7fc80affd700 1 -- 172.21.15.64:0/4262241036 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc810012400 con 0x7fc81c0663e0 2024-02-16T23:06:08.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.248+0000 7fc80affd700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc80406d7e0 0x7fc80406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.248+0000 7fc80affd700 1 -- 172.21.15.64:0/4262241036 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc81008cf90 con 0x7fc81c0663e0 2024-02-16T23:06:08.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.248+0000 7fc819d9b700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc80406d7e0 0x7fc80406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.249+0000 7fc819d9b700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc80406d7e0 0x7fc80406fc90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc80c004890 tx=0x7fc80c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:08.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.253+0000 7fc80affd700 1 -- 172.21.15.64:0/4262241036 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc8100c0d20 con 0x7fc81c0663e0 2024-02-16T23:06:08.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.505+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc81c105a10 con 0x7fc81c0663e0 2024-02-16T23:06:08.506 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:08 smithi064 ceph-mon[93757]: pgmap v250: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:08.506 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:08 smithi064 ceph-mon[93757]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:08.507 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:08.507 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:08 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:08.507 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:08 smithi064 ceph-mon[93757]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:08.507 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:08 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1917412409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:08.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.505+0000 7fc80affd700 1 -- 172.21.15.64:0/4262241036 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fc81005a290 con 0x7fc81c0663e0 2024-02-16T23:06:08.507 INFO:teuthology.orchestra.run.smithi064.stdout:e35 2024-02-16T23:06:08.507 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:06:08.507 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 35 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T23:04:58.106175+0000 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:06:08.508 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 85 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=34404} 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:06:08.509 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:06:08.510 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:34404} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:06:08.510 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:06:08.510 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:06:08.510 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:06:08.510 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:06:08.510 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.64:6828/2551190588,v1:172.21.15.64:6829/2551190588] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:06:08.510 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/2028541376,v1:172.21.15.192:6825/2028541376] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:06:08.510 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/1878053024,v1:172.21.15.192:6827/1878053024] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc80406d7e0 msgr2=0x7fc80406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc80406d7e0 0x7fc80406fc90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc80c004890 tx=0x7fc80c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0663e0 msgr2=0x7fc81c10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0663e0 0x7fc81c10d970 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc81000c8c0 tx=0x7fc8100044c0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 shutdown_connections 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc80406d7e0 0x7fc80406fc90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0663e0 0x7fc81c10d970 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 --2- 172.21.15.64:0/4262241036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc81c1000c0 0x7fc81c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 >> 172.21.15.64:0/4262241036 conn(0x7fc81c0f74e0 msgr2=0x7fc81c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 shutdown_connections 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.508+0000 7fc820cc1700 1 -- 172.21.15.64:0/4262241036 wait complete. 2024-02-16T23:06:08.511 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2024-02-16T23:06:08.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:08 smithi192 ceph-mon[82652]: pgmap v250: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:08 smithi192 ceph-mon[82652]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:08 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:08 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:08 smithi192 ceph-mon[82652]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:08.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:08 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1917412409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 -- 172.21.15.64:0/1372773642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 msgr2=0x7fa0fc102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 --2- 172.21.15.64:0/1372773642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 0x7fa0fc102520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec004940 tx=0x7fa0ec009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 -- 172.21.15.64:0/1372773642 shutdown_connections 2024-02-16T23:06:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 --2- 172.21.15.64:0/1372773642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 0x7fa0fc102520 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 --2- 172.21.15.64:0/1372773642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0fc0663e0 0x7fa0fc0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 -- 172.21.15.64:0/1372773642 >> 172.21.15.64:0/1372773642 conn(0x7fa0fc0f74e0 msgr2=0x7fa0fc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 -- 172.21.15.64:0/1372773642 shutdown_connections 2024-02-16T23:06:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 -- 172.21.15.64:0/1372773642 wait complete. 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 Processor -- start 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.615+0000 7fa1017d5700 1 -- start start 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa1017d5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0fc0663e0 0x7fa0fc10fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa1017d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 0x7fa0fc110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa1017d5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc110690 con 0x7fa0fc1000c0 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa1017d5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc110f30 con 0x7fa0fc0663e0 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa0fa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 0x7fa0fc110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa0fa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 0x7fa0fc110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54436/0 (socket says 172.21.15.64:54436) 2024-02-16T23:06:08.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa0fa7fc700 1 -- 172.21.15.64:0/3859979948 learned_addr learned my addr 172.21.15.64:0/3859979948 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa0faffd700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0fc0663e0 0x7fa0fc10fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa0fa7fc700 1 -- 172.21.15.64:0/3859979948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0fc0663e0 msgr2=0x7fa0fc10fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa0fa7fc700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0fc0663e0 0x7fa0fc10fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa0fa7fc700 1 -- 172.21.15.64:0/3859979948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec005040 con 0x7fa0fc1000c0 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.616+0000 7fa0fa7fc700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 0x7fa0fc110090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec000c00 tx=0x7fa0ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.617+0000 7fa0e37fe700 1 -- 172.21.15.64:0/3859979948 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0ec00e040 con 0x7fa0fc1000c0 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.617+0000 7fa0e37fe700 1 -- 172.21.15.64:0/3859979948 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0ec00bbf0 con 0x7fa0fc1000c0 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.617+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc1111b0 con 0x7fa0fc1000c0 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.617+0000 7fa0e37fe700 1 -- 172.21.15.64:0/3859979948 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0ec01c630 con 0x7fa0fc1000c0 2024-02-16T23:06:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.617+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc1116f0 con 0x7fa0fc1000c0 2024-02-16T23:06:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.618+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc04bed0 con 0x7fa0fc1000c0 2024-02-16T23:06:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.618+0000 7fa0e37fe700 1 -- 172.21.15.64:0/3859979948 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa0ec00bd60 con 0x7fa0fc1000c0 2024-02-16T23:06:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.619+0000 7fa0e37fe700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa0e406d7e0 0x7fa0e406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.619+0000 7fa0e37fe700 1 -- 172.21.15.64:0/3859979948 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa0ec08e350 con 0x7fa0fc1000c0 2024-02-16T23:06:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.619+0000 7fa0faffd700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa0e406d7e0 0x7fa0e406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.619+0000 7fa0faffd700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa0e406d7e0 0x7fa0e406fc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0007fd0 tx=0x7fa0f0009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:08.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.628+0000 7fa0e37fe700 1 -- 172.21.15.64:0/3859979948 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa0ec05b650 con 0x7fa0fc1000c0 2024-02-16T23:06:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.853+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0fc111b40 con 0x7fa0e406d7e0 2024-02-16T23:06:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.854+0000 7fa0e37fe700 1 -- 172.21.15.64:0/3859979948 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fa0fc111b40 con 0x7fa0e406d7e0 2024-02-16T23:06:08.855 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:06:08.855 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T23:06:08.855 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": true, 2024-02-16T23:06:08.855 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T23:06:08.855 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [ 2024-02-16T23:06:08.855 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-02-16T23:06:08.855 INFO:teuthology.orchestra.run.smithi064.stdout: "mon", 2024-02-16T23:06:08.856 INFO:teuthology.orchestra.run.smithi064.stdout: "mds", 2024-02-16T23:06:08.856 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr", 2024-02-16T23:06:08.856 INFO:teuthology.orchestra.run.smithi064.stdout: "osd" 2024-02-16T23:06:08.856 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-02-16T23:06:08.856 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": "16/21 daemons upgraded", 2024-02-16T23:06:08.856 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "Currently upgrading grafana daemons", 2024-02-16T23:06:08.856 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:06:08.856 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:06:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa0e406d7e0 msgr2=0x7fa0e406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa0e406d7e0 0x7fa0e406fc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0007fd0 tx=0x7fa0f0009040 comp rx=0 tx=0).stop 2024-02-16T23:06:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 msgr2=0x7fa0fc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 0x7fa0fc110090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec000c00 tx=0x7fa0ec009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 shutdown_connections 2024-02-16T23:06:08.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa0e406d7e0 0x7fa0e406fc90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0fc0663e0 0x7fa0fc10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 --2- 172.21.15.64:0/3859979948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc1000c0 0x7fa0fc110090 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 >> 172.21.15.64:0/3859979948 conn(0x7fa0fc0f74e0 msgr2=0x7fa0fc0fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:08.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 shutdown_connections 2024-02-16T23:06:08.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.857+0000 7fa1017d5700 1 -- 172.21.15.64:0/3859979948 wait complete. 2024-02-16T23:06:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.973+0000 7f528fd7d700 1 -- 172.21.15.64:0/2346888671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fbe40 msgr2=0x7f52880fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.973+0000 7f528fd7d700 1 --2- 172.21.15.64:0/2346888671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fbe40 0x7f52880fc250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f527c004940 tx=0x7f527c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.973+0000 7f528fd7d700 1 -- 172.21.15.64:0/2346888671 shutdown_connections 2024-02-16T23:06:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.973+0000 7f528fd7d700 1 --2- 172.21.15.64:0/2346888671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52880fcfb0 0x7f52880fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.973+0000 7f528fd7d700 1 --2- 172.21.15.64:0/2346888671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fbe40 0x7f52880fc250 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.973+0000 7f528fd7d700 1 -- 172.21.15.64:0/2346888671 >> 172.21.15.64:0/2346888671 conn(0x7f52880f74e0 msgr2=0x7f52880f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.973+0000 7f528fd7d700 1 -- 172.21.15.64:0/2346888671 shutdown_connections 2024-02-16T23:06:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.973+0000 7f528fd7d700 1 -- 172.21.15.64:0/2346888671 wait complete. 2024-02-16T23:06:08.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528fd7d700 1 Processor -- start 2024-02-16T23:06:08.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528fd7d700 1 -- start start 2024-02-16T23:06:08.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528fd7d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52880fbe40 0x7f52880fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528fd7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fcfb0 0x7f52880fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528fd7d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52880fe980 con 0x7f52880fcfb0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528fd7d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52880feaf0 con 0x7f52880fbe40 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528d318700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fcfb0 0x7f52880fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528d318700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fcfb0 0x7f52880fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54454/0 (socket says 172.21.15.64:54454) 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528d318700 1 -- 172.21.15.64:0/3362924779 learned_addr learned my addr 172.21.15.64:0/3362924779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528db19700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52880fbe40 0x7f52880fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528d318700 1 -- 172.21.15.64:0/3362924779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52880fbe40 msgr2=0x7f52880fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528d318700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52880fbe40 0x7f52880fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528d318700 1 -- 172.21.15.64:0/3362924779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c005040 con 0x7f52880fcfb0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.974+0000 7f528db19700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52880fbe40 0x7f52880fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.975+0000 7f528d318700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fcfb0 0x7f52880fe380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f528400c8c0 tx=0x7f52840044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.975+0000 7f527a7fc700 1 -- 172.21.15.64:0/3362924779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528400a980 con 0x7f52880fcfb0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.975+0000 7f527a7fc700 1 -- 172.21.15.64:0/3362924779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f528400ce60 con 0x7f52880fcfb0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.975+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52880fedd0 con 0x7f52880fcfb0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.975+0000 7f527a7fc700 1 -- 172.21.15.64:0/3362924779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52840115e0 con 0x7f52880fcfb0 2024-02-16T23:06:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.975+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5288115970 con 0x7f52880fcfb0 2024-02-16T23:06:08.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.976+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528804bed0 con 0x7f52880fcfb0 2024-02-16T23:06:08.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.976+0000 7f527a7fc700 1 -- 172.21.15.64:0/3362924779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5284012400 con 0x7f52880fcfb0 2024-02-16T23:06:08.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.977+0000 7f527a7fc700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f527406d7e0 0x7f527406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:08.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.977+0000 7f527a7fc700 1 -- 172.21.15.64:0/3362924779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f528408d050 con 0x7f52880fcfb0 2024-02-16T23:06:08.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.977+0000 7f528db19700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f527406d7e0 0x7f527406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:08.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.977+0000 7f528db19700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f527406d7e0 0x7f527406fc90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f527c009fd0 tx=0x7f527c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:08.982+0000 7f527a7fc700 1 -- 172.21.15.64:0/3362924779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f528405a350 con 0x7f52880fcfb0 2024-02-16T23:06:09.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.277+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5288104930 con 0x7f52880fcfb0 2024-02-16T23:06:09.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.278+0000 7f527a7fc700 1 -- 172.21.15.64:0/3362924779 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f52840126b0 con 0x7f52880fcfb0 2024-02-16T23:06:09.279 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:06:09.279 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:06:09.279 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f527406d7e0 msgr2=0x7f527406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f527406d7e0 0x7f527406fc90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f527c009fd0 tx=0x7f527c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fcfb0 msgr2=0x7f52880fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fcfb0 0x7f52880fe380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f528400c8c0 tx=0x7f52840044c0 comp rx=0 tx=0).stop 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 shutdown_connections 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f527406d7e0 0x7f527406fc90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52880fbe40 0x7f52880fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 --2- 172.21.15.64:0/3362924779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880fcfb0 0x7f52880fe380 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 >> 172.21.15.64:0/3362924779 conn(0x7f52880f74e0 msgr2=0x7f52880656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:09.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 shutdown_connections 2024-02-16T23:06:09.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:09.280+0000 7f528fd7d700 1 -- 172.21.15.64:0/3362924779 wait complete. 2024-02-16T23:06:09.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:09 smithi064 ceph-mon[93757]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:09.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:09.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:09 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:09.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:09 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/4262241036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:06:09.627 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:09 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3362924779' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:06:09.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:09 smithi192 ceph-mon[82652]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:09.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:09.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:09 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:09.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:09 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/4262241036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:06:09.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:09 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3362924779' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:06:10.686 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:10 smithi192 ceph-mon[82652]: pgmap v251: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:10.686 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:10 smithi192 ceph-mon[82652]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:10.711 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:10 smithi064 ceph-mon[93757]: pgmap v251: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:10.711 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:10 smithi064 ceph-mon[93757]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: pgmap v252: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:06:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:06:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T23:06:12.618 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T23:06:12.620 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.621 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:06:12.622 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:06:12.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: pgmap v252: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:12.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.785 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T23:06:12.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T23:06:12.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:06:12.788 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:12.789 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:06:12.789 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:06:12.789 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:06:13.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:13 smithi192 ceph-mon[82652]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T23:06:13.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:13 smithi192 ceph-mon[82652]: Upgrade: Finalizing container_image settings 2024-02-16T23:06:13.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:13 smithi192 ceph-mon[82652]: Upgrade: Complete! 2024-02-16T23:06:13.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:13.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:13.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:13 smithi064 ceph-mon[93757]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T23:06:13.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:13 smithi064 ceph-mon[93757]: Upgrade: Finalizing container_image settings 2024-02-16T23:06:13.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:13 smithi064 ceph-mon[93757]: Upgrade: Complete! 2024-02-16T23:06:13.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:13.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:06:14.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:14 smithi192 ceph-mon[82652]: pgmap v253: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:14 smithi064 ceph-mon[93757]: pgmap v253: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:16.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:16 smithi192 ceph-mon[82652]: pgmap v254: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:16.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:16 smithi064 ceph-mon[93757]: pgmap v254: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:18 smithi192 ceph-mon[82652]: pgmap v255: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:06:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:18 smithi064 ceph-mon[93757]: pgmap v255: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:06:20.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:20 smithi192 ceph-mon[82652]: pgmap v256: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:20.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:20 smithi064 ceph-mon[93757]: pgmap v256: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:22.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:22 smithi064 ceph-mon[93757]: pgmap v257: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:22.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:22 smithi192 ceph-mon[82652]: pgmap v257: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:24.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:24 smithi192 ceph-mon[82652]: pgmap v258: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:24.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:24 smithi064 ceph-mon[93757]: pgmap v258: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:26.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:26 smithi192 ceph-mon[82652]: pgmap v259: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:26.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:26 smithi064 ceph-mon[93757]: pgmap v259: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:28.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:28 smithi192 ceph-mon[82652]: pgmap v260: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:28.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:28 smithi064 ceph-mon[93757]: pgmap v260: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:30.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:30 smithi192 ceph-mon[82652]: pgmap v261: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:30.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:30 smithi064 ceph-mon[93757]: pgmap v261: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:32.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:32 smithi064 ceph-mon[93757]: pgmap v262: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:32.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:32 smithi192 ceph-mon[82652]: pgmap v262: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:33.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:06:33.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:06:34.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:34 smithi192 ceph-mon[82652]: pgmap v263: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:34.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:34 smithi064 ceph-mon[93757]: pgmap v263: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:36.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:36 smithi192 ceph-mon[82652]: pgmap v264: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:36.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:36 smithi064 ceph-mon[93757]: pgmap v264: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:38.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:38 smithi192 ceph-mon[82652]: pgmap v265: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:38.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:38 smithi064 ceph-mon[93757]: pgmap v265: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:39.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.402+0000 7fa293b11700 1 -- 172.21.15.64:0/2232553501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 msgr2=0x7fa28c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.402+0000 7fa293b11700 1 --2- 172.21.15.64:0/2232553501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 0x7fa28c0fe550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004940 tx=0x7fa27c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.402+0000 7fa293b11700 1 -- 172.21.15.64:0/2232553501 shutdown_connections 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.402+0000 7fa293b11700 1 --2- 172.21.15.64:0/2232553501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 0x7fa28c0fe550 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.402+0000 7fa293b11700 1 --2- 172.21.15.64:0/2232553501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcef0 0x7fa28c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.402+0000 7fa293b11700 1 -- 172.21.15.64:0/2232553501 >> 172.21.15.64:0/2232553501 conn(0x7fa28c0f8620 msgr2=0x7fa28c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.402+0000 7fa293b11700 1 -- 172.21.15.64:0/2232553501 shutdown_connections 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.402+0000 7fa293b11700 1 -- 172.21.15.64:0/2232553501 wait complete. 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa293b11700 1 Processor -- start 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa293b11700 1 -- start start 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa293b11700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcef0 0x7fa28c10d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa293b11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 0x7fa28c10de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa293b11700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c10e430 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa293b11700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c10ecd0 con 0x7fa28c0fcef0 2024-02-16T23:06:39.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa2910ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 0x7fa28c10de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa2910ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 0x7fa28c10de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58944/0 (socket says 172.21.15.64:58944) 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa2910ac700 1 -- 172.21.15.64:0/4143663747 learned_addr learned my addr 172.21.15.64:0/4143663747 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.403+0000 7fa2918ad700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcef0 0x7fa28c10d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa2910ac700 1 -- 172.21.15.64:0/4143663747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcef0 msgr2=0x7fa28c10d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa2910ac700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcef0 0x7fa28c10d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa2910ac700 1 -- 172.21.15.64:0/4143663747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c005040 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa2918ad700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcef0 0x7fa28c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa2910ac700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 0x7fa28c10de30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000c00 tx=0x7fa27c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa2827fc700 1 -- 172.21.15.64:0/4143663747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa27c00e070 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa2827fc700 1 -- 172.21.15.64:0/4143663747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa27c01c400 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c10ef50 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa2827fc700 1 -- 172.21.15.64:0/4143663747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa27c00b650 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.404+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c10f430 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.405+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c04bed0 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.406+0000 7fa2827fc700 1 -- 172.21.15.64:0/4143663747 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa27c01ca50 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.406+0000 7fa2827fc700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa27806d7e0 0x7fa27806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:39.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.406+0000 7fa2827fc700 1 -- 172.21.15.64:0/4143663747 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa27c08e620 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.406+0000 7fa2918ad700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa27806d7e0 0x7fa27806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:39.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.407+0000 7fa2918ad700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa27806d7e0 0x7fa27806fc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa288009b80 tx=0x7fa28800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:39.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.411+0000 7fa2827fc700 1 -- 172.21.15.64:0/4143663747 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa27c05b920 con 0x7fa28c0fe0e0 2024-02-16T23:06:39.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.634+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa28c10f880 con 0x7fa27806d7e0 2024-02-16T23:06:39.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.635+0000 7fa2827fc700 1 -- 172.21.15.64:0/4143663747 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fa28c10f880 con 0x7fa27806d7e0 2024-02-16T23:06:39.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa27806d7e0 msgr2=0x7fa27806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:39.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa27806d7e0 0x7fa27806fc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa288009b80 tx=0x7fa28800b040 comp rx=0 tx=0).stop 2024-02-16T23:06:39.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 msgr2=0x7fa28c10de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 0x7fa28c10de30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000c00 tx=0x7fa27c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 shutdown_connections 2024-02-16T23:06:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa27806d7e0 0x7fa27806fc90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa28c0fcef0 0x7fa28c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 --2- 172.21.15.64:0/4143663747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28c0fe0e0 0x7fa28c10de30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 >> 172.21.15.64:0/4143663747 conn(0x7fa28c0f8620 msgr2=0x7fa28c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 shutdown_connections 2024-02-16T23:06:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:39.637+0000 7fa293b11700 1 -- 172.21.15.64:0/4143663747 wait complete. 2024-02-16T23:06:40.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T23:06:40.616 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:06:40.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:40 smithi064 ceph-mon[93757]: pgmap v266: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:40.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:40 smithi192 ceph-mon[82652]: pgmap v266: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:41.687 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:41 smithi064 ceph-mon[93757]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:41.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:41 smithi192 ceph-mon[82652]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 -- 172.21.15.64:0/224461792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 msgr2=0x7f895c103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 --2- 172.21.15.64:0/224461792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 0x7f895c103510 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f894c004940 tx=0x7f894c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 -- 172.21.15.64:0/224461792 shutdown_connections 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 --2- 172.21.15.64:0/224461792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895c0fb570 0x7f895c0fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 --2- 172.21.15.64:0/224461792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 0x7f895c103510 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 -- 172.21.15.64:0/224461792 >> 172.21.15.64:0/224461792 conn(0x7f895c071560 msgr2=0x7f895c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 -- 172.21.15.64:0/224461792 shutdown_connections 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 -- 172.21.15.64:0/224461792 wait complete. 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.207+0000 7f89621c3700 1 Processor -- start 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f89621c3700 1 -- start start 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f89621c3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895c0fb570 0x7f895c0fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:42.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f89621c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 0x7f895c0fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f89621c3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0fd290 con 0x7f895c103140 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f89621c3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0fd400 con 0x7f895c0fb570 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f895affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 0x7f895c0fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f895affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 0x7f895c0fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56494/0 (socket says 172.21.15.64:56494) 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f895affd700 1 -- 172.21.15.64:0/1855826670 learned_addr learned my addr 172.21.15.64:0/1855826670 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f895b7fe700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895c0fb570 0x7f895c0fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f895affd700 1 -- 172.21.15.64:0/1855826670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895c0fb570 msgr2=0x7f895c0fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f895affd700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895c0fb570 0x7f895c0fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f895affd700 1 -- 172.21.15.64:0/1855826670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c005040 con 0x7f895c103140 2024-02-16T23:06:42.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.208+0000 7f895affd700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 0x7f895c0fcc90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f895000c830 tx=0x7f8950004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.209+0000 7f8943fff700 1 -- 172.21.15.64:0/1855826670 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f895000a760 con 0x7f895c103140 2024-02-16T23:06:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.209+0000 7f8943fff700 1 -- 172.21.15.64:0/1855826670 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f895000ad90 con 0x7f895c103140 2024-02-16T23:06:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.209+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c119da0 con 0x7f895c103140 2024-02-16T23:06:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.209+0000 7f8943fff700 1 -- 172.21.15.64:0/1855826670 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89500113a0 con 0x7f895c103140 2024-02-16T23:06:42.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.209+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c11a280 con 0x7f895c103140 2024-02-16T23:06:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.210+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c100310 con 0x7f895c103140 2024-02-16T23:06:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.210+0000 7f8943fff700 1 -- 172.21.15.64:0/1855826670 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f895000a8c0 con 0x7f895c103140 2024-02-16T23:06:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.211+0000 7f8943fff700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f894406d7e0 0x7f894406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.211+0000 7f8943fff700 1 -- 172.21.15.64:0/1855826670 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f895008cff0 con 0x7f895c103140 2024-02-16T23:06:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.211+0000 7f895b7fe700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f894406d7e0 0x7f894406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:42.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.211+0000 7f895b7fe700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f894406d7e0 0x7f894406fc90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f894c004f90 tx=0x7f894c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:42.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.215+0000 7f8943fff700 1 -- 172.21.15.64:0/1855826670 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89500567c0 con 0x7f895c103140 2024-02-16T23:06:42.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.429+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f895c04bed0 con 0x7f894406d7e0 2024-02-16T23:06:42.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.440+0000 7f8943fff700 1 -- 172.21.15.64:0/1855826670 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f895c04bed0 con 0x7f894406d7e0 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (56s) 35s ago 18m 16.3M - 0.25.0 c8568f914cd2 30b410320c0e 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (6m) 35s ago 18m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (6m) 87s ago 17m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (40s) 35s ago 18m 55.2M - 9.4.12 a0ea5e3c46d9 302d36ce61ee 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (2m) 35s ago 12m 12.2M - 19.0.0-1450-gf78a58c0 f081f847070a d4dd49dc2ce8 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (2m) 35s ago 12m 87.9M - 19.0.0-1450-gf78a58c0 f081f847070a bd35cb99fa04 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (105s) 87s ago 12m 14.0M - 19.0.0-1450-gf78a58c0 f081f847070a 8d8b30783f08 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (2m) 87s ago 12m 12.4M - 19.0.0-1450-gf78a58c0 f081f847070a ca63fdf53f50 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (8m) 35s ago 19m 511M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (8m) 87s ago 17m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:06:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (7m) 35s ago 19m 54.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (7m) 87s ago 17m 43.2M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (94s) 35s ago 18m 8220k - 1.5.0 0da6a335fe13 cbac3ca3c432 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (90s) 87s ago 17m 7906k - 1.5.0 0da6a335fe13 dd661245993a 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (6m) 35s ago 16m 107M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (5m) 35s ago 16m 100M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (5m) 35s ago 15m 86.6M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (4m) 87s ago 15m 121M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (3m) 87s ago 14m 108M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (3m) 87s ago 14m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 7bb5b265fc66 2024-02-16T23:06:42.441 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (73s) 35s ago 17m 44.0M - 2.43.0 a07b618ecd1d 51136d8386ab 2024-02-16T23:06:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f894406d7e0 msgr2=0x7f894406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f894406d7e0 0x7f894406fc90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f894c004f90 tx=0x7f894c009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 msgr2=0x7f895c0fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 0x7f895c0fcc90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f895000c830 tx=0x7f8950004470 comp rx=0 tx=0).stop 2024-02-16T23:06:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 shutdown_connections 2024-02-16T23:06:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f894406d7e0 0x7f894406fc90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895c0fb570 0x7f895c0fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 --2- 172.21.15.64:0/1855826670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f895c103140 0x7f895c0fcc90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 >> 172.21.15.64:0/1855826670 conn(0x7f895c071560 msgr2=0x7f895c0fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 shutdown_connections 2024-02-16T23:06:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:42.442+0000 7f89621c3700 1 -- 172.21.15.64:0/1855826670 wait complete. 2024-02-16T23:06:42.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:42 smithi064 ceph-mon[93757]: pgmap v267: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:42.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:42 smithi192 ceph-mon[82652]: pgmap v267: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:43.030 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade status' 2024-02-16T23:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:06:43.698 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:43 smithi064 ceph-mon[93757]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:43.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:43 smithi192 ceph-mon[82652]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:44.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:44 smithi192 ceph-mon[82652]: pgmap v268: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:44.825 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:44 smithi064 ceph-mon[93757]: pgmap v268: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 -- 172.21.15.64:0/2501941177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fdfe0 msgr2=0x7f0b4c0fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 --2- 172.21.15.64:0/2501941177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fdfe0 0x7f0b4c0fe3b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0b34004970 tx=0x7f0b34009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 -- 172.21.15.64:0/2501941177 shutdown_connections 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 --2- 172.21.15.64:0/2501941177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b4c0fe8f0 0x7f0b4c107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 --2- 172.21.15.64:0/2501941177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fdfe0 0x7f0b4c0fe3b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 -- 172.21.15.64:0/2501941177 >> 172.21.15.64:0/2501941177 conn(0x7f0b4c071d90 msgr2=0x7f0b4c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 -- 172.21.15.64:0/2501941177 shutdown_connections 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 -- 172.21.15.64:0/2501941177 wait complete. 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.026+0000 7f0b51226700 1 Processor -- start 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b51226700 1 -- start start 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b51226700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b4c0fdfe0 0x7f0b4c0fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b51226700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fe8f0 0x7f0b4c0fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b51226700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0fc9e0 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b51226700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0fcb50 con 0x7f0b4c0fdfe0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b4a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fe8f0 0x7f0b4c0fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b4a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fe8f0 0x7f0b4c0fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56516/0 (socket says 172.21.15.64:56516) 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b4a59c700 1 -- 172.21.15.64:0/1393491440 learned_addr learned my addr 172.21.15.64:0/1393491440 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b4ad9d700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b4c0fdfe0 0x7f0b4c0fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b4a59c700 1 -- 172.21.15.64:0/1393491440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b4c0fdfe0 msgr2=0x7f0b4c0fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b4a59c700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b4c0fdfe0 0x7f0b4c0fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b4a59c700 1 -- 172.21.15.64:0/1393491440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34005040 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.027+0000 7f0b4a59c700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fe8f0 0x7f0b4c0fc3e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00c8c0 tx=0x7f0b3c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.028+0000 7f0b437fe700 1 -- 172.21.15.64:0/1393491440 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b3c00a760 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.028+0000 7f0b437fe700 1 -- 172.21.15.64:0/1393491440 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b3c00ad90 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.028+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c119c90 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.028+0000 7f0b437fe700 1 -- 172.21.15.64:0/1393491440 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b3c0113a0 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.028+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c11a170 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.029+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c1046c0 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.029+0000 7f0b437fe700 1 -- 172.21.15.64:0/1393491440 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0b3c012400 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.030+0000 7f0b437fe700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f0b3806d7f0 0x7f0b3806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.030+0000 7f0b437fe700 1 -- 172.21.15.64:0/1393491440 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0b3c08cf10 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.030+0000 7f0b4ad9d700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f0b3806d7f0 0x7f0b3806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.030+0000 7f0b4ad9d700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f0b3806d7f0 0x7f0b3806fca0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b34004940 tx=0x7f0b34009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:45.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.034+0000 7f0b437fe700 1 -- 172.21.15.64:0/1393491440 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b3c0567a0 con 0x7f0b4c0fe8f0 2024-02-16T23:06:45.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.257+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b4c0fd5e0 con 0x7f0b3806d7f0 2024-02-16T23:06:45.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.258+0000 7f0b437fe700 1 -- 172.21.15.64:0/1393491440 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f0b4c0fd5e0 con 0x7f0b3806d7f0 2024-02-16T23:06:45.258 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:06:45.258 INFO:teuthology.orchestra.run.smithi064.stdout: "target_image": null, 2024-02-16T23:06:45.258 INFO:teuthology.orchestra.run.smithi064.stdout: "in_progress": false, 2024-02-16T23:06:45.258 INFO:teuthology.orchestra.run.smithi064.stdout: "which": "", 2024-02-16T23:06:45.258 INFO:teuthology.orchestra.run.smithi064.stdout: "services_complete": [], 2024-02-16T23:06:45.259 INFO:teuthology.orchestra.run.smithi064.stdout: "progress": null, 2024-02-16T23:06:45.259 INFO:teuthology.orchestra.run.smithi064.stdout: "message": "", 2024-02-16T23:06:45.259 INFO:teuthology.orchestra.run.smithi064.stdout: "is_paused": false 2024-02-16T23:06:45.259 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:06:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f0b3806d7f0 msgr2=0x7f0b3806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f0b3806d7f0 0x7f0b3806fca0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b34004940 tx=0x7f0b34009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fe8f0 msgr2=0x7f0b4c0fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fe8f0 0x7f0b4c0fc3e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00c8c0 tx=0x7f0b3c0044c0 comp rx=0 tx=0).stop 2024-02-16T23:06:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 shutdown_connections 2024-02-16T23:06:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f0b3806d7f0 0x7f0b3806fca0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b4c0fdfe0 0x7f0b4c0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 --2- 172.21.15.64:0/1393491440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c0fe8f0 0x7f0b4c0fc3e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 >> 172.21.15.64:0/1393491440 conn(0x7f0b4c071d90 msgr2=0x7f0b4c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 shutdown_connections 2024-02-16T23:06:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:45.260+0000 7f0b51226700 1 -- 172.21.15.64:0/1393491440 wait complete. 2024-02-16T23:06:45.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph health detail' 2024-02-16T23:06:46.232 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:06:46.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:46 smithi064 ceph-mon[93757]: pgmap v269: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:46.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:46 smithi064 ceph-mon[93757]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:46.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:46 smithi192 ceph-mon[82652]: pgmap v269: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:46.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:46 smithi192 ceph-mon[82652]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.952+0000 7f97b4e18700 1 -- 172.21.15.64:0/2277758809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 msgr2=0x7f97b0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.952+0000 7f97b4e18700 1 --2- 172.21.15.64:0/2277758809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 0x7f97b0103320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009ce0 tx=0x7f97a8004d10 comp rx=0 tx=0).stop 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.952+0000 7f97b4e18700 1 -- 172.21.15.64:0/2277758809 shutdown_connections 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.952+0000 7f97b4e18700 1 --2- 172.21.15.64:0/2277758809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97b01038f0 0x7f97b00fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.952+0000 7f97b4e18700 1 --2- 172.21.15.64:0/2277758809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 0x7f97b0103320 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.952+0000 7f97b4e18700 1 -- 172.21.15.64:0/2277758809 >> 172.21.15.64:0/2277758809 conn(0x7f97b00f8ae0 msgr2=0x7f97b00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.952+0000 7f97b4e18700 1 -- 172.21.15.64:0/2277758809 shutdown_connections 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.952+0000 7f97b4e18700 1 -- 172.21.15.64:0/2277758809 wait complete. 2024-02-16T23:06:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97b4e18700 1 Processor -- start 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97b4e18700 1 -- start start 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97b4e18700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 0x7f97b0115e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97b4e18700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97b01038f0 0x7f97b0116360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97b4e18700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b0116a40 con 0x7f97b0102f50 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97b4e18700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b010fcc0 con 0x7f97b01038f0 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97ae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 0x7f97b0115e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97ae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 0x7f97b0115e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56540/0 (socket says 172.21.15.64:56540) 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97ae59c700 1 -- 172.21.15.64:0/431573200 learned_addr learned my addr 172.21.15.64:0/431573200 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97ae59c700 1 -- 172.21.15.64:0/431573200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97b01038f0 msgr2=0x7f97b0116360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97ae59c700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97b01038f0 0x7f97b0116360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97ae59c700 1 -- 172.21.15.64:0/431573200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009940 con 0x7f97b0102f50 2024-02-16T23:06:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.953+0000 7f97ae59c700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 0x7f97b0115e20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004b70 tx=0x7f97a8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.954+0000 7f97a6ffd700 1 -- 172.21.15.64:0/431573200 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a800e040 con 0x7f97b0102f50 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.954+0000 7f97a6ffd700 1 -- 172.21.15.64:0/431573200 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97a8004610 con 0x7f97b0102f50 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.954+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b010fee0 con 0x7f97b0102f50 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.954+0000 7f97a6ffd700 1 -- 172.21.15.64:0/431573200 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a801c810 con 0x7f97b0102f50 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.954+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b0110420 con 0x7f97b0102f50 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.954+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b00fc6e0 con 0x7f97b0102f50 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.955+0000 7f97a6ffd700 1 -- 172.21.15.64:0/431573200 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f97a8012c00 con 0x7f97b0102f50 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.955+0000 7f97a6ffd700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f979c06d7f0 0x7f979c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.955+0000 7f97a6ffd700 1 -- 172.21.15.64:0/431573200 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f97a808ec10 con 0x7f97b0102f50 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.955+0000 7f97add9b700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f979c06d7f0 0x7f979c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.956+0000 7f97add9b700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f979c06d7f0 0x7f979c06fca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9798006fd0 tx=0x7f9798006db0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:47.960+0000 7f97a6ffd700 1 -- 172.21.15.64:0/431573200 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97a805be90 con 0x7f97b0102f50 2024-02-16T23:06:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.246+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f97b01114a0 con 0x7f97b0102f50 2024-02-16T23:06:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.247+0000 7f97a6ffd700 1 -- 172.21.15.64:0/431573200 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f97a805ba20 con 0x7f97b0102f50 2024-02-16T23:06:48.247 INFO:teuthology.orchestra.run.smithi064.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:06:48.247 INFO:teuthology.orchestra.run.smithi064.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T23:06:48.247 INFO:teuthology.orchestra.run.smithi064.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T23:06:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f979c06d7f0 msgr2=0x7f979c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f979c06d7f0 0x7f979c06fca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9798006fd0 tx=0x7f9798006db0 comp rx=0 tx=0).stop 2024-02-16T23:06:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 msgr2=0x7f97b0115e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 0x7f97b0115e20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004b70 tx=0x7f97a8004d10 comp rx=0 tx=0).stop 2024-02-16T23:06:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 shutdown_connections 2024-02-16T23:06:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f979c06d7f0 0x7f979c06fca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:48.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b0102f50 0x7f97b0115e20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:48.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 --2- 172.21.15.64:0/431573200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97b01038f0 0x7f97b0116360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:48.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 >> 172.21.15.64:0/431573200 conn(0x7f97b00f8ae0 msgr2=0x7f97b0070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:48.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 shutdown_connections 2024-02-16T23:06:48.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:48.249+0000 7f97b4e18700 1 -- 172.21.15.64:0/431573200 wait complete. 2024-02-16T23:06:48.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:48 smithi192 ceph-mon[82652]: pgmap v270: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:48.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:06:48.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:48 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/431573200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:06:48.854 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:48 smithi064 ceph-mon[93757]: pgmap v270: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:48.854 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:06:48.854 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:48 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/431573200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T23:06:48.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-16T23:06:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:06:50.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.718+0000 7fee8c055700 1 -- 172.21.15.64:0/57542890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 msgr2=0x7fee840fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.718+0000 7fee8c055700 1 --2- 172.21.15.64:0/57542890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 0x7fee840fcf60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fee78004940 tx=0x7fee78009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.718+0000 7fee8c055700 1 -- 172.21.15.64:0/57542890 shutdown_connections 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.718+0000 7fee8c055700 1 --2- 172.21.15.64:0/57542890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 0x7fee840fcf60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.718+0000 7fee8c055700 1 --2- 172.21.15.64:0/57542890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee84102f50 0x7fee84103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.718+0000 7fee8c055700 1 -- 172.21.15.64:0/57542890 >> 172.21.15.64:0/57542890 conn(0x7fee840f8ae0 msgr2=0x7fee840faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.718+0000 7fee8c055700 1 -- 172.21.15.64:0/57542890 shutdown_connections 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.718+0000 7fee8c055700 1 -- 172.21.15.64:0/57542890 wait complete. 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee8c055700 1 Processor -- start 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee8c055700 1 -- start start 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee8c055700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee84102f50 0x7fee84113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee8c055700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 0x7fee84113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee8c055700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84114620 con 0x7fee841038f0 2024-02-16T23:06:50.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee8c055700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8410d950 con 0x7fee84102f50 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee895f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 0x7fee84113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee895f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 0x7fee84113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52926/0 (socket says 172.21.15.64:52926) 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee895f0700 1 -- 172.21.15.64:0/1442394741 learned_addr learned my addr 172.21.15.64:0/1442394741 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee89df1700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee84102f50 0x7fee84113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee895f0700 1 -- 172.21.15.64:0/1442394741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee84102f50 msgr2=0x7fee84113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee895f0700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee84102f50 0x7fee84113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.719+0000 7fee895f0700 1 -- 172.21.15.64:0/1442394741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78005040 con 0x7fee841038f0 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.720+0000 7fee895f0700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 0x7fee84113fd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee78009100 tx=0x7fee78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.720+0000 7fee827fc700 1 -- 172.21.15.64:0/1442394741 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee7800e040 con 0x7fee841038f0 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.720+0000 7fee827fc700 1 -- 172.21.15.64:0/1442394741 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee78004290 con 0x7fee841038f0 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.720+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8410dbd0 con 0x7fee841038f0 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.720+0000 7fee827fc700 1 -- 172.21.15.64:0/1442394741 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee7801c3f0 con 0x7fee841038f0 2024-02-16T23:06:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.720+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8410e110 con 0x7fee841038f0 2024-02-16T23:06:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.721+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee84105420 con 0x7fee841038f0 2024-02-16T23:06:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.721+0000 7fee827fc700 1 -- 172.21.15.64:0/1442394741 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fee7800bbf0 con 0x7fee841038f0 2024-02-16T23:06:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.721+0000 7fee827fc700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fee6c06d7f0 0x7fee6c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:06:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.722+0000 7fee827fc700 1 -- 172.21.15.64:0/1442394741 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fee7808e0f0 con 0x7fee841038f0 2024-02-16T23:06:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.722+0000 7fee89df1700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fee6c06d7f0 0x7fee6c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:06:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.722+0000 7fee89df1700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fee6c06d7f0 0x7fee6c06fca0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee7c001450 tx=0x7fee7c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:06:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:50.726+0000 7fee827fc700 1 -- 172.21.15.64:0/1442394741 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee7805b4b0 con 0x7fee841038f0 2024-02-16T23:06:50.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:50 smithi192 ceph-mon[82652]: pgmap v271: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:50.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:50 smithi064 ceph-mon[93757]: pgmap v271: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:51.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.013+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fee8410f070 con 0x7fee841038f0 2024-02-16T23:06:51.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.014+0000 7fee827fc700 1 -- 172.21.15.64:0/1442394741 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fee7805b040 con 0x7fee841038f0 2024-02-16T23:06:51.014 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:06:51.014 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:06:51.014 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:06:51.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:06:51.014 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:06:51.014 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:06:51.015 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:06:51.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fee6c06d7f0 msgr2=0x7fee6c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:51.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fee6c06d7f0 0x7fee6c06fca0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee7c001450 tx=0x7fee7c00b040 comp rx=0 tx=0).stop 2024-02-16T23:06:51.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 msgr2=0x7fee84113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:06:51.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 0x7fee84113fd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee78009100 tx=0x7fee78009e30 comp rx=0 tx=0).stop 2024-02-16T23:06:51.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 shutdown_connections 2024-02-16T23:06:51.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fee6c06d7f0 0x7fee6c06fca0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:51.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee84102f50 0x7fee84113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:51.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 --2- 172.21.15.64:0/1442394741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee841038f0 0x7fee84113fd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:06:51.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 >> 172.21.15.64:0/1442394741 conn(0x7fee840f8ae0 msgr2=0x7fee840fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:06:51.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 shutdown_connections 2024-02-16T23:06:51.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:06:51.016+0000 7fee8c055700 1 -- 172.21.15.64:0/1442394741 wait complete. 2024-02-16T23:06:51.612 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:51 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1442394741' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:51.636 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-16T23:06:51.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:51 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1442394741' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:06:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:06:52.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:52 smithi064 ceph-mon[93757]: pgmap v272: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:52.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:52 smithi192 ceph-mon[82652]: pgmap v272: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:53.583 INFO:teuthology.orchestra.run.smithi064.stdout:wait for servicemap items w/ changing names to refresh 2024-02-16T23:06:54.276 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'sleep 60' 2024-02-16T23:06:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:06:54.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:54 smithi064 ceph-mon[93757]: pgmap v273: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:54.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:54 smithi192 ceph-mon[82652]: pgmap v273: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:56.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:56 smithi192 ceph-mon[82652]: pgmap v274: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:56.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:56 smithi064 ceph-mon[93757]: pgmap v274: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:58.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:06:58 smithi192 ceph-mon[82652]: pgmap v275: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:06:58.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:06:58 smithi064 ceph-mon[93757]: pgmap v275: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:00.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:00 smithi192 ceph-mon[82652]: pgmap v276: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:00.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:00 smithi064 ceph-mon[93757]: pgmap v276: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:02.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:02 smithi064 ceph-mon[93757]: pgmap v277: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:02.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:02 smithi192 ceph-mon[82652]: pgmap v277: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:03.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:07:03.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:07:04.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:04 smithi192 ceph-mon[82652]: pgmap v278: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:04.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:04 smithi064 ceph-mon[93757]: pgmap v278: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:06.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:06 smithi192 ceph-mon[82652]: pgmap v279: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:06.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:06 smithi064 ceph-mon[93757]: pgmap v279: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:08.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:08 smithi064 ceph-mon[93757]: pgmap v280: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:09.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:08 smithi192 ceph-mon[82652]: pgmap v280: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:10.836 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:10 smithi192 ceph-mon[82652]: pgmap v281: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:10.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:10 smithi064 ceph-mon[93757]: pgmap v281: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:12.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:12 smithi064 ceph-mon[93757]: pgmap v282: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:12.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:12 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:07:13.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:12 smithi192 ceph-mon[82652]: pgmap v282: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:13.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:12 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:07:13.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:07:13.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:07:13.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:07:14.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:07:14.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:07:14.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:07:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:14 smithi064 ceph-mon[93757]: pgmap v283: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:15.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:14 smithi192 ceph-mon[82652]: pgmap v283: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:16.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:16 smithi064 ceph-mon[93757]: pgmap v284: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:17.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:16 smithi192 ceph-mon[82652]: pgmap v284: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:18 smithi064 ceph-mon[93757]: pgmap v285: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:07:19.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:18 smithi192 ceph-mon[82652]: pgmap v285: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:19.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:07:20.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:20 smithi064 ceph-mon[93757]: pgmap v286: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:21.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:20 smithi192 ceph-mon[82652]: pgmap v286: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:22.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:22 smithi064 ceph-mon[93757]: pgmap v287: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:23.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:22 smithi192 ceph-mon[82652]: pgmap v287: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:24.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:24 smithi064 ceph-mon[93757]: pgmap v288: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:25.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:24 smithi192 ceph-mon[82652]: pgmap v288: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:26.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:26 smithi064 ceph-mon[93757]: pgmap v289: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:27.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:26 smithi192 ceph-mon[82652]: pgmap v289: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:28.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:28 smithi064 ceph-mon[93757]: pgmap v290: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:29.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:28 smithi192 ceph-mon[82652]: pgmap v290: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:30.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:30 smithi064 ceph-mon[93757]: pgmap v291: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:31.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:30 smithi192 ceph-mon[82652]: pgmap v291: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:32.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:32 smithi064 ceph-mon[93757]: pgmap v292: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:33.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:32 smithi192 ceph-mon[82652]: pgmap v292: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:33.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:07:34.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:07:34.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:34 smithi064 ceph-mon[93757]: pgmap v293: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:35.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:34 smithi192 ceph-mon[82652]: pgmap v293: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:36.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:36 smithi064 ceph-mon[93757]: pgmap v294: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:37.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:36 smithi192 ceph-mon[82652]: pgmap v294: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:38.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:38 smithi064 ceph-mon[93757]: pgmap v295: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:39.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:38 smithi192 ceph-mon[82652]: pgmap v295: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:40.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:40 smithi064 ceph-mon[93757]: pgmap v296: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:41.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:40 smithi192 ceph-mon[82652]: pgmap v296: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:42.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:42 smithi064 ceph-mon[93757]: pgmap v297: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:43.023 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:42 smithi192 ceph-mon[82652]: pgmap v297: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:44.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:44 smithi064 ceph-mon[93757]: pgmap v298: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:45.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:44 smithi192 ceph-mon[82652]: pgmap v298: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:46.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:46 smithi064 ceph-mon[93757]: pgmap v299: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:47.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:46 smithi192 ceph-mon[82652]: pgmap v299: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:48.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:48 smithi064 ceph-mon[93757]: pgmap v300: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:48.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:07:49.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:48 smithi192 ceph-mon[82652]: pgmap v300: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:49.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:07:50.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:50 smithi064 ceph-mon[93757]: pgmap v301: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:51.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:50 smithi192 ceph-mon[82652]: pgmap v301: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:52.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:52 smithi192 ceph-mon[82652]: pgmap v302: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:52.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:52 smithi064 ceph-mon[93757]: pgmap v302: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:54.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:54 smithi064 ceph-mon[93757]: pgmap v303: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:55.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:54 smithi192 ceph-mon[82652]: pgmap v303: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:56.606 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T23:07:56.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:56 smithi064 ceph-mon[93757]: pgmap v304: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:56.876 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:07:57.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:56 smithi192 ceph-mon[82652]: pgmap v304: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.513+0000 7f17bb3e9700 1 -- 172.21.15.64:0/959747694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4101ee0 msgr2=0x7f17b41022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:07:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.513+0000 7f17bb3e9700 1 --2- 172.21.15.64:0/959747694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4101ee0 0x7f17b41022b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004940 tx=0x7f17b0009e30 comp rx=0 tx=0).stop 2024-02-16T23:07:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.513+0000 7f17bb3e9700 1 -- 172.21.15.64:0/959747694 shutdown_connections 2024-02-16T23:07:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.513+0000 7f17bb3e9700 1 --2- 172.21.15.64:0/959747694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b4102880 0x7f17b40fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:07:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.513+0000 7f17bb3e9700 1 --2- 172.21.15.64:0/959747694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4101ee0 0x7f17b41022b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:07:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.513+0000 7f17bb3e9700 1 -- 172.21.15.64:0/959747694 >> 172.21.15.64:0/959747694 conn(0x7f17b4071560 msgr2=0x7f17b4071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.513+0000 7f17bb3e9700 1 -- 172.21.15.64:0/959747694 shutdown_connections 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.513+0000 7f17bb3e9700 1 -- 172.21.15.64:0/959747694 wait complete. 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17bb3e9700 1 Processor -- start 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17bb3e9700 1 -- start start 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17bb3e9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b4101ee0 0x7f17b410eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17bb3e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4102880 0x7f17b410f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17bb3e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b410f620 con 0x7f17b4102880 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17bb3e9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b410f790 con 0x7f17b4101ee0 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17b8984700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4102880 0x7f17b410f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17b8984700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4102880 0x7f17b410f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57794/0 (socket says 172.21.15.64:57794) 2024-02-16T23:07:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.514+0000 7f17b8984700 1 -- 172.21.15.64:0/466879199 learned_addr learned my addr 172.21.15.64:0/466879199 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17b9185700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b4101ee0 0x7f17b410eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17b8984700 1 -- 172.21.15.64:0/466879199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b4101ee0 msgr2=0x7f17b410eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17b8984700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b4101ee0 0x7f17b410eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17b8984700 1 -- 172.21.15.64:0/466879199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0005040 con 0x7f17b4102880 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17b8984700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4102880 0x7f17b410f000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f17a400b7c0 tx=0x7f17a4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17a9ffb700 1 -- 172.21.15.64:0/466879199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a400f6f0 con 0x7f17b4102880 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b4117ae0 con 0x7f17b4102880 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17a9ffb700 1 -- 172.21.15.64:0/466879199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17a400fd20 con 0x7f17b4102880 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17a9ffb700 1 -- 172.21.15.64:0/466879199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a40103a0 con 0x7f17b4102880 2024-02-16T23:07:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.515+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b4117ff0 con 0x7f17b4102880 2024-02-16T23:07:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.516+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b4073080 con 0x7f17b4102880 2024-02-16T23:07:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.517+0000 7f17a9ffb700 1 -- 172.21.15.64:0/466879199 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f17a4011400 con 0x7f17b4102880 2024-02-16T23:07:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.517+0000 7f17a9ffb700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f17a006d7e0 0x7f17a006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:07:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.517+0000 7f17a9ffb700 1 -- 172.21.15.64:0/466879199 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f17a408cdb0 con 0x7f17b4102880 2024-02-16T23:07:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.517+0000 7f17b9185700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f17a006d7e0 0x7f17a006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:07:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.518+0000 7f17b9185700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f17a006d7e0 0x7f17a006fc90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004f90 tx=0x7f17b0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:07:58.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.522+0000 7f17a9ffb700 1 -- 172.21.15.64:0/466879199 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17a4056580 con 0x7f17b4102880 2024-02-16T23:07:58.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.734+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 --> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f17b40fbea0 con 0x7f17a006d7e0 2024-02-16T23:07:58.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.744+0000 7f17a9ffb700 1 -- 172.21.15.64:0/466879199 <== mgr.34100 v2:172.21.15.64:6800/1480939122 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f17b40fbea0 con 0x7f17a006d7e0 2024-02-16T23:07:58.744 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T23:07:58.744 INFO:teuthology.orchestra.run.smithi064.stdout:alertmanager.smithi064 smithi064 *:9093,9094 running (2m) 111s ago 19m 16.3M - 0.25.0 c8568f914cd2 30b410320c0e 2024-02-16T23:07:58.744 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi064 smithi064 running (8m) 111s ago 19m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 0d2d8ae0068f 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:crash.smithi192 smithi192 running (7m) 2m ago 18m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a fd7e6b594b96 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:grafana.smithi064 smithi064 *:3000 running (116s) 111s ago 19m 55.2M - 9.4.12 a0ea5e3c46d9 302d36ce61ee 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.bmcyuf smithi064 running (3m) 111s ago 14m 12.2M - 19.0.0-1450-gf78a58c0 f081f847070a d4dd49dc2ce8 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi064.tqesxx smithi064 running (4m) 111s ago 14m 87.9M - 19.0.0-1450-gf78a58c0 f081f847070a bd35cb99fa04 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.luiybj smithi192 running (3m) 2m ago 14m 14.0M - 19.0.0-1450-gf78a58c0 f081f847070a 8d8b30783f08 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:mds.cephfs.smithi192.tnppti smithi192 running (3m) 2m ago 14m 12.4M - 19.0.0-1450-gf78a58c0 f081f847070a ca63fdf53f50 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi064.kubmlk smithi064 *:8443,9283,8765 running (10m) 111s ago 20m 511M - 19.0.0-1450-gf78a58c0 f081f847070a 0631eaed21d9 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.smithi192.pauvih smithi192 *:8443,9283,8765 running (9m) 2m ago 18m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5270e0cac621 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi064 smithi064 running (9m) 111s ago 21m 54.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a19a30728466 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:mon.smithi192 smithi192 running (8m) 2m ago 18m 43.2M 2048M 19.0.0-1450-gf78a58c0 f081f847070a eb1d711c4dcd 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi064 smithi064 *:9100 running (2m) 111s ago 19m 8220k - 1.5.0 0da6a335fe13 cbac3ca3c432 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:node-exporter.smithi192 smithi192 *:9100 running (2m) 2m ago 18m 7906k - 1.5.0 0da6a335fe13 dd661245993a 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (7m) 111s ago 17m 107M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 3d93cd9ed3d9 2024-02-16T23:07:58.745 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi064 running (6m) 111s ago 17m 100M 1557M 19.0.0-1450-gf78a58c0 f081f847070a 7044a8328d9a 2024-02-16T23:07:58.746 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi064 running (6m) 111s ago 16m 86.6M 1557M 19.0.0-1450-gf78a58c0 f081f847070a a73d025bae8b 2024-02-16T23:07:58.746 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi192 running (5m) 2m ago 16m 121M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 3b60e36c8803 2024-02-16T23:07:58.746 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi192 running (5m) 2m ago 16m 108M 2581M 19.0.0-1450-gf78a58c0 f081f847070a f93a7734ffc4 2024-02-16T23:07:58.746 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi192 running (4m) 2m ago 15m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 7bb5b265fc66 2024-02-16T23:07:58.746 INFO:teuthology.orchestra.run.smithi064.stdout:prometheus.smithi064 smithi064 *:9095 running (2m) 111s ago 19m 44.0M - 2.43.0 a07b618ecd1d 51136d8386ab 2024-02-16T23:07:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f17a006d7e0 msgr2=0x7f17a006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:07:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f17a006d7e0 0x7f17a006fc90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004f90 tx=0x7f17b0009e30 comp rx=0 tx=0).stop 2024-02-16T23:07:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4102880 msgr2=0x7f17b410f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:07:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4102880 0x7f17b410f000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f17a400b7c0 tx=0x7f17a4009220 comp rx=0 tx=0).stop 2024-02-16T23:07:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 shutdown_connections 2024-02-16T23:07:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f17a006d7e0 0x7f17a006fc90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:07:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b4101ee0 0x7f17b410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:07:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 --2- 172.21.15.64:0/466879199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17b4102880 0x7f17b410f000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:07:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 >> 172.21.15.64:0/466879199 conn(0x7f17b4071560 msgr2=0x7f17b40fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:07:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 shutdown_connections 2024-02-16T23:07:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:07:58.747+0000 7f17bb3e9700 1 -- 172.21.15.64:0/466879199 wait complete. 2024-02-16T23:07:59.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:07:58 smithi192 ceph-mon[82652]: pgmap v305: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:59.073 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:07:58 smithi064 ceph-mon[93757]: pgmap v305: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:07:59.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-16T23:07:59.665 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:08:00.905 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:00 smithi064 ceph-mon[93757]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:08:00.906 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:00 smithi064 ceph-mon[93757]: pgmap v306: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:01.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:00 smithi192 ceph-mon[82652]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T23:08:01.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:00 smithi192 ceph-mon[82652]: pgmap v306: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.405+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/3454571088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 msgr2=0x7fb5a4106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.405+0000 7fb5a9fb4700 1 --2- 172.21.15.64:0/3454571088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 0x7fb5a4106f90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb594004940 tx=0x7fb594009e30 comp rx=0 tx=0).stop 2024-02-16T23:08:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.405+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/3454571088 shutdown_connections 2024-02-16T23:08:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.405+0000 7fb5a9fb4700 1 --2- 172.21.15.64:0/3454571088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 0x7fb5a4106f90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.405+0000 7fb5a9fb4700 1 --2- 172.21.15.64:0/3454571088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a40fdfb0 0x7fb5a40fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.405+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/3454571088 >> 172.21.15.64:0/3454571088 conn(0x7fb5a40f79e0 msgr2=0x7fb5a40f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.405+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/3454571088 shutdown_connections 2024-02-16T23:08:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.405+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/3454571088 wait complete. 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a9fb4700 1 Processor -- start 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a9fb4700 1 -- start start 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a9fb4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a40fdfb0 0x7fb5a40fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a9fb4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 0x7fb5a40fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a9fb4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a40fc920 con 0x7fb5a40fe950 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a9fb4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a40fca90 con 0x7fb5a40fdfb0 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 0x7fb5a40fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 0x7fb5a40fc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34710/0 (socket says 172.21.15.64:34710) 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a2ffd700 1 -- 172.21.15.64:0/979433767 learned_addr learned my addr 172.21.15.64:0/979433767 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.406+0000 7fb5a37fe700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a40fdfb0 0x7fb5a40fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb5a2ffd700 1 -- 172.21.15.64:0/979433767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a40fdfb0 msgr2=0x7fb5a40fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb5a2ffd700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a40fdfb0 0x7fb5a40fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb5a2ffd700 1 -- 172.21.15.64:0/979433767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594005040 con 0x7fb5a40fe950 2024-02-16T23:08:01.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb5a2ffd700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 0x7fb5a40fc320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb594009180 tx=0x7fb594009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:01.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb58bfff700 1 -- 172.21.15.64:0/979433767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb59400e040 con 0x7fb5a40fe950 2024-02-16T23:08:01.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb58bfff700 1 -- 172.21.15.64:0/979433767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb594004290 con 0x7fb5a40fe950 2024-02-16T23:08:01.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb58bfff700 1 -- 172.21.15.64:0/979433767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb59401c630 con 0x7fb5a40fe950 2024-02-16T23:08:01.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a4119c80 con 0x7fb5a40fe950 2024-02-16T23:08:01.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.407+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a411a160 con 0x7fb5a40fe950 2024-02-16T23:08:01.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.408+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a41045a0 con 0x7fb5a40fe950 2024-02-16T23:08:01.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.409+0000 7fb58bfff700 1 -- 172.21.15.64:0/979433767 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb59400bbf0 con 0x7fb5a40fe950 2024-02-16T23:08:01.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.409+0000 7fb58bfff700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb58c06d7f0 0x7fb58c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:01.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.409+0000 7fb58bfff700 1 -- 172.21.15.64:0/979433767 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb59408e130 con 0x7fb5a40fe950 2024-02-16T23:08:01.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.409+0000 7fb5a37fe700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb58c06d7f0 0x7fb58c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:01.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.410+0000 7fb5a37fe700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb58c06d7f0 0x7fb58c06fca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb5980041d0 tx=0x7fb5980043c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:01.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.414+0000 7fb58bfff700 1 -- 172.21.15.64:0/979433767 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5940579c0 con 0x7fb5a40fe950 2024-02-16T23:08:01.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.703+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb5a4064b00 con 0x7fb5a40fe950 2024-02-16T23:08:01.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.703+0000 7fb58bfff700 1 -- 172.21.15.64:0/979433767 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb594004db0 con 0x7fb5a40fe950 2024-02-16T23:08:01.703 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-16T23:08:01.703 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": { 2024-02-16T23:08:01.703 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:08:01.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": { 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": { 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: "overall": { 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-02-16T23:08:01.704 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-16T23:08:01.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.705+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb58c06d7f0 msgr2=0x7fb58c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:01.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.705+0000 7fb5a9fb4700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb58c06d7f0 0x7fb58c06fca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb5980041d0 tx=0x7fb5980043c0 comp rx=0 tx=0).stop 2024-02-16T23:08:01.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.705+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 msgr2=0x7fb5a40fc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.705+0000 7fb5a9fb4700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 0x7fb5a40fc320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb594009180 tx=0x7fb594009e30 comp rx=0 tx=0).stop 2024-02-16T23:08:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.705+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 shutdown_connections 2024-02-16T23:08:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.705+0000 7fb5a9fb4700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb58c06d7f0 0x7fb58c06fca0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.705+0000 7fb5a9fb4700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a40fdfb0 0x7fb5a40fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.705+0000 7fb5a9fb4700 1 --2- 172.21.15.64:0/979433767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a40fe950 0x7fb5a40fc320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.706+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 >> 172.21.15.64:0/979433767 conn(0x7fb5a40f79e0 msgr2=0x7fb5a4101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.706+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 shutdown_connections 2024-02-16T23:08:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:01.706+0000 7fb5a9fb4700 1 -- 172.21.15.64:0/979433767 wait complete. 2024-02-16T23:08:02.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-16T23:08:02.673 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:08:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:02 smithi064 ceph-mon[93757]: pgmap v307: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:02.795 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:02 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/979433767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:08:03.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:02 smithi192 ceph-mon[82652]: pgmap v307: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:03.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:02 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/979433767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:08:03.754 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:08:04.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:08:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.248+0000 7fa30968e700 1 -- 172.21.15.64:0/1176419031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 msgr2=0x7fa3040fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.248+0000 7fa30968e700 1 --2- 172.21.15.64:0/1176419031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 0x7fa3040fbe70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004940 tx=0x7fa2f8009e30 comp rx=0 tx=0).stop 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.248+0000 7fa30968e700 1 -- 172.21.15.64:0/1176419031 shutdown_connections 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.248+0000 7fa30968e700 1 --2- 172.21.15.64:0/1176419031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 0x7fa3040fbe70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.248+0000 7fa30968e700 1 --2- 172.21.15.64:0/1176419031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa304101e60 0x7fa304102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.248+0000 7fa30968e700 1 -- 172.21.15.64:0/1176419031 >> 172.21.15.64:0/1176419031 conn(0x7fa3040f7a10 msgr2=0x7fa3040f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.248+0000 7fa30968e700 1 -- 172.21.15.64:0/1176419031 shutdown_connections 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.248+0000 7fa30968e700 1 -- 172.21.15.64:0/1176419031 wait complete. 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa30968e700 1 Processor -- start 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa30968e700 1 -- start start 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa30968e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa304101e60 0x7fa3040fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa30968e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 0x7fa3040fe020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa30968e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040fe620 con 0x7fa304102800 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa30968e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040fe790 con 0x7fa304101e60 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa3027fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 0x7fa3040fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa3027fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 0x7fa3040fe020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34730/0 (socket says 172.21.15.64:34730) 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa3027fc700 1 -- 172.21.15.64:0/2527796937 learned_addr learned my addr 172.21.15.64:0/2527796937 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa302ffd700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa304101e60 0x7fa3040fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa3027fc700 1 -- 172.21.15.64:0/2527796937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa304101e60 msgr2=0x7fa3040fdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa3027fc700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa304101e60 0x7fa3040fdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa3027fc700 1 -- 172.21.15.64:0/2527796937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f8005040 con 0x7fa304102800 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.249+0000 7fa3027fc700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 0x7fa3040fe020 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80090d0 tx=0x7fa2f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.250+0000 7fa2eb7fe700 1 -- 172.21.15.64:0/2527796937 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2f800e040 con 0x7fa304102800 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.250+0000 7fa2eb7fe700 1 -- 172.21.15.64:0/2527796937 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2f800bbf0 con 0x7fa304102800 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.250+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa304119d20 con 0x7fa304102800 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.250+0000 7fa2eb7fe700 1 -- 172.21.15.64:0/2527796937 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2f801c630 con 0x7fa304102800 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.250+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30411a180 con 0x7fa304102800 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.250+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3040fb610 con 0x7fa304102800 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.251+0000 7fa2eb7fe700 1 -- 172.21.15.64:0/2527796937 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa2f800bd60 con 0x7fa304102800 2024-02-16T23:08:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.251+0000 7fa2eb7fe700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa2ec06d7e0 0x7fa2ec06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.252+0000 7fa2eb7fe700 1 -- 172.21.15.64:0/2527796937 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa2f808e2d0 con 0x7fa304102800 2024-02-16T23:08:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.252+0000 7fa302ffd700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa2ec06d7e0 0x7fa2ec06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.252+0000 7fa302ffd700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa2ec06d7e0 0x7fa2ec06fc90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40041d0 tx=0x7fa2f4009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.256+0000 7fa2eb7fe700 1 -- 172.21.15.64:0/2527796937 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2f8057aa0 con 0x7fa304102800 2024-02-16T23:08:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.544+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa304108f40 con 0x7fa304102800 2024-02-16T23:08:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.544+0000 7fa2eb7fe700 1 -- 172.21.15.64:0/2527796937 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa2f8013e60 con 0x7fa304102800 2024-02-16T23:08:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa2ec06d7e0 msgr2=0x7fa2ec06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa2ec06d7e0 0x7fa2ec06fc90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40041d0 tx=0x7fa2f4009040 comp rx=0 tx=0).stop 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 msgr2=0x7fa3040fe020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 0x7fa3040fe020 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80090d0 tx=0x7fa2f8009e30 comp rx=0 tx=0).stop 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 shutdown_connections 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa2ec06d7e0 0x7fa2ec06fc90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa304101e60 0x7fa3040fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 --2- 172.21.15.64:0/2527796937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa304102800 0x7fa3040fe020 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.546+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 >> 172.21.15.64:0/2527796937 conn(0x7fa3040f7a10 msgr2=0x7fa304066010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.547+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 shutdown_connections 2024-02-16T23:08:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:04.547+0000 7fa30968e700 1 -- 172.21.15.64:0/2527796937 wait complete. 2024-02-16T23:08:04.557 INFO:teuthology.orchestra.run.smithi064.stdout:true 2024-02-16T23:08:04.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:04 smithi064 ceph-mon[93757]: pgmap v308: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:04.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:04 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2527796937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:08:05.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:04 smithi192 ceph-mon[82652]: pgmap v308: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:05.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:04 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2527796937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:08:05.170 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-16T23:08:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:08:05.859 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:05 smithi064 ceph-mon[93757]: pgmap v309: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:06.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:05 smithi192 ceph-mon[82652]: pgmap v309: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:07.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.109+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/1999036237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 msgr2=0x7f4f440fb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:07.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 --2- 172.21.15.64:0/1999036237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 0x7f4f440fb7b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f38004940 tx=0x7f4f38009e30 comp rx=0 tx=0).stop 2024-02-16T23:08:07.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/1999036237 shutdown_connections 2024-02-16T23:08:07.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 --2- 172.21.15.64:0/1999036237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 0x7f4f440fb7b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 --2- 172.21.15.64:0/1999036237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f44102df0 0x7f4f441031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/1999036237 >> 172.21.15.64:0/1999036237 conn(0x7f4f44071340 msgr2=0x7f4f44071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/1999036237 shutdown_connections 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/1999036237 wait complete. 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 Processor -- start 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 -- start start 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 0x7f4f441137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f44102df0 0x7f4f44113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f44114330 con 0x7f4f440fb360 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f4a1a2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4410d660 con 0x7f4f44102df0 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.110+0000 7f4f491a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 0x7f4f441137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f491a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 0x7f4f441137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34754/0 (socket says 172.21.15.64:34754) 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f491a0700 1 -- 172.21.15.64:0/916376722 learned_addr learned my addr 172.21.15.64:0/916376722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f4899f700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f44102df0 0x7f4f44113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f491a0700 1 -- 172.21.15.64:0/916376722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f44102df0 msgr2=0x7f4f44113ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f491a0700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f44102df0 0x7f4f44113ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f491a0700 1 -- 172.21.15.64:0/916376722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38005040 con 0x7f4f440fb360 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f491a0700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 0x7f4f441137a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400ec30 tx=0x7f4f3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f41ffb700 1 -- 172.21.15.64:0/916376722 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f34007b30 con 0x7f4f440fb360 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f41ffb700 1 -- 172.21.15.64:0/916376722 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f3400f040 con 0x7f4f440fb360 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4410d940 con 0x7f4f440fb360 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f41ffb700 1 -- 172.21.15.64:0/916376722 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f3400cc90 con 0x7f4f440fb360 2024-02-16T23:08:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.111+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4410de80 con 0x7f4f440fb360 2024-02-16T23:08:07.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.113+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4405f5c0 con 0x7f4f440fb360 2024-02-16T23:08:07.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.113+0000 7f4f41ffb700 1 -- 172.21.15.64:0/916376722 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4f34007c90 con 0x7f4f440fb360 2024-02-16T23:08:07.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.113+0000 7f4f41ffb700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4f2c06d7f0 0x7f4f2c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:07.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.113+0000 7f4f41ffb700 1 -- 172.21.15.64:0/916376722 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4f3408d3b0 con 0x7f4f440fb360 2024-02-16T23:08:07.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.113+0000 7f4f4899f700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4f2c06d7f0 0x7f4f2c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:07.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.114+0000 7f4f4899f700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4f2c06d7f0 0x7f4f2c06fca0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f4410efd0 tx=0x7f4f38009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.118+0000 7f4f41ffb700 1 -- 172.21.15.64:0/916376722 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f3405a6b0 con 0x7f4f440fb360 2024-02-16T23:08:07.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.412+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4f4410ed80 con 0x7f4f440fb360 2024-02-16T23:08:07.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.413+0000 7f4f41ffb700 1 -- 172.21.15.64:0/916376722 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4f3405a240 con 0x7f4f440fb360 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4f2c06d7f0 msgr2=0x7f4f2c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4f2c06d7f0 0x7f4f2c06fca0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f4410efd0 tx=0x7f4f38009e30 comp rx=0 tx=0).stop 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 msgr2=0x7f4f441137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 0x7f4f441137a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400ec30 tx=0x7f4f3400b040 comp rx=0 tx=0).stop 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 shutdown_connections 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4f2c06d7f0 0x7f4f2c06fca0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f440fb360 0x7f4f441137a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 --2- 172.21.15.64:0/916376722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f44102df0 0x7f4f44113ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 >> 172.21.15.64:0/916376722 conn(0x7f4f44071340 msgr2=0x7f4f440fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 shutdown_connections 2024-02-16T23:08:07.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:07.415+0000 7f4f4a1a2700 1 -- 172.21.15.64:0/916376722 wait complete. 2024-02-16T23:08:07.425 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)" 2024-02-16T23:08:08.079 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:07 smithi064 ceph-mon[93757]: pgmap v310: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:08.080 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:07 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/916376722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:08:08.106 DEBUG:teuthology.parallel:result is None 2024-02-16T23:08:08.106 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T23:08:08.117 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T23:08:08.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T23:08:08.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:07 smithi192 ceph-mon[82652]: pgmap v310: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:08.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:07 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/916376722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T23:08:08.382 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/mon.smithi064/config 2024-02-16T23:08:09.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.982+0000 7fce0db5b700 1 -- 172.21.15.64:0/2045209890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08102f50 msgr2=0x7fce08103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:09.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.982+0000 7fce0db5b700 1 --2- 172.21.15.64:0/2045209890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08102f50 0x7fce08103320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8004940 tx=0x7fcdf8009e30 comp rx=0 tx=0).stop 2024-02-16T23:08:09.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 -- 172.21.15.64:0/2045209890 shutdown_connections 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 --2- 172.21.15.64:0/2045209890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce081038f0 0x7fce080fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 --2- 172.21.15.64:0/2045209890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08102f50 0x7fce08103320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 -- 172.21.15.64:0/2045209890 >> 172.21.15.64:0/2045209890 conn(0x7fce080f8ae0 msgr2=0x7fce080faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 -- 172.21.15.64:0/2045209890 shutdown_connections 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 -- 172.21.15.64:0/2045209890 wait complete. 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 Processor -- start 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 -- start start 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.983+0000 7fce0db5b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce08102f50 0x7fce08113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce0db5b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce081038f0 0x7fce08113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce0db5b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce081145f0 con 0x7fce081038f0 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce0db5b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0810d920 con 0x7fce08102f50 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce06ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce081038f0 0x7fce08113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:09.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce06ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce081038f0 0x7fce08113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34776/0 (socket says 172.21.15.64:34776) 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce06ffd700 1 -- 172.21.15.64:0/1193654375 learned_addr learned my addr 172.21.15.64:0/1193654375 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce077fe700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce08102f50 0x7fce08113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce06ffd700 1 -- 172.21.15.64:0/1193654375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce08102f50 msgr2=0x7fce08113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce06ffd700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce08102f50 0x7fce08113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce06ffd700 1 -- 172.21.15.64:0/1193654375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8005040 con 0x7fce081038f0 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce06ffd700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce081038f0 0x7fce08113fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc002a50 tx=0x7fcdfc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fcdeffff700 1 -- 172.21.15.64:0/1193654375 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc007cc0 con 0x7fce081038f0 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0810dc00 con 0x7fce081038f0 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fcdeffff700 1 -- 172.21.15.64:0/1193654375 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdfc00f040 con 0x7fce081038f0 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fcdeffff700 1 -- 172.21.15.64:0/1193654375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc00d9c0 con 0x7fce081038f0 2024-02-16T23:08:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.984+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0810e140 con 0x7fce081038f0 2024-02-16T23:08:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.985+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce08100260 con 0x7fce081038f0 2024-02-16T23:08:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.986+0000 7fcdeffff700 1 -- 172.21.15.64:0/1193654375 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcdfc00db20 con 0x7fce081038f0 2024-02-16T23:08:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.986+0000 7fcdeffff700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcdf006d7e0 0x7fcdf006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.986+0000 7fcdeffff700 1 -- 172.21.15.64:0/1193654375 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcdfc08cd40 con 0x7fce081038f0 2024-02-16T23:08:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.986+0000 7fce077fe700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcdf006d7e0 0x7fcdf006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.987+0000 7fce077fe700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcdf006d7e0 0x7fcdf006fc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8004f90 tx=0x7fcdf8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:09.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:09.991+0000 7fcdeffff700 1 -- 172.21.15.64:0/1193654375 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdfc090030 con 0x7fce081038f0 2024-02-16T23:08:10.086 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:09 smithi192 ceph-mon[82652]: pgmap v311: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:10.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:09 smithi064 ceph-mon[93757]: pgmap v311: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.235+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce0804bed0 con 0x7fce081038f0 2024-02-16T23:08:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.236+0000 7fcdeffff700 1 -- 172.21.15.64:0/1193654375 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fcdfc059d30 con 0x7fce081038f0 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:e35 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 35 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-02-16T22:53:41.360879+0000 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-02-16T23:04:58.106175+0000 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-02-16T23:08:10.236 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 60 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 85 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 1 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:in 0 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=34404} 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-02-16T23:08:10.237 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data enabled 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 1 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.tqesxx{0:34404} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.64:6826/2289872563,v1:172.21.15.64:6827/2289872563] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi064.bmcyuf{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.64:6828/2551190588,v1:172.21.15.64:6829/2551190588] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.tnppti{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/2028541376,v1:172.21.15.192:6825/2028541376] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:08:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.cephfs.smithi192.luiybj{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/1878053024,v1:172.21.15.192:6827/1878053024] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.237+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcdf006d7e0 msgr2=0x7fcdf006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.237+0000 7fce0db5b700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcdf006d7e0 0x7fcdf006fc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8004f90 tx=0x7fcdf8009e30 comp rx=0 tx=0).stop 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce081038f0 msgr2=0x7fce08113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce081038f0 0x7fce08113fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc002a50 tx=0x7fcdfc004600 comp rx=0 tx=0).stop 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 shutdown_connections 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcdf006d7e0 0x7fcdf006fc90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce08102f50 0x7fce08113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 --2- 172.21.15.64:0/1193654375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce081038f0 0x7fce08113fa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 >> 172.21.15.64:0/1193654375 conn(0x7fce080f8ae0 msgr2=0x7fce080fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 shutdown_connections 2024-02-16T23:08:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:10.238+0000 7fce0db5b700 1 -- 172.21.15.64:0/1193654375 wait complete. 2024-02-16T23:08:10.240 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2024-02-16T23:08:10.936 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-16T23:08:10.947 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T23:08:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 -- 172.21.15.64:0/3663736194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc101b40 msgr2=0x7f05dc0fb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 --2- 172.21.15.64:0/3663736194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc101b40 0x7f05dc0fb230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f05c80048d0 tx=0x7f05c8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 -- 172.21.15.64:0/3663736194 shutdown_connections 2024-02-16T23:08:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 --2- 172.21.15.64:0/3663736194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc101b40 0x7f05dc0fb230 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 --2- 172.21.15.64:0/3663736194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05dc1011a0 0x7f05dc101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 -- 172.21.15.64:0/3663736194 >> 172.21.15.64:0/3663736194 conn(0x7f05dc071290 msgr2=0x7f05dc071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 -- 172.21.15.64:0/3663736194 shutdown_connections 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 -- 172.21.15.64:0/3663736194 wait complete. 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 Processor -- start 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.049+0000 7f05e2b9d640 1 -- start start 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e2b9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc1011a0 0x7f05dc10dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e2b9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05dc101b40 0x7f05dc10e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e2b9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc10e720 con 0x7f05dc1011a0 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e2b9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc10e890 con 0x7f05dc101b40 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e0939640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc1011a0 0x7f05dc10dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e0939640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc1011a0 0x7f05dc10dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34746/0 (socket says 172.21.15.64:34746) 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e0939640 1 -- 172.21.15.64:0/2609814559 learned_addr learned my addr 172.21.15.64:0/2609814559 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05dbfff640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05dc101b40 0x7f05dc10e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e0939640 1 -- 172.21.15.64:0/2609814559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05dc101b40 msgr2=0x7f05dc10e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e0939640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05dc101b40 0x7f05dc10e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e0939640 1 -- 172.21.15.64:0/2609814559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8005040 con 0x7f05dc1011a0 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05dbfff640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05dc101b40 0x7f05dc10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.050+0000 7f05e0939640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc1011a0 0x7f05dc10dca0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f05d000eb70 tx=0x7f05d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.051+0000 7f05d97fa640 1 -- 172.21.15.64:0/2609814559 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05d0007c30 con 0x7f05dc1011a0 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.051+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc10eb10 con 0x7f05dc1011a0 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.051+0000 7f05d97fa640 1 -- 172.21.15.64:0/2609814559 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05d000f040 con 0x7f05dc1011a0 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.051+0000 7f05d97fa640 1 -- 172.21.15.64:0/2609814559 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05d0009680 con 0x7f05dc1011a0 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.051+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc117010 con 0x7f05dc1011a0 2024-02-16T23:08:11.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.052+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05dc064030 con 0x7f05dc1011a0 2024-02-16T23:08:11.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.052+0000 7f05d97fa640 1 -- 172.21.15.64:0/2609814559 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f05d000cd40 con 0x7f05dc1011a0 2024-02-16T23:08:11.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.052+0000 7f05d97fa640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05cc06d750 0x7f05cc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.053+0000 7f05d97fa640 1 -- 172.21.15.64:0/2609814559 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f05d008cb00 con 0x7f05dc1011a0 2024-02-16T23:08:11.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.053+0000 7f05dbfff640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05cc06d750 0x7f05cc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.053+0000 7f05dbfff640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05cc06d750 0x7f05cc06fc00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f05c802bda0 tx=0x7f05c8009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:11.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.057+0000 7f05d97fa640 1 -- 172.21.15.64:0/2609814559 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05d0056300 con 0x7f05dc1011a0 2024-02-16T23:08:11.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:10 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1193654375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:08:11.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.239+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05dc10eab0 con 0x7f05dc1011a0 2024-02-16T23:08:11.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.240+0000 7f05d97fa640 1 -- 172.21.15.64:0/2609814559 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16913 (secure 0 0 0) 0x7f05d00599c0 con 0x7f05dc1011a0 2024-02-16T23:08:11.241 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:11.241 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","created":"2024-02-16T22:46:56.779041+0000","modified":"2024-02-16T23:04:47.133740+0000","last_up_change":"2024-02-16T23:03:26.379108+0000","last_in_change":"2024-02-16T22:52:04.211979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:51:12.402108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T22:53:40.294617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T22:53:40.597634+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7a680d8c-0559-46c6-9bfc-95fca252a657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6803","nonce":3681251107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6805","nonce":3681251107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6809","nonce":3681251107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6807","nonce":3681251107}]},"public_addr":"172.21.15.64:6803/3681251107","cluster_addr":"172.21.15.64:6805/3681251107","heartbeat_back_addr":"172.21.15.64:6809/3681251107","heartbeat_front_addr":"172.21.15.64:6807/3681251107","state":["exists","up"]},{"osd":1,"uuid":"21a8757d-9f77-4f46-83d3-ff904c9e50b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6811","nonce":1635803238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6813","nonce":1635803238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6817","nonce":1635803238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6815","nonce":1635803238}]},"public_addr":"172.21.15.64:6811/1635803238","cluster_addr":"172.21.15.64:6813/1635803238","heartbeat_back_addr":"172.21.15.64:6817/1635803238","heartbeat_front_addr":"172.21.15.64:6815/1635803238","state":["exists","up"]},{"osd":2,"uuid":"3fcb2e12-54a0-46db-af11-d047cf25d50b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6819","nonce":530939834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6821","nonce":530939834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6825","nonce":530939834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6823","nonce":530939834}]},"public_addr":"172.21.15.64:6819/530939834","cluster_addr":"172.21.15.64:6821/530939834","heartbeat_back_addr":"172.21.15.64:6825/530939834","heartbeat_front_addr":"172.21.15.64:6823/530939834","state":["exists","up"]},{"osd":3,"uuid":"a7a58c77-96fa-4835-9ea4-d16e6d995620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6801","nonce":2367052696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6803","nonce":2367052696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6807","nonce":2367052696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6805","nonce":2367052696}]},"public_addr":"172.21.15.192:6801/2367052696","cluster_addr":"172.21.15.192:6803/2367052696","heartbeat_back_addr":"172.21.15.192:6807/2367052696","heartbeat_front_addr":"172.21.15.192:6805/2367052696","state":["exists","up"]},{"osd":4,"uuid":"04e094c5-2494-457a-a4a5-7b2742be9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6809","nonce":2408603569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6811","nonce":2408603569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6815","nonce":2408603569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6813","nonce":2408603569}]},"public_addr":"172.21.15.192:6809/2408603569","cluster_addr":"172.21.15.192:6811/2408603569","heartbeat_back_addr":"172.21.15.192:6815/2408603569","heartbeat_front_addr":"172.21.15.192:6813/2408603569","state":["exists","up"]},{"osd":5,"uuid":"bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6817","nonce":2198637799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6819","nonce":2198637799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6823","nonce":2198637799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6821","nonce":2198637799}]},"public_addr":"172.21.15.192:6817/2198637799","cluster_addr":"172.21.15.192:6819/2198637799","heartbeat_back_addr":"172.21.15.192:6823/2198637799","heartbeat_front_addr":"172.21.15.192:6821/2198637799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:14.481189+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:42.243983+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:09.715768+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:32.358421+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:56.431975+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:52:21.204162+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6824/1462436420":"2024-02-17T23:04:23.398629+0000","172.21.15.64:6829/168979118":"2024-02-17T23:03:59.013169+0000","172.21.15.64:6827/3220035256":"2024-02-17T23:03:32.044676+0000","172.21.15.192:6826/2853522025":"2024-02-17T23:04:47.133716+0000","172.21.15.64:0/1250745016":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1538951056":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1222682641":"2024-02-17T22:48:58.067040+0000","172.21.15.192:6828/2862437082":"2024-02-17T22:58:16.122252+0000","172.21.15.64:6800/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6801/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.192:0/2602543520":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/407404719":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1881977004":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1804571669":"2024-02-17T22:47:54.261948+0000","172.21.15.192:0/733087388":"2024-02-17T22:58:16.122252+0000","172.21.15.192:0/1972327284":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/2780081097":"2024-02-17T22:55:55.330401+0000","172.21.15.64:0/4258467456":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6828/168979118":"2024-02-17T23:03:59.013169+0000","172.21.15.64:0/4290875509":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1046482636":"2024-02-17T22:55:55.330401+0000","172.21.15.192:6829/2862437082":"2024-02-17T22:58:16.122252+0000","172.21.15.192:0/3116412139":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/3827352044":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/3005970232":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/700343021":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6800/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/743141814":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6801/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:6800/700343021":"2024-02-17T22:55:55.330401+0000","172.21.15.64:0/4194870513":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1988317603":"2024-02-17T22:59:02.601780+0000","172.21.15.64:6801/1223017301":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1580822017":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1671562709":"2024-02-17T22:59:02.601780+0000","172.21.15.192:6827/2853522025":"2024-02-17T23:04:47.133716+0000","172.21.15.192:6825/1462436420":"2024-02-17T23:04:23.398629+0000","172.21.15.64:6800/1223017301":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/347371753":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/741447403":"2024-02-17T22:59:02.601780+0000","172.21.15.64:6800/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6826/3220035256":"2024-02-17T23:03:32.044676+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T23:08:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.242+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05cc06d750 msgr2=0x7f05cc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.242+0000 7f05e2b9d640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05cc06d750 0x7f05cc06fc00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f05c802bda0 tx=0x7f05c8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.242+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc1011a0 msgr2=0x7f05dc10dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.242+0000 7f05e2b9d640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc1011a0 0x7f05dc10dca0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f05d000eb70 tx=0x7f05d000b040 comp rx=0 tx=0).stop 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.243+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 shutdown_connections 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.243+0000 7f05e2b9d640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05cc06d750 0x7f05cc06fc00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.243+0000 7f05e2b9d640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05dc101b40 0x7f05dc10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.243+0000 7f05e2b9d640 1 --2- 172.21.15.64:0/2609814559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc1011a0 0x7f05dc10dca0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.243+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 >> 172.21.15.64:0/2609814559 conn(0x7f05dc071290 msgr2=0x7f05dc0fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.243+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 shutdown_connections 2024-02-16T23:08:11.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.243+0000 7f05e2b9d640 1 -- 172.21.15.64:0/2609814559 wait complete. 2024-02-16T23:08:11.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:51:12.402108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T22:53:40.294617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T22:53:40.597634+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T23:08:11.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T23:08:11.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:10 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1193654375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T23:08:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.356+0000 7fcadb91d640 1 -- 172.21.15.64:0/3106121593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 msgr2=0x7fcad41027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.356+0000 7fcadb91d640 1 --2- 172.21.15.64:0/3106121593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 0x7fcad41027d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcad00048d0 tx=0x7fcad0009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.356+0000 7fcadb91d640 1 -- 172.21.15.64:0/3106121593 shutdown_connections 2024-02-16T23:08:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.356+0000 7fcadb91d640 1 --2- 172.21.15.64:0/3106121593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4102d10 0x7fcad40faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.356+0000 7fcadb91d640 1 --2- 172.21.15.64:0/3106121593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 0x7fcad41027d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.356+0000 7fcadb91d640 1 -- 172.21.15.64:0/3106121593 >> 172.21.15.64:0/3106121593 conn(0x7fcad4070a10 msgr2=0x7fcad4070e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.356+0000 7fcadb91d640 1 -- 172.21.15.64:0/3106121593 shutdown_connections 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.356+0000 7fcadb91d640 1 -- 172.21.15.64:0/3106121593 wait complete. 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcadb91d640 1 Processor -- start 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcadb91d640 1 -- start start 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcadb91d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 0x7fcad4112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcadb91d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4102d10 0x7fcad4113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcadb91d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4113740 con 0x7fcad4102400 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcadb91d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad410ccd0 con 0x7fcad4102d10 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcad96b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 0x7fcad4112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcad96b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 0x7fcad4112c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34778/0 (socket says 172.21.15.64:34778) 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcad96b9640 1 -- 172.21.15.64:0/1597848785 learned_addr learned my addr 172.21.15.64:0/1597848785 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcad8eb8640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4102d10 0x7fcad4113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcad96b9640 1 -- 172.21.15.64:0/1597848785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4102d10 msgr2=0x7fcad4113140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcad96b9640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4102d10 0x7fcad4113140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.357+0000 7fcad96b9640 1 -- 172.21.15.64:0/1597848785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad0005040 con 0x7fcad4102400 2024-02-16T23:08:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.358+0000 7fcad96b9640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 0x7fcad4112c00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcad00043d0 tx=0x7fcad0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.358+0000 7fcac9ffb640 1 -- 172.21.15.64:0/1597848785 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcad0030e60 con 0x7fcad4102400 2024-02-16T23:08:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.358+0000 7fcac9ffb640 1 -- 172.21.15.64:0/1597848785 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcad00092e0 con 0x7fcad4102400 2024-02-16T23:08:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.358+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad410cf50 con 0x7fcad4102400 2024-02-16T23:08:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.358+0000 7fcac9ffb640 1 -- 172.21.15.64:0/1597848785 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcad003a5f0 con 0x7fcad4102400 2024-02-16T23:08:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.358+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad410d470 con 0x7fcad4102400 2024-02-16T23:08:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.359+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad4064480 con 0x7fcad4102400 2024-02-16T23:08:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.359+0000 7fcac9ffb640 1 -- 172.21.15.64:0/1597848785 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcad0009450 con 0x7fcad4102400 2024-02-16T23:08:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.360+0000 7fcac9ffb640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcac006d750 0x7fcac006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.360+0000 7fcad8eb8640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcac006d750 0x7fcac006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.360+0000 7fcac9ffb640 1 -- 172.21.15.64:0/1597848785 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcad00afc50 con 0x7fcad4102400 2024-02-16T23:08:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.360+0000 7fcad8eb8640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcac006d750 0x7fcac006fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcac4007fd0 tx=0x7fcac4009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:11.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.365+0000 7fcac9ffb640 1 -- 172.21.15.64:0/1597848785 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcad00794e0 con 0x7fcad4102400 2024-02-16T23:08:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.544+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcad410e230 con 0x7fcad4102400 2024-02-16T23:08:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.544+0000 7fcac9ffb640 1 -- 172.21.15.64:0/1597848785 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcad007cba0 con 0x7fcad4102400 2024-02-16T23:08:11.544 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcac006d750 msgr2=0x7fcac006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcac006d750 0x7fcac006fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcac4007fd0 tx=0x7fcac4009e90 comp rx=0 tx=0).stop 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 msgr2=0x7fcad4112c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 0x7fcad4112c00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcad00043d0 tx=0x7fcad0009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 shutdown_connections 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcac006d750 0x7fcac006fc00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4102d10 0x7fcad4113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 --2- 172.21.15.64:0/1597848785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcad4102400 0x7fcad4112c00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 >> 172.21.15.64:0/1597848785 conn(0x7fcad4070a10 msgr2=0x7fcad40f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 shutdown_connections 2024-02-16T23:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.547+0000 7fcadb91d640 1 -- 172.21.15.64:0/1597848785 wait complete. 2024-02-16T23:08:11.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T23:08:11.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 -- 172.21.15.64:0/3965966950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 msgr2=0x7f8bb01027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 --2- 172.21.15.64:0/3965966950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 0x7f8bb01027c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00048d0 tx=0x7f8ba0009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 -- 172.21.15.64:0/3965966950 shutdown_connections 2024-02-16T23:08:11.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 --2- 172.21.15.64:0/3965966950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb00faa60 0x7f8bb00faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 --2- 172.21.15.64:0/3965966950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 0x7f8bb01027c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 -- 172.21.15.64:0/3965966950 >> 172.21.15.64:0/3965966950 conn(0x7f8bb0071880 msgr2=0x7f8bb0071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:11.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 -- 172.21.15.64:0/3965966950 shutdown_connections 2024-02-16T23:08:11.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 -- 172.21.15.64:0/3965966950 wait complete. 2024-02-16T23:08:11.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 Processor -- start 2024-02-16T23:08:11.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.659+0000 7f8bb6807640 1 -- start start 2024-02-16T23:08:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bb6807640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb00faa60 0x7f8bb0112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bb6807640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 0x7f8bb0113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bb6807640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0113770 con 0x7f8bb01023f0 2024-02-16T23:08:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bb6807640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb010cd00 con 0x7f8bb00faa60 2024-02-16T23:08:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 0x7f8bb0113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 0x7f8bb0113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34792/0 (socket says 172.21.15.64:34792) 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baf7fe640 1 -- 172.21.15.64:0/1629998170 learned_addr learned my addr 172.21.15.64:0/1629998170 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baffff640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb00faa60 0x7f8bb0112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baf7fe640 1 -- 172.21.15.64:0/1629998170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb00faa60 msgr2=0x7f8bb0112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baf7fe640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb00faa60 0x7f8bb0112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baf7fe640 1 -- 172.21.15.64:0/1629998170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0005040 con 0x7f8bb01023f0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baffff640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb00faa60 0x7f8bb0112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8baf7fe640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 0x7f8bb0113170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b98002a50 tx=0x7f8b98004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bacff9640 1 -- 172.21.15.64:0/1629998170 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b98007c30 con 0x7f8bb01023f0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bacff9640 1 -- 172.21.15.64:0/1629998170 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b9800f040 con 0x7f8bb01023f0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb010cfe0 con 0x7f8bb01023f0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bacff9640 1 -- 172.21.15.64:0/1629998170 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b9800db50 con 0x7f8bb01023f0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.660+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb010d500 con 0x7f8bb01023f0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.661+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb0064480 con 0x7f8bb01023f0 2024-02-16T23:08:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.662+0000 7f8bacff9640 1 -- 172.21.15.64:0/1629998170 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8b9801c400 con 0x7f8bb01023f0 2024-02-16T23:08:11.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.662+0000 7f8bacff9640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8b9c06d750 0x7f8b9c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.662+0000 7f8bacff9640 1 -- 172.21.15.64:0/1629998170 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8b9808c8b0 con 0x7f8bb01023f0 2024-02-16T23:08:11.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.662+0000 7f8baffff640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8b9c06d750 0x7f8b9c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.663+0000 7f8baffff640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8b9c06d750 0x7f8b9c06fc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0000c00 tx=0x7f8ba0009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.667+0000 7f8bacff9640 1 -- 172.21.15.64:0/1629998170 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b98056000 con 0x7f8bb01023f0 2024-02-16T23:08:11.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.848+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f8bb010d720 con 0x7f8bb01023f0 2024-02-16T23:08:11.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.849+0000 7f8bacff9640 1 -- 172.21.15.64:0/1629998170 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8b980596c0 con 0x7f8bb01023f0 2024-02-16T23:08:11.849 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-02-16T23:08:11.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8b9c06d750 msgr2=0x7f8b9c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8b9c06d750 0x7f8b9c06fc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0000c00 tx=0x7f8ba0009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 msgr2=0x7f8bb0113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 0x7f8bb0113170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b98002a50 tx=0x7f8b98004600 comp rx=0 tx=0).stop 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 shutdown_connections 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb01023f0 0x7f8bb0113170 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8b9c06d750 0x7f8b9c06fc00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 --2- 172.21.15.64:0/1629998170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb00faa60 0x7f8bb0112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.850+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 >> 172.21.15.64:0/1629998170 conn(0x7f8bb0071880 msgr2=0x7f8bb00f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.851+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 shutdown_connections 2024-02-16T23:08:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.851+0000 7f8bb6807640 1 -- 172.21.15.64:0/1629998170 wait complete. 2024-02-16T23:08:11.860 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:11 smithi064 ceph-mon[93757]: pgmap v312: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:11.860 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:11 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2609814559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T23:08:11.860 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:11 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1597848785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T23:08:11.861 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T23:08:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.961+0000 7f805502e640 1 -- 172.21.15.64:0/1576736684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 msgr2=0x7f80500fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.961+0000 7f805502e640 1 --2- 172.21.15.64:0/1576736684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 0x7f80500fb700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f80400048d0 tx=0x7f8040009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.961+0000 7f805502e640 1 -- 172.21.15.64:0/1576736684 shutdown_connections 2024-02-16T23:08:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.961+0000 7f805502e640 1 --2- 172.21.15.64:0/1576736684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 0x7f80500fb700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.961+0000 7f805502e640 1 --2- 172.21.15.64:0/1576736684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80501012d0 0x7f80501016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.961+0000 7f805502e640 1 -- 172.21.15.64:0/1576736684 >> 172.21.15.64:0/1576736684 conn(0x7f8050071230 msgr2=0x7f8050071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.961+0000 7f805502e640 1 -- 172.21.15.64:0/1576736684 shutdown_connections 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f805502e640 1 -- 172.21.15.64:0/1576736684 wait complete. 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f805502e640 1 Processor -- start 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f805502e640 1 -- start start 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f805502e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 0x7f8050112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f805502e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80501012d0 0x7f8050113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f805502e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8050113820 con 0x7f80500fb2b0 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f805502e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805010cd20 con 0x7f80501012d0 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f804ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 0x7f8050112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f804ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 0x7f8050112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34800/0 (socket says 172.21.15.64:34800) 2024-02-16T23:08:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.962+0000 7f804ed9d640 1 -- 172.21.15.64:0/1330017480 learned_addr learned my addr 172.21.15.64:0/1330017480 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f804e59c640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80501012d0 0x7f8050113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f804ed9d640 1 -- 172.21.15.64:0/1330017480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80501012d0 msgr2=0x7f8050113190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f804ed9d640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80501012d0 0x7f8050113190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f804ed9d640 1 -- 172.21.15.64:0/1330017480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040005040 con 0x7f80500fb2b0 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f804e59c640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80501012d0 0x7f8050113190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f804ed9d640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 0x7f8050112c50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f803800b790 tx=0x7f80380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f80477fe640 1 -- 172.21.15.64:0/1330017480 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f803800c700 con 0x7f80500fb2b0 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f80477fe640 1 -- 172.21.15.64:0/1330017480 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f803800cd30 con 0x7f80500fb2b0 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805010d000 con 0x7f80500fb2b0 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f80477fe640 1 -- 172.21.15.64:0/1330017480 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8038013370 con 0x7f80500fb2b0 2024-02-16T23:08:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.963+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805010d520 con 0x7f80500fb2b0 2024-02-16T23:08:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.964+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8050064030 con 0x7f80500fb2b0 2024-02-16T23:08:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.964+0000 7f80477fe640 1 -- 172.21.15.64:0/1330017480 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8038012070 con 0x7f80500fb2b0 2024-02-16T23:08:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.965+0000 7f80477fe640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f803c06d760 0x7f803c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.965+0000 7f80477fe640 1 -- 172.21.15.64:0/1330017480 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f803808cc20 con 0x7f80500fb2b0 2024-02-16T23:08:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.965+0000 7f804e59c640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f803c06d760 0x7f803c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.965+0000 7f804e59c640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f803c06d760 0x7f803c06fc10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f804002bda0 tx=0x7f8040009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:11.969+0000 7f80477fe640 1 -- 172.21.15.64:0/1330017480 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8038056370 con 0x7f80500fb2b0 2024-02-16T23:08:12.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.149+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f8050108420 con 0x7f80500fb2b0 2024-02-16T23:08:12.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.149+0000 7f80477fe640 1 -- 172.21.15.64:0/1330017480 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8038059a30 con 0x7f80500fb2b0 2024-02-16T23:08:12.149 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-02-16T23:08:12.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f803c06d760 msgr2=0x7f803c06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f803c06d760 0x7f803c06fc10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f804002bda0 tx=0x7f8040009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 msgr2=0x7f8050112c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 0x7f8050112c50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f803800b790 tx=0x7f80380091d0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 shutdown_connections 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f803c06d760 0x7f803c06fc10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80501012d0 0x7f8050113190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 --2- 172.21.15.64:0/1330017480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80500fb2b0 0x7f8050112c50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 >> 172.21.15.64:0/1330017480 conn(0x7f8050071230 msgr2=0x7f80500644e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 shutdown_connections 2024-02-16T23:08:12.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.151+0000 7f805502e640 1 -- 172.21.15.64:0/1330017480 wait complete. 2024-02-16T23:08:12.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-16T23:08:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 -- 172.21.15.64:0/1061393654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 msgr2=0x7f3278101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 --2- 172.21.15.64:0/1061393654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 0x7f3278101870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f32740048d0 tx=0x7f3274009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 -- 172.21.15.64:0/1061393654 shutdown_connections 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 --2- 172.21.15.64:0/1061393654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32780fb460 0x7f32780fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 --2- 172.21.15.64:0/1061393654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 0x7f3278101870 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 -- 172.21.15.64:0/1061393654 >> 172.21.15.64:0/1061393654 conn(0x7f32780f6fe0 msgr2=0x7f32780f93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 -- 172.21.15.64:0/1061393654 shutdown_connections 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 -- 172.21.15.64:0/1061393654 wait complete. 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 Processor -- start 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 -- start start 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32780fb460 0x7f327810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 0x7f327810e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327810e940 con 0x7f32781014a0 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.261+0000 7f327f79d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327810eab0 con 0x7f32780fb460 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327cd38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 0x7f327810e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327cd38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 0x7f327810e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34820/0 (socket says 172.21.15.64:34820) 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327cd38640 1 -- 172.21.15.64:0/2326936903 learned_addr learned my addr 172.21.15.64:0/2326936903 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327d539640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32780fb460 0x7f327810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327cd38640 1 -- 172.21.15.64:0/2326936903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32780fb460 msgr2=0x7f327810dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327cd38640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32780fb460 0x7f327810dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327cd38640 1 -- 172.21.15.64:0/2326936903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3274005040 con 0x7f32781014a0 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327d539640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32780fb460 0x7f327810dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327cd38640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 0x7f327810e400 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f326800b790 tx=0x7f32680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f326dffb640 1 -- 172.21.15.64:0/2326936903 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326800c700 con 0x7f32781014a0 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f326dffb640 1 -- 172.21.15.64:0/2326936903 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f326800cd30 con 0x7f32781014a0 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3278116f30 con 0x7f32781014a0 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f326dffb640 1 -- 172.21.15.64:0/2326936903 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3268013370 con 0x7f32781014a0 2024-02-16T23:08:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.262+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32781172b0 con 0x7f32781014a0 2024-02-16T23:08:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.263+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3278064480 con 0x7f32781014a0 2024-02-16T23:08:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.264+0000 7f326dffb640 1 -- 172.21.15.64:0/2326936903 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3268012070 con 0x7f32781014a0 2024-02-16T23:08:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.264+0000 7f326dffb640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f326406d750 0x7f326406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.264+0000 7f326dffb640 1 -- 172.21.15.64:0/2326936903 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f326808cc20 con 0x7f32781014a0 2024-02-16T23:08:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.264+0000 7f327d539640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f326406d750 0x7f326406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.264+0000 7f327d539640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f326406d750 0x7f326406fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f327402bda0 tx=0x7f3274009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:12.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.269+0000 7f326dffb640 1 -- 172.21.15.64:0/2326936903 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3268056370 con 0x7f32781014a0 2024-02-16T23:08:12.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:11 smithi192 ceph-mon[82652]: pgmap v312: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:12.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:11 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2609814559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T23:08:12.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:11 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1597848785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T23:08:12.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.490+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32781085f0 con 0x7f32781014a0 2024-02-16T23:08:12.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.491+0000 7f326dffb640 1 -- 172.21.15.64:0/2326936903 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5002 (secure 0 0 0) 0x7f3268059a30 con 0x7f32781014a0 2024-02-16T23:08:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:12.491 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44349,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/2028541376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2028541376},{"type":"v1","addr":"172.21.15.192:6825","nonce":2028541376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44353,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/1878053024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1878053024},{"type":"v1","addr":"172.21.15.192:6827","nonce":1878053024}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:58.106175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":0,"incarnation":32,"state":"up:active","state_seq":21,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:12.491 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2024-02-16T23:08:12.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f326406d750 msgr2=0x7f326406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f326406d750 0x7f326406fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f327402bda0 tx=0x7f3274009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 msgr2=0x7f327810e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 0x7f327810e400 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f326800b790 tx=0x7f32680091d0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 shutdown_connections 2024-02-16T23:08:12.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32781014a0 0x7f327810e400 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f326406d750 0x7f326406fc00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 --2- 172.21.15.64:0/2326936903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32780fb460 0x7f327810dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 >> 172.21.15.64:0/2326936903 conn(0x7f32780f6fe0 msgr2=0x7f32780f7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:12.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 shutdown_connections 2024-02-16T23:08:12.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.493+0000 7f327f79d640 1 -- 172.21.15.64:0/2326936903 wait complete. 2024-02-16T23:08:12.503 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 10, 'max_mds': 1, 'flags': 18} 2024-02-16T23:08:12.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 11 2024-02-16T23:08:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 -- 172.21.15.64:0/848583908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101240 msgr2=0x7f9550101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 --2- 172.21.15.64:0/848583908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101240 0x7f9550101610 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f95400048d0 tx=0x7f9540009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 -- 172.21.15.64:0/848583908 shutdown_connections 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 --2- 172.21.15.64:0/848583908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550101be0 0x7f95500fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 --2- 172.21.15.64:0/848583908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101240 0x7f9550101610 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 -- 172.21.15.64:0/848583908 >> 172.21.15.64:0/848583908 conn(0x7f95500f6e10 msgr2=0x7f95500f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 -- 172.21.15.64:0/848583908 shutdown_connections 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 -- 172.21.15.64:0/848583908 wait complete. 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 Processor -- start 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.604+0000 7f9555407640 1 -- start start 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f9555407640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550101240 0x7f955010dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f9555407640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101be0 0x7f955010e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f9555407640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955010e7e0 con 0x7f9550101be0 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f9555407640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955010e950 con 0x7f9550101240 2024-02-16T23:08:12.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f954e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101be0 0x7f955010e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f954e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101be0 0x7f955010e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34838/0 (socket says 172.21.15.64:34838) 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f954e7fc640 1 -- 172.21.15.64:0/3511964531 learned_addr learned my addr 172.21.15.64:0/3511964531 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f954e7fc640 1 -- 172.21.15.64:0/3511964531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550101240 msgr2=0x7f955010dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f954e7fc640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550101240 0x7f955010dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f954e7fc640 1 -- 172.21.15.64:0/3511964531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540005040 con 0x7f9550101be0 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f954e7fc640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101be0 0x7f955010e2a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f953800b790 tx=0x7f95380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f95477fe640 1 -- 172.21.15.64:0/3511964531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953800c700 con 0x7f9550101be0 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f95477fe640 1 -- 172.21.15.64:0/3511964531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f953800cd30 con 0x7f9550101be0 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9550116dd0 con 0x7f9550101be0 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f95477fe640 1 -- 172.21.15.64:0/3511964531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9538013370 con 0x7f9550101be0 2024-02-16T23:08:12.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.605+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95501171b0 con 0x7f9550101be0 2024-02-16T23:08:12.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.606+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9550064480 con 0x7f9550101be0 2024-02-16T23:08:12.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.607+0000 7f95477fe640 1 -- 172.21.15.64:0/3511964531 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9538012070 con 0x7f9550101be0 2024-02-16T23:08:12.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.607+0000 7f95477fe640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f953c06d750 0x7f953c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.607+0000 7f95477fe640 1 -- 172.21.15.64:0/3511964531 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f953808cbc0 con 0x7f9550101be0 2024-02-16T23:08:12.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.607+0000 7f954effd640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f953c06d750 0x7f953c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:12.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.608+0000 7f954effd640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f953c06d750 0x7f953c06fc00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9540034040 tx=0x7f9540009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:12.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.612+0000 7f95477fe640 1 -- 172.21.15.64:0/3511964531 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9538056310 con 0x7f9550101be0 2024-02-16T23:08:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.837+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 11, "format": "json"} v 0) v1 -- 0x7f9550108400 con 0x7f9550101be0 2024-02-16T23:08:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.837+0000 7f95477fe640 1 -- 172.21.15.64:0/3511964531 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 11, "format": "json"}]=0 dumped fsmap epoch 11 v35) v1 ==== 107+0+4827 (secure 0 0 0) 0x7f95380599d0 con 0x7f9550101be0 2024-02-16T23:08:12.837 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:12.837 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T22:53:56.166293+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi064.tqesxx","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6827/3220035256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3220035256},{"type":"v1","addr":"172.21.15.64:6827","nonce":3220035256}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 11 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.839+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f953c06d750 msgr2=0x7f953c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.839+0000 7f9555407640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f953c06d750 0x7f953c06fc00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9540034040 tx=0x7f9540009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.839+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101be0 msgr2=0x7f955010e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.839+0000 7f9555407640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101be0 0x7f955010e2a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f953800b790 tx=0x7f95380091d0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.840+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 shutdown_connections 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.840+0000 7f9555407640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9550101be0 0x7f955010e2a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.840+0000 7f9555407640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f953c06d750 0x7f953c06fc00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.840+0000 7f9555407640 1 --2- 172.21.15.64:0/3511964531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550101240 0x7f955010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.840+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 >> 172.21.15.64:0/3511964531 conn(0x7f95500f6e10 msgr2=0x7f95500f8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.840+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 shutdown_connections 2024-02-16T23:08:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.840+0000 7f9555407640 1 -- 172.21.15.64:0/3511964531 wait complete. 2024-02-16T23:08:12.850 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 12 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.950+0000 7fb2d939d640 1 -- 172.21.15.64:0/1078685758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 msgr2=0x7fb2d40fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.950+0000 7fb2d939d640 1 --2- 172.21.15.64:0/1078685758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 0x7fb2d40fd590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40048d0 tx=0x7fb2c4009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.950+0000 7fb2d939d640 1 -- 172.21.15.64:0/1078685758 shutdown_connections 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.950+0000 7fb2d939d640 1 --2- 172.21.15.64:0/1078685758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d40fdad0 0x7fb2d40fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.950+0000 7fb2d939d640 1 --2- 172.21.15.64:0/1078685758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 0x7fb2d40fd590 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.950+0000 7fb2d939d640 1 -- 172.21.15.64:0/1078685758 >> 172.21.15.64:0/1078685758 conn(0x7fb2d4071950 msgr2=0x7fb2d4071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.950+0000 7fb2d939d640 1 -- 172.21.15.64:0/1078685758 shutdown_connections 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.950+0000 7fb2d939d640 1 -- 172.21.15.64:0/1078685758 wait complete. 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d939d640 1 Processor -- start 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d939d640 1 -- start start 2024-02-16T23:08:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d939d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 0x7fb2d4114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d939d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d40fdad0 0x7fb2d4115450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d939d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4115ae0 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d939d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d410efe0 con 0x7fb2d40fdad0 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 0x7fb2d4114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 0x7fb2d4114f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34852/0 (socket says 172.21.15.64:34852) 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d2ffd640 1 -- 172.21.15.64:0/2977923766 learned_addr learned my addr 172.21.15.64:0/2977923766 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d27fc640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d40fdad0 0x7fb2d4115450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d2ffd640 1 -- 172.21.15.64:0/2977923766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d40fdad0 msgr2=0x7fb2d4115450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d2ffd640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d40fdad0 0x7fb2d4115450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.951+0000 7fb2d2ffd640 1 -- 172.21.15.64:0/2977923766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4005040 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.952+0000 7fb2d2ffd640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 0x7fb2d4114f10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000c00 tx=0x7fb2c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.952+0000 7fb2bb7fe640 1 -- 172.21.15.64:0/2977923766 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2c40092e0 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.952+0000 7fb2bb7fe640 1 -- 172.21.15.64:0/2977923766 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2c4009440 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.952+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d410f260 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.952+0000 7fb2bb7fe640 1 -- 172.21.15.64:0/2977923766 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2c4043640 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.952+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d410f780 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.953+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d4064480 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.953+0000 7fb2bb7fe640 1 -- 172.21.15.64:0/2977923766 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb2c403b420 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.953+0000 7fb2bb7fe640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2bc06d750 0x7fb2bc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.954+0000 7fb2d27fc640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2bc06d750 0x7fb2bc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.954+0000 7fb2bb7fe640 1 -- 172.21.15.64:0/2977923766 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb2c40afe10 con 0x7fb2d40fd1c0 2024-02-16T23:08:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.954+0000 7fb2d27fc640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2bc06d750 0x7fb2bc06fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006fd0 tx=0x7fb2c8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:12.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:12.958+0000 7fb2bb7fe640 1 -- 172.21.15.64:0/2977923766 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2c40795e0 con 0x7fb2d40fd1c0 2024-02-16T23:08:13.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:12 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1629998170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T23:08:13.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:12 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1330017480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T23:08:13.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:12 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2326936903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T23:08:13.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.179+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 12, "format": "json"} v 0) v1 -- 0x7fb2d410f9a0 con 0x7fb2d40fd1c0 2024-02-16T23:08:13.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.180+0000 7fb2bb7fe640 1 -- 172.21.15.64:0/2977923766 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 12, "format": "json"}]=0 dumped fsmap epoch 12 v35) v1 ==== 107+0+4827 (secure 0 0 0) 0x7fb2c407cca0 con 0x7fb2d40fd1c0 2024-02-16T23:08:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:13.180 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T22:53:56.166293+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi064.tqesxx","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6827/3220035256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3220035256},{"type":"v1","addr":"172.21.15.64:6827","nonce":3220035256}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 12 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2bc06d750 msgr2=0x7fb2bc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2bc06d750 0x7fb2bc06fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006fd0 tx=0x7fb2c8008040 comp rx=0 tx=0).stop 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 msgr2=0x7fb2d4114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 0x7fb2d4114f10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000c00 tx=0x7fb2c4009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 shutdown_connections 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fb2bc06d750 0x7fb2bc06fc00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d40fdad0 0x7fb2d4115450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 --2- 172.21.15.64:0/2977923766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2d40fd1c0 0x7fb2d4114f10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 >> 172.21.15.64:0/2977923766 conn(0x7fb2d4071950 msgr2=0x7fb2d40faa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:13.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 shutdown_connections 2024-02-16T23:08:13.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.182+0000 7fb2d939d640 1 -- 172.21.15.64:0/2977923766 wait complete. 2024-02-16T23:08:13.193 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 13 2024-02-16T23:08:13.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:12 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1629998170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T23:08:13.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:12 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1330017480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T23:08:13.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:12 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2326936903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T23:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/3332525448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 msgr2=0x7f3e88102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 --2- 172.21.15.64:0/3332525448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 0x7f3e88102770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e780048d0 tx=0x7f3e78009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/3332525448 shutdown_connections 2024-02-16T23:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 --2- 172.21.15.64:0/3332525448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e88102d40 0x7f3e880fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 --2- 172.21.15.64:0/3332525448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 0x7f3e88102770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/3332525448 >> 172.21.15.64:0/3332525448 conn(0x7f3e880f7f90 msgr2=0x7f3e880fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/3332525448 shutdown_connections 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/3332525448 wait complete. 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 Processor -- start 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 -- start start 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 0x7f3e8806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e88102d40 0x7f3e8806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88071740 con 0x7f3e881023a0 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8cf9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88070220 con 0x7f3e88102d40 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 0x7f3e8806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 0x7f3e8806f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34874/0 (socket says 172.21.15.64:34874) 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.293+0000 7f3e8659c640 1 -- 172.21.15.64:0/1223958735 learned_addr learned my addr 172.21.15.64:0/1223958735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e7ffff640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e88102d40 0x7f3e8806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e8659c640 1 -- 172.21.15.64:0/1223958735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e88102d40 msgr2=0x7f3e8806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e8659c640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e88102d40 0x7f3e8806fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e8659c640 1 -- 172.21.15.64:0/1223958735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78005040 con 0x7f3e881023a0 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e7ffff640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e88102d40 0x7f3e8806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e8659c640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 0x7f3e8806f7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3e78031720 tx=0x7f3e78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e7effd640 1 -- 172.21.15.64:0/1223958735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e780094c0 con 0x7f3e881023a0 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e7effd640 1 -- 172.21.15.64:0/1223958735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e78009620 con 0x7f3e881023a0 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e88070470 con 0x7f3e881023a0 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e7effd640 1 -- 172.21.15.64:0/1223958735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e7803b640 con 0x7f3e881023a0 2024-02-16T23:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.294+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e88119550 con 0x7f3e881023a0 2024-02-16T23:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.296+0000 7f3e7effd640 1 -- 172.21.15.64:0/1223958735 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3e7802f070 con 0x7f3e881023a0 2024-02-16T23:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.296+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e88064030 con 0x7f3e881023a0 2024-02-16T23:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.296+0000 7f3e7effd640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3e7406d750 0x7f3e7406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.296+0000 7f3e7effd640 1 -- 172.21.15.64:0/1223958735 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3e780af5d0 con 0x7f3e881023a0 2024-02-16T23:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.296+0000 7f3e7ffff640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3e7406d750 0x7f3e7406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.296+0000 7f3e7ffff640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3e7406d750 0x7f3e7406fc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3e88070f90 tx=0x7f3e70009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:13.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.301+0000 7f3e7effd640 1 -- 172.21.15.64:0/1223958735 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e78078d20 con 0x7f3e881023a0 2024-02-16T23:08:13.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.527+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 13, "format": "json"} v 0) v1 -- 0x7f3e881094e0 con 0x7f3e881023a0 2024-02-16T23:08:13.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.528+0000 7f3e7effd640 1 -- 172.21.15.64:0/1223958735 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 13, "format": "json"}]=0 dumped fsmap epoch 13 v35) v1 ==== 107+0+4826 (secure 0 0 0) 0x7f3e7807c3e0 con 0x7f3e881023a0 2024-02-16T23:08:13.528 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:13.528 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T22:53:56.166293+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi064.tqesxx","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6827/3220035256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3220035256},{"type":"v1","addr":"172.21.15.64:6827","nonce":3220035256}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:13.529 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-02-16T23:08:13.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3e7406d750 msgr2=0x7f3e7406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3e7406d750 0x7f3e7406fc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3e88070f90 tx=0x7f3e70009040 comp rx=0 tx=0).stop 2024-02-16T23:08:13.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 msgr2=0x7f3e8806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 0x7f3e8806f7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3e78031720 tx=0x7f3e78009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 shutdown_connections 2024-02-16T23:08:13.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f3e7406d750 0x7f3e7406fc00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e88102d40 0x7f3e8806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 --2- 172.21.15.64:0/1223958735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e881023a0 0x7f3e8806f7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 >> 172.21.15.64:0/1223958735 conn(0x7f3e880f7f90 msgr2=0x7f3e880f9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:13.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 shutdown_connections 2024-02-16T23:08:13.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.530+0000 7f3e8cf9d640 1 -- 172.21.15.64:0/1223958735 wait complete. 2024-02-16T23:08:13.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-02-16T23:08:13.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 -- 172.21.15.64:0/702017607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 msgr2=0x7f57fc0648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 --2- 172.21.15.64:0/702017607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 0x7f57fc0648e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0048d0 tx=0x7f57ec009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 -- 172.21.15.64:0/702017607 shutdown_connections 2024-02-16T23:08:13.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 --2- 172.21.15.64:0/702017607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 0x7f57fc0648e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 --2- 172.21.15.64:0/702017607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57fc1023e0 0x7f57fc1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 -- 172.21.15.64:0/702017607 >> 172.21.15.64:0/702017607 conn(0x7f57fc071230 msgr2=0x7f57fc071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 -- 172.21.15.64:0/702017607 shutdown_connections 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 -- 172.21.15.64:0/702017607 wait complete. 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 Processor -- start 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.643+0000 7f5801d1d640 1 -- start start 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f5801d1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 0x7f57fc0fb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f5801d1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57fc1023e0 0x7f57fc0fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f5801d1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0fbe00 con 0x7f57fc0644b0 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f5801d1d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0fbf70 con 0x7f57fc1023e0 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 0x7f57fc0fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 0x7f57fc0fb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34892/0 (socket says 172.21.15.64:34892) 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57fb7fe640 1 -- 172.21.15.64:0/646793674 learned_addr learned my addr 172.21.15.64:0/646793674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:13.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57faffd640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57fc1023e0 0x7f57fc0fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57fb7fe640 1 -- 172.21.15.64:0/646793674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57fc1023e0 msgr2=0x7f57fc0fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57fb7fe640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57fc1023e0 0x7f57fc0fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57fb7fe640 1 -- 172.21.15.64:0/646793674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec005040 con 0x7f57fc0644b0 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57fb7fe640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 0x7f57fc0fb380 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f57f0002a50 tx=0x7f57f0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57e3fff640 1 -- 172.21.15.64:0/646793674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f0007c30 con 0x7f57fc0644b0 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc1190b0 con 0x7f57fc0644b0 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57e3fff640 1 -- 172.21.15.64:0/646793674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57f000f040 con 0x7f57fc0644b0 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f57e3fff640 1 -- 172.21.15.64:0/646793674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f000dbe0 con 0x7f57fc0644b0 2024-02-16T23:08:13.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.644+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc1193e0 con 0x7f57fc0644b0 2024-02-16T23:08:13.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.645+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc064030 con 0x7f57fc0644b0 2024-02-16T23:08:13.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.646+0000 7f57e3fff640 1 -- 172.21.15.64:0/646793674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f57f000dd40 con 0x7f57fc0644b0 2024-02-16T23:08:13.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.646+0000 7f57e3fff640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f57e406d750 0x7f57e406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.646+0000 7f57e3fff640 1 -- 172.21.15.64:0/646793674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f57f008cef0 con 0x7f57fc0644b0 2024-02-16T23:08:13.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.646+0000 7f57faffd640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f57e406d750 0x7f57e406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.647+0000 7f57faffd640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f57e406d750 0x7f57e406fc00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0fcb70 tx=0x7f57ec009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:13.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.651+0000 7f57e3fff640 1 -- 172.21.15.64:0/646793674 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f57f0056640 con 0x7f57fc0644b0 2024-02-16T23:08:13.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.875+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7f57fc109490 con 0x7f57fc0644b0 2024-02-16T23:08:13.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.875+0000 7f57e3fff640 1 -- 172.21.15.64:0/646793674 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v35) v1 ==== 107+0+4038 (secure 0 0 0) 0x7f57f0059d00 con 0x7f57fc0644b0 2024-02-16T23:08:13.875 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:13.875 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:03:32.044688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:13.876 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-02-16T23:08:13.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f57e406d750 msgr2=0x7f57e406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f57e406d750 0x7f57e406fc00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0fcb70 tx=0x7f57ec009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 msgr2=0x7f57fc0fb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 0x7f57fc0fb380 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f57f0002a50 tx=0x7f57f0004600 comp rx=0 tx=0).stop 2024-02-16T23:08:13.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 shutdown_connections 2024-02-16T23:08:13.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f57e406d750 0x7f57e406fc00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57fc1023e0 0x7f57fc0fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 --2- 172.21.15.64:0/646793674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57fc0644b0 0x7f57fc0fb380 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 >> 172.21.15.64:0/646793674 conn(0x7f57fc071230 msgr2=0x7f57fc0fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:13.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.877+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 shutdown_connections 2024-02-16T23:08:13.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.878+0000 7f5801d1d640 1 -- 172.21.15.64:0/646793674 wait complete. 2024-02-16T23:08:13.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-02-16T23:08:13.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.988+0000 7fcc7139d640 1 -- 172.21.15.64:0/2377482887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 msgr2=0x7fcc6c0648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.988+0000 7fcc7139d640 1 --2- 172.21.15.64:0/2377482887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 0x7fcc6c0648b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0048d0 tx=0x7fcc5c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 -- 172.21.15.64:0/2377482887 shutdown_connections 2024-02-16T23:08:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 --2- 172.21.15.64:0/2377482887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 0x7fcc6c0648b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 --2- 172.21.15.64:0/2377482887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc6c1022b0 0x7fcc6c102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 -- 172.21.15.64:0/2377482887 >> 172.21.15.64:0/2377482887 conn(0x7fcc6c071190 msgr2=0x7fcc6c071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 -- 172.21.15.64:0/2377482887 shutdown_connections 2024-02-16T23:08:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 -- 172.21.15.64:0/2377482887 wait complete. 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 Processor -- start 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 -- start start 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.989+0000 7fcc7139d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 0x7fcc6c0fb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc7139d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc6c1022b0 0x7fcc6c0fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc7139d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0fd220 con 0x7fcc6c064480 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc7139d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0fbd00 con 0x7fcc6c1022b0 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc6affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 0x7fcc6c0fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc6affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 0x7fcc6c0fb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34904/0 (socket says 172.21.15.64:34904) 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc6affd640 1 -- 172.21.15.64:0/2863888772 learned_addr learned my addr 172.21.15.64:0/2863888772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc6a7fc640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc6c1022b0 0x7fcc6c0fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc6affd640 1 -- 172.21.15.64:0/2863888772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc6c1022b0 msgr2=0x7fcc6c0fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc6affd640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc6c1022b0 0x7fcc6c0fb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc6affd640 1 -- 172.21.15.64:0/2863888772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c005040 con 0x7fcc6c064480 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc6affd640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 0x7fcc6c0fb280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000eb70 tx=0x7fcc6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:13.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc537fe640 1 -- 172.21.15.64:0/2863888772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc60007c30 con 0x7fcc6c064480 2024-02-16T23:08:13.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c0fbf50 con 0x7fcc6c064480 2024-02-16T23:08:13.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc537fe640 1 -- 172.21.15.64:0/2863888772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc6000f040 con 0x7fcc6c064480 2024-02-16T23:08:13.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc537fe640 1 -- 172.21.15.64:0/2863888772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc600095b0 con 0x7fcc6c064480 2024-02-16T23:08:13.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.990+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c119070 con 0x7fcc6c064480 2024-02-16T23:08:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.992+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c04bcc0 con 0x7fcc6c064480 2024-02-16T23:08:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.992+0000 7fcc537fe640 1 -- 172.21.15.64:0/2863888772 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcc60009730 con 0x7fcc6c064480 2024-02-16T23:08:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.992+0000 7fcc537fe640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc5406d810 0x7fcc5406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.993+0000 7fcc537fe640 1 -- 172.21.15.64:0/2863888772 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcc6008cd30 con 0x7fcc6c064480 2024-02-16T23:08:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.993+0000 7fcc6a7fc640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc5406d810 0x7fcc5406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:13.993+0000 7fcc6a7fc640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc5406d810 0x7fcc5406fcc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0fca70 tx=0x7fcc5c009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.001+0000 7fcc537fe640 1 -- 172.21.15.64:0/2863888772 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc60056480 con 0x7fcc6c064480 2024-02-16T23:08:14.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:13 smithi064 ceph-mon[93757]: pgmap v313: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:14.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:13 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3511964531' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 11, "format": "json"}]: dispatch 2024-02-16T23:08:14.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:13 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2977923766' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 12, "format": "json"}]: dispatch 2024-02-16T23:08:14.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:13 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:08:14.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:13 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1223958735' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-02-16T23:08:14.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.221+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7fcc6c109380 con 0x7fcc6c064480 2024-02-16T23:08:14.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.222+0000 7fcc537fe640 1 -- 172.21.15.64:0/2863888772 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v35) v1 ==== 107+0+4049 (secure 0 0 0) 0x7fcc60059b40 con 0x7fcc6c064480 2024-02-16T23:08:14.222 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:14.222 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:03:32.049606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":0,"incarnation":15,"state":"up:replay","state_seq":2,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 15 2024-02-16T23:08:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc5406d810 msgr2=0x7fcc5406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc5406d810 0x7fcc5406fcc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0fca70 tx=0x7fcc5c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 msgr2=0x7fcc6c0fb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 0x7fcc6c0fb280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000eb70 tx=0x7fcc6000b040 comp rx=0 tx=0).stop 2024-02-16T23:08:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 shutdown_connections 2024-02-16T23:08:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcc5406d810 0x7fcc5406fcc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc6c1022b0 0x7fcc6c0fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 --2- 172.21.15.64:0/2863888772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc6c064480 0x7fcc6c0fb280 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 >> 172.21.15.64:0/2863888772 conn(0x7fcc6c071190 msgr2=0x7fcc6c0f9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 shutdown_connections 2024-02-16T23:08:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.224+0000 7fcc7139d640 1 -- 172.21.15.64:0/2863888772 wait complete. 2024-02-16T23:08:14.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-02-16T23:08:14.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:13 smithi192 ceph-mon[82652]: pgmap v313: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:14.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:13 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3511964531' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 11, "format": "json"}]: dispatch 2024-02-16T23:08:14.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:13 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2977923766' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 12, "format": "json"}]: dispatch 2024-02-16T23:08:14.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:13 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:08:14.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:13 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1223958735' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-02-16T23:08:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.336+0000 7ffa9b19d640 1 -- 172.21.15.64:0/4010026483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102d20 msgr2=0x7ffa940fc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.336+0000 7ffa9b19d640 1 --2- 172.21.15.64:0/4010026483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102d20 0x7ffa940fc3b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa800048d0 tx=0x7ffa80009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.336+0000 7ffa9b19d640 1 -- 172.21.15.64:0/4010026483 shutdown_connections 2024-02-16T23:08:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.336+0000 7ffa9b19d640 1 --2- 172.21.15.64:0/4010026483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102d20 0x7ffa940fc3b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.336+0000 7ffa9b19d640 1 --2- 172.21.15.64:0/4010026483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa94102380 0x7ffa94102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.336+0000 7ffa9b19d640 1 -- 172.21.15.64:0/4010026483 >> 172.21.15.64:0/4010026483 conn(0x7ffa940f7fb0 msgr2=0x7ffa940fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.336+0000 7ffa9b19d640 1 -- 172.21.15.64:0/4010026483 shutdown_connections 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.336+0000 7ffa9b19d640 1 -- 172.21.15.64:0/4010026483 wait complete. 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa9b19d640 1 Processor -- start 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa9b19d640 1 -- start start 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa9b19d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102380 0x7ffa94112ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa9b19d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa94102d20 0x7ffa941130e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa9b19d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941136e0 con 0x7ffa94102380 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa9b19d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94113850 con 0x7ffa94102d20 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa98f39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102380 0x7ffa94112ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa98f39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102380 0x7ffa94112ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34924/0 (socket says 172.21.15.64:34924) 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa98f39640 1 -- 172.21.15.64:0/1501617739 learned_addr learned my addr 172.21.15.64:0/1501617739 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa93fff640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa94102d20 0x7ffa941130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa98f39640 1 -- 172.21.15.64:0/1501617739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa94102d20 msgr2=0x7ffa941130e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa98f39640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa94102d20 0x7ffa941130e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa98f39640 1 -- 172.21.15.64:0/1501617739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa80005040 con 0x7ffa94102380 2024-02-16T23:08:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.337+0000 7ffa93fff640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa94102d20 0x7ffa941130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:08:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.338+0000 7ffa98f39640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102380 0x7ffa94112ba0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa88002a50 tx=0x7ffa88004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.338+0000 7ffa917fa640 1 -- 172.21.15.64:0/1501617739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa88007c30 con 0x7ffa94102380 2024-02-16T23:08:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.338+0000 7ffa917fa640 1 -- 172.21.15.64:0/1501617739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa8800f040 con 0x7ffa94102380 2024-02-16T23:08:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.338+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9410ce20 con 0x7ffa94102380 2024-02-16T23:08:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.338+0000 7ffa917fa640 1 -- 172.21.15.64:0/1501617739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa8800db50 con 0x7ffa94102380 2024-02-16T23:08:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.338+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9410d340 con 0x7ffa94102380 2024-02-16T23:08:14.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.339+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa94064030 con 0x7ffa94102380 2024-02-16T23:08:14.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.339+0000 7ffa917fa640 1 -- 172.21.15.64:0/1501617739 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ffa8801c400 con 0x7ffa94102380 2024-02-16T23:08:14.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.340+0000 7ffa917fa640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ffa8406d750 0x7ffa8406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:14.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.340+0000 7ffa93fff640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ffa8406d750 0x7ffa8406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:14.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.340+0000 7ffa917fa640 1 -- 172.21.15.64:0/1501617739 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ffa8808c8b0 con 0x7ffa94102380 2024-02-16T23:08:14.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.340+0000 7ffa93fff640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ffa8406d750 0x7ffa8406fc00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa9410e4a0 tx=0x7ffa80009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.345+0000 7ffa917fa640 1 -- 172.21.15.64:0/1501617739 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa88056000 con 0x7ffa94102380 2024-02-16T23:08:14.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.566+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7ffa9410d560 con 0x7ffa94102380 2024-02-16T23:08:14.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.567+0000 7ffa917fa640 1 -- 172.21.15.64:0/1501617739 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v35) v1 ==== 107+0+4054 (secure 0 0 0) 0x7ffa880596c0 con 0x7ffa94102380 2024-02-16T23:08:14.567 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:14.567 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:03:42.179762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":0,"incarnation":15,"state":"up:reconnect","state_seq":149,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:14.567 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2024-02-16T23:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ffa8406d750 msgr2=0x7ffa8406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ffa8406d750 0x7ffa8406fc00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa9410e4a0 tx=0x7ffa80009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102380 msgr2=0x7ffa94112ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102380 0x7ffa94112ba0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa88002a50 tx=0x7ffa88004600 comp rx=0 tx=0).stop 2024-02-16T23:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 shutdown_connections 2024-02-16T23:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ffa8406d750 0x7ffa8406fc00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa94102d20 0x7ffa941130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 --2- 172.21.15.64:0/1501617739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa94102380 0x7ffa94112ba0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 >> 172.21.15.64:0/1501617739 conn(0x7ffa940f7fb0 msgr2=0x7ffa940f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 shutdown_connections 2024-02-16T23:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.569+0000 7ffa9b19d640 1 -- 172.21.15.64:0/1501617739 wait complete. 2024-02-16T23:08:14.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-02-16T23:08:14.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.679+0000 7f05cfaae640 1 -- 172.21.15.64:0/604586439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101c10 msgr2=0x7f05c80fb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.679+0000 7f05cfaae640 1 --2- 172.21.15.64:0/604586439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101c10 0x7f05c80fb2a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f05b80048d0 tx=0x7f05b8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.679+0000 7f05cfaae640 1 -- 172.21.15.64:0/604586439 shutdown_connections 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.679+0000 7f05cfaae640 1 --2- 172.21.15.64:0/604586439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101c10 0x7f05c80fb2a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.679+0000 7f05cfaae640 1 --2- 172.21.15.64:0/604586439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c8101270 0x7f05c8101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.679+0000 7f05cfaae640 1 -- 172.21.15.64:0/604586439 >> 172.21.15.64:0/604586439 conn(0x7f05c80f6e80 msgr2=0x7f05c80f9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.679+0000 7f05cfaae640 1 -- 172.21.15.64:0/604586439 shutdown_connections 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.679+0000 7f05cfaae640 1 -- 172.21.15.64:0/604586439 wait complete. 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cfaae640 1 Processor -- start 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cfaae640 1 -- start start 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cfaae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101270 0x7f05c8111b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cfaae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c8101c10 0x7f05c81120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cfaae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81126a0 con 0x7f05c8101270 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cfaae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8112810 con 0x7f05c8101c10 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cd84a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101270 0x7f05c8111b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cd84a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101270 0x7f05c8111b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34934/0 (socket says 172.21.15.64:34934) 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cd84a640 1 -- 172.21.15.64:0/1029266551 learned_addr learned my addr 172.21.15.64:0/1029266551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cd049640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c8101c10 0x7f05c81120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cd84a640 1 -- 172.21.15.64:0/1029266551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c8101c10 msgr2=0x7f05c81120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cd84a640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c8101c10 0x7f05c81120a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cd84a640 1 -- 172.21.15.64:0/1029266551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8005040 con 0x7f05c8101270 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.680+0000 7f05cd049640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c8101c10 0x7f05c81120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.681+0000 7f05cd84a640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101270 0x7f05c8111b60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f05c400c790 tx=0x7f05c40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.681+0000 7f05be7fc640 1 -- 172.21.15.64:0/1029266551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c400a820 con 0x7f05c8101270 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.681+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c810bde0 con 0x7f05c8101270 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.681+0000 7f05be7fc640 1 -- 172.21.15.64:0/1029266551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05c400ae50 con 0x7f05c8101270 2024-02-16T23:08:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.681+0000 7f05be7fc640 1 -- 172.21.15.64:0/1029266551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c40115e0 con 0x7f05c8101270 2024-02-16T23:08:14.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.681+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c810c300 con 0x7f05c8101270 2024-02-16T23:08:14.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.682+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c8064480 con 0x7f05c8101270 2024-02-16T23:08:14.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.682+0000 7f05be7fc640 1 -- 172.21.15.64:0/1029266551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f05c4012400 con 0x7f05c8101270 2024-02-16T23:08:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.683+0000 7f05be7fc640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05b406d750 0x7f05b406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.683+0000 7f05be7fc640 1 -- 172.21.15.64:0/1029266551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f05c408d150 con 0x7f05c8101270 2024-02-16T23:08:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.683+0000 7f05cd049640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05b406d750 0x7f05b406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.683+0000 7f05cd049640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05b406d750 0x7f05b406fc00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05c810d460 tx=0x7f05b8009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:14.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.687+0000 7f05be7fc640 1 -- 172.21.15.64:0/1029266551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05c4056920 con 0x7f05c8101270 2024-02-16T23:08:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:08:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:08:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:14 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/646793674' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-02-16T23:08:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:14 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2863888772' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-02-16T23:08:14.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:08:14.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:08:14.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:14 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:08:14.868 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:14 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1501617739' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-16T23:08:14.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.913+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f05c81083d0 con 0x7f05c8101270 2024-02-16T23:08:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.913+0000 7f05be7fc640 1 -- 172.21.15.64:0/1029266551 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v35) v1 ==== 107+0+4051 (secure 0 0 0) 0x7f05c4059fe0 con 0x7f05c8101270 2024-02-16T23:08:14.913 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:14.913 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:03:42.768792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":0,"incarnation":15,"state":"up:rejoin","state_seq":150,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:14.913 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 17 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.915+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05b406d750 msgr2=0x7f05b406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.915+0000 7f05cfaae640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05b406d750 0x7f05b406fc00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05c810d460 tx=0x7f05b8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.915+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101270 msgr2=0x7f05c8111b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.915+0000 7f05cfaae640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101270 0x7f05c8111b60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f05c400c790 tx=0x7f05c40043c0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.916+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 shutdown_connections 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.916+0000 7f05cfaae640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f05b406d750 0x7f05b406fc00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.916+0000 7f05cfaae640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c8101c10 0x7f05c81120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.916+0000 7f05cfaae640 1 --2- 172.21.15.64:0/1029266551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8101270 0x7f05c8111b60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.916+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 >> 172.21.15.64:0/1029266551 conn(0x7f05c80f6e80 msgr2=0x7f05c80f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.916+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 shutdown_connections 2024-02-16T23:08:14.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:14.916+0000 7f05cfaae640 1 -- 172.21.15.64:0/1029266551 wait complete. 2024-02-16T23:08:14.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.026+0000 7ff7ad507640 1 -- 172.21.15.64:0/374133807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 msgr2=0x7ff7a81027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.026+0000 7ff7ad507640 1 --2- 172.21.15.64:0/374133807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 0x7ff7a81027d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00048d0 tx=0x7ff7a0009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.026+0000 7ff7ad507640 1 -- 172.21.15.64:0/374133807 shutdown_connections 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.026+0000 7ff7ad507640 1 --2- 172.21.15.64:0/374133807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a8102d10 0x7ff7a80faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.026+0000 7ff7ad507640 1 --2- 172.21.15.64:0/374133807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 0x7ff7a81027d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.026+0000 7ff7ad507640 1 -- 172.21.15.64:0/374133807 >> 172.21.15.64:0/374133807 conn(0x7ff7a8070a10 msgr2=0x7ff7a8070e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.026+0000 7ff7ad507640 1 -- 172.21.15.64:0/374133807 shutdown_connections 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.026+0000 7ff7ad507640 1 -- 172.21.15.64:0/374133807 wait complete. 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7ad507640 1 Processor -- start 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7ad507640 1 -- start start 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7ad507640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 0x7ff7a8112bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7ad507640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a8102d10 0x7ff7a81130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7ad507640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a81136f0 con 0x7ff7a8102400 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7ad507640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a810cc80 con 0x7ff7a8102d10 2024-02-16T23:08:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 0x7ff7a8112bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 0x7ff7a8112bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34952/0 (socket says 172.21.15.64:34952) 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7a6ffd640 1 -- 172.21.15.64:0/2060339169 learned_addr learned my addr 172.21.15.64:0/2060339169 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7a67fc640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a8102d10 0x7ff7a81130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7a6ffd640 1 -- 172.21.15.64:0/2060339169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a8102d10 msgr2=0x7ff7a81130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7a6ffd640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a8102d10 0x7ff7a81130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7a6ffd640 1 -- 172.21.15.64:0/2060339169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a0005040 con 0x7ff7a8102400 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.027+0000 7ff7a67fc640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a8102d10 0x7ff7a81130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.028+0000 7ff7a6ffd640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 0x7ff7a8112bb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00043d0 tx=0x7ff7a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.028+0000 7ff78f7fe640 1 -- 172.21.15.64:0/2060339169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7a0030e60 con 0x7ff7a8102400 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.028+0000 7ff78f7fe640 1 -- 172.21.15.64:0/2060339169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7a00092e0 con 0x7ff7a8102400 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.028+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a810cf00 con 0x7ff7a8102400 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.028+0000 7ff78f7fe640 1 -- 172.21.15.64:0/2060339169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7a003a5f0 con 0x7ff7a8102400 2024-02-16T23:08:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.028+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a810d420 con 0x7ff7a8102400 2024-02-16T23:08:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.029+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a8064480 con 0x7ff7a8102400 2024-02-16T23:08:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.029+0000 7ff78f7fe640 1 -- 172.21.15.64:0/2060339169 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff7a0009450 con 0x7ff7a8102400 2024-02-16T23:08:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.029+0000 7ff78f7fe640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff79006d750 0x7ff79006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.030+0000 7ff78f7fe640 1 -- 172.21.15.64:0/2060339169 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff7a00afd70 con 0x7ff7a8102400 2024-02-16T23:08:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.030+0000 7ff7a67fc640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff79006d750 0x7ff79006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.030+0000 7ff7a67fc640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff79006d750 0x7ff79006fc00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff79c007fd0 tx=0x7ff79c009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:15.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.034+0000 7ff78f7fe640 1 -- 172.21.15.64:0/2060339169 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7a0079540 con 0x7ff7a8102400 2024-02-16T23:08:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.255+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7ff7a810e1e0 con 0x7ff7a8102400 2024-02-16T23:08:15.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.256+0000 7ff78f7fe640 1 -- 172.21.15.64:0/2060339169 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v35) v1 ==== 107+0+4051 (secure 0 0 0) 0x7ff7a007cc00 con 0x7ff7a8102400 2024-02-16T23:08:15.256 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:15.256 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:03:43.908124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":0,"incarnation":15,"state":"up:active","state_seq":151,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:15.256 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 18 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.257+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff79006d750 msgr2=0x7ff79006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.257+0000 7ff7ad507640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff79006d750 0x7ff79006fc00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff79c007fd0 tx=0x7ff79c009e90 comp rx=0 tx=0).stop 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 msgr2=0x7ff7a8112bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 0x7ff7a8112bb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00043d0 tx=0x7ff7a0009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 shutdown_connections 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7ff79006d750 0x7ff79006fc00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a8102d10 0x7ff7a81130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 --2- 172.21.15.64:0/2060339169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a8102400 0x7ff7a8112bb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 >> 172.21.15.64:0/2060339169 conn(0x7ff7a8070a10 msgr2=0x7ff7a80f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 shutdown_connections 2024-02-16T23:08:15.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.258+0000 7ff7ad507640 1 -- 172.21.15.64:0/2060339169 wait complete. 2024-02-16T23:08:15.268 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-02-16T23:08:15.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:08:15.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:08:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:14 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/646793674' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-02-16T23:08:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:14 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2863888772' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-02-16T23:08:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:08:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:08:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:14 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:08:15.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:14 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1501617739' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-16T23:08:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 -- 172.21.15.64:0/2249342536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fd1c0 msgr2=0x7f632c0fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 --2- 172.21.15.64:0/2249342536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fd1c0 0x7f632c0fd590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f63240048d0 tx=0x7f6324009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 -- 172.21.15.64:0/2249342536 shutdown_connections 2024-02-16T23:08:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 --2- 172.21.15.64:0/2249342536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f632c0fdad0 0x7f632c0fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 --2- 172.21.15.64:0/2249342536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fd1c0 0x7f632c0fd590 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 -- 172.21.15.64:0/2249342536 >> 172.21.15.64:0/2249342536 conn(0x7f632c071950 msgr2=0x7f632c071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 -- 172.21.15.64:0/2249342536 shutdown_connections 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 -- 172.21.15.64:0/2249342536 wait complete. 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 Processor -- start 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.368+0000 7f6333807640 1 -- start start 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6333807640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f632c0fd1c0 0x7f632c107d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6333807640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fdad0 0x7f632c1082d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6333807640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c109d30 con 0x7f632c0fdad0 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6333807640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c108810 con 0x7f632c0fd1c0 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6330da2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fdad0 0x7f632c1082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6330da2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fdad0 0x7f632c1082d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34978/0 (socket says 172.21.15.64:34978) 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6330da2640 1 -- 172.21.15.64:0/2330492008 learned_addr learned my addr 172.21.15.64:0/2330492008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f63315a3640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f632c0fd1c0 0x7f632c107d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6330da2640 1 -- 172.21.15.64:0/2330492008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f632c0fd1c0 msgr2=0x7f632c107d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6330da2640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f632c0fd1c0 0x7f632c107d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6330da2640 1 -- 172.21.15.64:0/2330492008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324005040 con 0x7f632c0fdad0 2024-02-16T23:08:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f63315a3640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f632c0fd1c0 0x7f632c107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:08:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6330da2640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fdad0 0x7f632c1082d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f631c002f40 tx=0x7f631c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6321ffb640 1 -- 172.21.15.64:0/2330492008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631c007c30 con 0x7f632c0fdad0 2024-02-16T23:08:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6321ffb640 1 -- 172.21.15.64:0/2330492008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f631c00f040 con 0x7f632c0fdad0 2024-02-16T23:08:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c108a60 con 0x7f632c0fdad0 2024-02-16T23:08:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6321ffb640 1 -- 172.21.15.64:0/2330492008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631c00ccb0 con 0x7f632c0fdad0 2024-02-16T23:08:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.369+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c1192b0 con 0x7f632c0fdad0 2024-02-16T23:08:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.370+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632c064480 con 0x7f632c0fdad0 2024-02-16T23:08:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.371+0000 7f6321ffb640 1 -- 172.21.15.64:0/2330492008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f631c0090d0 con 0x7f632c0fdad0 2024-02-16T23:08:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.371+0000 7f6321ffb640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f631806d750 0x7f631806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.371+0000 7f6321ffb640 1 -- 172.21.15.64:0/2330492008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f631c08cbd0 con 0x7f632c0fdad0 2024-02-16T23:08:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.371+0000 7f63315a3640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f631806d750 0x7f631806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.372+0000 7f63315a3640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f631806d750 0x7f631806fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6324009fa0 tx=0x7f6324009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.376+0000 7f6321ffb640 1 -- 172.21.15.64:0/2330492008 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f631c0563a0 con 0x7f632c0fdad0 2024-02-16T23:08:15.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.600+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f632c1092c0 con 0x7f632c0fdad0 2024-02-16T23:08:15.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.600+0000 7f6321ffb640 1 -- 172.21.15.64:0/2330492008 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v35) v1 ==== 107+0+4864 (secure 0 0 0) 0x7f631c060020 con 0x7f632c0fdad0 2024-02-16T23:08:15.600 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:15.601 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:03:43.908124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi064.bmcyuf","rank":0,"incarnation":15,"state":"up:active","state_seq":151,"addr":"172.21.15.64:6829/168979118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":168979118},{"type":"v1","addr":"172.21.15.64:6829","nonce":168979118}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 19 2024-02-16T23:08:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f631806d750 msgr2=0x7f631806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f631806d750 0x7f631806fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6324009fa0 tx=0x7f6324009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fdad0 msgr2=0x7f632c1082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fdad0 0x7f632c1082d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f631c002f40 tx=0x7f631c00b040 comp rx=0 tx=0).stop 2024-02-16T23:08:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 shutdown_connections 2024-02-16T23:08:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f632c0fdad0 0x7f632c1082d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f631806d750 0x7f631806fc00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 --2- 172.21.15.64:0/2330492008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f632c0fd1c0 0x7f632c107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 >> 172.21.15.64:0/2330492008 conn(0x7f632c071950 msgr2=0x7f632c0f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.602+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 shutdown_connections 2024-02-16T23:08:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.603+0000 7f6333807640 1 -- 172.21.15.64:0/2330492008 wait complete. 2024-02-16T23:08:15.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-02-16T23:08:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.712+0000 7fa0b879d640 1 -- 172.21.15.64:0/3550406518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 msgr2=0x7fa0b00644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.712+0000 7fa0b879d640 1 --2- 172.21.15.64:0/3550406518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 0x7fa0b00644a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00048d0 tx=0x7fa0a0009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 -- 172.21.15.64:0/3550406518 shutdown_connections 2024-02-16T23:08:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 --2- 172.21.15.64:0/3550406518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 0x7fa0b00644a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 --2- 172.21.15.64:0/3550406518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0b0102370 0x7fa0b0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 -- 172.21.15.64:0/3550406518 >> 172.21.15.64:0/3550406518 conn(0x7fa0b0071230 msgr2=0x7fa0b0071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 -- 172.21.15.64:0/3550406518 shutdown_connections 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 -- 172.21.15.64:0/3550406518 wait complete. 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 Processor -- start 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 -- start start 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0b0102370 0x7fa0b00fb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 0x7fa0b00fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00fbe00 con 0x7fa0b0102d10 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b879d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00fbf70 con 0x7fa0b0102370 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b5d38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 0x7fa0b00fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b5d38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 0x7fa0b00fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35000/0 (socket says 172.21.15.64:35000) 2024-02-16T23:08:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b5d38640 1 -- 172.21.15.64:0/1385293373 learned_addr learned my addr 172.21.15.64:0/1385293373 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.713+0000 7fa0b6539640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0b0102370 0x7fa0b00fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0b5d38640 1 -- 172.21.15.64:0/1385293373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0b0102370 msgr2=0x7fa0b00fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0b5d38640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0b0102370 0x7fa0b00fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0b5d38640 1 -- 172.21.15.64:0/1385293373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a0005040 con 0x7fa0b0102d10 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0b5d38640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 0x7fa0b00fb8c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0034040 tx=0x7fa0a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0a6ffd640 1 -- 172.21.15.64:0/1385293373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0a00094c0 con 0x7fa0b0102d10 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0a6ffd640 1 -- 172.21.15.64:0/1385293373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0a0009a80 con 0x7fa0b0102d10 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b00fc1f0 con 0x7fa0b0102d10 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0a6ffd640 1 -- 172.21.15.64:0/1385293373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0a003b400 con 0x7fa0b0102d10 2024-02-16T23:08:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.714+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b0119400 con 0x7fa0b0102d10 2024-02-16T23:08:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.715+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b0064030 con 0x7fa0b0102d10 2024-02-16T23:08:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.716+0000 7fa0a6ffd640 1 -- 172.21.15.64:0/1385293373 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa0a002f070 con 0x7fa0b0102d10 2024-02-16T23:08:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.716+0000 7fa0a6ffd640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa09c06d750 0x7fa09c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.716+0000 7fa0a6ffd640 1 -- 172.21.15.64:0/1385293373 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa0a00af0f0 con 0x7fa0b0102d10 2024-02-16T23:08:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.716+0000 7fa0b6539640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa09c06d750 0x7fa09c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.716+0000 7fa0b6539640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa09c06d750 0x7fa09c06fc00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac004eb0 tx=0x7fa0ac00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.721+0000 7fa0a6ffd640 1 -- 172.21.15.64:0/1385293373 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa0a0078840 con 0x7fa0b0102d10 2024-02-16T23:08:15.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:15 smithi064 ceph-mon[93757]: pgmap v314: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:15.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:15 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1029266551' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-16T23:08:15.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:15 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2060339169' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-16T23:08:15.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:15 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2330492008' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-16T23:08:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.940+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fa0b01094d0 con 0x7fa0b0102d10 2024-02-16T23:08:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.940+0000 7fa0a6ffd640 1 -- 172.21.15.64:0/1385293373 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v35) v1 ==== 107+0+4075 (secure 0 0 0) 0x7fa0a007bf00 con 0x7fa0b0102d10 2024-02-16T23:08:15.940 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:15.941 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:03:59.013187+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 20 2024-02-16T23:08:15.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa09c06d750 msgr2=0x7fa09c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa09c06d750 0x7fa09c06fc00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac004eb0 tx=0x7fa0ac00b040 comp rx=0 tx=0).stop 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 msgr2=0x7fa0b00fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 0x7fa0b00fb8c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0034040 tx=0x7fa0a0009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 shutdown_connections 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b0102d10 0x7fa0b00fb8c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa09c06d750 0x7fa09c06fc00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 --2- 172.21.15.64:0/1385293373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0b0102370 0x7fa0b00fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.942+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 >> 172.21.15.64:0/1385293373 conn(0x7fa0b0071230 msgr2=0x7fa0b00fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.943+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 shutdown_connections 2024-02-16T23:08:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:15.943+0000 7fa0b879d640 1 -- 172.21.15.64:0/1385293373 wait complete. 2024-02-16T23:08:15.953 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-16T23:08:16.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.052+0000 7fa53675d640 1 -- 172.21.15.64:0/2676331341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fd390 msgr2=0x7fa5300fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.052+0000 7fa53675d640 1 --2- 172.21.15.64:0/2676331341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fd390 0x7fa5300fd760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa5200048d0 tx=0x7fa520009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.052+0000 7fa53675d640 1 -- 172.21.15.64:0/2676331341 shutdown_connections 2024-02-16T23:08:16.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.052+0000 7fa53675d640 1 --2- 172.21.15.64:0/2676331341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fdd30 0x7fa530106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.052+0000 7fa53675d640 1 --2- 172.21.15.64:0/2676331341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fd390 0x7fa5300fd760 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.052+0000 7fa53675d640 1 -- 172.21.15.64:0/2676331341 >> 172.21.15.64:0/2676331341 conn(0x7fa5300f6de0 msgr2=0x7fa5300f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:16.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.052+0000 7fa53675d640 1 -- 172.21.15.64:0/2676331341 shutdown_connections 2024-02-16T23:08:16.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.052+0000 7fa53675d640 1 -- 172.21.15.64:0/2676331341 wait complete. 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa53675d640 1 Processor -- start 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa53675d640 1 -- start start 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa53675d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fd390 0x7fa530112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa53675d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fdd30 0x7fa5301131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa53675d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5301137c0 con 0x7fa5300fdd30 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa53675d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53010cd50 con 0x7fa5300fd390 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fdd30 0x7fa5301131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fdd30 0x7fa5301131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35022/0 (socket says 172.21.15.64:35022) 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa52f7fe640 1 -- 172.21.15.64:0/1515607419 learned_addr learned my addr 172.21.15.64:0/1515607419 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa52ffff640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fd390 0x7fa530112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa52f7fe640 1 -- 172.21.15.64:0/1515607419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fd390 msgr2=0x7fa530112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa52f7fe640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fd390 0x7fa530112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.053+0000 7fa52f7fe640 1 -- 172.21.15.64:0/1515607419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520005040 con 0x7fa5300fdd30 2024-02-16T23:08:16.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.054+0000 7fa52f7fe640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fdd30 0x7fa5301131c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa524002a50 tx=0x7fa524004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.054+0000 7fa52cff9640 1 -- 172.21.15.64:0/1515607419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa524007c30 con 0x7fa5300fdd30 2024-02-16T23:08:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.054+0000 7fa52cff9640 1 -- 172.21.15.64:0/1515607419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa52400f040 con 0x7fa5300fdd30 2024-02-16T23:08:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.054+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53010d030 con 0x7fa5300fdd30 2024-02-16T23:08:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.054+0000 7fa52cff9640 1 -- 172.21.15.64:0/1515607419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa52400db50 con 0x7fa5300fdd30 2024-02-16T23:08:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.054+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53010d550 con 0x7fa5300fdd30 2024-02-16T23:08:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.055+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa530064480 con 0x7fa5300fdd30 2024-02-16T23:08:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.055+0000 7fa52cff9640 1 -- 172.21.15.64:0/1515607419 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa52401c400 con 0x7fa5300fdd30 2024-02-16T23:08:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.056+0000 7fa52cff9640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa51806d750 0x7fa51806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.056+0000 7fa52cff9640 1 -- 172.21.15.64:0/1515607419 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa52408c800 con 0x7fa5300fdd30 2024-02-16T23:08:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.056+0000 7fa52ffff640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa51806d750 0x7fa51806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.056+0000 7fa52ffff640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa51806d750 0x7fa51806fc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa520003080 tx=0x7fa520009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:16.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.061+0000 7fa52cff9640 1 -- 172.21.15.64:0/1515607419 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa524056000 con 0x7fa5300fdd30 2024-02-16T23:08:16.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.278+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fa53010e280 con 0x7fa5300fdd30 2024-02-16T23:08:16.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.278+0000 7fa52cff9640 1 -- 172.21.15.64:0/1515607419 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v35) v1 ==== 107+0+4086 (secure 0 0 0) 0x7fa5240596c0 con 0x7fa5300fdd30 2024-02-16T23:08:16.278 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:16.279 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:03:59.018132+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":0,"incarnation":21,"state":"up:replay","state_seq":3,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:16.279 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa51806d750 msgr2=0x7fa51806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa51806d750 0x7fa51806fc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa520003080 tx=0x7fa520009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fdd30 msgr2=0x7fa5301131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fdd30 0x7fa5301131c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa524002a50 tx=0x7fa524004600 comp rx=0 tx=0).stop 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 shutdown_connections 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5300fdd30 0x7fa5301131c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa51806d750 0x7fa51806fc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 --2- 172.21.15.64:0/1515607419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5300fd390 0x7fa530112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 >> 172.21.15.64:0/1515607419 conn(0x7fa5300f6de0 msgr2=0x7fa530100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 shutdown_connections 2024-02-16T23:08:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.281+0000 7fa53675d640 1 -- 172.21.15.64:0/1515607419 wait complete. 2024-02-16T23:08:16.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:15 smithi192 ceph-mon[82652]: pgmap v314: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:16.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:15 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1029266551' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-16T23:08:16.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:15 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2060339169' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-16T23:08:16.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:15 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2330492008' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-16T23:08:16.291 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.390+0000 7f294c39d640 1 -- 172.21.15.64:0/2191539876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 msgr2=0x7f29440faa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.390+0000 7f294c39d640 1 --2- 172.21.15.64:0/2191539876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 0x7f29440faa50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29340048d0 tx=0x7f2934009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.390+0000 7f294c39d640 1 -- 172.21.15.64:0/2191539876 shutdown_connections 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.390+0000 7f294c39d640 1 --2- 172.21.15.64:0/2191539876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 0x7f29440faa50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.390+0000 7f294c39d640 1 --2- 172.21.15.64:0/2191539876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2944102400 0x7f29441027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.390+0000 7f294c39d640 1 -- 172.21.15.64:0/2191539876 >> 172.21.15.64:0/2191539876 conn(0x7f2944070a10 msgr2=0x7f2944070e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.390+0000 7f294c39d640 1 -- 172.21.15.64:0/2191539876 shutdown_connections 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.390+0000 7f294c39d640 1 -- 172.21.15.64:0/2191539876 wait complete. 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f294c39d640 1 Processor -- start 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f294c39d640 1 -- start start 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f294c39d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2944102400 0x7f2944112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f294c39d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 0x7f2944113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f294c39d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2944113800 con 0x7f2944102d10 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f294c39d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294410cd90 con 0x7f2944102400 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f2949938640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 0x7f2944113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f2949938640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 0x7f2944113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35040/0 (socket says 172.21.15.64:35040) 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f2949938640 1 -- 172.21.15.64:0/1662503113 learned_addr learned my addr 172.21.15.64:0/1662503113 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f2949938640 1 -- 172.21.15.64:0/1662503113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2944102400 msgr2=0x7f2944112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f2949938640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2944102400 0x7f2944112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.391+0000 7f2949938640 1 -- 172.21.15.64:0/1662503113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934005040 con 0x7f2944102d10 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.392+0000 7f2949938640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 0x7f2944113200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2934009fa0 tx=0x7f2934009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.392+0000 7f293affd640 1 -- 172.21.15.64:0/1662503113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29340090d0 con 0x7f2944102d10 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.392+0000 7f293affd640 1 -- 172.21.15.64:0/1662503113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f293403b420 con 0x7f2944102d10 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.392+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294410d010 con 0x7f2944102d10 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.392+0000 7f293affd640 1 -- 172.21.15.64:0/1662503113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2934043600 con 0x7f2944102d10 2024-02-16T23:08:16.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.392+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294410d530 con 0x7f2944102d10 2024-02-16T23:08:16.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.393+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2944064480 con 0x7f2944102d10 2024-02-16T23:08:16.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.393+0000 7f293affd640 1 -- 172.21.15.64:0/1662503113 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2934009230 con 0x7f2944102d10 2024-02-16T23:08:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.394+0000 7f293affd640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f293006d750 0x7f293006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.394+0000 7f293affd640 1 -- 172.21.15.64:0/1662503113 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f29340aff60 con 0x7f2944102d10 2024-02-16T23:08:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.394+0000 7f294a139640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f293006d750 0x7f293006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.394+0000 7f294a139640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f293006d750 0x7f293006fc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2940004d40 tx=0x7f294000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:16.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.398+0000 7f293affd640 1 -- 172.21.15.64:0/1662503113 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2934079770 con 0x7f2944102d10 2024-02-16T23:08:16.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.617+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f294410d750 con 0x7f2944102d10 2024-02-16T23:08:16.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.617+0000 7f293affd640 1 -- 172.21.15.64:0/1662503113 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v35) v1 ==== 107+0+4091 (secure 0 0 0) 0x7f293407ce30 con 0x7f2944102d10 2024-02-16T23:08:16.617 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:16.618 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:07.823782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":156,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 22 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.619+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f293006d750 msgr2=0x7f293006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.619+0000 7f294c39d640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f293006d750 0x7f293006fc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2940004d40 tx=0x7f294000b040 comp rx=0 tx=0).stop 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.619+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 msgr2=0x7f2944113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.619+0000 7f294c39d640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 0x7f2944113200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2934009fa0 tx=0x7f2934009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.620+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 shutdown_connections 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.620+0000 7f294c39d640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2944102d10 0x7f2944113200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.620+0000 7f294c39d640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f293006d750 0x7f293006fc00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.620+0000 7f294c39d640 1 --2- 172.21.15.64:0/1662503113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2944102400 0x7f2944112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.620+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 >> 172.21.15.64:0/1662503113 conn(0x7f2944070a10 msgr2=0x7f29440f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.620+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 shutdown_connections 2024-02-16T23:08:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.620+0000 7f294c39d640 1 -- 172.21.15.64:0/1662503113 wait complete. 2024-02-16T23:08:16.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-16T23:08:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.729+0000 7f11c599d640 1 -- 172.21.15.64:0/3367973002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 msgr2=0x7f11c01027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.729+0000 7f11c599d640 1 --2- 172.21.15.64:0/3367973002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 0x7f11c01027b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f11b80048d0 tx=0x7f11b8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.729+0000 7f11c599d640 1 -- 172.21.15.64:0/3367973002 shutdown_connections 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.729+0000 7f11c599d640 1 --2- 172.21.15.64:0/3367973002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11c00644b0 0x7f11c00648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.729+0000 7f11c599d640 1 --2- 172.21.15.64:0/3367973002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 0x7f11c01027b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.729+0000 7f11c599d640 1 -- 172.21.15.64:0/3367973002 >> 172.21.15.64:0/3367973002 conn(0x7f11c0071230 msgr2=0x7f11c0071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.729+0000 7f11c599d640 1 -- 172.21.15.64:0/3367973002 shutdown_connections 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.730+0000 7f11c599d640 1 -- 172.21.15.64:0/3367973002 wait complete. 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.730+0000 7f11c599d640 1 Processor -- start 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.730+0000 7f11c599d640 1 -- start start 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.730+0000 7f11c599d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11c00644b0 0x7f11c0112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.730+0000 7f11c599d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 0x7f11c0113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.730+0000 7f11c599d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0113720 con 0x7f11c01023e0 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.730+0000 7f11c599d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0113890 con 0x7f11c00644b0 2024-02-16T23:08:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.730+0000 7f11be7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 0x7f11c0113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11be7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 0x7f11c0113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35050/0 (socket says 172.21.15.64:35050) 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11be7fc640 1 -- 172.21.15.64:0/2787352178 learned_addr learned my addr 172.21.15.64:0/2787352178 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11beffd640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11c00644b0 0x7f11c0112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11be7fc640 1 -- 172.21.15.64:0/2787352178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11c00644b0 msgr2=0x7f11c0112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11be7fc640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11c00644b0 0x7f11c0112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11be7fc640 1 -- 172.21.15.64:0/2787352178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8005040 con 0x7f11c01023e0 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11be7fc640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 0x7f11c0113120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f11a800c790 tx=0x7f11a80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11b77fe640 1 -- 172.21.15.64:0/2787352178 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a800a820 con 0x7f11c01023e0 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11b77fe640 1 -- 172.21.15.64:0/2787352178 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11a800ae50 con 0x7f11c01023e0 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c010ce60 con 0x7f11c01023e0 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11b77fe640 1 -- 172.21.15.64:0/2787352178 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a80115e0 con 0x7f11c01023e0 2024-02-16T23:08:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.731+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c010d380 con 0x7f11c01023e0 2024-02-16T23:08:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.732+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c0064030 con 0x7f11c01023e0 2024-02-16T23:08:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.733+0000 7f11b77fe640 1 -- 172.21.15.64:0/2787352178 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f11a8012400 con 0x7f11c01023e0 2024-02-16T23:08:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.733+0000 7f11b77fe640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f11ac06d750 0x7f11ac06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.733+0000 7f11beffd640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f11ac06d750 0x7f11ac06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.733+0000 7f11b77fe640 1 -- 172.21.15.64:0/2787352178 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f11a808d090 con 0x7f11c01023e0 2024-02-16T23:08:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.734+0000 7f11beffd640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f11ac06d750 0x7f11ac06fc00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f11b8000c00 tx=0x7f11b8009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:16.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.738+0000 7f11b77fe640 1 -- 172.21.15.64:0/2787352178 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11a8056860 con 0x7f11c01023e0 2024-02-16T23:08:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.957+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f11c01094b0 con 0x7f11c01023e0 2024-02-16T23:08:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.957+0000 7f11b77fe640 1 -- 172.21.15.64:0/2787352178 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v35) v1 ==== 107+0+4088 (secure 0 0 0) 0x7f11a8059f20 con 0x7f11c01023e0 2024-02-16T23:08:16.957 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:16.957 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:08.708238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":157,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 23 2024-02-16T23:08:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f11ac06d750 msgr2=0x7f11ac06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f11ac06d750 0x7f11ac06fc00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f11b8000c00 tx=0x7f11b8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 msgr2=0x7f11c0113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 0x7f11c0113120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f11a800c790 tx=0x7f11a80043c0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 shutdown_connections 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11c01023e0 0x7f11c0113120 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f11ac06d750 0x7f11ac06fc00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 --2- 172.21.15.64:0/2787352178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11c00644b0 0x7f11c0112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 >> 172.21.15.64:0/2787352178 conn(0x7f11c0071230 msgr2=0x7f11c00fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 shutdown_connections 2024-02-16T23:08:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:16.959+0000 7f11c599d640 1 -- 172.21.15.64:0/2787352178 wait complete. 2024-02-16T23:08:16.969 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:16 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1385293373' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-16T23:08:16.969 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:16 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1515607419' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-16T23:08:16.969 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:16 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1662503113' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-16T23:08:16.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-16T23:08:17.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.069+0000 7f2a6275d640 1 -- 172.21.15.64:0/3347829814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101c10 msgr2=0x7f2a5c0fb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.069+0000 7f2a6275d640 1 --2- 172.21.15.64:0/3347829814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101c10 0x7f2a5c0fb2a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0048d0 tx=0x7f2a4c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.069+0000 7f2a6275d640 1 -- 172.21.15.64:0/3347829814 shutdown_connections 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.069+0000 7f2a6275d640 1 --2- 172.21.15.64:0/3347829814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101c10 0x7f2a5c0fb2a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.069+0000 7f2a6275d640 1 --2- 172.21.15.64:0/3347829814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a5c101270 0x7f2a5c101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.069+0000 7f2a6275d640 1 -- 172.21.15.64:0/3347829814 >> 172.21.15.64:0/3347829814 conn(0x7f2a5c0f6de0 msgr2=0x7f2a5c0f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.069+0000 7f2a6275d640 1 -- 172.21.15.64:0/3347829814 shutdown_connections 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.069+0000 7f2a6275d640 1 -- 172.21.15.64:0/3347829814 wait complete. 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a6275d640 1 Processor -- start 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a6275d640 1 -- start start 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a6275d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101270 0x7f2a5c112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a6275d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a5c101c10 0x7f2a5c113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a6275d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c113740 con 0x7f2a5c101270 2024-02-16T23:08:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a6275d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c10ccd0 con 0x7f2a5c101c10 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a5bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101270 0x7f2a5c112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a5bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101270 0x7f2a5c112c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35056/0 (socket says 172.21.15.64:35056) 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a5bfff640 1 -- 172.21.15.64:0/1010255156 learned_addr learned my addr 172.21.15.64:0/1010255156 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a5b7fe640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a5c101c10 0x7f2a5c113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a5bfff640 1 -- 172.21.15.64:0/1010255156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a5c101c10 msgr2=0x7f2a5c113140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a5bfff640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a5c101c10 0x7f2a5c113140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a5bfff640 1 -- 172.21.15.64:0/1010255156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c005040 con 0x7f2a5c101270 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.070+0000 7f2a5bfff640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101270 0x7f2a5c112c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2a44002a50 tx=0x7f2a44004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.071+0000 7f2a58ff9640 1 -- 172.21.15.64:0/1010255156 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a44007c30 con 0x7f2a5c101270 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.071+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c10cfb0 con 0x7f2a5c101270 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.071+0000 7f2a58ff9640 1 -- 172.21.15.64:0/1010255156 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a4400f040 con 0x7f2a5c101270 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.071+0000 7f2a58ff9640 1 -- 172.21.15.64:0/1010255156 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a4400db50 con 0x7f2a5c101270 2024-02-16T23:08:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.071+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c10d4a0 con 0x7f2a5c101270 2024-02-16T23:08:17.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.072+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c064480 con 0x7f2a5c101270 2024-02-16T23:08:17.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.072+0000 7f2a58ff9640 1 -- 172.21.15.64:0/1010255156 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2a4401c400 con 0x7f2a5c101270 2024-02-16T23:08:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.072+0000 7f2a58ff9640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a4806d750 0x7f2a4806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.073+0000 7f2a58ff9640 1 -- 172.21.15.64:0/1010255156 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2a4408c8b0 con 0x7f2a5c101270 2024-02-16T23:08:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.073+0000 7f2a5b7fe640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a4806d750 0x7f2a4806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.073+0000 7f2a5b7fe640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a4806d750 0x7f2a4806fc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c10de10 tx=0x7f2a4c009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:17.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.077+0000 7f2a58ff9640 1 -- 172.21.15.64:0/1010255156 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a44056000 con 0x7f2a5c101270 2024-02-16T23:08:17.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:16 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1385293373' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-16T23:08:17.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:16 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1515607419' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-16T23:08:17.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:16 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1662503113' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-16T23:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.294+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f2a5c10d6c0 con 0x7f2a5c101270 2024-02-16T23:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.295+0000 7f2a58ff9640 1 -- 172.21.15.64:0/1010255156 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v35) v1 ==== 107+0+4901 (secure 0 0 0) 0x7f2a440596c0 con 0x7f2a5c101270 2024-02-16T23:08:17.295 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:17.295 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:09.775012+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi192.tnppti","rank":0,"incarnation":21,"state":"up:active","state_seq":158,"addr":"172.21.15.192:6825/1462436420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1462436420},{"type":"v1","addr":"172.21.15.192:6825","nonce":1462436420}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:17.295 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a4806d750 msgr2=0x7f2a4806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a4806d750 0x7f2a4806fc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c10de10 tx=0x7f2a4c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101270 msgr2=0x7f2a5c112c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101270 0x7f2a5c112c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2a44002a50 tx=0x7f2a44004600 comp rx=0 tx=0).stop 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 shutdown_connections 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a4806d750 0x7f2a4806fc00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a5c101c10 0x7f2a5c113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 --2- 172.21.15.64:0/1010255156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5c101270 0x7f2a5c112c00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 >> 172.21.15.64:0/1010255156 conn(0x7f2a5c0f6de0 msgr2=0x7f2a5c065690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:17.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 shutdown_connections 2024-02-16T23:08:17.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.297+0000 7f2a6275d640 1 -- 172.21.15.64:0/1010255156 wait complete. 2024-02-16T23:08:17.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-16T23:08:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.406+0000 7f921fcae640 1 -- 172.21.15.64:0/3816122051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92181012d0 msgr2=0x7f92181016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.406+0000 7f921fcae640 1 --2- 172.21.15.64:0/3816122051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92181012d0 0x7f92181016a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f92080048d0 tx=0x7f9208009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.406+0000 7f921fcae640 1 -- 172.21.15.64:0/3816122051 shutdown_connections 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.406+0000 7f921fcae640 1 --2- 172.21.15.64:0/3816122051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92180fb2b0 0x7f92180fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.406+0000 7f921fcae640 1 --2- 172.21.15.64:0/3816122051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92181012d0 0x7f92181016a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.406+0000 7f921fcae640 1 -- 172.21.15.64:0/3816122051 >> 172.21.15.64:0/3816122051 conn(0x7f9218071230 msgr2=0x7f9218071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921fcae640 1 -- 172.21.15.64:0/3816122051 shutdown_connections 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921fcae640 1 -- 172.21.15.64:0/3816122051 wait complete. 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921fcae640 1 Processor -- start 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921fcae640 1 -- start start 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921fcae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92180fb2b0 0x7f92181020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921fcae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92181012d0 0x7f9218102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921fcae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218104080 con 0x7f92180fb2b0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921fcae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218102b60 con 0x7f92181012d0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921da4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92180fb2b0 0x7f92181020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921da4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92180fb2b0 0x7f92181020e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35082/0 (socket says 172.21.15.64:35082) 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.407+0000 7f921da4a640 1 -- 172.21.15.64:0/1605077769 learned_addr learned my addr 172.21.15.64:0/1605077769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f921d249640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92181012d0 0x7f9218102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f921da4a640 1 -- 172.21.15.64:0/1605077769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92181012d0 msgr2=0x7f9218102620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f921da4a640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92181012d0 0x7f9218102620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f921da4a640 1 -- 172.21.15.64:0/1605077769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9208005040 con 0x7f92180fb2b0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f921d249640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92181012d0 0x7f9218102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f921da4a640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92180fb2b0 0x7f92181020e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9208031720 tx=0x7f9208009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f920e7fc640 1 -- 172.21.15.64:0/1605077769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92080094c0 con 0x7f92180fb2b0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f920e7fc640 1 -- 172.21.15.64:0/1605077769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9208009620 con 0x7f92180fb2b0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9218102db0 con 0x7f92180fb2b0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f920e7fc640 1 -- 172.21.15.64:0/1605077769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f920803b640 con 0x7f92180fb2b0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.408+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9218119240 con 0x7f92180fb2b0 2024-02-16T23:08:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.409+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9218064030 con 0x7f92180fb2b0 2024-02-16T23:08:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.409+0000 7f920e7fc640 1 -- 172.21.15.64:0/1605077769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f920802f070 con 0x7f92180fb2b0 2024-02-16T23:08:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.410+0000 7f920e7fc640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f920406d750 0x7f920406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.410+0000 7f920e7fc640 1 -- 172.21.15.64:0/1605077769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f92080af430 con 0x7f92180fb2b0 2024-02-16T23:08:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.410+0000 7f921d249640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f920406d750 0x7f920406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.410+0000 7f921d249640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f920406d750 0x7f920406fc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f92181038d0 tx=0x7f921400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.414+0000 7f920e7fc640 1 -- 172.21.15.64:0/1605077769 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9208078c00 con 0x7f92180fb2b0 2024-02-16T23:08:17.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.633+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f9218119420 con 0x7f92180fb2b0 2024-02-16T23:08:17.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.633+0000 7f920e7fc640 1 -- 172.21.15.64:0/1605077769 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v35) v1 ==== 107+0+4106 (secure 0 0 0) 0x7f920807c2c0 con 0x7f92180fb2b0 2024-02-16T23:08:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:17.633 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:23.398645+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-02-16T23:08:17.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f920406d750 msgr2=0x7f920406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f920406d750 0x7f920406fc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f92181038d0 tx=0x7f921400b040 comp rx=0 tx=0).stop 2024-02-16T23:08:17.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92180fb2b0 msgr2=0x7f92181020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92180fb2b0 0x7f92181020e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9208031720 tx=0x7f9208009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 shutdown_connections 2024-02-16T23:08:17.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f920406d750 0x7f920406fc00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92181012d0 0x7f9218102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 --2- 172.21.15.64:0/1605077769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92180fb2b0 0x7f92181020e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 >> 172.21.15.64:0/1605077769 conn(0x7f9218071230 msgr2=0x7f9218064500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:17.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 shutdown_connections 2024-02-16T23:08:17.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.635+0000 7f921fcae640 1 -- 172.21.15.64:0/1605077769 wait complete. 2024-02-16T23:08:17.646 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-16T23:08:17.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 -- 172.21.15.64:0/1144652548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 msgr2=0x7f9f180fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 --2- 172.21.15.64:0/1144652548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 0x7f9f180fb270 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0048d0 tx=0x7f9f0c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 -- 172.21.15.64:0/1144652548 shutdown_connections 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 --2- 172.21.15.64:0/1144652548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 0x7f9f180fb270 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 --2- 172.21.15.64:0/1144652548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18101240 0x7f9f18101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 -- 172.21.15.64:0/1144652548 >> 172.21.15.64:0/1144652548 conn(0x7f9f180f6e10 msgr2=0x7f9f180f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 -- 172.21.15.64:0/1144652548 shutdown_connections 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 -- 172.21.15.64:0/1144652548 wait complete. 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 Processor -- start 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 -- start start 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18101240 0x7f9f1810dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 0x7f9f1810e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1810e7b0 con 0x7f9f18101be0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.746+0000 7f9f2075d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1810e920 con 0x7f9f18101240 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f1dcf8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 0x7f9f1810e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f1dcf8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 0x7f9f1810e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35110/0 (socket says 172.21.15.64:35110) 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f1dcf8640 1 -- 172.21.15.64:0/3767612378 learned_addr learned my addr 172.21.15.64:0/3767612378 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f1e4f9640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18101240 0x7f9f1810dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f1dcf8640 1 -- 172.21.15.64:0/3767612378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18101240 msgr2=0x7f9f1810dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f1dcf8640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18101240 0x7f9f1810dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f1dcf8640 1 -- 172.21.15.64:0/3767612378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c005040 con 0x7f9f18101be0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f1dcf8640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 0x7f9f1810e270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c031720 tx=0x7f9f0c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f16ffd640 1 -- 172.21.15.64:0/3767612378 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f0c0094c0 con 0x7f9f18101be0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f16ffd640 1 -- 172.21.15.64:0/3767612378 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f0c009620 con 0x7f9f18101be0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1810eba0 con 0x7f9f18101be0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f16ffd640 1 -- 172.21.15.64:0/3767612378 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f0c03b640 con 0x7f9f18101be0 2024-02-16T23:08:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.747+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f181170a0 con 0x7f9f18101be0 2024-02-16T23:08:17.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.748+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f18064480 con 0x7f9f18101be0 2024-02-16T23:08:17.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.749+0000 7f9f16ffd640 1 -- 172.21.15.64:0/3767612378 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9f0c02f070 con 0x7f9f18101be0 2024-02-16T23:08:17.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.749+0000 7f9f16ffd640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f9f0006d750 0x7f9f0006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:17.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.749+0000 7f9f16ffd640 1 -- 172.21.15.64:0/3767612378 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f9f0c0af1b0 con 0x7f9f18101be0 2024-02-16T23:08:17.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.749+0000 7f9f1e4f9640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f9f0006d750 0x7f9f0006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:17.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.750+0000 7f9f1e4f9640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f9f0006d750 0x7f9f0006fc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f10004ed0 tx=0x7f9f1000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:17.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.754+0000 7f9f16ffd640 1 -- 172.21.15.64:0/3767612378 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f0c078980 con 0x7f9f18101be0 2024-02-16T23:08:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.971+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f9f1810eb40 con 0x7f9f18101be0 2024-02-16T23:08:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.971+0000 7f9f16ffd640 1 -- 172.21.15.64:0/3767612378 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v35) v1 ==== 107+0+4117 (secure 0 0 0) 0x7f9f0c07c040 con 0x7f9f18101be0 2024-02-16T23:08:17.971 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:17.972 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:23.404386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24297":{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.973+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f9f0006d750 msgr2=0x7f9f0006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.973+0000 7f9f2075d640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f9f0006d750 0x7f9f0006fc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f10004ed0 tx=0x7f9f1000b040 comp rx=0 tx=0).stop 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.973+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 msgr2=0x7f9f1810e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.973+0000 7f9f2075d640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 0x7f9f1810e270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c031720 tx=0x7f9f0c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.973+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 shutdown_connections 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.974+0000 7f9f2075d640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f18101be0 0x7f9f1810e270 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.974+0000 7f9f2075d640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f9f0006d750 0x7f9f0006fc00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.974+0000 7f9f2075d640 1 --2- 172.21.15.64:0/3767612378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18101240 0x7f9f1810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.974+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 >> 172.21.15.64:0/3767612378 conn(0x7f9f180f6e10 msgr2=0x7f9f180f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.974+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 shutdown_connections 2024-02-16T23:08:17.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:17.974+0000 7f9f2075d640 1 -- 172.21.15.64:0/3767612378 wait complete. 2024-02-16T23:08:17.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:17 smithi064 ceph-mon[93757]: pgmap v315: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:17.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:17 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2787352178' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-16T23:08:17.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:17 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1010255156' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-16T23:08:17.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:17 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1605077769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-16T23:08:17.983 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:08:17.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-16T23:08:18.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.083+0000 7fa646c07640 1 -- 172.21.15.64:0/3849434559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 msgr2=0x7fa6400faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.083+0000 7fa646c07640 1 --2- 172.21.15.64:0/3849434559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 0x7fa6400faee0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa6280048d0 tx=0x7fa628009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.083+0000 7fa646c07640 1 -- 172.21.15.64:0/3849434559 shutdown_connections 2024-02-16T23:08:18.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.083+0000 7fa646c07640 1 --2- 172.21.15.64:0/3849434559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 0x7fa6400faee0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.083+0000 7fa646c07640 1 --2- 172.21.15.64:0/3849434559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6400fe2f0 0x7fa6400fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.083+0000 7fa646c07640 1 -- 172.21.15.64:0/3849434559 >> 172.21.15.64:0/3849434559 conn(0x7fa640071950 msgr2=0x7fa640071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.083+0000 7fa646c07640 1 -- 172.21.15.64:0/3849434559 shutdown_connections 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.083+0000 7fa646c07640 1 -- 172.21.15.64:0/3849434559 wait complete. 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa646c07640 1 Processor -- start 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa646c07640 1 -- start start 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa646c07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 0x7fa640112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa646c07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6400fe2f0 0x7fa640113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa646c07640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa640113750 con 0x7fa6400faa70 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa646c07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401138c0 con 0x7fa6400fe2f0 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa6449a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 0x7fa640112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa6449a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 0x7fa640112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35124/0 (socket says 172.21.15.64:35124) 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa6449a3640 1 -- 172.21.15.64:0/4150341556 learned_addr learned my addr 172.21.15.64:0/4150341556 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:18.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa63ffff640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6400fe2f0 0x7fa640113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa6449a3640 1 -- 172.21.15.64:0/4150341556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6400fe2f0 msgr2=0x7fa640113150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa6449a3640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6400fe2f0 0x7fa640113150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa6449a3640 1 -- 172.21.15.64:0/4150341556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628005040 con 0x7fa6400faa70 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.084+0000 7fa63ffff640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6400fe2f0 0x7fa640113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.085+0000 7fa6449a3640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 0x7fa640112c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa63800db50 tx=0x7fa638004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.085+0000 7fa63d7fa640 1 -- 172.21.15.64:0/4150341556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa63800b9d0 con 0x7fa6400faa70 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.085+0000 7fa63d7fa640 1 -- 172.21.15.64:0/4150341556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa63800f040 con 0x7fa6400faa70 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.085+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64010ce90 con 0x7fa6400faa70 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.085+0000 7fa63d7fa640 1 -- 172.21.15.64:0/4150341556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6380125d0 con 0x7fa6400faa70 2024-02-16T23:08:18.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.085+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64010d3b0 con 0x7fa6400faa70 2024-02-16T23:08:18.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.086+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa640064480 con 0x7fa6400faa70 2024-02-16T23:08:18.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.086+0000 7fa63d7fa640 1 -- 172.21.15.64:0/4150341556 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa638013430 con 0x7fa6400faa70 2024-02-16T23:08:18.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.087+0000 7fa63d7fa640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa62c06d750 0x7fa62c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.087+0000 7fa63d7fa640 1 -- 172.21.15.64:0/4150341556 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa63808d470 con 0x7fa6400faa70 2024-02-16T23:08:18.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.087+0000 7fa63ffff640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa62c06d750 0x7fa62c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.087+0000 7fa63ffff640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa62c06d750 0x7fa62c06fc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa62802bda0 tx=0x7fa628009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:18.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.091+0000 7fa63d7fa640 1 -- 172.21.15.64:0/4150341556 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa638056c40 con 0x7fa6400faa70 2024-02-16T23:08:18.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:17 smithi192 ceph-mon[82652]: pgmap v315: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:17 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2787352178' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-16T23:08:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:17 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1010255156' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-16T23:08:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:17 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1605077769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-16T23:08:18.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:08:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.309+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fa64010e250 con 0x7fa6400faa70 2024-02-16T23:08:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.310+0000 7fa63d7fa640 1 -- 172.21.15.64:0/4150341556 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v35) v1 ==== 107+0+4122 (secure 0 0 0) 0x7fa64010e250 con 0x7fa6400faa70 2024-02-16T23:08:18.309 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:18.310 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:32.644829+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24297":{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":161,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa62c06d750 msgr2=0x7fa62c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa62c06d750 0x7fa62c06fc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa62802bda0 tx=0x7fa628009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 msgr2=0x7fa640112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 0x7fa640112c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa63800db50 tx=0x7fa638004ee0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 shutdown_connections 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa62c06d750 0x7fa62c06fc00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6400fe2f0 0x7fa640113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 --2- 172.21.15.64:0/4150341556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6400faa70 0x7fa640112c10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 >> 172.21.15.64:0/4150341556 conn(0x7fa640071950 msgr2=0x7fa6400f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 shutdown_connections 2024-02-16T23:08:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.312+0000 7fa646c07640 1 -- 172.21.15.64:0/4150341556 wait complete. 2024-02-16T23:08:18.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-02-16T23:08:18.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.422+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1242127469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf01012d0 msgr2=0x7f4cf01016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.422+0000 7f4cf50ae640 1 --2- 172.21.15.64:0/1242127469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf01012d0 0x7f4cf01016a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80048d0 tx=0x7f4cd8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.422+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1242127469 shutdown_connections 2024-02-16T23:08:18.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.422+0000 7f4cf50ae640 1 --2- 172.21.15.64:0/1242127469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cf00fb2b0 0x7f4cf00fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.422+0000 7f4cf50ae640 1 --2- 172.21.15.64:0/1242127469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf01012d0 0x7f4cf01016a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.422+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1242127469 >> 172.21.15.64:0/1242127469 conn(0x7f4cf0071230 msgr2=0x7f4cf0071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:18.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.422+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1242127469 shutdown_connections 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.422+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1242127469 wait complete. 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4cf50ae640 1 Processor -- start 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4cf50ae640 1 -- start start 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4cf50ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf00fb2b0 0x7f4cf010dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4cf50ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cf01012d0 0x7f4cf010e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4cf50ae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf010e760 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4cf50ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf010e8d0 con 0x7f4cf01012d0 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4ceed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf00fb2b0 0x7f4cf010dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4ceed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf00fb2b0 0x7f4cf010dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35140/0 (socket says 172.21.15.64:35140) 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.423+0000 7f4ceed9d640 1 -- 172.21.15.64:0/1905707518 learned_addr learned my addr 172.21.15.64:0/1905707518 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4cee59c640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cf01012d0 0x7f4cf010e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4ceed9d640 1 -- 172.21.15.64:0/1905707518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cf01012d0 msgr2=0x7f4cf010e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4ceed9d640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cf01012d0 0x7f4cf010e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4ceed9d640 1 -- 172.21.15.64:0/1905707518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd8005040 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4ceed9d640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf00fb2b0 0x7f4cf010dce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8031720 tx=0x7f4cd8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:18.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4ce77fe640 1 -- 172.21.15.64:0/1905707518 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd80094c0 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4ce77fe640 1 -- 172.21.15.64:0/1905707518 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cd8009620 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf010eb50 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4ce77fe640 1 -- 172.21.15.64:0/1905707518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd803b640 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.424+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf0117050 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.425+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf0064030 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.426+0000 7f4ce77fe640 1 -- 172.21.15.64:0/1905707518 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4cd802f070 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.426+0000 7f4ce77fe640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4cdc06d750 0x7f4cdc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.426+0000 7f4ce77fe640 1 -- 172.21.15.64:0/1905707518 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4cd80af390 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.426+0000 7f4cee59c640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4cdc06d750 0x7f4cdc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.427+0000 7f4cee59c640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4cdc06d750 0x7f4cdc06fc00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0007fd0 tx=0x7f4ce0009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:18.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.431+0000 7f4ce77fe640 1 -- 172.21.15.64:0/1905707518 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4cd8078ae0 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.649+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f4cf010eaf0 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.650+0000 7f4ce77fe640 1 -- 172.21.15.64:0/1905707518 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v35) v1 ==== 107+0+4119 (secure 0 0 0) 0x7f4cd807c1a0 con 0x7f4cf00fb2b0 2024-02-16T23:08:18.649 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:18.650 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:33.577215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24297":{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":162,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 28 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4cdc06d750 msgr2=0x7f4cdc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4cdc06d750 0x7f4cdc06fc00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0007fd0 tx=0x7f4ce0009e90 comp rx=0 tx=0).stop 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf00fb2b0 msgr2=0x7f4cf010dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf00fb2b0 0x7f4cf010dce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8031720 tx=0x7f4cd8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 shutdown_connections 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f4cdc06d750 0x7f4cdc06fc00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cf01012d0 0x7f4cf010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 --2- 172.21.15.64:0/1905707518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf00fb2b0 0x7f4cf010dce0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 >> 172.21.15.64:0/1905707518 conn(0x7f4cf0071230 msgr2=0x7f4cf00fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 shutdown_connections 2024-02-16T23:08:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.652+0000 7f4cf50ae640 1 -- 172.21.15.64:0/1905707518 wait complete. 2024-02-16T23:08:18.662 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-02-16T23:08:18.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 -- 172.21.15.64:0/623488817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fdd30 msgr2=0x7f79a8106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 --2- 172.21.15.64:0/623488817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fdd30 0x7f79a8106370 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f799c0048d0 tx=0x7f799c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 -- 172.21.15.64:0/623488817 shutdown_connections 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 --2- 172.21.15.64:0/623488817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fdd30 0x7f79a8106370 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 --2- 172.21.15.64:0/623488817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80fd390 0x7f79a80fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 -- 172.21.15.64:0/623488817 >> 172.21.15.64:0/623488817 conn(0x7f79a80f6de0 msgr2=0x7f79a80f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 -- 172.21.15.64:0/623488817 shutdown_connections 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 -- 172.21.15.64:0/623488817 wait complete. 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.762+0000 7f79add9d640 1 Processor -- start 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79add9d640 1 -- start start 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79add9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fd390 0x7f79a8112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79add9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80fdd30 0x7f79a81131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79add9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a81137b0 con 0x7f79a80fd390 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79add9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a810cd40 con 0x7f79a80fdd30 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fd390 0x7f79a8112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fd390 0x7f79a8112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35156/0 (socket says 172.21.15.64:35156) 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a77fe640 1 -- 172.21.15.64:0/1977087657 learned_addr learned my addr 172.21.15.64:0/1977087657 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a77fe640 1 -- 172.21.15.64:0/1977087657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80fdd30 msgr2=0x7f79a81131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a6ffd640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80fdd30 0x7f79a81131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a77fe640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80fdd30 0x7f79a81131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a77fe640 1 -- 172.21.15.64:0/1977087657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c005040 con 0x7f79a80fd390 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a6ffd640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80fdd30 0x7f79a81131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f79a77fe640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fd390 0x7f79a8112c70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f799800c790 tx=0x7f79980043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.763+0000 7f798ffff640 1 -- 172.21.15.64:0/1977087657 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f799800a820 con 0x7f79a80fd390 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.764+0000 7f798ffff640 1 -- 172.21.15.64:0/1977087657 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f799800ae50 con 0x7f79a80fd390 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.764+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a810d020 con 0x7f79a80fd390 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.764+0000 7f798ffff640 1 -- 172.21.15.64:0/1977087657 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79980115e0 con 0x7f79a80fd390 2024-02-16T23:08:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.764+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a810d540 con 0x7f79a80fd390 2024-02-16T23:08:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.765+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a8064480 con 0x7f79a80fd390 2024-02-16T23:08:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.765+0000 7f798ffff640 1 -- 172.21.15.64:0/1977087657 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7998012400 con 0x7f79a80fd390 2024-02-16T23:08:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.765+0000 7f798ffff640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f799006d750 0x7f799006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.766+0000 7f79a6ffd640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f799006d750 0x7f799006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.766+0000 7f798ffff640 1 -- 172.21.15.64:0/1977087657 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f799808d060 con 0x7f79a80fd390 2024-02-16T23:08:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.766+0000 7f79a6ffd640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f799006d750 0x7f799006fc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f799c02bda0 tx=0x7f799c009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.770+0000 7f798ffff640 1 -- 172.21.15.64:0/1977087657 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79980568f0 con 0x7f79a80fd390 2024-02-16T23:08:18.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.988+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f79a810e270 con 0x7f79a80fd390 2024-02-16T23:08:18.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.988+0000 7f798ffff640 1 -- 172.21.15.64:0/1977087657 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v35) v1 ==== 107+0+4119 (secure 0 0 0) 0x7f7998059fb0 con 0x7f79a80fd390 2024-02-16T23:08:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:18.988 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:34.657247+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24297":{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":0,"incarnation":26,"state":"up:active","state_seq":163,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:18.988 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 29 2024-02-16T23:08:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f799006d750 msgr2=0x7f799006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f799006d750 0x7f799006fc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f799c02bda0 tx=0x7f799c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fd390 msgr2=0x7f79a8112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fd390 0x7f79a8112c70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f799800c790 tx=0x7f79980043c0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 shutdown_connections 2024-02-16T23:08:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f799006d750 0x7f799006fc00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80fdd30 0x7f79a81131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 --2- 172.21.15.64:0/1977087657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a80fd390 0x7f79a8112c70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 >> 172.21.15.64:0/1977087657 conn(0x7f79a80f6de0 msgr2=0x7f79a8100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 shutdown_connections 2024-02-16T23:08:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:18.990+0000 7f79add9d640 1 -- 172.21.15.64:0/1977087657 wait complete. 2024-02-16T23:08:18.999 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:18 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3767612378' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-16T23:08:18.999 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:18 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/4150341556' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-16T23:08:19.000 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:18 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1905707518' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-16T23:08:19.001 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-02-16T23:08:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 -- 172.21.15.64:0/615490864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101240 msgr2=0x7f025c101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 --2- 172.21.15.64:0/615490864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101240 0x7f025c101610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02500048d0 tx=0x7f0250009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 -- 172.21.15.64:0/615490864 shutdown_connections 2024-02-16T23:08:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 --2- 172.21.15.64:0/615490864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f025c101be0 0x7f025c0fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 --2- 172.21.15.64:0/615490864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101240 0x7f025c101610 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 -- 172.21.15.64:0/615490864 >> 172.21.15.64:0/615490864 conn(0x7f025c0f6e10 msgr2=0x7f025c0f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 -- 172.21.15.64:0/615490864 shutdown_connections 2024-02-16T23:08:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 -- 172.21.15.64:0/615490864 wait complete. 2024-02-16T23:08:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 Processor -- start 2024-02-16T23:08:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 -- start start 2024-02-16T23:08:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f025c101240 0x7f025c111ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101be0 0x7f025c112000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c112600 con 0x7f025c101be0 2024-02-16T23:08:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.100+0000 7f0262eae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c10bb90 con 0x7f025c101240 2024-02-16T23:08:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f025bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101be0 0x7f025c112000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f025bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101be0 0x7f025c112000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35178/0 (socket says 172.21.15.64:35178) 2024-02-16T23:08:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f025bfff640 1 -- 172.21.15.64:0/2720302459 learned_addr learned my addr 172.21.15.64:0/2720302459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f0260c4a640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f025c101240 0x7f025c111ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f025bfff640 1 -- 172.21.15.64:0/2720302459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f025c101240 msgr2=0x7f025c111ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f025bfff640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f025c101240 0x7f025c111ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f025bfff640 1 -- 172.21.15.64:0/2720302459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250005040 con 0x7f025c101be0 2024-02-16T23:08:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f025bfff640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101be0 0x7f025c112000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0248002a80 tx=0x7f0248004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f02597fa640 1 -- 172.21.15.64:0/2720302459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0248007c30 con 0x7f025c101be0 2024-02-16T23:08:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f02597fa640 1 -- 172.21.15.64:0/2720302459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f024800f040 con 0x7f025c101be0 2024-02-16T23:08:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c10be70 con 0x7f025c101be0 2024-02-16T23:08:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f02597fa640 1 -- 172.21.15.64:0/2720302459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02480096b0 con 0x7f025c101be0 2024-02-16T23:08:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.101+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c10c390 con 0x7f025c101be0 2024-02-16T23:08:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.102+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c064480 con 0x7f025c101be0 2024-02-16T23:08:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.103+0000 7f02597fa640 1 -- 172.21.15.64:0/2720302459 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f024800d6b0 con 0x7f025c101be0 2024-02-16T23:08:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.103+0000 7f02597fa640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f024c06d750 0x7f024c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.103+0000 7f02597fa640 1 -- 172.21.15.64:0/2720302459 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f024808c930 con 0x7f025c101be0 2024-02-16T23:08:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.103+0000 7f0260c4a640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f024c06d750 0x7f024c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.104+0000 7f0260c4a640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f024c06d750 0x7f024c06fc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f0250009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:19.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.108+0000 7f02597fa640 1 -- 172.21.15.64:0/2720302459 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0248056100 con 0x7f025c101be0 2024-02-16T23:08:19.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:18 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3767612378' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-16T23:08:19.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:18 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/4150341556' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-16T23:08:19.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:18 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1905707518' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-16T23:08:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.328+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f025c10c600 con 0x7f025c101be0 2024-02-16T23:08:19.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.328+0000 7f02597fa640 1 -- 172.21.15.64:0/2720302459 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v35) v1 ==== 107+0+4935 (secure 0 0 0) 0x7f02480597c0 con 0x7f025c101be0 2024-02-16T23:08:19.328 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:19.328 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44349,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/2028541376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2028541376},{"type":"v1","addr":"172.21.15.192:6825","nonce":2028541376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:34.657247+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24297":{"gid":24297,"name":"cephfs.smithi192.luiybj","rank":0,"incarnation":26,"state":"up:active","state_seq":163,"addr":"172.21.15.192:6827/2853522025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2853522025},{"type":"v1","addr":"172.21.15.192:6827","nonce":2853522025}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 30 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f024c06d750 msgr2=0x7f024c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f024c06d750 0x7f024c06fc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f0250009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101be0 msgr2=0x7f025c112000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101be0 0x7f025c112000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0248002a80 tx=0x7f0248004600 comp rx=0 tx=0).stop 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 shutdown_connections 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f025c101be0 0x7f025c112000 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f024c06d750 0x7f024c06fc00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 --2- 172.21.15.64:0/2720302459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f025c101240 0x7f025c111ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 >> 172.21.15.64:0/2720302459 conn(0x7f025c0f6e10 msgr2=0x7f025c0f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 shutdown_connections 2024-02-16T23:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.330+0000 7f0262eae640 1 -- 172.21.15.64:0/2720302459 wait complete. 2024-02-16T23:08:19.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.440+0000 7f5b8db14640 1 -- 172.21.15.64:0/1640487410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b881012d0 msgr2=0x7f5b881016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.440+0000 7f5b8db14640 1 --2- 172.21.15.64:0/1640487410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b881012d0 0x7f5b881016a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5b780048d0 tx=0x7f5b78009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.440+0000 7f5b8db14640 1 -- 172.21.15.64:0/1640487410 shutdown_connections 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.440+0000 7f5b8db14640 1 --2- 172.21.15.64:0/1640487410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b880fb2b0 0x7f5b880fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.440+0000 7f5b8db14640 1 --2- 172.21.15.64:0/1640487410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b881012d0 0x7f5b881016a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.440+0000 7f5b8db14640 1 -- 172.21.15.64:0/1640487410 >> 172.21.15.64:0/1640487410 conn(0x7f5b88071230 msgr2=0x7f5b88071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.440+0000 7f5b8db14640 1 -- 172.21.15.64:0/1640487410 shutdown_connections 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.440+0000 7f5b8db14640 1 -- 172.21.15.64:0/1640487410 wait complete. 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b8db14640 1 Processor -- start 2024-02-16T23:08:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b8db14640 1 -- start start 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b8db14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b880fb2b0 0x7f5b8810dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b8db14640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b881012d0 0x7f5b8810e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b8db14640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8810e700 con 0x7f5b880fb2b0 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b8db14640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8810e870 con 0x7f5b881012d0 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b880fb2b0 0x7f5b8810dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b880fb2b0 0x7f5b8810dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35198/0 (socket says 172.21.15.64:35198) 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b877fe640 1 -- 172.21.15.64:0/1797106395 learned_addr learned my addr 172.21.15.64:0/1797106395 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b86ffd640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b881012d0 0x7f5b8810e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b877fe640 1 -- 172.21.15.64:0/1797106395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b881012d0 msgr2=0x7f5b8810e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.441+0000 7f5b877fe640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b881012d0 0x7f5b8810e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.442+0000 7f5b877fe640 1 -- 172.21.15.64:0/1797106395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b78005040 con 0x7f5b880fb2b0 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.442+0000 7f5b877fe640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b880fb2b0 0x7f5b8810dc80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5b78031720 tx=0x7f5b78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.442+0000 7f5b6ffff640 1 -- 172.21.15.64:0/1797106395 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b780094c0 con 0x7f5b880fb2b0 2024-02-16T23:08:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.442+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8810eaf0 con 0x7f5b880fb2b0 2024-02-16T23:08:19.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.442+0000 7f5b6ffff640 1 -- 172.21.15.64:0/1797106395 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b78009620 con 0x7f5b880fb2b0 2024-02-16T23:08:19.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.442+0000 7f5b6ffff640 1 -- 172.21.15.64:0/1797106395 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b7803b640 con 0x7f5b880fb2b0 2024-02-16T23:08:19.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.442+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b88116ff0 con 0x7f5b880fb2b0 2024-02-16T23:08:19.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.443+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b88064030 con 0x7f5b880fb2b0 2024-02-16T23:08:19.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.443+0000 7f5b6ffff640 1 -- 172.21.15.64:0/1797106395 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5b7802f070 con 0x7f5b880fb2b0 2024-02-16T23:08:19.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.444+0000 7f5b6ffff640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5b7006d750 0x7f5b7006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.444+0000 7f5b6ffff640 1 -- 172.21.15.64:0/1797106395 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5b780af460 con 0x7f5b880fb2b0 2024-02-16T23:08:19.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.444+0000 7f5b86ffd640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5b7006d750 0x7f5b7006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.444+0000 7f5b86ffd640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5b7006d750 0x7f5b7006fc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5b88064d50 tx=0x7f5b7c009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:19.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.448+0000 7f5b6ffff640 1 -- 172.21.15.64:0/1797106395 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b78078bb0 con 0x7f5b880fb2b0 2024-02-16T23:08:19.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.666+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f5b881083a0 con 0x7f5b880fb2b0 2024-02-16T23:08:19.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.667+0000 7f5b6ffff640 1 -- 172.21.15.64:0/1797106395 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v35) v1 ==== 107+0+4140 (secure 0 0 0) 0x7f5b7807c270 con 0x7f5b880fb2b0 2024-02-16T23:08:19.667 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:19.667 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44349,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/2028541376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2028541376},{"type":"v1","addr":"172.21.15.192:6825","nonce":2028541376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:47.133727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:19.667 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 31 2024-02-16T23:08:19.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5b7006d750 msgr2=0x7f5b7006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5b7006d750 0x7f5b7006fc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5b88064d50 tx=0x7f5b7c009040 comp rx=0 tx=0).stop 2024-02-16T23:08:19.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b880fb2b0 msgr2=0x7f5b8810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b880fb2b0 0x7f5b8810dc80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5b78031720 tx=0x7f5b78009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 shutdown_connections 2024-02-16T23:08:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f5b7006d750 0x7f5b7006fc00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b881012d0 0x7f5b8810e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 --2- 172.21.15.64:0/1797106395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b880fb2b0 0x7f5b8810dc80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 >> 172.21.15.64:0/1797106395 conn(0x7f5b88071230 msgr2=0x7f5b880fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 shutdown_connections 2024-02-16T23:08:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.669+0000 7f5b8db14640 1 -- 172.21.15.64:0/1797106395 wait complete. 2024-02-16T23:08:19.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 -- 172.21.15.64:0/1083902143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102360 msgr2=0x7f365c102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 --2- 172.21.15.64:0/1083902143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102360 0x7f365c102730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f364c0048d0 tx=0x7f364c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 -- 172.21.15.64:0/1083902143 shutdown_connections 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 --2- 172.21.15.64:0/1083902143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c102d00 0x7f365c0fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 --2- 172.21.15.64:0/1083902143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102360 0x7f365c102730 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 -- 172.21.15.64:0/1083902143 >> 172.21.15.64:0/1083902143 conn(0x7f365c0f7ef0 msgr2=0x7f365c0fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 -- 172.21.15.64:0/1083902143 shutdown_connections 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 -- 172.21.15.64:0/1083902143 wait complete. 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 Processor -- start 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.779+0000 7f3661d9d640 1 -- start start 2024-02-16T23:08:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3661d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c102360 0x7f365c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3661d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102d00 0x7f365c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3661d9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c070220 con 0x7f365c102d00 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3661d9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c070390 con 0x7f365c102360 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f365affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102d00 0x7f365c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f365affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102d00 0x7f365c06fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35216/0 (socket says 172.21.15.64:35216) 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f365affd640 1 -- 172.21.15.64:0/882460028 learned_addr learned my addr 172.21.15.64:0/882460028 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f365b7fe640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c102360 0x7f365c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f365affd640 1 -- 172.21.15.64:0/882460028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c102360 msgr2=0x7f365c06f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f365affd640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c102360 0x7f365c06f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f365affd640 1 -- 172.21.15.64:0/882460028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c005040 con 0x7f365c102d00 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f365affd640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102d00 0x7f365c06fce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f365000b790 tx=0x7f36500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:19.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3643fff640 1 -- 172.21.15.64:0/882460028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365000c700 con 0x7f365c102d00 2024-02-16T23:08:19.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3643fff640 1 -- 172.21.15.64:0/882460028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365000cd30 con 0x7f365c102d00 2024-02-16T23:08:19.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c119100 con 0x7f365c102d00 2024-02-16T23:08:19.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3643fff640 1 -- 172.21.15.64:0/882460028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3650013370 con 0x7f365c102d00 2024-02-16T23:08:19.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.780+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365c119430 con 0x7f365c102d00 2024-02-16T23:08:19.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.781+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365c064480 con 0x7f365c102d00 2024-02-16T23:08:19.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.782+0000 7f3643fff640 1 -- 172.21.15.64:0/882460028 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3650012070 con 0x7f365c102d00 2024-02-16T23:08:19.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.782+0000 7f3643fff640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f364406d750 0x7f364406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:19.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.782+0000 7f3643fff640 1 -- 172.21.15.64:0/882460028 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f365008cc20 con 0x7f365c102d00 2024-02-16T23:08:19.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.782+0000 7f365b7fe640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f364406d750 0x7f364406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:19.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.783+0000 7f365b7fe640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f364406d750 0x7f364406fc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f364c000c00 tx=0x7f364c009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:19.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:19.787+0000 7f3643fff640 1 -- 172.21.15.64:0/882460028 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3650056370 con 0x7f365c102d00 2024-02-16T23:08:20.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.006+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f365c109520 con 0x7f365c102d00 2024-02-16T23:08:20.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.007+0000 7f3643fff640 1 -- 172.21.15.64:0/882460028 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v35) v1 ==== 107+0+4185 (secure 0 0 0) 0x7f3650059a30 con 0x7f365c102d00 2024-02-16T23:08:20.007 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:20.007 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44349,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/2028541376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2028541376},{"type":"v1","addr":"172.21.15.192:6825","nonce":2028541376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:47.141100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:20.007 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 32 2024-02-16T23:08:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f364406d750 msgr2=0x7f364406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f364406d750 0x7f364406fc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f364c000c00 tx=0x7f364c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102d00 msgr2=0x7f365c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102d00 0x7f365c06fce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f365000b790 tx=0x7f36500091d0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 shutdown_connections 2024-02-16T23:08:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c102d00 0x7f365c06fce0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f364406d750 0x7f364406fc00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 --2- 172.21.15.64:0/882460028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c102360 0x7f365c06f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 >> 172.21.15.64:0/882460028 conn(0x7f365c0f7ef0 msgr2=0x7f365c0f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 shutdown_connections 2024-02-16T23:08:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.009+0000 7f3661d9d640 1 -- 172.21.15.64:0/882460028 wait complete. 2024-02-16T23:08:20.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:19 smithi064 ceph-mon[93757]: pgmap v316: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:20.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:19 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1977087657' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-16T23:08:20.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:19 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2720302459' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-16T23:08:20.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:19 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1797106395' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-16T23:08:20.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-02-16T23:08:20.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.119+0000 7f13d0d1d640 1 -- 172.21.15.64:0/3130582172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 msgr2=0x7f13cc101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.119+0000 7f13d0d1d640 1 --2- 172.21.15.64:0/3130582172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 0x7f13cc101660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0048d0 tx=0x7f13bc009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 -- 172.21.15.64:0/3130582172 shutdown_connections 2024-02-16T23:08:20.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 --2- 172.21.15.64:0/3130582172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc101c30 0x7f13cc0fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 --2- 172.21.15.64:0/3130582172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 0x7f13cc101660 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 -- 172.21.15.64:0/3130582172 >> 172.21.15.64:0/3130582172 conn(0x7f13cc0f6e00 msgr2=0x7f13cc0f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 -- 172.21.15.64:0/3130582172 shutdown_connections 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 -- 172.21.15.64:0/3130582172 wait complete. 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 Processor -- start 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 -- start start 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 0x7f13cc111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc101c30 0x7f13cc1120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc1126c0 con 0x7f13cc101290 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13d0d1d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc10bc50 con 0x7f13cc101c30 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 0x7f13cc111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 0x7f13cc111b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35228/0 (socket says 172.21.15.64:35228) 2024-02-16T23:08:20.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.120+0000 7f13ca59c640 1 -- 172.21.15.64:0/4167312299 learned_addr learned my addr 172.21.15.64:0/4167312299 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13c9d9b640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc101c30 0x7f13cc1120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13ca59c640 1 -- 172.21.15.64:0/4167312299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc101c30 msgr2=0x7f13cc1120c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13ca59c640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc101c30 0x7f13cc1120c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13ca59c640 1 -- 172.21.15.64:0/4167312299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc005040 con 0x7f13cc101290 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13c9d9b640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc101c30 0x7f13cc1120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13ca59c640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 0x7f13cc111b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f13bc031720 tx=0x7f13bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13baffd640 1 -- 172.21.15.64:0/4167312299 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13bc0094c0 con 0x7f13cc101290 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13baffd640 1 -- 172.21.15.64:0/4167312299 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13bc009620 con 0x7f13cc101290 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13cc10bed0 con 0x7f13cc101290 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13baffd640 1 -- 172.21.15.64:0/4167312299 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13bc03b640 con 0x7f13cc101290 2024-02-16T23:08:20.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.121+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc10c3f0 con 0x7f13cc101290 2024-02-16T23:08:20.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.122+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13cc064480 con 0x7f13cc101290 2024-02-16T23:08:20.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.122+0000 7f13baffd640 1 -- 172.21.15.64:0/4167312299 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f13bc02f070 con 0x7f13cc101290 2024-02-16T23:08:20.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.123+0000 7f13baffd640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f13b406d750 0x7f13b406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.123+0000 7f13baffd640 1 -- 172.21.15.64:0/4167312299 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f13bc0af570 con 0x7f13cc101290 2024-02-16T23:08:20.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.123+0000 7f13c9d9b640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f13b406d750 0x7f13b406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:20.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.123+0000 7f13c9d9b640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f13b406d750 0x7f13b406fc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13cc10d440 tx=0x7f13c0009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:20.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.127+0000 7f13baffd640 1 -- 172.21.15.64:0/4167312299 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13bc078cc0 con 0x7f13cc101290 2024-02-16T23:08:20.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:19 smithi192 ceph-mon[82652]: pgmap v316: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:20.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:19 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1977087657' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-16T23:08:20.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:19 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2720302459' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-16T23:08:20.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:19 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1797106395' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-16T23:08:20.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.349+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f13cc10c580 con 0x7f13cc101290 2024-02-16T23:08:20.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.349+0000 7f13baffd640 1 -- 172.21.15.64:0/4167312299 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v35) v1 ==== 107+0+4189 (secure 0 0 0) 0x7f13bc07c380 con 0x7f13cc101290 2024-02-16T23:08:20.349 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:20.349 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44349,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/2028541376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2028541376},{"type":"v1","addr":"172.21.15.192:6825","nonce":2028541376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:56.158329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:20.350 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 33 2024-02-16T23:08:20.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f13b406d750 msgr2=0x7f13b406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f13b406d750 0x7f13b406fc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13cc10d440 tx=0x7f13c0009040 comp rx=0 tx=0).stop 2024-02-16T23:08:20.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 msgr2=0x7f13cc111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 0x7f13cc111b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f13bc031720 tx=0x7f13bc009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 shutdown_connections 2024-02-16T23:08:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f13b406d750 0x7f13b406fc00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc101c30 0x7f13cc1120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 --2- 172.21.15.64:0/4167312299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13cc101290 0x7f13cc111b80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 >> 172.21.15.64:0/4167312299 conn(0x7f13cc0f6e00 msgr2=0x7f13cc0f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 shutdown_connections 2024-02-16T23:08:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.351+0000 7f13d0d1d640 1 -- 172.21.15.64:0/4167312299 wait complete. 2024-02-16T23:08:20.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.461+0000 7f45ddeae640 1 -- 172.21.15.64:0/3730840199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fdd30 msgr2=0x7f45d8106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.461+0000 7f45ddeae640 1 --2- 172.21.15.64:0/3730840199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fdd30 0x7f45d8106370 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0048d0 tx=0x7f45cc009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.461+0000 7f45ddeae640 1 -- 172.21.15.64:0/3730840199 shutdown_connections 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.461+0000 7f45ddeae640 1 --2- 172.21.15.64:0/3730840199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fdd30 0x7f45d8106370 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.461+0000 7f45ddeae640 1 --2- 172.21.15.64:0/3730840199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d80fd390 0x7f45d80fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.461+0000 7f45ddeae640 1 -- 172.21.15.64:0/3730840199 >> 172.21.15.64:0/3730840199 conn(0x7f45d80f6de0 msgr2=0x7f45d80f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.461+0000 7f45ddeae640 1 -- 172.21.15.64:0/3730840199 shutdown_connections 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.461+0000 7f45ddeae640 1 -- 172.21.15.64:0/3730840199 wait complete. 2024-02-16T23:08:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45ddeae640 1 Processor -- start 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45ddeae640 1 -- start start 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45ddeae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fd390 0x7f45d8112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45ddeae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d80fdd30 0x7f45d81131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45ddeae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d81137b0 con 0x7f45d80fd390 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45ddeae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d810cd40 con 0x7f45d80fdd30 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fd390 0x7f45d8112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fd390 0x7f45d8112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35244/0 (socket says 172.21.15.64:35244) 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45d77fe640 1 -- 172.21.15.64:0/2913621889 learned_addr learned my addr 172.21.15.64:0/2913621889 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45d77fe640 1 -- 172.21.15.64:0/2913621889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d80fdd30 msgr2=0x7f45d81131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45d77fe640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d80fdd30 0x7f45d81131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45d77fe640 1 -- 172.21.15.64:0/2913621889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc005040 con 0x7f45d80fd390 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.462+0000 7f45d77fe640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fd390 0x7f45d8112c70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c790 tx=0x7f45c80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.463+0000 7f45bffff640 1 -- 172.21.15.64:0/2913621889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c800a820 con 0x7f45d80fd390 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.463+0000 7f45bffff640 1 -- 172.21.15.64:0/2913621889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45c800ae50 con 0x7f45d80fd390 2024-02-16T23:08:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.463+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d810d020 con 0x7f45d80fd390 2024-02-16T23:08:20.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.463+0000 7f45bffff640 1 -- 172.21.15.64:0/2913621889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c80115e0 con 0x7f45d80fd390 2024-02-16T23:08:20.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.463+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d810d540 con 0x7f45d80fd390 2024-02-16T23:08:20.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.464+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8064480 con 0x7f45d80fd390 2024-02-16T23:08:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.464+0000 7f45bffff640 1 -- 172.21.15.64:0/2913621889 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f45c8012400 con 0x7f45d80fd390 2024-02-16T23:08:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.465+0000 7f45bffff640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f45c006d750 0x7f45c006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.465+0000 7f45bffff640 1 -- 172.21.15.64:0/2913621889 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f45c808d150 con 0x7f45d80fd390 2024-02-16T23:08:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.465+0000 7f45d6ffd640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f45c006d750 0x7f45c006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.465+0000 7f45d6ffd640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f45c006d750 0x7f45c006fc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45cc034040 tx=0x7f45cc009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:20.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.469+0000 7f45bffff640 1 -- 172.21.15.64:0/2913621889 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f45c8056920 con 0x7f45d80fd390 2024-02-16T23:08:20.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.688+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f45d810e270 con 0x7f45d80fd390 2024-02-16T23:08:20.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.688+0000 7f45bffff640 1 -- 172.21.15.64:0/2913621889 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v35) v1 ==== 107+0+5002 (secure 0 0 0) 0x7f45c8059fe0 con 0x7f45d80fd390 2024-02-16T23:08:20.688 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:20.688 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi064.bmcyuf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6829/2551190588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2551190588},{"type":"v1","addr":"172.21.15.64:6829","nonce":2551190588}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44349,"name":"cephfs.smithi192.tnppti","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/2028541376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2028541376},{"type":"v1","addr":"172.21.15.192:6825","nonce":2028541376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44353,"name":"cephfs.smithi192.luiybj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/1878053024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1878053024},{"type":"v1","addr":"172.21.15.192:6827","nonce":1878053024}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:53:41.360879+0000","modified":"2024-02-16T23:04:57.036248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi064.tqesxx","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.64:6827/2289872563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2289872563},{"type":"v1","addr":"172.21.15.64:6827","nonce":2289872563}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T23:08:20.688 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 34 2024-02-16T23:08:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f45c006d750 msgr2=0x7f45c006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f45c006d750 0x7f45c006fc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45cc034040 tx=0x7f45cc009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fd390 msgr2=0x7f45d8112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fd390 0x7f45d8112c70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c790 tx=0x7f45c80043c0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 shutdown_connections 2024-02-16T23:08:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f45c006d750 0x7f45c006fc00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d80fdd30 0x7f45d81131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 --2- 172.21.15.64:0/2913621889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d80fd390 0x7f45d8112c70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 >> 172.21.15.64:0/2913621889 conn(0x7f45d80f6de0 msgr2=0x7f45d8100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:20.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 shutdown_connections 2024-02-16T23:08:20.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.690+0000 7f45ddeae640 1 -- 172.21.15.64:0/2913621889 wait complete. 2024-02-16T23:08:20.701 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-02-16T23:08:20.712 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-02-16T23:08:20.713 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T23:08:20.713 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T23:08:20.729 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T23:08:20.729 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T23:08:20.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T23:08:20.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.923+0000 7f20f579d640 1 -- 172.21.15.64:0/1749548688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102280 msgr2=0x7f20f0102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.923+0000 7f20f579d640 1 --2- 172.21.15.64:0/1749548688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102280 0x7f20f0102650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20e80048d0 tx=0x7f20e8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.923+0000 7f20f579d640 1 -- 172.21.15.64:0/1749548688 shutdown_connections 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.923+0000 7f20f579d640 1 --2- 172.21.15.64:0/1749548688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f0102c20 0x7f20f00fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.923+0000 7f20f579d640 1 --2- 172.21.15.64:0/1749548688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102280 0x7f20f0102650 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.923+0000 7f20f579d640 1 -- 172.21.15.64:0/1749548688 >> 172.21.15.64:0/1749548688 conn(0x7f20f00f7e50 msgr2=0x7f20f00fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.923+0000 7f20f579d640 1 -- 172.21.15.64:0/1749548688 shutdown_connections 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.923+0000 7f20f579d640 1 -- 172.21.15.64:0/1749548688 wait complete. 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20f579d640 1 Processor -- start 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20f579d640 1 -- start start 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20f579d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f0102280 0x7f20f0112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20f579d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102c20 0x7f20f0113080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20f579d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0113680 con 0x7f20f0102c20 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20f579d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f010cc10 con 0x7f20f0102280 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20ee7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102c20 0x7f20f0113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20ee7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102c20 0x7f20f0113080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36926/0 (socket says 172.21.15.64:36926) 2024-02-16T23:08:20.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.924+0000 7f20ee7fc640 1 -- 172.21.15.64:0/3298716223 learned_addr learned my addr 172.21.15.64:0/3298716223 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20eeffd640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f0102280 0x7f20f0112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20ee7fc640 1 -- 172.21.15.64:0/3298716223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f0102280 msgr2=0x7f20f0112b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20ee7fc640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f0102280 0x7f20f0112b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20ee7fc640 1 -- 172.21.15.64:0/3298716223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8005040 con 0x7f20f0102c20 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20eeffd640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f0102280 0x7f20f0112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20ee7fc640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102c20 0x7f20f0113080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20d800c790 tx=0x7f20d80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20e77fe640 1 -- 172.21.15.64:0/3298716223 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d800a820 con 0x7f20f0102c20 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20e77fe640 1 -- 172.21.15.64:0/3298716223 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20d800ae50 con 0x7f20f0102c20 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f010cef0 con 0x7f20f0102c20 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20e77fe640 1 -- 172.21.15.64:0/3298716223 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d80115e0 con 0x7f20f0102c20 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.925+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f010d410 con 0x7f20f0102c20 2024-02-16T23:08:20.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.926+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f0064030 con 0x7f20f0102c20 2024-02-16T23:08:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.926+0000 7f20e77fe640 1 -- 172.21.15.64:0/3298716223 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f20d8012400 con 0x7f20f0102c20 2024-02-16T23:08:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.927+0000 7f20e77fe640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f20dc06d750 0x7f20dc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.927+0000 7f20e77fe640 1 -- 172.21.15.64:0/3298716223 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f20d808d150 con 0x7f20f0102c20 2024-02-16T23:08:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.927+0000 7f20eeffd640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f20dc06d750 0x7f20dc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:20.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.927+0000 7f20eeffd640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f20dc06d750 0x7f20dc06fc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f20e802bda0 tx=0x7f20e8009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:20.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:20.931+0000 7f20e77fe640 1 -- 172.21.15.64:0/3298716223 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20d8056920 con 0x7f20f0102c20 2024-02-16T23:08:21.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.112+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20f010d690 con 0x7f20f0102c20 2024-02-16T23:08:21.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.113+0000 7f20e77fe640 1 -- 172.21.15.64:0/3298716223 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16913 (secure 0 0 0) 0x7f20d8059fe0 con 0x7f20f0102c20 2024-02-16T23:08:21.113 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:21.114 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","created":"2024-02-16T22:46:56.779041+0000","modified":"2024-02-16T23:04:47.133740+0000","last_up_change":"2024-02-16T23:03:26.379108+0000","last_in_change":"2024-02-16T22:52:04.211979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:51:12.402108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T22:53:40.294617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T22:53:40.597634+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7a680d8c-0559-46c6-9bfc-95fca252a657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6803","nonce":3681251107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6805","nonce":3681251107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6809","nonce":3681251107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6807","nonce":3681251107}]},"public_addr":"172.21.15.64:6803/3681251107","cluster_addr":"172.21.15.64:6805/3681251107","heartbeat_back_addr":"172.21.15.64:6809/3681251107","heartbeat_front_addr":"172.21.15.64:6807/3681251107","state":["exists","up"]},{"osd":1,"uuid":"21a8757d-9f77-4f46-83d3-ff904c9e50b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6811","nonce":1635803238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6813","nonce":1635803238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6817","nonce":1635803238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6815","nonce":1635803238}]},"public_addr":"172.21.15.64:6811/1635803238","cluster_addr":"172.21.15.64:6813/1635803238","heartbeat_back_addr":"172.21.15.64:6817/1635803238","heartbeat_front_addr":"172.21.15.64:6815/1635803238","state":["exists","up"]},{"osd":2,"uuid":"3fcb2e12-54a0-46db-af11-d047cf25d50b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6819","nonce":530939834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6821","nonce":530939834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6825","nonce":530939834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6823","nonce":530939834}]},"public_addr":"172.21.15.64:6819/530939834","cluster_addr":"172.21.15.64:6821/530939834","heartbeat_back_addr":"172.21.15.64:6825/530939834","heartbeat_front_addr":"172.21.15.64:6823/530939834","state":["exists","up"]},{"osd":3,"uuid":"a7a58c77-96fa-4835-9ea4-d16e6d995620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6801","nonce":2367052696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6803","nonce":2367052696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6807","nonce":2367052696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6805","nonce":2367052696}]},"public_addr":"172.21.15.192:6801/2367052696","cluster_addr":"172.21.15.192:6803/2367052696","heartbeat_back_addr":"172.21.15.192:6807/2367052696","heartbeat_front_addr":"172.21.15.192:6805/2367052696","state":["exists","up"]},{"osd":4,"uuid":"04e094c5-2494-457a-a4a5-7b2742be9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6809","nonce":2408603569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6811","nonce":2408603569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6815","nonce":2408603569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6813","nonce":2408603569}]},"public_addr":"172.21.15.192:6809/2408603569","cluster_addr":"172.21.15.192:6811/2408603569","heartbeat_back_addr":"172.21.15.192:6815/2408603569","heartbeat_front_addr":"172.21.15.192:6813/2408603569","state":["exists","up"]},{"osd":5,"uuid":"bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6817","nonce":2198637799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6819","nonce":2198637799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6823","nonce":2198637799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6821","nonce":2198637799}]},"public_addr":"172.21.15.192:6817/2198637799","cluster_addr":"172.21.15.192:6819/2198637799","heartbeat_back_addr":"172.21.15.192:6823/2198637799","heartbeat_front_addr":"172.21.15.192:6821/2198637799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:14.481189+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:42.243983+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:09.715768+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:32.358421+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:56.431975+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:52:21.204162+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6824/1462436420":"2024-02-17T23:04:23.398629+0000","172.21.15.64:6829/168979118":"2024-02-17T23:03:59.013169+0000","172.21.15.64:6827/3220035256":"2024-02-17T23:03:32.044676+0000","172.21.15.192:6826/2853522025":"2024-02-17T23:04:47.133716+0000","172.21.15.64:0/1250745016":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1538951056":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1222682641":"2024-02-17T22:48:58.067040+0000","172.21.15.192:6828/2862437082":"2024-02-17T22:58:16.122252+0000","172.21.15.64:6800/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6801/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.192:0/2602543520":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/407404719":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1881977004":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1804571669":"2024-02-17T22:47:54.261948+0000","172.21.15.192:0/733087388":"2024-02-17T22:58:16.122252+0000","172.21.15.192:0/1972327284":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/2780081097":"2024-02-17T22:55:55.330401+0000","172.21.15.64:0/4258467456":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6828/168979118":"2024-02-17T23:03:59.013169+0000","172.21.15.64:0/4290875509":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1046482636":"2024-02-17T22:55:55.330401+0000","172.21.15.192:6829/2862437082":"2024-02-17T22:58:16.122252+0000","172.21.15.192:0/3116412139":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/3827352044":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/3005970232":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/700343021":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6800/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/743141814":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6801/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:6800/700343021":"2024-02-17T22:55:55.330401+0000","172.21.15.64:0/4194870513":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1988317603":"2024-02-17T22:59:02.601780+0000","172.21.15.64:6801/1223017301":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1580822017":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1671562709":"2024-02-17T22:59:02.601780+0000","172.21.15.192:6827/2853522025":"2024-02-17T23:04:47.133716+0000","172.21.15.192:6825/1462436420":"2024-02-17T23:04:23.398629+0000","172.21.15.64:6800/1223017301":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/347371753":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/741447403":"2024-02-17T22:59:02.601780+0000","172.21.15.64:6800/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6826/3220035256":"2024-02-17T23:03:32.044676+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T23:08:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f20dc06d750 msgr2=0x7f20dc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f20dc06d750 0x7f20dc06fc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f20e802bda0 tx=0x7f20e8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102c20 msgr2=0x7f20f0113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102c20 0x7f20f0113080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20d800c790 tx=0x7f20d80043c0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 shutdown_connections 2024-02-16T23:08:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20f0102c20 0x7f20f0113080 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f20dc06d750 0x7f20dc06fc00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 --2- 172.21.15.64:0/3298716223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f0102280 0x7f20f0112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 >> 172.21.15.64:0/3298716223 conn(0x7f20f00f7e50 msgr2=0x7f20f00f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 shutdown_connections 2024-02-16T23:08:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.115+0000 7f20f579d640 1 -- 172.21.15.64:0/3298716223 wait complete. 2024-02-16T23:08:21.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:20 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/882460028' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-16T23:08:21.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:20 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/4167312299' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-16T23:08:21.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:20 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2913621889' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-16T23:08:21.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:51:12.402108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T22:53:40.294617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T22:53:40.597634+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T23:08:21.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T23:08:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.227+0000 7f4924014640 1 -- 172.21.15.64:0/658436201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101be0 msgr2=0x7f491c0fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.227+0000 7f4924014640 1 --2- 172.21.15.64:0/658436201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101be0 0x7f491c0fb270 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f49100048d0 tx=0x7f4910009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 -- 172.21.15.64:0/658436201 shutdown_connections 2024-02-16T23:08:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 --2- 172.21.15.64:0/658436201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101be0 0x7f491c0fb270 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 --2- 172.21.15.64:0/658436201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f491c101240 0x7f491c101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 -- 172.21.15.64:0/658436201 >> 172.21.15.64:0/658436201 conn(0x7f491c0f6e10 msgr2=0x7f491c0f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 -- 172.21.15.64:0/658436201 shutdown_connections 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 -- 172.21.15.64:0/658436201 wait complete. 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 Processor -- start 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 -- start start 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101240 0x7f491c114db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f491c101be0 0x7f491c1152f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1158f0 con 0x7f491c101240 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4924014640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10ee80 con 0x7f491c101be0 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4921db0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101240 0x7f491c114db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4921db0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101240 0x7f491c114db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36944/0 (socket says 172.21.15.64:36944) 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.228+0000 7f4921db0640 1 -- 172.21.15.64:0/2626381420 learned_addr learned my addr 172.21.15.64:0/2626381420 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f4921db0640 1 -- 172.21.15.64:0/2626381420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f491c101be0 msgr2=0x7f491c1152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f4921db0640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f491c101be0 0x7f491c1152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f4921db0640 1 -- 172.21.15.64:0/2626381420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910005040 con 0x7f491c101240 2024-02-16T23:08:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f4921db0640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101240 0x7f491c114db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f491400c790 tx=0x7f49140043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f491a7fc640 1 -- 172.21.15.64:0/2626381420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491400a820 con 0x7f491c101240 2024-02-16T23:08:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f491a7fc640 1 -- 172.21.15.64:0/2626381420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f491400ae50 con 0x7f491c101240 2024-02-16T23:08:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c10f160 con 0x7f491c101240 2024-02-16T23:08:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f491a7fc640 1 -- 172.21.15.64:0/2626381420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49140115e0 con 0x7f491c101240 2024-02-16T23:08:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.229+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c10f680 con 0x7f491c101240 2024-02-16T23:08:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.230+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c064480 con 0x7f491c101240 2024-02-16T23:08:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.230+0000 7f491a7fc640 1 -- 172.21.15.64:0/2626381420 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4914012400 con 0x7f491c101240 2024-02-16T23:08:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.231+0000 7f491a7fc640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f490406d750 0x7f490406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.231+0000 7f491a7fc640 1 -- 172.21.15.64:0/2626381420 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f491408d150 con 0x7f491c101240 2024-02-16T23:08:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.231+0000 7f49215af640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f490406d750 0x7f490406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.231+0000 7f49215af640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f490406d750 0x7f490406fc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4910034040 tx=0x7f4910009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:21.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.236+0000 7f491a7fc640 1 -- 172.21.15.64:0/2626381420 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4914056920 con 0x7f491c101240 2024-02-16T23:08:21.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:20 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/882460028' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-16T23:08:21.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:20 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/4167312299' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-16T23:08:21.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:20 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2913621889' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-16T23:08:21.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.415+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f491c10f900 con 0x7f491c101240 2024-02-16T23:08:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.415+0000 7f491a7fc640 1 -- 172.21.15.64:0/2626381420 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4914059fe0 con 0x7f491c101240 2024-02-16T23:08:21.415 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f490406d750 msgr2=0x7f490406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f490406d750 0x7f490406fc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4910034040 tx=0x7f4910009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101240 msgr2=0x7f491c114db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101240 0x7f491c114db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f491400c790 tx=0x7f49140043c0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 shutdown_connections 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f490406d750 0x7f490406fc00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f491c101be0 0x7f491c1152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 --2- 172.21.15.64:0/2626381420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c101240 0x7f491c114db0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 >> 172.21.15.64:0/2626381420 conn(0x7f491c0f6e10 msgr2=0x7f491c0656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 shutdown_connections 2024-02-16T23:08:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.417+0000 7f4924014640 1 -- 172.21.15.64:0/2626381420 wait complete. 2024-02-16T23:08:21.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 -- 172.21.15.64:0/221848839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101be0 msgr2=0x7fcf180fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 --2- 172.21.15.64:0/221848839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101be0 0x7fcf180fb270 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0048d0 tx=0x7fcf0c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 -- 172.21.15.64:0/221848839 shutdown_connections 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 --2- 172.21.15.64:0/221848839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101be0 0x7fcf180fb270 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 --2- 172.21.15.64:0/221848839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf18101240 0x7fcf18101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 -- 172.21.15.64:0/221848839 >> 172.21.15.64:0/221848839 conn(0x7fcf180f6e10 msgr2=0x7fcf180f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 -- 172.21.15.64:0/221848839 shutdown_connections 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 -- 172.21.15.64:0/221848839 wait complete. 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.529+0000 7fcf1d26e640 1 Processor -- start 2024-02-16T23:08:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf1d26e640 1 -- start start 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf1d26e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101240 0x7fcf180fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf1d26e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf18101be0 0x7fcf180fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf1d26e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180fee80 con 0x7fcf18101240 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf1d26e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180fd960 con 0x7fcf18101be0 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101240 0x7fcf180fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101240 0x7fcf180fcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36970/0 (socket says 172.21.15.64:36970) 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf16ffd640 1 -- 172.21.15.64:0/4160872242 learned_addr learned my addr 172.21.15.64:0/4160872242 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf167fc640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf18101be0 0x7fcf180fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf16ffd640 1 -- 172.21.15.64:0/4160872242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf18101be0 msgr2=0x7fcf180fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf16ffd640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf18101be0 0x7fcf180fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf16ffd640 1 -- 172.21.15.64:0/4160872242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c005040 con 0x7fcf18101240 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf167fc640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf18101be0 0x7fcf180fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fcf16ffd640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101240 0x7fcf180fcee0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c830 tx=0x7fcf08004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fceff7fe640 1 -- 172.21.15.64:0/4160872242 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0800a850 con 0x7fcf18101240 2024-02-16T23:08:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.530+0000 7fceff7fe640 1 -- 172.21.15.64:0/4160872242 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf0800ae80 con 0x7fcf18101240 2024-02-16T23:08:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.531+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf180fdc10 con 0x7fcf18101240 2024-02-16T23:08:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.531+0000 7fceff7fe640 1 -- 172.21.15.64:0/4160872242 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf08012ea0 con 0x7fcf18101240 2024-02-16T23:08:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.531+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf18119250 con 0x7fcf18101240 2024-02-16T23:08:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.531+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf18064480 con 0x7fcf18101240 2024-02-16T23:08:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.532+0000 7fceff7fe640 1 -- 172.21.15.64:0/4160872242 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcf08012400 con 0x7fcf18101240 2024-02-16T23:08:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.532+0000 7fceff7fe640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf0006d750 0x7fcf0006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.533+0000 7fcf167fc640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf0006d750 0x7fcf0006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.533+0000 7fceff7fe640 1 -- 172.21.15.64:0/4160872242 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcf0808d130 con 0x7fcf18101240 2024-02-16T23:08:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.533+0000 7fcf167fc640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf0006d750 0x7fcf0006fc00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf180fe6d0 tx=0x7fcf0c009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.537+0000 7fceff7fe640 1 -- 172.21.15.64:0/4160872242 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf080569c0 con 0x7fcf18101240 2024-02-16T23:08:21.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.723+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fcf18108380 con 0x7fcf18101240 2024-02-16T23:08:21.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.723+0000 7fceff7fe640 1 -- 172.21.15.64:0/4160872242 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcf080126b0 con 0x7fcf18101240 2024-02-16T23:08:21.723 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf0006d750 msgr2=0x7fcf0006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf0006d750 0x7fcf0006fc00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf180fe6d0 tx=0x7fcf0c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101240 msgr2=0x7fcf180fcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101240 0x7fcf180fcee0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c830 tx=0x7fcf08004430 comp rx=0 tx=0).stop 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 shutdown_connections 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fcf0006d750 0x7fcf0006fc00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf18101be0 0x7fcf180fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 --2- 172.21.15.64:0/4160872242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf18101240 0x7fcf180fcee0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 >> 172.21.15.64:0/4160872242 conn(0x7fcf180f6e10 msgr2=0x7fcf180656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 shutdown_connections 2024-02-16T23:08:21.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.725+0000 7fcf1d26e640 1 -- 172.21.15.64:0/4160872242 wait complete. 2024-02-16T23:08:21.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T23:08:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 -- 172.21.15.64:0/3313488986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 msgr2=0x7f8fd81027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 --2- 172.21.15.64:0/3313488986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 0x7f8fd81027b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80048d0 tx=0x7f8fc8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 -- 172.21.15.64:0/3313488986 shutdown_connections 2024-02-16T23:08:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 --2- 172.21.15.64:0/3313488986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fd80644b0 0x7f8fd80648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 --2- 172.21.15.64:0/3313488986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 0x7f8fd81027b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 -- 172.21.15.64:0/3313488986 >> 172.21.15.64:0/3313488986 conn(0x7f8fd8071230 msgr2=0x7f8fd8071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 -- 172.21.15.64:0/3313488986 shutdown_connections 2024-02-16T23:08:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 -- 172.21.15.64:0/3313488986 wait complete. 2024-02-16T23:08:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.838+0000 7f8fdd59d640 1 Processor -- start 2024-02-16T23:08:21.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fdd59d640 1 -- start start 2024-02-16T23:08:21.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fdd59d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fd80644b0 0x7f8fd80fb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fdd59d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 0x7f8fd80fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fdd59d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd80fd320 con 0x7f8fd81023e0 2024-02-16T23:08:21.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fdd59d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd80fbe00 con 0x7f8fd80644b0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fce7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 0x7f8fd80fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fce7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 0x7f8fd80fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36982/0 (socket says 172.21.15.64:36982) 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fce7fc640 1 -- 172.21.15.64:0/1169445293 learned_addr learned my addr 172.21.15.64:0/1169445293 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fd6ffd640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fd80644b0 0x7f8fd80fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fce7fc640 1 -- 172.21.15.64:0/1169445293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fd80644b0 msgr2=0x7f8fd80fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fce7fc640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fd80644b0 0x7f8fd80fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fce7fc640 1 -- 172.21.15.64:0/1169445293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc8005040 con 0x7f8fd81023e0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.839+0000 7f8fce7fc640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 0x7f8fd80fb8c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000eb70 tx=0x7f8fc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.840+0000 7f8fcffff640 1 -- 172.21.15.64:0/1169445293 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc0007c30 con 0x7f8fd81023e0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.840+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd80fc050 con 0x7f8fd81023e0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.840+0000 7f8fcffff640 1 -- 172.21.15.64:0/1169445293 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fc000f040 con 0x7f8fd81023e0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.840+0000 7f8fcffff640 1 -- 172.21.15.64:0/1169445293 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc00095b0 con 0x7f8fd81023e0 2024-02-16T23:08:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.840+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd8119210 con 0x7f8fd81023e0 2024-02-16T23:08:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.841+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd8064030 con 0x7f8fd81023e0 2024-02-16T23:08:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.841+0000 7f8fcffff640 1 -- 172.21.15.64:0/1169445293 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8fc0009730 con 0x7f8fd81023e0 2024-02-16T23:08:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.841+0000 7f8fcffff640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc406d750 0x7f8fc406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.842+0000 7f8fcffff640 1 -- 172.21.15.64:0/1169445293 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8fc008ccb0 con 0x7f8fd81023e0 2024-02-16T23:08:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.842+0000 7f8fd6ffd640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc406d750 0x7f8fc406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.842+0000 7f8fd6ffd640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc406d750 0x7f8fc406fc00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8000c00 tx=0x7f8fc8009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:21.846+0000 7f8fcffff640 1 -- 172.21.15.64:0/1169445293 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fc00564b0 con 0x7f8fd81023e0 2024-02-16T23:08:22.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.026+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f8fd81094b0 con 0x7f8fd81023e0 2024-02-16T23:08:22.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.026+0000 7f8fcffff640 1 -- 172.21.15.64:0/1169445293 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8fc0059b70 con 0x7f8fd81023e0 2024-02-16T23:08:22.026 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-02-16T23:08:22.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc406d750 msgr2=0x7f8fc406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc406d750 0x7f8fc406fc00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8000c00 tx=0x7f8fc8009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 msgr2=0x7f8fd80fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 0x7f8fd80fb8c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000eb70 tx=0x7f8fc000b040 comp rx=0 tx=0).stop 2024-02-16T23:08:22.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 shutdown_connections 2024-02-16T23:08:22.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd81023e0 0x7f8fd80fb8c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f8fc406d750 0x7f8fc406fc00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 --2- 172.21.15.64:0/1169445293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fd80644b0 0x7f8fd80fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 >> 172.21.15.64:0/1169445293 conn(0x7f8fd8071230 msgr2=0x7f8fd80fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:22.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 shutdown_connections 2024-02-16T23:08:22.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.028+0000 7f8fdd59d640 1 -- 172.21.15.64:0/1169445293 wait complete. 2024-02-16T23:08:22.038 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:21 smithi064 ceph-mon[93757]: pgmap v317: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:22.038 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:21 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3298716223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T23:08:22.038 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:21 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2626381420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T23:08:22.038 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:21 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/4160872242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T23:08:22.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.138+0000 7f2a4339d640 1 -- 172.21.15.64:0/3290993342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c0fb2b0 msgr2=0x7f2a3c0fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.138+0000 7f2a4339d640 1 --2- 172.21.15.64:0/3290993342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c0fb2b0 0x7f2a3c0fb700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0048d0 tx=0x7f2a2c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.138+0000 7f2a4339d640 1 -- 172.21.15.64:0/3290993342 shutdown_connections 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.138+0000 7f2a4339d640 1 --2- 172.21.15.64:0/3290993342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c0fb2b0 0x7f2a3c0fb700 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.138+0000 7f2a4339d640 1 --2- 172.21.15.64:0/3290993342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a3c1012d0 0x7f2a3c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.138+0000 7f2a4339d640 1 -- 172.21.15.64:0/3290993342 >> 172.21.15.64:0/3290993342 conn(0x7f2a3c071230 msgr2=0x7f2a3c071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a4339d640 1 -- 172.21.15.64:0/3290993342 shutdown_connections 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a4339d640 1 -- 172.21.15.64:0/3290993342 wait complete. 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a4339d640 1 Processor -- start 2024-02-16T23:08:22.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a4339d640 1 -- start start 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a4339d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a3c0fb2b0 0x7f2a3c10dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a4339d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c1012d0 0x7f2a3c10e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a4339d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c10e790 con 0x7f2a3c1012d0 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a4339d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c10e900 con 0x7f2a3c0fb2b0 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a40938640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c1012d0 0x7f2a3c10e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a40938640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c1012d0 0x7f2a3c10e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37010/0 (socket says 172.21.15.64:37010) 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.139+0000 7f2a40938640 1 -- 172.21.15.64:0/1076314272 learned_addr learned my addr 172.21.15.64:0/1076314272 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a40938640 1 -- 172.21.15.64:0/1076314272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a3c0fb2b0 msgr2=0x7f2a3c10dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a41139640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a3c0fb2b0 0x7f2a3c10dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a40938640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a3c0fb2b0 0x7f2a3c10dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a40938640 1 -- 172.21.15.64:0/1076314272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c005040 con 0x7f2a3c1012d0 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a41139640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a3c0fb2b0 0x7f2a3c10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a40938640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c1012d0 0x7f2a3c10e200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c031720 tx=0x7f2a2c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a31ffb640 1 -- 172.21.15.64:0/1076314272 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2c0094c0 con 0x7f2a3c1012d0 2024-02-16T23:08:22.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a31ffb640 1 -- 172.21.15.64:0/1076314272 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a2c009620 con 0x7f2a3c1012d0 2024-02-16T23:08:22.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c116d30 con 0x7f2a3c1012d0 2024-02-16T23:08:22.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a31ffb640 1 -- 172.21.15.64:0/1076314272 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2c03b640 con 0x7f2a3c1012d0 2024-02-16T23:08:22.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.140+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c117110 con 0x7f2a3c1012d0 2024-02-16T23:08:22.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.141+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c064030 con 0x7f2a3c1012d0 2024-02-16T23:08:22.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.141+0000 7f2a31ffb640 1 -- 172.21.15.64:0/1076314272 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2a2c02f070 con 0x7f2a3c1012d0 2024-02-16T23:08:22.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.142+0000 7f2a31ffb640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a2806d750 0x7f2a2806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.142+0000 7f2a31ffb640 1 -- 172.21.15.64:0/1076314272 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2a2c0af550 con 0x7f2a3c1012d0 2024-02-16T23:08:22.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.142+0000 7f2a41139640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a2806d750 0x7f2a2806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.142+0000 7f2a41139640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a2806d750 0x7f2a2806fc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a380041d0 tx=0x7f2a38009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:22.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.147+0000 7f2a31ffb640 1 -- 172.21.15.64:0/1076314272 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a2c078ca0 con 0x7f2a3c1012d0 2024-02-16T23:08:22.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:21 smithi192 ceph-mon[82652]: pgmap v317: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:22.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:21 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3298716223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T23:08:22.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:21 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2626381420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T23:08:22.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:21 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/4160872242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T23:08:22.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.341+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2a3c1083c0 con 0x7f2a3c1012d0 2024-02-16T23:08:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.341+0000 7f2a31ffb640 1 -- 172.21.15.64:0/1076314272 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v85) v1 ==== 81+0+2553 (secure 0 0 0) 0x7f2a2c07c360 con 0x7f2a3c1012d0 2024-02-16T23:08:22.341 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6824/1462436420 2024-02-17T23:04:23.398629+0000 2024-02-16T23:08:22.341 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6829/168979118 2024-02-17T23:03:59.013169+0000 2024-02-16T23:08:22.341 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6827/3220035256 2024-02-17T23:03:32.044676+0000 2024-02-16T23:08:22.341 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6826/2853522025 2024-02-17T23:04:47.133716+0000 2024-02-16T23:08:22.341 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1250745016 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1538951056 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1222682641 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6828/2862437082 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3482665154 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3482665154 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:0/2602543520 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/407404719 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1881977004 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1804571669 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:0/733087388 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:0/1972327284 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2780081097 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4258467456 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6828/168979118 2024-02-17T23:03:59.013169+0000 2024-02-16T23:08:22.342 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4290875509 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1046482636 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6829/2862437082 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:0/3116412139 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3827352044 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3005970232 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/700343021 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/600121841 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/743141814 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/600121841 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/700343021 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4194870513 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1988317603 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/1223017301 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1580822017 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1671562709 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:22.343 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6827/2853522025 2024-02-17T23:04:47.133716+0000 2024-02-16T23:08:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6825/1462436420 2024-02-17T23:04:23.398629+0000 2024-02-16T23:08:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/1223017301 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/347371753 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/741447403 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3542090188 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3542090188 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6826/3220035256 2024-02-17T23:03:32.044676+0000 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:listed 43 entries 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a2806d750 msgr2=0x7f2a2806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a2806d750 0x7f2a2806fc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a380041d0 tx=0x7f2a38009040 comp rx=0 tx=0).stop 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c1012d0 msgr2=0x7f2a3c10e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c1012d0 0x7f2a3c10e200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c031720 tx=0x7f2a2c009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 shutdown_connections 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c1012d0 0x7f2a3c10e200 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2a2806d750 0x7f2a2806fc00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 --2- 172.21.15.64:0/1076314272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a3c0fb2b0 0x7f2a3c10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 >> 172.21.15.64:0/1076314272 conn(0x7f2a3c071230 msgr2=0x7f2a3c0fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 shutdown_connections 2024-02-16T23:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.343+0000 7f2a4339d640 1 -- 172.21.15.64:0/1076314272 wait complete. 2024-02-16T23:08:22.354 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-16T23:08:22.354 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T23:08:22.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T23:08:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.507+0000 7f2b53b5d640 1 -- 172.21.15.64:0/1917423581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 msgr2=0x7f2b4c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.507+0000 7f2b53b5d640 1 --2- 172.21.15.64:0/1917423581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 0x7f2b4c1028f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b440048d0 tx=0x7f2b44009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.507+0000 7f2b53b5d640 1 -- 172.21.15.64:0/1917423581 shutdown_connections 2024-02-16T23:08:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.507+0000 7f2b53b5d640 1 --2- 172.21.15.64:0/1917423581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c0fc4e0 0x7f2b4c0fc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.507+0000 7f2b53b5d640 1 --2- 172.21.15.64:0/1917423581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 0x7f2b4c1028f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.507+0000 7f2b53b5d640 1 -- 172.21.15.64:0/1917423581 >> 172.21.15.64:0/1917423581 conn(0x7f2b4c0f8060 msgr2=0x7f2b4c0fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b53b5d640 1 -- 172.21.15.64:0/1917423581 shutdown_connections 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b53b5d640 1 -- 172.21.15.64:0/1917423581 wait complete. 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b53b5d640 1 Processor -- start 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b53b5d640 1 -- start start 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b53b5d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c0fc4e0 0x7f2b4c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b53b5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 0x7f2b4c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b53b5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c071740 con 0x7f2b4c102520 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b53b5d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c070220 con 0x7f2b4c0fc4e0 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b510f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 0x7f2b4c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b510f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 0x7f2b4c06fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37024/0 (socket says 172.21.15.64:37024) 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.508+0000 7f2b510f8640 1 -- 172.21.15.64:0/942838576 learned_addr learned my addr 172.21.15.64:0/942838576 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b518f9640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c0fc4e0 0x7f2b4c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b510f8640 1 -- 172.21.15.64:0/942838576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c0fc4e0 msgr2=0x7f2b4c06f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b510f8640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c0fc4e0 0x7f2b4c06f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b510f8640 1 -- 172.21.15.64:0/942838576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b44005040 con 0x7f2b4c102520 2024-02-16T23:08:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b510f8640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 0x7f2b4c06fce0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2b40002a80 tx=0x7f2b40004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b4a7fc640 1 -- 172.21.15.64:0/942838576 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b40007c30 con 0x7f2b4c102520 2024-02-16T23:08:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b4a7fc640 1 -- 172.21.15.64:0/942838576 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b4000f040 con 0x7f2b4c102520 2024-02-16T23:08:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c0704d0 con 0x7f2b4c102520 2024-02-16T23:08:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b4a7fc640 1 -- 172.21.15.64:0/942838576 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b4000db70 con 0x7f2b4c102520 2024-02-16T23:08:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.509+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4c1193f0 con 0x7f2b4c102520 2024-02-16T23:08:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.510+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4c064480 con 0x7f2b4c102520 2024-02-16T23:08:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.510+0000 7f2b4a7fc640 1 -- 172.21.15.64:0/942838576 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2b4000dcd0 con 0x7f2b4c102520 2024-02-16T23:08:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.511+0000 7f2b4a7fc640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2b3406d760 0x7f2b3406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.511+0000 7f2b4a7fc640 1 -- 172.21.15.64:0/942838576 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2b4008c920 con 0x7f2b4c102520 2024-02-16T23:08:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.511+0000 7f2b518f9640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2b3406d760 0x7f2b3406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.511+0000 7f2b518f9640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2b3406d760 0x7f2b3406fc10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2b44034040 tx=0x7f2b44009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.516+0000 7f2b4a7fc640 1 -- 172.21.15.64:0/942838576 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b40056070 con 0x7f2b4c102520 2024-02-16T23:08:22.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.699+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b4c1095f0 con 0x7f2b4c102520 2024-02-16T23:08:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.701+0000 7f2b4a7fc640 1 -- 172.21.15.64:0/942838576 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16913 (secure 0 0 0) 0x7f2b40059730 con 0x7f2b4c102520 2024-02-16T23:08:22.701 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-16T23:08:22.701 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"2912dc76-cd1d-11ee-95ba-87774f69a715","created":"2024-02-16T22:46:56.779041+0000","modified":"2024-02-16T23:04:47.133740+0000","last_up_change":"2024-02-16T23:03:26.379108+0000","last_in_change":"2024-02-16T22:52:04.211979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:51:12.402108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T22:53:40.294617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T22:53:40.597634+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7a680d8c-0559-46c6-9bfc-95fca252a657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6803","nonce":3681251107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6805","nonce":3681251107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6809","nonce":3681251107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3681251107},{"type":"v1","addr":"172.21.15.64:6807","nonce":3681251107}]},"public_addr":"172.21.15.64:6803/3681251107","cluster_addr":"172.21.15.64:6805/3681251107","heartbeat_back_addr":"172.21.15.64:6809/3681251107","heartbeat_front_addr":"172.21.15.64:6807/3681251107","state":["exists","up"]},{"osd":1,"uuid":"21a8757d-9f77-4f46-83d3-ff904c9e50b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6811","nonce":1635803238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6813","nonce":1635803238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6817","nonce":1635803238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1635803238},{"type":"v1","addr":"172.21.15.64:6815","nonce":1635803238}]},"public_addr":"172.21.15.64:6811/1635803238","cluster_addr":"172.21.15.64:6813/1635803238","heartbeat_back_addr":"172.21.15.64:6817/1635803238","heartbeat_front_addr":"172.21.15.64:6815/1635803238","state":["exists","up"]},{"osd":2,"uuid":"3fcb2e12-54a0-46db-af11-d047cf25d50b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6819","nonce":530939834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6821","nonce":530939834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6825","nonce":530939834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":530939834},{"type":"v1","addr":"172.21.15.64:6823","nonce":530939834}]},"public_addr":"172.21.15.64:6819/530939834","cluster_addr":"172.21.15.64:6821/530939834","heartbeat_back_addr":"172.21.15.64:6825/530939834","heartbeat_front_addr":"172.21.15.64:6823/530939834","state":["exists","up"]},{"osd":3,"uuid":"a7a58c77-96fa-4835-9ea4-d16e6d995620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6801","nonce":2367052696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6803","nonce":2367052696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6807","nonce":2367052696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2367052696},{"type":"v1","addr":"172.21.15.192:6805","nonce":2367052696}]},"public_addr":"172.21.15.192:6801/2367052696","cluster_addr":"172.21.15.192:6803/2367052696","heartbeat_back_addr":"172.21.15.192:6807/2367052696","heartbeat_front_addr":"172.21.15.192:6805/2367052696","state":["exists","up"]},{"osd":4,"uuid":"04e094c5-2494-457a-a4a5-7b2742be9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6809","nonce":2408603569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6811","nonce":2408603569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6815","nonce":2408603569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2408603569},{"type":"v1","addr":"172.21.15.192:6813","nonce":2408603569}]},"public_addr":"172.21.15.192:6809/2408603569","cluster_addr":"172.21.15.192:6811/2408603569","heartbeat_back_addr":"172.21.15.192:6815/2408603569","heartbeat_front_addr":"172.21.15.192:6813/2408603569","state":["exists","up"]},{"osd":5,"uuid":"bab1c12a-4e32-48e9-8a24-32ac3c4ba4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6817","nonce":2198637799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6819","nonce":2198637799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6823","nonce":2198637799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2198637799},{"type":"v1","addr":"172.21.15.192:6821","nonce":2198637799}]},"public_addr":"172.21.15.192:6817/2198637799","cluster_addr":"172.21.15.192:6819/2198637799","heartbeat_back_addr":"172.21.15.192:6823/2198637799","heartbeat_front_addr":"172.21.15.192:6821/2198637799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:14.481189+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:50:42.243983+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:09.715768+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:32.358421+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:51:56.431975+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:52:21.204162+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6824/1462436420":"2024-02-17T23:04:23.398629+0000","172.21.15.64:6829/168979118":"2024-02-17T23:03:59.013169+0000","172.21.15.64:6827/3220035256":"2024-02-17T23:03:32.044676+0000","172.21.15.192:6826/2853522025":"2024-02-17T23:04:47.133716+0000","172.21.15.64:0/1250745016":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1538951056":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1222682641":"2024-02-17T22:48:58.067040+0000","172.21.15.192:6828/2862437082":"2024-02-17T22:58:16.122252+0000","172.21.15.64:6800/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.64:6801/3482665154":"2024-02-17T22:47:54.261948+0000","172.21.15.192:0/2602543520":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/407404719":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1881977004":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1804571669":"2024-02-17T22:47:54.261948+0000","172.21.15.192:0/733087388":"2024-02-17T22:58:16.122252+0000","172.21.15.192:0/1972327284":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/2780081097":"2024-02-17T22:55:55.330401+0000","172.21.15.64:0/4258467456":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6828/168979118":"2024-02-17T23:03:59.013169+0000","172.21.15.64:0/4290875509":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/1046482636":"2024-02-17T22:55:55.330401+0000","172.21.15.192:6829/2862437082":"2024-02-17T22:58:16.122252+0000","172.21.15.192:0/3116412139":"2024-02-17T22:58:16.122252+0000","172.21.15.64:0/3827352044":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/3005970232":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/700343021":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6800/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:0/743141814":"2024-02-17T22:55:55.330401+0000","172.21.15.64:6801/600121841":"2024-02-17T22:48:58.067040+0000","172.21.15.64:6800/700343021":"2024-02-17T22:55:55.330401+0000","172.21.15.64:0/4194870513":"2024-02-17T22:47:54.261948+0000","172.21.15.64:0/1988317603":"2024-02-17T22:59:02.601780+0000","172.21.15.64:6801/1223017301":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/1580822017":"2024-02-17T22:47:24.032877+0000","172.21.15.64:0/1671562709":"2024-02-17T22:59:02.601780+0000","172.21.15.192:6827/2853522025":"2024-02-17T23:04:47.133716+0000","172.21.15.192:6825/1462436420":"2024-02-17T23:04:23.398629+0000","172.21.15.64:6800/1223017301":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/347371753":"2024-02-17T22:59:02.601780+0000","172.21.15.64:0/741447403":"2024-02-17T22:59:02.601780+0000","172.21.15.64:6800/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6801/3542090188":"2024-02-17T22:47:24.032877+0000","172.21.15.64:6826/3220035256":"2024-02-17T23:03:32.044676+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2b3406d760 msgr2=0x7f2b3406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2b3406d760 0x7f2b3406fc10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2b44034040 tx=0x7f2b44009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 msgr2=0x7f2b4c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 0x7f2b4c06fce0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2b40002a80 tx=0x7f2b40004600 comp rx=0 tx=0).stop 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 shutdown_connections 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b4c102520 0x7f2b4c06fce0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f2b3406d760 0x7f2b3406fc10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 --2- 172.21.15.64:0/942838576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c0fc4e0 0x7f2b4c06f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 >> 172.21.15.64:0/942838576 conn(0x7f2b4c0f8060 msgr2=0x7f2b4c0f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 shutdown_connections 2024-02-16T23:08:22.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.703+0000 7f2b53b5d640 1 -- 172.21.15.64:0/942838576 wait complete. 2024-02-16T23:08:22.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:51:12.402108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T22:53:40.294617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T22:53:40.597634+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T23:08:22.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T23:08:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.815+0000 7f9730d5d640 1 -- 172.21.15.64:0/4188318431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c0fb2b0 msgr2=0x7f972c0fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.815+0000 7f9730d5d640 1 --2- 172.21.15.64:0/4188318431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c0fb2b0 0x7f972c0fb700 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f97140048d0 tx=0x7f9714009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 -- 172.21.15.64:0/4188318431 shutdown_connections 2024-02-16T23:08:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 --2- 172.21.15.64:0/4188318431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c0fb2b0 0x7f972c0fb700 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 --2- 172.21.15.64:0/4188318431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f972c1012d0 0x7f972c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 -- 172.21.15.64:0/4188318431 >> 172.21.15.64:0/4188318431 conn(0x7f972c071230 msgr2=0x7f972c071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 -- 172.21.15.64:0/4188318431 shutdown_connections 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 -- 172.21.15.64:0/4188318431 wait complete. 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 Processor -- start 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 -- start start 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f972c0fb2b0 0x7f972c112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c1012d0 0x7f972c113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c113740 con 0x7f972c1012d0 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9730d5d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c10ccd0 con 0x7f972c0fb2b0 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9729d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c1012d0 0x7f972c113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9729d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c1012d0 0x7f972c113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37048/0 (socket says 172.21.15.64:37048) 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f9729d9b640 1 -- 172.21.15.64:0/3148873464 learned_addr learned my addr 172.21.15.64:0/3148873464 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.816+0000 7f972a59c640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f972c0fb2b0 0x7f972c112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9729d9b640 1 -- 172.21.15.64:0/3148873464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f972c0fb2b0 msgr2=0x7f972c112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9729d9b640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f972c0fb2b0 0x7f972c112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:22.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9729d9b640 1 -- 172.21.15.64:0/3148873464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714005040 con 0x7f972c1012d0 2024-02-16T23:08:22.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9729d9b640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c1012d0 0x7f972c113140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9714031720 tx=0x7f9714009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:22.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9722ffd640 1 -- 172.21.15.64:0/3148873464 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97140094c0 con 0x7f972c1012d0 2024-02-16T23:08:22.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9722ffd640 1 -- 172.21.15.64:0/3148873464 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9714009620 con 0x7f972c1012d0 2024-02-16T23:08:22.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c10cf50 con 0x7f972c1012d0 2024-02-16T23:08:22.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9722ffd640 1 -- 172.21.15.64:0/3148873464 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f971403b640 con 0x7f972c1012d0 2024-02-16T23:08:22.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.817+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c10d470 con 0x7f972c1012d0 2024-02-16T23:08:22.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.818+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c064030 con 0x7f972c1012d0 2024-02-16T23:08:22.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.818+0000 7f9722ffd640 1 -- 172.21.15.64:0/3148873464 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f971402f070 con 0x7f972c1012d0 2024-02-16T23:08:22.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.819+0000 7f9722ffd640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f971806d750 0x7f971806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:22.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.819+0000 7f9722ffd640 1 -- 172.21.15.64:0/3148873464 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f97140af430 con 0x7f972c1012d0 2024-02-16T23:08:22.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.819+0000 7f972a59c640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f971806d750 0x7f971806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:22.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.819+0000 7f972a59c640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f971806d750 0x7f971806fc00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f971c009fd0 tx=0x7f971c009e50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:22.823+0000 7f9722ffd640 1 -- 172.21.15.64:0/3148873464 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9714078c00 con 0x7f972c1012d0 2024-02-16T23:08:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.003+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f972c10d650 con 0x7f972c1012d0 2024-02-16T23:08:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.003+0000 7f9722ffd640 1 -- 172.21.15.64:0/3148873464 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f971407c2c0 con 0x7f972c1012d0 2024-02-16T23:08:23.003 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-02-16T23:08:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f971806d750 msgr2=0x7f971806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f971806d750 0x7f971806fc00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f971c009fd0 tx=0x7f971c009e50 comp rx=0 tx=0).stop 2024-02-16T23:08:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c1012d0 msgr2=0x7f972c113140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c1012d0 0x7f972c113140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9714031720 tx=0x7f9714009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 shutdown_connections 2024-02-16T23:08:23.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f972c1012d0 0x7f972c113140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f971806d750 0x7f971806fc00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 --2- 172.21.15.64:0/3148873464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f972c0fb2b0 0x7f972c112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 >> 172.21.15.64:0/3148873464 conn(0x7f972c071230 msgr2=0x7f972c064520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:23.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 shutdown_connections 2024-02-16T23:08:23.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.005+0000 7f9730d5d640 1 -- 172.21.15.64:0/3148873464 wait complete. 2024-02-16T23:08:23.015 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:22 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1169445293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T23:08:23.015 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:22 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1076314272' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T23:08:23.015 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:22 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/942838576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T23:08:23.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T23:08:23.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.117+0000 7fc40111d640 1 -- 172.21.15.64:0/1533451860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 msgr2=0x7fc3fc102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.117+0000 7fc40111d640 1 --2- 172.21.15.64:0/1533451860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 0x7fc3fc102730 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40048d0 tx=0x7fc3e4009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.117+0000 7fc40111d640 1 -- 172.21.15.64:0/1533451860 shutdown_connections 2024-02-16T23:08:23.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.117+0000 7fc40111d640 1 --2- 172.21.15.64:0/1533451860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3fc102d00 0x7fc3fc0fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.117+0000 7fc40111d640 1 --2- 172.21.15.64:0/1533451860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 0x7fc3fc102730 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.117+0000 7fc40111d640 1 -- 172.21.15.64:0/1533451860 >> 172.21.15.64:0/1533451860 conn(0x7fc3fc0f7ef0 msgr2=0x7fc3fc0fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.117+0000 7fc40111d640 1 -- 172.21.15.64:0/1533451860 shutdown_connections 2024-02-16T23:08:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.117+0000 7fc40111d640 1 -- 172.21.15.64:0/1533451860 wait complete. 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc40111d640 1 Processor -- start 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc40111d640 1 -- start start 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc40111d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 0x7fc3fc112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc40111d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3fc102d00 0x7fc3fc113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc40111d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc113750 con 0x7fc3fc102360 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc40111d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc10cce0 con 0x7fc3fc102d00 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc3fad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 0x7fc3fc112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc3fad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 0x7fc3fc112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37064/0 (socket says 172.21.15.64:37064) 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc3fad9d640 1 -- 172.21.15.64:0/503933552 learned_addr learned my addr 172.21.15.64:0/503933552 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc3fad9d640 1 -- 172.21.15.64:0/503933552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3fc102d00 msgr2=0x7fc3fc113150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.118+0000 7fc3fa59c640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3fc102d00 0x7fc3fc113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc3fad9d640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3fc102d00 0x7fc3fc113150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc3fad9d640 1 -- 172.21.15.64:0/503933552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e4005040 con 0x7fc3fc102360 2024-02-16T23:08:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc3fa59c640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3fc102d00 0x7fc3fc113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc3fad9d640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 0x7fc3fc112c10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4031720 tx=0x7fc3e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc3f37fe640 1 -- 172.21.15.64:0/503933552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3e40094c0 con 0x7fc3fc102360 2024-02-16T23:08:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc10cf60 con 0x7fc3fc102360 2024-02-16T23:08:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc3f37fe640 1 -- 172.21.15.64:0/503933552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3e4009620 con 0x7fc3fc102360 2024-02-16T23:08:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc3f37fe640 1 -- 172.21.15.64:0/503933552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3e403b640 con 0x7fc3fc102360 2024-02-16T23:08:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.119+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc10d480 con 0x7fc3fc102360 2024-02-16T23:08:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.120+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc064480 con 0x7fc3fc102360 2024-02-16T23:08:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.120+0000 7fc3f37fe640 1 -- 172.21.15.64:0/503933552 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc3e402f070 con 0x7fc3fc102360 2024-02-16T23:08:23.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.121+0000 7fc3f37fe640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc3e806d750 0x7fc3e806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.121+0000 7fc3f37fe640 1 -- 172.21.15.64:0/503933552 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc3e40af5d0 con 0x7fc3fc102360 2024-02-16T23:08:23.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.121+0000 7fc3fa59c640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc3e806d750 0x7fc3e806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.121+0000 7fc3fa59c640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc3e806d750 0x7fc3e806fc00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc10e4d0 tx=0x7fc3ec009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:23.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.125+0000 7fc3f37fe640 1 -- 172.21.15.64:0/503933552 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3e4078d20 con 0x7fc3fc102360 2024-02-16T23:08:23.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:22 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1169445293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T23:08:23.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:22 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1076314272' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T23:08:23.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:22 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/942838576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T23:08:23.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.306+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fc3fc10d610 con 0x7fc3fc102360 2024-02-16T23:08:23.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.306+0000 7fc3f37fe640 1 -- 172.21.15.64:0/503933552 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc3e407c3e0 con 0x7fc3fc102360 2024-02-16T23:08:23.306 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-02-16T23:08:23.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc3e806d750 msgr2=0x7fc3e806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc3e806d750 0x7fc3e806fc00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc10e4d0 tx=0x7fc3ec009040 comp rx=0 tx=0).stop 2024-02-16T23:08:23.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 msgr2=0x7fc3fc112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 0x7fc3fc112c10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4031720 tx=0x7fc3e4009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 shutdown_connections 2024-02-16T23:08:23.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fc3e806d750 0x7fc3e806fc00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3fc102d00 0x7fc3fc113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 --2- 172.21.15.64:0/503933552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3fc102360 0x7fc3fc112c10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 >> 172.21.15.64:0/503933552 conn(0x7fc3fc0f7ef0 msgr2=0x7fc3fc0f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:23.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 shutdown_connections 2024-02-16T23:08:23.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.308+0000 7fc40111d640 1 -- 172.21.15.64:0/503933552 wait complete. 2024-02-16T23:08:23.319 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T23:08:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 -- 172.21.15.64:0/3923636212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 msgr2=0x7fa2540fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 --2- 172.21.15.64:0/3923636212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 0x7fa2540fb290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa2480048d0 tx=0x7fa248009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 -- 172.21.15.64:0/3923636212 shutdown_connections 2024-02-16T23:08:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 --2- 172.21.15.64:0/3923636212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 0x7fa2540fb290 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 --2- 172.21.15.64:0/3923636212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa254101260 0x7fa254101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 -- 172.21.15.64:0/3923636212 >> 172.21.15.64:0/3923636212 conn(0x7fa2540f6e50 msgr2=0x7fa2540f9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 -- 172.21.15.64:0/3923636212 shutdown_connections 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 -- 172.21.15.64:0/3923636212 wait complete. 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 Processor -- start 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 -- start start 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa254101260 0x7fa2540fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 0x7fa2540fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540fde30 con 0x7fa254101c00 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.421+0000 7fa25906e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540fdfa0 con 0x7fa254101260 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 0x7fa2540fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 0x7fa2540fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37088/0 (socket says 172.21.15.64:37088) 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25259c640 1 -- 172.21.15.64:0/2180663191 learned_addr learned my addr 172.21.15.64:0/2180663191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa252d9d640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa254101260 0x7fa2540fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25259c640 1 -- 172.21.15.64:0/2180663191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa254101260 msgr2=0x7fa2540fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25259c640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa254101260 0x7fa2540fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25259c640 1 -- 172.21.15.64:0/2180663191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa248005040 con 0x7fa254101c00 2024-02-16T23:08:23.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa252d9d640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa254101260 0x7fa2540fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:23.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25259c640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 0x7fa2540fd8f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa248031720 tx=0x7fa248009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:23.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa23b7fe640 1 -- 172.21.15.64:0/2180663191 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2480094c0 con 0x7fa254101c00 2024-02-16T23:08:23.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2540fe220 con 0x7fa254101c00 2024-02-16T23:08:23.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa23b7fe640 1 -- 172.21.15.64:0/2180663191 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa248009620 con 0x7fa254101c00 2024-02-16T23:08:23.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa23b7fe640 1 -- 172.21.15.64:0/2180663191 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa24803b640 con 0x7fa254101c00 2024-02-16T23:08:23.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.422+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa254119380 con 0x7fa254101c00 2024-02-16T23:08:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.423+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa254064480 con 0x7fa254101c00 2024-02-16T23:08:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.424+0000 7fa23b7fe640 1 -- 172.21.15.64:0/2180663191 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa24802f070 con 0x7fa254101c00 2024-02-16T23:08:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.424+0000 7fa23b7fe640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa23c06d750 0x7fa23c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.424+0000 7fa23b7fe640 1 -- 172.21.15.64:0/2180663191 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa2480af270 con 0x7fa254101c00 2024-02-16T23:08:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.424+0000 7fa252d9d640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa23c06d750 0x7fa23c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.425+0000 7fa252d9d640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa23c06d750 0x7fa23c06fc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa244009af0 tx=0x7fa24400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.429+0000 7fa23b7fe640 1 -- 172.21.15.64:0/2180663191 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2480789c0 con 0x7fa254101c00 2024-02-16T23:08:23.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.609+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa2540fe220 con 0x7fa254101c00 2024-02-16T23:08:23.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.610+0000 7fa23b7fe640 1 -- 172.21.15.64:0/2180663191 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa2540fe220 con 0x7fa254101c00 2024-02-16T23:08:23.610 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.611+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa23c06d750 msgr2=0x7fa23c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.611+0000 7fa25906e640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa23c06d750 0x7fa23c06fc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa244009af0 tx=0x7fa24400b040 comp rx=0 tx=0).stop 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.611+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 msgr2=0x7fa2540fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.611+0000 7fa25906e640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 0x7fa2540fd8f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa248031720 tx=0x7fa248009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.611+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 shutdown_connections 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.611+0000 7fa25906e640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254101c00 0x7fa2540fd8f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.612+0000 7fa25906e640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7fa23c06d750 0x7fa23c06fc00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.612+0000 7fa25906e640 1 --2- 172.21.15.64:0/2180663191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa254101260 0x7fa2540fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.612+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 >> 172.21.15.64:0/2180663191 conn(0x7fa2540f6e50 msgr2=0x7fa2540ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.612+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 shutdown_connections 2024-02-16T23:08:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.612+0000 7fa25906e640 1 -- 172.21.15.64:0/2180663191 wait complete. 2024-02-16T23:08:23.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-16T23:08:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.721+0000 7f338e51d640 1 -- 172.21.15.64:0/293040058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 msgr2=0x7f33881027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.721+0000 7f338e51d640 1 --2- 172.21.15.64:0/293040058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 0x7f33881027b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f33700048d0 tx=0x7f3370009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.721+0000 7f338e51d640 1 -- 172.21.15.64:0/293040058 shutdown_connections 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.721+0000 7f338e51d640 1 --2- 172.21.15.64:0/293040058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880644b0 0x7f33880648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.721+0000 7f338e51d640 1 --2- 172.21.15.64:0/293040058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 0x7f33881027b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.721+0000 7f338e51d640 1 -- 172.21.15.64:0/293040058 >> 172.21.15.64:0/293040058 conn(0x7f3388071230 msgr2=0x7f3388071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.721+0000 7f338e51d640 1 -- 172.21.15.64:0/293040058 shutdown_connections 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.721+0000 7f338e51d640 1 -- 172.21.15.64:0/293040058 wait complete. 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f338e51d640 1 Processor -- start 2024-02-16T23:08:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f338e51d640 1 -- start start 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f338e51d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880644b0 0x7f3388112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f338e51d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 0x7f3388113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f338e51d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33881137b0 con 0x7f33881023e0 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f338e51d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338810ccb0 con 0x7f33880644b0 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f33877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 0x7f3388113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f33877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 0x7f3388113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37114/0 (socket says 172.21.15.64:37114) 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.722+0000 7f33877fe640 1 -- 172.21.15.64:0/1852562667 learned_addr learned my addr 172.21.15.64:0/1852562667 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f3387fff640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880644b0 0x7f3388112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f33877fe640 1 -- 172.21.15.64:0/1852562667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880644b0 msgr2=0x7f3388112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f33877fe640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880644b0 0x7f3388112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f33877fe640 1 -- 172.21.15.64:0/1852562667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3370005040 con 0x7f33881023e0 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f3387fff640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880644b0 0x7f3388112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T23:08:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f33877fe640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 0x7f3388113120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f337800b790 tx=0x7f33780091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:23.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f3384ff9640 1 -- 172.21.15.64:0/1852562667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f337800c700 con 0x7f33881023e0 2024-02-16T23:08:23.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f3384ff9640 1 -- 172.21.15.64:0/1852562667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f337800cd30 con 0x7f33881023e0 2024-02-16T23:08:23.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338810cf90 con 0x7f33881023e0 2024-02-16T23:08:23.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f3384ff9640 1 -- 172.21.15.64:0/1852562667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3378013370 con 0x7f33881023e0 2024-02-16T23:08:23.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.723+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338810d4b0 con 0x7f33881023e0 2024-02-16T23:08:23.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.724+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3388064030 con 0x7f33881023e0 2024-02-16T23:08:23.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.724+0000 7f3384ff9640 1 -- 172.21.15.64:0/1852562667 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3378012070 con 0x7f33881023e0 2024-02-16T23:08:23.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.725+0000 7f3384ff9640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f337406d750 0x7f337406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T23:08:23.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.725+0000 7f3384ff9640 1 -- 172.21.15.64:0/1852562667 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f337808cc20 con 0x7f33881023e0 2024-02-16T23:08:23.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.725+0000 7f3387fff640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f337406d750 0x7f337406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T23:08:23.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.725+0000 7f3387fff640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f337406d750 0x7f337406fc00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f337002bda0 tx=0x7f3370009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T23:08:23.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.729+0000 7f3384ff9640 1 -- 172.21.15.64:0/1852562667 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3378056370 con 0x7f33881023e0 2024-02-16T23:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.922+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3388109510 con 0x7f33881023e0 2024-02-16T23:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.923+0000 7f3384ff9640 1 -- 172.21.15.64:0/1852562667 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v85) v1 ==== 81+0+2553 (secure 0 0 0) 0x7f3378059a30 con 0x7f33881023e0 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6824/1462436420 2024-02-17T23:04:23.398629+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6829/168979118 2024-02-17T23:03:59.013169+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6827/3220035256 2024-02-17T23:03:32.044676+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6826/2853522025 2024-02-17T23:04:47.133716+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1250745016 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1538951056 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1222682641 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6828/2862437082 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3482665154 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3482665154 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:0/2602543520 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:23.924 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/407404719 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1881977004 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1804571669 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:0/733087388 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:0/1972327284 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2780081097 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4258467456 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6828/168979118 2024-02-17T23:03:59.013169+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4290875509 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1046482636 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6829/2862437082 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:0/3116412139 2024-02-17T22:58:16.122252+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3827352044 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3005970232 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/700343021 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/600121841 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:23.925 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/743141814 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/600121841 2024-02-17T22:48:58.067040+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/700343021 2024-02-17T22:55:55.330401+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4194870513 2024-02-17T22:47:54.261948+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1988317603 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/1223017301 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1580822017 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1671562709 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6827/2853522025 2024-02-17T23:04:47.133716+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.192:6825/1462436420 2024-02-17T23:04:23.398629+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/1223017301 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/347371753 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/741447403 2024-02-17T22:59:02.601780+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3542090188 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3542090188 2024-02-17T22:47:24.032877+0000 2024-02-16T23:08:23.926 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6826/3220035256 2024-02-17T23:03:32.044676+0000 2024-02-16T23:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:listed 43 entries 2024-02-16T23:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f337406d750 msgr2=0x7f337406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f337406d750 0x7f337406fc00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f337002bda0 tx=0x7f3370009da0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 msgr2=0x7f3388113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 0x7f3388113120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f337800b790 tx=0x7f33780091d0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 shutdown_connections 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33881023e0 0x7f3388113120 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.64:6800/1480939122,v1:172.21.15.64:6801/1480939122] conn(0x7f337406d750 0x7f337406fc00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 --2- 172.21.15.64:0/1852562667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880644b0 0x7f3388112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 >> 172.21.15.64:0/1852562667 conn(0x7f3388071230 msgr2=0x7f33880fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 shutdown_connections 2024-02-16T23:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-16T23:08:23.925+0000 7f338e51d640 1 -- 172.21.15.64:0/1852562667 wait complete. 2024-02-16T23:08:23.934 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:23 smithi064 ceph-mon[93757]: pgmap v318: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:23.934 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:23 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/3148873464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T23:08:23.935 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:23 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/503933552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T23:08:23.935 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:23 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/2180663191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T23:08:23.936 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-02-16T23:08:23.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T23:08:23.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-02-16T23:08:23.962 INFO:teuthology.orchestra.run:waiting for 300 2024-02-16T23:08:24.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:23 smithi192 ceph-mon[82652]: pgmap v318: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:24.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:23 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/3148873464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T23:08:24.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:23 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/503933552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T23:08:24.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:23 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/2180663191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T23:08:25.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:24 smithi064 ceph-mon[93757]: from='client.? 172.21.15.64:0/1852562667' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T23:08:25.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:24 smithi192 ceph-mon[82652]: from='client.? 172.21.15.64:0/1852562667' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T23:08:26.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:25 smithi064 ceph-mon[93757]: pgmap v319: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:26.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:25 smithi192 ceph-mon[82652]: pgmap v319: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:28.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:27 smithi064 ceph-mon[93757]: pgmap v320: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:28.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:27 smithi192 ceph-mon[82652]: pgmap v320: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:30.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:29 smithi064 ceph-mon[93757]: pgmap v321: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:30.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:29 smithi192 ceph-mon[82652]: pgmap v321: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:32.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:31 smithi064 ceph-mon[93757]: pgmap v322: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:32.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:31 smithi192 ceph-mon[82652]: pgmap v322: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:33.118 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:08:33.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:08:34.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:33 smithi192 ceph-mon[82652]: pgmap v323: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:34.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:33 smithi064 ceph-mon[93757]: pgmap v323: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:36.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:35 smithi192 ceph-mon[82652]: pgmap v324: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:36.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:35 smithi064 ceph-mon[93757]: pgmap v324: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:38.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:37 smithi192 ceph-mon[82652]: pgmap v325: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:38.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:37 smithi064 ceph-mon[93757]: pgmap v325: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:40.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:39 smithi192 ceph-mon[82652]: pgmap v326: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:40.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:39 smithi064 ceph-mon[93757]: pgmap v326: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:42.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:41 smithi192 ceph-mon[82652]: pgmap v327: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:42.293 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:41 smithi064 ceph-mon[93757]: pgmap v327: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:44.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:43 smithi192 ceph-mon[82652]: pgmap v328: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:44.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:43 smithi064 ceph-mon[93757]: pgmap v328: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:46.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:45 smithi192 ceph-mon[82652]: pgmap v329: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:46.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:45 smithi064 ceph-mon[93757]: pgmap v329: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:48.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:47 smithi192 ceph-mon[82652]: pgmap v330: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:48.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:08:48.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:47 smithi064 ceph-mon[93757]: pgmap v330: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:48.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:08:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:49 smithi192 ceph-mon[82652]: pgmap v331: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:50.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:49 smithi064 ceph-mon[93757]: pgmap v331: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:52.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:51 smithi192 ceph-mon[82652]: pgmap v332: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:52.293 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:51 smithi064 ceph-mon[93757]: pgmap v332: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:54.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:53 smithi192 ceph-mon[82652]: pgmap v333: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:54.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:53 smithi064 ceph-mon[93757]: pgmap v333: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:56.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:55 smithi192 ceph-mon[82652]: pgmap v334: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:56.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:55 smithi064 ceph-mon[93757]: pgmap v334: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:58.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:08:57 smithi192 ceph-mon[82652]: pgmap v335: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:08:58.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:08:57 smithi064 ceph-mon[93757]: pgmap v335: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:00.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:00 smithi192 ceph-mon[82652]: pgmap v336: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:00.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:00 smithi064 ceph-mon[93757]: pgmap v336: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:02.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:02 smithi192 ceph-mon[82652]: pgmap v337: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:02.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:02 smithi064 ceph-mon[93757]: pgmap v337: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:03.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:09:03.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:09:04.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:04 smithi192 ceph-mon[82652]: pgmap v338: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:04.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:04 smithi064 ceph-mon[93757]: pgmap v338: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:06.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:06 smithi192 ceph-mon[82652]: pgmap v339: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:06.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:06 smithi064 ceph-mon[93757]: pgmap v339: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:08.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:08 smithi192 ceph-mon[82652]: pgmap v340: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:08.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:08 smithi064 ceph-mon[93757]: pgmap v340: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:10.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:10 smithi192 ceph-mon[82652]: pgmap v341: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:10.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:10 smithi064 ceph-mon[93757]: pgmap v341: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:12.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:12 smithi192 ceph-mon[82652]: pgmap v342: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:12.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:12 smithi064 ceph-mon[93757]: pgmap v342: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:14.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:14 smithi192 ceph-mon[82652]: pgmap v343: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:14.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:14 smithi064 ceph-mon[93757]: pgmap v343: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:15.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:15 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:09:15.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:15 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:09:15.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:15 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:09:15.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:15 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:09:15.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:15 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:09:15.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:15 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:09:16.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:16 smithi192 ceph-mon[82652]: pgmap v344: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:16.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:09:16.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:09:16.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:09:16.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:16 smithi064 ceph-mon[93757]: pgmap v344: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:16.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:09:16.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:09:16.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:09:18.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:18 smithi192 ceph-mon[82652]: pgmap v345: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:09:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:18 smithi064 ceph-mon[93757]: pgmap v345: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:09:20.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:20 smithi192 ceph-mon[82652]: pgmap v346: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:20.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:20 smithi064 ceph-mon[93757]: pgmap v346: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:22.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:22 smithi192 ceph-mon[82652]: pgmap v347: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:22.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:22 smithi064 ceph-mon[93757]: pgmap v347: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:24.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:24 smithi192 ceph-mon[82652]: pgmap v348: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:24.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:24 smithi064 ceph-mon[93757]: pgmap v348: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:26.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:26 smithi192 ceph-mon[82652]: pgmap v349: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:26.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:26 smithi064 ceph-mon[93757]: pgmap v349: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:28.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:28 smithi192 ceph-mon[82652]: pgmap v350: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:28.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:28 smithi064 ceph-mon[93757]: pgmap v350: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:30.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:30 smithi192 ceph-mon[82652]: pgmap v351: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:30.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:30 smithi064 ceph-mon[93757]: pgmap v351: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:32.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:32 smithi192 ceph-mon[82652]: pgmap v352: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:32.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:32 smithi064 ceph-mon[93757]: pgmap v352: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:33.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:09:33.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:09:34.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:34 smithi192 ceph-mon[82652]: pgmap v353: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:34.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:34 smithi064 ceph-mon[93757]: pgmap v353: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:36.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:36 smithi192 ceph-mon[82652]: pgmap v354: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:36.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:36 smithi064 ceph-mon[93757]: pgmap v354: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:38.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:38 smithi192 ceph-mon[82652]: pgmap v355: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:38.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:38 smithi064 ceph-mon[93757]: pgmap v355: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:40.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:40 smithi192 ceph-mon[82652]: pgmap v356: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:40.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:40 smithi064 ceph-mon[93757]: pgmap v356: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:42.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:42 smithi192 ceph-mon[82652]: pgmap v357: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:42.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:42 smithi064 ceph-mon[93757]: pgmap v357: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:44.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:44 smithi192 ceph-mon[82652]: pgmap v358: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:44.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:44 smithi064 ceph-mon[93757]: pgmap v358: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:46.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:46 smithi192 ceph-mon[82652]: pgmap v359: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:46.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:46 smithi064 ceph-mon[93757]: pgmap v359: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:48.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:48 smithi192 ceph-mon[82652]: pgmap v360: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:48.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:09:48.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:48 smithi064 ceph-mon[93757]: pgmap v360: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:48.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:09:50.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:50 smithi192 ceph-mon[82652]: pgmap v361: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:50.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:50 smithi064 ceph-mon[93757]: pgmap v361: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:52.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:52 smithi192 ceph-mon[82652]: pgmap v362: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:52.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:52 smithi064 ceph-mon[93757]: pgmap v362: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:54.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:54 smithi192 ceph-mon[82652]: pgmap v363: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:54.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:54 smithi064 ceph-mon[93757]: pgmap v363: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:56.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:56 smithi192 ceph-mon[82652]: pgmap v364: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:56.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:56 smithi064 ceph-mon[93757]: pgmap v364: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:58.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:09:58 smithi192 ceph-mon[82652]: pgmap v365: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:09:58.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:09:58 smithi064 ceph-mon[93757]: pgmap v365: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:00.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:00 smithi064 ceph-mon[93757]: pgmap v366: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:00.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:00 smithi064 ceph-mon[93757]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:10:00.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:00 smithi192 ceph-mon[82652]: pgmap v366: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:00.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:00 smithi192 ceph-mon[82652]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T23:10:02.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:02 smithi064 ceph-mon[93757]: pgmap v367: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:02.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:02 smithi192 ceph-mon[82652]: pgmap v367: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:03.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:10:03.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:10:04.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:04 smithi192 ceph-mon[82652]: pgmap v368: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:04.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:04 smithi064 ceph-mon[93757]: pgmap v368: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:06.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:06 smithi064 ceph-mon[93757]: pgmap v369: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:06.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:06 smithi192 ceph-mon[82652]: pgmap v369: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:08.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:08 smithi064 ceph-mon[93757]: pgmap v370: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:08.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:08 smithi192 ceph-mon[82652]: pgmap v370: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:10.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:10 smithi192 ceph-mon[82652]: pgmap v371: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:10.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:10 smithi064 ceph-mon[93757]: pgmap v371: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:12.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:12 smithi064 ceph-mon[93757]: pgmap v372: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:12.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:12 smithi192 ceph-mon[82652]: pgmap v372: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:14.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:14 smithi064 ceph-mon[93757]: pgmap v373: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:14.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:14 smithi192 ceph-mon[82652]: pgmap v373: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:16.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:16 smithi064 ceph-mon[93757]: pgmap v374: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:16.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:16 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:10:16.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:16 smithi192 ceph-mon[82652]: pgmap v374: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:16.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:16 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:10:17.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:10:17.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:10:17.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:10:17.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:10:17.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:10:17.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:10:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:18 smithi064 ceph-mon[93757]: pgmap v375: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:18.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:10:19.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:18 smithi192 ceph-mon[82652]: pgmap v375: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:19.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:10:20.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:20 smithi064 ceph-mon[93757]: pgmap v376: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:21.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:20 smithi192 ceph-mon[82652]: pgmap v376: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:22.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:22 smithi064 ceph-mon[93757]: pgmap v377: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:23.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:22 smithi192 ceph-mon[82652]: pgmap v377: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:24.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:24 smithi064 ceph-mon[93757]: pgmap v378: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:25.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:24 smithi192 ceph-mon[82652]: pgmap v378: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:26.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:26 smithi064 ceph-mon[93757]: pgmap v379: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:27.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:26 smithi192 ceph-mon[82652]: pgmap v379: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:28.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:28 smithi064 ceph-mon[93757]: pgmap v380: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:29.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:28 smithi192 ceph-mon[82652]: pgmap v380: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:30.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:30 smithi064 ceph-mon[93757]: pgmap v381: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:31.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:30 smithi192 ceph-mon[82652]: pgmap v381: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:32.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:32 smithi064 ceph-mon[93757]: pgmap v382: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:33.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:32 smithi192 ceph-mon[82652]: pgmap v382: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:33.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:10:34.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:10:34.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:34 smithi064 ceph-mon[93757]: pgmap v383: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:35.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:34 smithi192 ceph-mon[82652]: pgmap v383: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:36.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:36 smithi064 ceph-mon[93757]: pgmap v384: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:37.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:36 smithi192 ceph-mon[82652]: pgmap v384: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:38.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:38 smithi064 ceph-mon[93757]: pgmap v385: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:39.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:38 smithi192 ceph-mon[82652]: pgmap v385: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:40.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:40 smithi064 ceph-mon[93757]: pgmap v386: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:41.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:40 smithi192 ceph-mon[82652]: pgmap v386: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:42.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:42 smithi064 ceph-mon[93757]: pgmap v387: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:43.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:42 smithi192 ceph-mon[82652]: pgmap v387: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:44.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:44 smithi064 ceph-mon[93757]: pgmap v388: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:45.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:44 smithi192 ceph-mon[82652]: pgmap v388: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:46.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:46 smithi064 ceph-mon[93757]: pgmap v389: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:47.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:46 smithi192 ceph-mon[82652]: pgmap v389: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:48.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:48 smithi064 ceph-mon[93757]: pgmap v390: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:48.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:10:49.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:48 smithi192 ceph-mon[82652]: pgmap v390: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:49.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:10:50.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:50 smithi064 ceph-mon[93757]: pgmap v391: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:51.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:50 smithi192 ceph-mon[82652]: pgmap v391: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:52.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:52 smithi064 ceph-mon[93757]: pgmap v392: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:53.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:52 smithi192 ceph-mon[82652]: pgmap v392: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:54.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:54 smithi064 ceph-mon[93757]: pgmap v393: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:55.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:54 smithi192 ceph-mon[82652]: pgmap v393: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:56.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:56 smithi064 ceph-mon[93757]: pgmap v394: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:57.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:56 smithi192 ceph-mon[82652]: pgmap v394: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:58.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:10:58 smithi064 ceph-mon[93757]: pgmap v395: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:10:59.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:10:58 smithi192 ceph-mon[82652]: pgmap v395: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:01.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:00 smithi192 ceph-mon[82652]: pgmap v396: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:01.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:00 smithi064 ceph-mon[93757]: pgmap v396: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:03.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:02 smithi192 ceph-mon[82652]: pgmap v397: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:03.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:02 smithi064 ceph-mon[93757]: pgmap v397: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:04.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:11:04.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:11:05.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:04 smithi192 ceph-mon[82652]: pgmap v398: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:05.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:04 smithi064 ceph-mon[93757]: pgmap v398: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:07.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:06 smithi192 ceph-mon[82652]: pgmap v399: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:07.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:06 smithi064 ceph-mon[93757]: pgmap v399: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:09.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:08 smithi192 ceph-mon[82652]: pgmap v400: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:09.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:08 smithi064 ceph-mon[93757]: pgmap v400: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:11.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:10 smithi192 ceph-mon[82652]: pgmap v401: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:11.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:10 smithi064 ceph-mon[93757]: pgmap v401: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:13.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:12 smithi192 ceph-mon[82652]: pgmap v402: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:13.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:12 smithi064 ceph-mon[93757]: pgmap v402: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:15.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:14 smithi192 ceph-mon[82652]: pgmap v403: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:15.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:14 smithi064 ceph-mon[93757]: pgmap v403: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:17.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:16 smithi192 ceph-mon[82652]: pgmap v404: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:17.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:16 smithi064 ceph-mon[93757]: pgmap v404: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:18.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:11:18.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:11:18.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:11:18.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:17 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:11:18.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:11:18.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:11:18.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:11:18.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:17 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:11:19.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:18 smithi192 ceph-mon[82652]: pgmap v405: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:19.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:11:19.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:18 smithi064 ceph-mon[93757]: pgmap v405: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:19.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:11:21.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:20 smithi192 ceph-mon[82652]: pgmap v406: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:21.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:20 smithi064 ceph-mon[93757]: pgmap v406: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:23.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:22 smithi192 ceph-mon[82652]: pgmap v407: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:23.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:22 smithi064 ceph-mon[93757]: pgmap v407: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:25.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:24 smithi192 ceph-mon[82652]: pgmap v408: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:25.120 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:24 smithi064 ceph-mon[93757]: pgmap v408: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:27.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:26 smithi192 ceph-mon[82652]: pgmap v409: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:27.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:26 smithi064 ceph-mon[93757]: pgmap v409: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:29.033 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:28 smithi192 ceph-mon[82652]: pgmap v410: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:29.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:28 smithi064 ceph-mon[93757]: pgmap v410: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:30.018 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:29 smithi064 ceph-mon[93757]: pgmap v411: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:30.036 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:29 smithi192 ceph-mon[82652]: pgmap v411: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:32.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:31 smithi064 ceph-mon[93757]: pgmap v412: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:32.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:31 smithi192 ceph-mon[82652]: pgmap v412: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:33.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:32 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:11:33.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:32 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:11:34.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:33 smithi064 ceph-mon[93757]: pgmap v413: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:34.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:33 smithi192 ceph-mon[82652]: pgmap v413: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:36.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:35 smithi064 ceph-mon[93757]: pgmap v414: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:36.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:35 smithi192 ceph-mon[82652]: pgmap v414: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:38.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:37 smithi064 ceph-mon[93757]: pgmap v415: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:38.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:37 smithi192 ceph-mon[82652]: pgmap v415: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:40.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:39 smithi064 ceph-mon[93757]: pgmap v416: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:40.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:39 smithi192 ceph-mon[82652]: pgmap v416: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:42.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:41 smithi064 ceph-mon[93757]: pgmap v417: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:42.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:41 smithi192 ceph-mon[82652]: pgmap v417: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:44.117 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:43 smithi064 ceph-mon[93757]: pgmap v418: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:44.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:43 smithi192 ceph-mon[82652]: pgmap v418: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:46.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:45 smithi192 ceph-mon[82652]: pgmap v419: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:46.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:45 smithi064 ceph-mon[93757]: pgmap v419: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:48.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:47 smithi192 ceph-mon[82652]: pgmap v420: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:48.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:47 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:11:48.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:47 smithi064 ceph-mon[93757]: pgmap v420: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:48.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:47 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:11:50.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:49 smithi192 ceph-mon[82652]: pgmap v421: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:50.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:49 smithi064 ceph-mon[93757]: pgmap v421: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:52.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:51 smithi192 ceph-mon[82652]: pgmap v422: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:52.293 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:51 smithi064 ceph-mon[93757]: pgmap v422: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:54.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:53 smithi192 ceph-mon[82652]: pgmap v423: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:54.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:53 smithi064 ceph-mon[93757]: pgmap v423: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:56.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:55 smithi192 ceph-mon[82652]: pgmap v424: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:56.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:55 smithi064 ceph-mon[93757]: pgmap v424: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:58.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:57 smithi192 ceph-mon[82652]: pgmap v425: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:11:58.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:57 smithi064 ceph-mon[93757]: pgmap v425: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:00.169 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:11:59 smithi064 ceph-mon[93757]: pgmap v426: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:00.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:11:59 smithi192 ceph-mon[82652]: pgmap v426: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:02.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:01 smithi192 ceph-mon[82652]: pgmap v427: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:02.292 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:01 smithi064 ceph-mon[93757]: pgmap v427: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:03.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:02 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:12:03.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:02 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:12:04.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:03 smithi192 ceph-mon[82652]: pgmap v428: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:04.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:03 smithi064 ceph-mon[93757]: pgmap v428: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:06.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:05 smithi192 ceph-mon[82652]: pgmap v429: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:06.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:05 smithi064 ceph-mon[93757]: pgmap v429: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:08.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:07 smithi192 ceph-mon[82652]: pgmap v430: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:08.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:07 smithi064 ceph-mon[93757]: pgmap v430: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:10.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:09 smithi192 ceph-mon[82652]: pgmap v431: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:10.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:09 smithi064 ceph-mon[93757]: pgmap v431: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:12.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:11 smithi192 ceph-mon[82652]: pgmap v432: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:12.292 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:11 smithi064 ceph-mon[93757]: pgmap v432: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:14.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:13 smithi192 ceph-mon[82652]: pgmap v433: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:14.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:13 smithi064 ceph-mon[93757]: pgmap v433: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:16.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:16 smithi192 ceph-mon[82652]: pgmap v434: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:16.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:16 smithi064 ceph-mon[93757]: pgmap v434: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:18.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:18 smithi192 ceph-mon[82652]: pgmap v435: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:18.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:12:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:12:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:12:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:12:18.534 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:12:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:18 smithi064 ceph-mon[93757]: pgmap v435: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:12:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:12:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:12:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:12:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:12:20.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:20 smithi192 ceph-mon[82652]: pgmap v436: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:20.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:20 smithi064 ceph-mon[93757]: pgmap v436: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:22.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:22 smithi192 ceph-mon[82652]: pgmap v437: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:22.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:22 smithi064 ceph-mon[93757]: pgmap v437: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:24.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:24 smithi192 ceph-mon[82652]: pgmap v438: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:24.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:24 smithi064 ceph-mon[93757]: pgmap v438: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:26.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:26 smithi192 ceph-mon[82652]: pgmap v439: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:26.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:26 smithi064 ceph-mon[93757]: pgmap v439: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:28.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:28 smithi192 ceph-mon[82652]: pgmap v440: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:28.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:28 smithi064 ceph-mon[93757]: pgmap v440: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:30.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:30 smithi192 ceph-mon[82652]: pgmap v441: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:30.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:30 smithi064 ceph-mon[93757]: pgmap v441: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:32.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:32 smithi192 ceph-mon[82652]: pgmap v442: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:32.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:32 smithi064 ceph-mon[93757]: pgmap v442: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:33.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:33 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:12:33.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:33 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:12:34.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:34 smithi192 ceph-mon[82652]: pgmap v443: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:34.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:34 smithi064 ceph-mon[93757]: pgmap v443: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:36.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:36 smithi192 ceph-mon[82652]: pgmap v444: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:36.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:36 smithi064 ceph-mon[93757]: pgmap v444: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:38.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:38 smithi192 ceph-mon[82652]: pgmap v445: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:38.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:38 smithi064 ceph-mon[93757]: pgmap v445: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:40.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:40 smithi192 ceph-mon[82652]: pgmap v446: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:40.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:40 smithi064 ceph-mon[93757]: pgmap v446: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:42.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:42 smithi192 ceph-mon[82652]: pgmap v447: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:42.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:42 smithi064 ceph-mon[93757]: pgmap v447: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:44.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:44 smithi192 ceph-mon[82652]: pgmap v448: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:44.616 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:44 smithi064 ceph-mon[93757]: pgmap v448: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:46.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:46 smithi192 ceph-mon[82652]: pgmap v449: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:46.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:46 smithi064 ceph-mon[93757]: pgmap v449: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:48.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:48 smithi192 ceph-mon[82652]: pgmap v450: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:48.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:48 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:12:48.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:48 smithi064 ceph-mon[93757]: pgmap v450: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:48.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:48 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:12:50.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:50 smithi192 ceph-mon[82652]: pgmap v451: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:50.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:50 smithi064 ceph-mon[93757]: pgmap v451: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:52.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:52 smithi192 ceph-mon[82652]: pgmap v452: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:52.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:52 smithi064 ceph-mon[93757]: pgmap v452: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:54.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:54 smithi192 ceph-mon[82652]: pgmap v453: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:54.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:54 smithi064 ceph-mon[93757]: pgmap v453: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:56.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:56 smithi192 ceph-mon[82652]: pgmap v454: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:56.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:56 smithi064 ceph-mon[93757]: pgmap v454: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:58.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:12:58 smithi192 ceph-mon[82652]: pgmap v455: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:12:58.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:12:58 smithi064 ceph-mon[93757]: pgmap v455: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:00.533 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:00 smithi192 ceph-mon[82652]: pgmap v456: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:00.619 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:00 smithi064 ceph-mon[93757]: pgmap v456: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:02.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:02 smithi064 ceph-mon[93757]: pgmap v457: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:02.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:02 smithi192 ceph-mon[82652]: pgmap v457: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:03.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:03 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:13:03.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:03 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:13:04.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:04 smithi064 ceph-mon[93757]: pgmap v458: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:04.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:04 smithi192 ceph-mon[82652]: pgmap v458: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:06.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:06 smithi064 ceph-mon[93757]: pgmap v459: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:06.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:06 smithi192 ceph-mon[82652]: pgmap v459: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:08.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:08 smithi064 ceph-mon[93757]: pgmap v460: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:08.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:08 smithi192 ceph-mon[82652]: pgmap v460: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:10.586 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:10 smithi192 ceph-mon[82652]: pgmap v461: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:10.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:10 smithi064 ceph-mon[93757]: pgmap v461: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:12.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:12 smithi064 ceph-mon[93757]: pgmap v462: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:12.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:12 smithi192 ceph-mon[82652]: pgmap v462: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:14.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:14 smithi064 ceph-mon[93757]: pgmap v463: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:14.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:14 smithi192 ceph-mon[82652]: pgmap v463: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:16.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:16 smithi064 ceph-mon[93757]: pgmap v464: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:16.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:16 smithi192 ceph-mon[82652]: pgmap v464: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:18 smithi064 ceph-mon[93757]: pgmap v465: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:13:18.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:18 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:13:18.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:18 smithi192 ceph-mon[82652]: pgmap v465: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:18.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T23:13:18.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:18 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T23:13:19.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:19 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:13:19.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:19 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:13:19.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:19 smithi064 ceph-mon[93757]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:13:19.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:19 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T23:13:19.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:19 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T23:13:19.784 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:19 smithi192 ceph-mon[82652]: from='mgr.34100 172.21.15.64:0/3102244709' entity='mgr.smithi064.kubmlk' 2024-02-16T23:13:20.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:20 smithi064 ceph-mon[93757]: pgmap v466: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:20.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:20 smithi192 ceph-mon[82652]: pgmap v466: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:22.617 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:22 smithi064 ceph-mon[93757]: pgmap v467: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:22.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:22 smithi192 ceph-mon[82652]: pgmap v467: 65 pgs: 65 active+clean; 251 MiB data, 999 MiB used, 535 GiB / 536 GiB avail 2024-02-16T23:13:23.028 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-02-16T23:13:23.028 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T23:13:23.030 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-16T23:13:23.043 ERROR:teuthology.run_tasks:Manager failed: full_sequential_finally Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 38, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 43, in task ((taskname, confg),) = entry.items() ValueError: too many values to unpack (expected 1) 2024-02-16T23:13:23.043 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-16T23:13:23.097 INFO:tasks.cephadm:Teardown begin 2024-02-16T23:13:23.097 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T23:13:23.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T23:13:23.133 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T23:13:23.168 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-16T23:13:23.168 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-16T23:13:23.191 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-16T23:13:23.225 INFO:tasks.cephadm:Stopping all daemons... 2024-02-16T23:13:23.225 INFO:tasks.cephadm.mon.smithi064:Stopping mon.smithi064... 2024-02-16T23:13:23.225 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064 2024-02-16T23:13:23.526 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:23 smithi064 systemd[1]: Stopping Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:13:23.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[93753]: 2024-02-16T23:13:23.526+0000 7fef11857700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi064 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:13:23.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:23 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064[93753]: 2024-02-16T23:13:23.526+0000 7fef11857700 -1 mon.smithi064@0(leader) e3 *** Got Signal Terminated *** 2024-02-16T23:13:23.867 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:23 smithi064 podman[149977]: 2024-02-16 23:13:23.587959597 +0000 UTC m=+0.161311879 container died a19a307284664cb3a9e8202f3ba0310e0276082286160bf572cf3bc53d50ae88 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214) 2024-02-16T23:13:24.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:24 smithi064 podman[149977]: 2024-02-16 23:13:24.221796882 +0000 UTC m=+0.795149172 container cleanup a19a307284664cb3a9e8202f3ba0310e0276082286160bf572cf3bc53d50ae88 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-16T23:13:24.367 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:24 smithi064 bash[149977]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064 2024-02-16T23:13:24.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service' 2024-02-16T23:13:24.675 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:24 smithi064 podman[150002]: 2024-02-16 23:13:24.41140782 +0000 UTC m=+0.819281966 container remove a19a307284664cb3a9e8202f3ba0310e0276082286160bf572cf3bc53d50ae88 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi064, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:13:24.675 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:24 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service: Deactivated successfully. 2024-02-16T23:13:24.675 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:24 smithi064 systemd[1]: Stopped Ceph mon.smithi064 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:13:24.675 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 16 23:13:24 smithi064 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi064.service: Consumed 11.491s CPU time. 2024-02-16T23:13:24.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T23:13:24.706 INFO:tasks.cephadm.mon.smithi064:Stopped mon.smithi064 2024-02-16T23:13:24.706 INFO:tasks.cephadm.mon.smithi192:Stopping mon.smithi192... 2024-02-16T23:13:24.706 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi192 2024-02-16T23:13:25.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:24 smithi192 systemd[1]: Stopping Ceph mon.smithi192 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:13:25.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:25 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192[82648]: 2024-02-16T23:13:25.009+0000 7fef13572700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi192 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:13:25.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:25 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192[82648]: 2024-02-16T23:13:25.009+0000 7fef13572700 -1 mon.smithi192@1(peon) e3 *** Got Signal Terminated *** 2024-02-16T23:13:25.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:25 smithi192 podman[119574]: 2024-02-16 23:13:25.235059666 +0000 UTC m=+0.333303600 container died eb1d711c4dcd147eae17f04420f53e97f0cae3fbb708e143d97fb2fdb73441fa (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T23:13:25.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:25 smithi192 podman[119574]: 2024-02-16 23:13:25.660706652 +0000 UTC m=+0.758950584 container cleanup eb1d711c4dcd147eae17f04420f53e97f0cae3fbb708e143d97fb2fdb73441fa (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-16T23:13:25.783 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:25 smithi192 bash[119574]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192 2024-02-16T23:13:26.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi192.service' 2024-02-16T23:13:26.151 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:25 smithi192 podman[119588]: 2024-02-16 23:13:25.895711915 +0000 UTC m=+0.659225216 container remove eb1d711c4dcd147eae17f04420f53e97f0cae3fbb708e143d97fb2fdb73441fa (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-mon-smithi192, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-16T23:13:26.151 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:26 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi192.service: Deactivated successfully. 2024-02-16T23:13:26.151 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:26 smithi192 systemd[1]: Stopped Ceph mon.smithi192 for 2912dc76-cd1d-11ee-95ba-87774f69a715. 2024-02-16T23:13:26.152 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Feb 16 23:13:26 smithi192 systemd[1]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@mon.smithi192.service: Consumed 6.623s CPU time. 2024-02-16T23:13:26.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T23:13:26.183 INFO:tasks.cephadm.mon.smithi192:Stopped mon.smithi192 2024-02-16T23:13:26.183 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-16T23:13:26.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.0 2024-02-16T23:13:26.614 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:26 smithi064 systemd[1]: Stopping Ceph osd.0 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:13:26.867 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:26 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[102605]: 2024-02-16T23:13:26.614+0000 7f2aefd3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:13:26.867 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:26 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[102605]: 2024-02-16T23:13:26.614+0000 7f2aefd3a700 -1 osd.0 85 *** Got signal Terminated *** 2024-02-16T23:13:26.867 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:26 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0[102605]: 2024-02-16T23:13:26.614+0000 7f2aefd3a700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:13:32.293 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:32 smithi064 podman[150110]: 2024-02-16 23:13:32.041783903 +0000 UTC m=+5.658741657 container died 3d93cd9ed3d9e8a3e2a6bd5264902706e2cf3cb84bcc5b69f287d9a00b4615fc (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-16T23:13:32.617 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:32 smithi064 podman[150110]: 2024-02-16 23:13:32.398261306 +0000 UTC m=+6.015219059 container cleanup 3d93cd9ed3d9e8a3e2a6bd5264902706e2cf3cb84bcc5b69f287d9a00b4615fc (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:13:32.617 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:32 smithi064 bash[150110]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0 2024-02-16T23:13:33.117 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:32 smithi064 podman[150128]: 2024-02-16 23:13:32.816555415 +0000 UTC m=+0.778264243 container remove 3d93cd9ed3d9e8a3e2a6bd5264902706e2cf3cb84bcc5b69f287d9a00b4615fc (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5) 2024-02-16T23:13:33.445 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:33 smithi064 podman[150223]: 2024-02-16 23:13:33.069527945 +0000 UTC m=+0.017981531 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:13:33.445 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:33 smithi064 podman[150223]: 2024-02-16 23:13:33.200935089 +0000 UTC m=+0.149388672 container create 94d21a1d76304cbe747e273d153acd0ab966b77305f19bec7fa7874f390c87ac (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-16T23:13:33.445 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:33 smithi064 podman[150223]: 2024-02-16 23:13:33.445649559 +0000 UTC m=+0.394103153 container init 94d21a1d76304cbe747e273d153acd0ab966b77305f19bec7fa7874f390c87ac (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-16T23:13:33.744 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:33 smithi064 podman[150223]: 2024-02-16 23:13:33.449392381 +0000 UTC m=+0.397845965 container start 94d21a1d76304cbe747e273d153acd0ab966b77305f19bec7fa7874f390c87ac (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:13:33.745 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:33 smithi064 podman[150223]: 2024-02-16 23:13:33.561821496 +0000 UTC m=+0.510275094 container attach 94d21a1d76304cbe747e273d153acd0ab966b77305f19bec7fa7874f390c87ac (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-16T23:13:34.117 INFO:journalctl@ceph.osd.0.smithi064.stdout:Feb 16 23:13:33 smithi064 podman[150223]: 2024-02-16 23:13:33.746277134 +0000 UTC m=+0.694730722 container died 94d21a1d76304cbe747e273d153acd0ab966b77305f19bec7fa7874f390c87ac (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD) 2024-02-16T23:13:34.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.0.service' 2024-02-16T23:13:34.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T23:13:34.553 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-16T23:13:34.554 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-16T23:13:34.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.1 2024-02-16T23:13:34.788 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:34 smithi064 systemd[1]: Stopping Ceph osd.1 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:13:35.117 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:34 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[108258]: 2024-02-16T23:13:34.876+0000 7f2d6a081700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:13:35.117 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:34 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[108258]: 2024-02-16T23:13:34.876+0000 7f2d6a081700 -1 osd.1 85 *** Got signal Terminated *** 2024-02-16T23:13:35.117 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:34 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1[108258]: 2024-02-16T23:13:34.876+0000 7f2d6a081700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:13:40.544 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:40 smithi064 podman[150329]: 2024-02-16 23:13:40.269135409 +0000 UTC m=+5.521611833 container died 7044a8328d9a666a4164d60801330c6d4795cef8e2050baeabef0cb0ebb298ba (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-16T23:13:40.867 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:40 smithi064 podman[150329]: 2024-02-16 23:13:40.635533735 +0000 UTC m=+5.888010164 container cleanup 7044a8328d9a666a4164d60801330c6d4795cef8e2050baeabef0cb0ebb298ba (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-16T23:13:40.867 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:40 smithi064 bash[150329]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1 2024-02-16T23:13:41.250 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:40 smithi064 podman[150357]: 2024-02-16 23:13:40.929067235 +0000 UTC m=+0.651091933 container remove 7044a8328d9a666a4164d60801330c6d4795cef8e2050baeabef0cb0ebb298ba (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-16T23:13:41.501 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:41 smithi064 podman[150452]: 2024-02-16 23:13:41.154679023 +0000 UTC m=+0.017993423 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:13:41.501 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:41 smithi064 podman[150452]: 2024-02-16 23:13:41.288458181 +0000 UTC m=+0.151772578 container create 4d9c6737155d169d87e3807cdd74daea01fdbd5d12ed457f804a93c257063e01 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD) 2024-02-16T23:13:41.501 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:41 smithi064 podman[150452]: 2024-02-16 23:13:41.49935635 +0000 UTC m=+0.362670759 container init 4d9c6737155d169d87e3807cdd74daea01fdbd5d12ed457f804a93c257063e01 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:13:41.501 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:41 smithi064 podman[150452]: 2024-02-16 23:13:41.50244415 +0000 UTC m=+0.365758555 container start 4d9c6737155d169d87e3807cdd74daea01fdbd5d12ed457f804a93c257063e01 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:13:41.797 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:41 smithi064 podman[150452]: 2024-02-16 23:13:41.600037398 +0000 UTC m=+0.463351797 container attach 4d9c6737155d169d87e3807cdd74daea01fdbd5d12ed457f804a93c257063e01 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-16T23:13:42.117 INFO:journalctl@ceph.osd.1.smithi064.stdout:Feb 16 23:13:41 smithi064 podman[150452]: 2024-02-16 23:13:41.799062083 +0000 UTC m=+0.662376487 container died 4d9c6737155d169d87e3807cdd74daea01fdbd5d12ed457f804a93c257063e01 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-02-16T23:13:42.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.1.service' 2024-02-16T23:13:42.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T23:13:42.476 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-16T23:13:42.477 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-16T23:13:42.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.2 2024-02-16T23:13:42.830 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:42 smithi064 systemd[1]: Stopping Ceph osd.2 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:13:43.117 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:42 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[113975]: 2024-02-16T23:13:42.831+0000 7f77acab6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:13:43.117 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:42 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[113975]: 2024-02-16T23:13:42.831+0000 7f77acab6700 -1 osd.2 85 *** Got signal Terminated *** 2024-02-16T23:13:43.117 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:42 smithi064 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2[113975]: 2024-02-16T23:13:42.831+0000 7f77acab6700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:13:48.399 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:48 smithi064 podman[150558]: 2024-02-16 23:13:48.141615539 +0000 UTC m=+5.438131097 container died a73d025bae8ba90de081fa338e4d08e4abd5d73b46132828c84ef86936e8bb46 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:13:48.714 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:48 smithi064 podman[150558]: 2024-02-16 23:13:48.489773018 +0000 UTC m=+5.786288575 container cleanup a73d025bae8ba90de081fa338e4d08e4abd5d73b46132828c84ef86936e8bb46 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-16T23:13:48.714 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:48 smithi064 bash[150558]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2 2024-02-16T23:13:49.116 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:48 smithi064 podman[150576]: 2024-02-16 23:13:48.715874853 +0000 UTC m=+0.569492926 container remove a73d025bae8ba90de081fa338e4d08e4abd5d73b46132828c84ef86936e8bb46 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:13:49.377 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:49 smithi064 podman[150670]: 2024-02-16 23:13:49.02234181 +0000 UTC m=+0.018464246 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:13:49.377 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:49 smithi064 podman[150670]: 2024-02-16 23:13:49.166937821 +0000 UTC m=+0.163060243 container create fa36fdc87cc6d003b529f4663732eeaf9b9a96ea0572c3ebbd300ce30d8242f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-16T23:13:49.680 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:49 smithi064 podman[150670]: 2024-02-16 23:13:49.378291576 +0000 UTC m=+0.374414005 container init fa36fdc87cc6d003b529f4663732eeaf9b9a96ea0572c3ebbd300ce30d8242f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-16T23:13:49.681 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:49 smithi064 podman[150670]: 2024-02-16 23:13:49.381799444 +0000 UTC m=+0.377921874 container start fa36fdc87cc6d003b529f4663732eeaf9b9a96ea0572c3ebbd300ce30d8242f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T23:13:49.681 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:49 smithi064 podman[150670]: 2024-02-16 23:13:49.452689461 +0000 UTC m=+0.448811891 container attach fa36fdc87cc6d003b529f4663732eeaf9b9a96ea0572c3ebbd300ce30d8242f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T23:13:49.963 INFO:journalctl@ceph.osd.2.smithi064.stdout:Feb 16 23:13:49 smithi064 podman[150670]: 2024-02-16 23:13:49.68196596 +0000 UTC m=+0.678088391 container died fa36fdc87cc6d003b529f4663732eeaf9b9a96ea0572c3ebbd300ce30d8242f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:13:50.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.2.service' 2024-02-16T23:13:50.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T23:13:50.306 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-16T23:13:50.307 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-16T23:13:50.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.3 2024-02-16T23:13:50.671 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:50 smithi192 systemd[1]: Stopping Ceph osd.3 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:13:50.932 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:50 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[92529]: 2024-02-16T23:13:50.671+0000 7fde7e992700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:13:50.932 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:50 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[92529]: 2024-02-16T23:13:50.671+0000 7fde7e992700 -1 osd.3 85 *** Got signal Terminated *** 2024-02-16T23:13:50.932 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:50 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3[92529]: 2024-02-16T23:13:50.671+0000 7fde7e992700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:13:53.284 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:52 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:52.955+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:13:54.283 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:53 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:53.909+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:13:54.938 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:13:54 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:13:54.656+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:13:55.283 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:54 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:54.939+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:13:55.925 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:13:55 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:13:55.654+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:13:56.246 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:55 smithi192 podman[119695]: 2024-02-16 23:13:55.998346875 +0000 UTC m=+5.486652759 container died 3b60e36c880384ee3dd9f06789c55c8f66d9a3e4a64f2fcc21d93a5da041170a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-16T23:13:56.246 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:55 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:55.926+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:13:56.505 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:56 smithi192 podman[119695]: 2024-02-16 23:13:56.330339176 +0000 UTC m=+5.818645070 container cleanup 3b60e36c880384ee3dd9f06789c55c8f66d9a3e4a64f2fcc21d93a5da041170a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:13:56.505 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:56 smithi192 bash[119695]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3 2024-02-16T23:13:56.783 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:56 smithi192 podman[119724]: 2024-02-16 23:13:56.506345073 +0000 UTC m=+0.504128017 container remove 3b60e36c880384ee3dd9f06789c55c8f66d9a3e4a64f2fcc21d93a5da041170a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-16T23:13:56.783 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:13:56 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:13:56.661+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:13:57.183 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:56 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:56.908+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:13:57.183 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:56 smithi192 podman[119814]: 2024-02-16 23:13:56.940775119 +0000 UTC m=+0.111783033 container create 792ad5dd88013258c7f7df124ded89c4e33cbbfc9b61bfa09bca4734e1eb05ad (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-16T23:13:57.183 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:56 smithi192 podman[119814]: 2024-02-16 23:13:56.846927563 +0000 UTC m=+0.017935440 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:13:57.183 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:57 smithi192 podman[119814]: 2024-02-16 23:13:57.184705062 +0000 UTC m=+0.355712934 container init 792ad5dd88013258c7f7df124ded89c4e33cbbfc9b61bfa09bca4734e1eb05ad (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:13:57.472 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:57 smithi192 podman[119814]: 2024-02-16 23:13:57.188195611 +0000 UTC m=+0.359203483 container start 792ad5dd88013258c7f7df124ded89c4e33cbbfc9b61bfa09bca4734e1eb05ad (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-16T23:13:57.473 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:57 smithi192 podman[119814]: 2024-02-16 23:13:57.301667957 +0000 UTC m=+0.472675847 container attach 792ad5dd88013258c7f7df124ded89c4e33cbbfc9b61bfa09bca4734e1eb05ad (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-16T23:13:57.783 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:13:57 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:13:57.671+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:13:57.783 INFO:journalctl@ceph.osd.3.smithi192.stdout:Feb 16 23:13:57 smithi192 podman[119814]: 2024-02-16 23:13:57.47429743 +0000 UTC m=+0.645305297 container died 792ad5dd88013258c7f7df124ded89c4e33cbbfc9b61bfa09bca4734e1eb05ad (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-16T23:13:58.046 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:57 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:57.947+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:13:58.070 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.3.service' 2024-02-16T23:13:58.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T23:13:58.114 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-16T23:13:58.114 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-16T23:13:58.115 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.4 2024-02-16T23:13:58.424 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:58 smithi192 systemd[1]: Stopping Ceph osd.4 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:13:58.783 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:13:58 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:13:58.673+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:13:58.783 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:58 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:58.424+0000 7f84cb87f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:13:58.783 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:58 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:58.424+0000 7f84cb87f700 -1 osd.4 85 *** Got signal Terminated *** 2024-02-16T23:13:58.784 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:58 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:58.424+0000 7f84cb87f700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:13:59.283 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:13:58 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:13:58.982+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:14:00.011 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:13:59 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:13:59.690+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:00.283 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:00 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:14:00.012+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:14:01.033 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:00 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:14:00.975+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:14:01.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:00 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:00.735+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:02.033 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:01 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:14:01.946+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:14:02.033 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:01 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:14:01.946+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:36.431325+0000 front 2024-02-16T23:13:36.431145+0000 (oldest deadline 2024-02-16T23:14:01.131053+0000) 2024-02-16T23:14:02.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:01 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:01.706+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:02.974 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:02 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:02.662+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:02.974 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:02 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:02.662+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:03.283 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:02 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:14:02.974+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.230457+0000 front 2024-02-16T23:13:28.230516+0000 (oldest deadline 2024-02-16T23:13:52.930403+0000) 2024-02-16T23:14:03.283 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:02 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4[97404]: 2024-02-16T23:14:02.974+0000 7f84c466c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:36.431325+0000 front 2024-02-16T23:13:36.431145+0000 (oldest deadline 2024-02-16T23:14:01.131053+0000) 2024-02-16T23:14:04.033 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:03 smithi192 podman[119918]: 2024-02-16 23:14:03.879464441 +0000 UTC m=+5.534390679 container died f93a7734ffc482c2df81816227752ae0cb88ab4e063cd4ac0694a7a4ad7ed9b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-16T23:14:04.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:03 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:03.708+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:04.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:03 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:03.708+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:04.525 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:04 smithi192 podman[119918]: 2024-02-16 23:14:04.241901837 +0000 UTC m=+5.896828075 container cleanup f93a7734ffc482c2df81816227752ae0cb88ab4e063cd4ac0694a7a4ad7ed9b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-16T23:14:04.525 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:04 smithi192 bash[119918]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4 2024-02-16T23:14:04.783 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:04 smithi192 podman[119936]: 2024-02-16 23:14:04.526437437 +0000 UTC m=+0.644443405 container remove f93a7734ffc482c2df81816227752ae0cb88ab4e063cd4ac0694a7a4ad7ed9b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T23:14:04.784 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:04 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:04.739+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:04.784 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:04 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:04.739+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:05.170 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:04 smithi192 podman[120027]: 2024-02-16 23:14:04.764106492 +0000 UTC m=+0.017829184 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:14:05.170 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:04 smithi192 podman[120027]: 2024-02-16 23:14:04.885623973 +0000 UTC m=+0.139346668 container create cd37ea9babfbf5ce99db49e6b46ed5f672b8f0a51fb8451dfd4f7ff349e92ef3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True) 2024-02-16T23:14:05.170 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:05 smithi192 podman[120027]: 2024-02-16 23:14:05.096553406 +0000 UTC m=+0.350276097 container init cd37ea9babfbf5ce99db49e6b46ed5f672b8f0a51fb8451dfd4f7ff349e92ef3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-16T23:14:05.170 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:05 smithi192 podman[120027]: 2024-02-16 23:14:05.099666215 +0000 UTC m=+0.353388907 container start cd37ea9babfbf5ce99db49e6b46ed5f672b8f0a51fb8451dfd4f7ff349e92ef3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-16T23:14:05.170 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:05 smithi192 podman[120027]: 2024-02-16 23:14:05.171433099 +0000 UTC m=+0.425155791 container attach cd37ea9babfbf5ce99db49e6b46ed5f672b8f0a51fb8451dfd4f7ff349e92ef3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:14:05.533 INFO:journalctl@ceph.osd.4.smithi192.stdout:Feb 16 23:14:05 smithi192 podman[120027]: 2024-02-16 23:14:05.39124035 +0000 UTC m=+0.644963041 container died cd37ea9babfbf5ce99db49e6b46ed5f672b8f0a51fb8451dfd4f7ff349e92ef3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2) 2024-02-16T23:14:05.974 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:05 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:05.778+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:05.974 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:05 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:05.778+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:05.996 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.4.service' 2024-02-16T23:14:06.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T23:14:06.037 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-16T23:14:06.038 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-16T23:14:06.038 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.5 2024-02-16T23:14:06.283 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:06 smithi192 systemd[1]: Stopping Ceph osd.5 for 2912dc76-cd1d-11ee-95ba-87774f69a715... 2024-02-16T23:14:06.738 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:06 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:06.385+0000 7f1741f42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T23:14:06.738 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:06 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:06.385+0000 7f1741f42700 -1 osd.5 85 *** Got signal Terminated *** 2024-02-16T23:14:06.738 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:06 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:06.385+0000 7f1741f42700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T23:14:06.738 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:06 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:06.739+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:06.738 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:06 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:06.739+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:08.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:07 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:07.765+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:08.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:07 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:07.765+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:09.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:08 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:08.728+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:09.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:08 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:08.728+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:09.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:08 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:08.728+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2024-02-16T23:13:42.618562+0000 front 2024-02-16T23:13:42.618543+0000 (oldest deadline 2024-02-16T23:14:08.418948+0000) 2024-02-16T23:14:10.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:09 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:09.754+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:10.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:09 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:09.754+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:10.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:09 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:09.754+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2024-02-16T23:13:42.618562+0000 front 2024-02-16T23:13:42.618543+0000 (oldest deadline 2024-02-16T23:14:08.418948+0000) 2024-02-16T23:14:11.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:10 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:10.750+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-02-16T23:13:28.517164+0000 front 2024-02-16T23:13:28.517282+0000 (oldest deadline 2024-02-16T23:13:53.817325+0000) 2024-02-16T23:14:11.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:10 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:10.750+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2024-02-16T23:13:37.918300+0000 front 2024-02-16T23:13:37.918291+0000 (oldest deadline 2024-02-16T23:14:02.618141+0000) 2024-02-16T23:14:11.033 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:10 smithi192 ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5[102687]: 2024-02-16T23:14:10.750+0000 7f173ad2f700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2024-02-16T23:13:42.618562+0000 front 2024-02-16T23:13:42.618543+0000 (oldest deadline 2024-02-16T23:14:08.418948+0000) 2024-02-16T23:14:11.993 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:11 smithi192 podman[120133]: 2024-02-16 23:14:11.712946221 +0000 UTC m=+5.449824537 container stop 7bb5b265fc665280365fb5136240ab3ca9137ad452f41944fc0e9d04136a7c21 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-16T23:14:11.993 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:11 smithi192 podman[120133]: 2024-02-16 23:14:11.738789274 +0000 UTC m=+5.475667597 container died 7bb5b265fc665280365fb5136240ab3ca9137ad452f41944fc0e9d04136a7c21 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-16T23:14:12.283 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:12 smithi192 podman[120133]: 2024-02-16 23:14:12.2319695 +0000 UTC m=+5.968847857 container cleanup 7bb5b265fc665280365fb5136240ab3ca9137ad452f41944fc0e9d04136a7c21 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-16T23:14:12.283 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:12 smithi192 bash[120133]: ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5 2024-02-16T23:14:12.783 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:12 smithi192 podman[120150]: 2024-02-16 23:14:12.391513021 +0000 UTC m=+0.671346351 container remove 7bb5b265fc665280365fb5136240ab3ca9137ad452f41944fc0e9d04136a7c21 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5) 2024-02-16T23:14:13.068 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:12 smithi192 podman[120241]: 2024-02-16 23:14:12.690943871 +0000 UTC m=+0.017529557 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T23:14:13.068 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:12 smithi192 podman[120241]: 2024-02-16 23:14:12.800658373 +0000 UTC m=+0.127244102 container create 5b514e3859e6d8869b7512ebd0961159f30a7f96fc378285dca429c43eead232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T23:14:13.068 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:12 smithi192 podman[120241]: 2024-02-16 23:14:12.995464887 +0000 UTC m=+0.322050586 container init 5b514e3859e6d8869b7512ebd0961159f30a7f96fc378285dca429c43eead232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-16T23:14:13.068 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:12 smithi192 podman[120241]: 2024-02-16 23:14:12.998470406 +0000 UTC m=+0.325056097 container start 5b514e3859e6d8869b7512ebd0961159f30a7f96fc378285dca429c43eead232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-16T23:14:13.068 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:13 smithi192 podman[120241]: 2024-02-16 23:14:13.069719107 +0000 UTC m=+0.396304806 container attach 5b514e3859e6d8869b7512ebd0961159f30a7f96fc378285dca429c43eead232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-16T23:14:13.533 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:13 smithi192 podman[120241]: 2024-02-16 23:14:13.289967406 +0000 UTC m=+0.616553094 container died 5b514e3859e6d8869b7512ebd0961159f30a7f96fc378285dca429c43eead232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-16T23:14:13.833 INFO:journalctl@ceph.osd.5.smithi192.stdout:Feb 16 23:14:13 smithi192 podman[120267]: 2024-02-16 23:14:13.814982432 +0000 UTC m=+0.519250201 container remove 5b514e3859e6d8869b7512ebd0961159f30a7f96fc378285dca429c43eead232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2912dc76-cd1d-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-16T23:14:13.838 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2912dc76-cd1d-11ee-95ba-87774f69a715@osd.5.service' 2024-02-16T23:14:13.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T23:14:13.879 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-16T23:14:13.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-16T23:14:17.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[76752]: fuse finished with error 0 and tester_r 0 2024-02-16T23:14:35.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-16T23:14:50.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T23:14:50.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T23:14:50.263 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-16T23:14:50.265 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738/remote/smithi064/crash 2024-02-16T23:14:50.266 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/crash -- . 2024-02-16T23:14:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-16T23:14:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-02-16T23:14:50.305 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738/remote/smithi192/crash 2024-02-16T23:14:50.310 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/crash -- . 2024-02-16T23:14:50.337 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-16T23:14:50.337 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-02-16T23:14:50.338 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-16T23:14:50.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-16T23:14:50.377 INFO:tasks.cephadm:Compressing logs... 2024-02-16T23:14:50.377 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-16T23:14:50.419 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-16T23:14:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-16T23:14:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:: No such file or directory 2024-02-16T23:14:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-02-16T23:14:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mon.smithi064.log 2024-02-16T23:14:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi064.bmcyuf.log 2024-02-16T23:14:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mon.smithi064.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mgr.smithi064.kubmlk.log 2024-02-16T23:14:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-16T23:14:50.447 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi064.bmcyuf.log: /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mgr.smithi064.kubmlk.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:: No such file or directory 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mgr.smithi192.pauvih.log 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mon.smithi192.log 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mgr.smithi192.pauvih.log: -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi192.luiybj.log 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mon.smithi192.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi192.luiybj.log: /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-16T23:14:50.450 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-16T23:14:50.451 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi192.tnppti.log 2024-02-16T23:14:50.451 INFO:teuthology.orchestra.run.smithi192.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-16T23:14:50.451 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-16T23:14:50.451 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-16T23:14:50.452 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.3.log: 91.2% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-16T23:14:50.452 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.0.log: 91.1% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-16T23:14:50.454 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-16T23:14:50.454 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi192.tnppti.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.log 2024-02-16T23:14:50.454 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-16T23:14:50.454 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-16T23:14:50.457 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.log: 88.1% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-16T23:14:50.458 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-02-16T23:14:50.459 INFO:teuthology.orchestra.run.smithi064.stderr: 90.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-16T23:14:50.460 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.log 2024-02-16T23:14:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-16T23:14:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.log: 85.2% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-16T23:14:50.464 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-16T23:14:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-16T23:14:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi064.tqesxx.log 2024-02-16T23:14:50.489 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.2.log: /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi064.tqesxx.log: 91.0% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-16T23:14:50.501 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/ceph-client.1.log: 91.0% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-16T23:14:50.507 INFO:teuthology.orchestra.run.smithi192.stderr: 89.0% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mgr.smithi192.pauvih.log.gz 2024-02-16T23:14:50.830 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mgr.smithi064.kubmlk.log.gz 2024-02-16T23:14:50.909 INFO:teuthology.orchestra.run.smithi192.stderr: 92.5% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mon.smithi192.log.gz 2024-02-16T23:14:51.704 INFO:teuthology.orchestra.run.smithi064.stderr: 90.5% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mon.smithi064.log.gz 2024-02-16T23:14:54.576 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-16T23:14:54.646 INFO:teuthology.orchestra.run.smithi192.stderr: 93.7% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-16T23:14:55.433 INFO:teuthology.orchestra.run.smithi192.stderr: 93.5% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-16T23:14:55.554 INFO:teuthology.orchestra.run.smithi192.stderr: 93.7% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-16T23:14:55.895 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-16T23:14:56.402 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-16T23:14:57.873 INFO:teuthology.orchestra.run.smithi192.stderr: 94.8% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi192.luiybj.log.gz 2024-02-16T23:14:58.116 INFO:teuthology.orchestra.run.smithi192.stderr: 94.8% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi192.tnppti.log.gz 2024-02-16T23:14:58.499 INFO:teuthology.orchestra.run.smithi064.stderr: 94.8% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi064.bmcyuf.log.gz 2024-02-16T23:15:05.380 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-02-16T23:15:05.380 INFO:teuthology.orchestra.run.smithi192.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-02-16T23:15:05.382 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-02-16T23:15:05.382 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m14.948s 2024-02-16T23:15:05.382 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m43.451s 2024-02-16T23:15:05.382 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m1.327s 2024-02-16T23:15:06.513 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-02-16T23:16:27.635 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/2912dc76-cd1d-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi064.tqesxx.log.gz 2024-02-16T23:16:27.637 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-16T23:16:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:real 1m37.204s 2024-02-16T23:16:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:user 2m14.431s 2024-02-16T23:16:27.637 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m3.814s 2024-02-16T23:16:27.638 INFO:tasks.cephadm:Archiving logs... 2024-02-16T23:16:27.638 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738/remote/smithi064/log 2024-02-16T23:16:27.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-16T23:16:46.447 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738/remote/smithi192/log 2024-02-16T23:16:46.449 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-16T23:16:51.880 INFO:tasks.cephadm:Removing cluster... 2024-02-16T23:16:51.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 --force 2024-02-16T23:16:52.509 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2912dc76-cd1d-11ee-95ba-87774f69a715 --force 2024-02-16T23:16:52.996 INFO:tasks.cephadm:Removing cephadm ... 2024-02-16T23:16:52.996 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-16T23:16:53.011 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-16T23:16:53.026 INFO:tasks.cephadm:Teardown complete 2024-02-16T23:16:53.026 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-16T23:16:53.038 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T23:16:53.039 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-16T23:16:53.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-16T23:16:53.053 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-16T23:16:53.143 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-16T23:16:53.154 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-16T23:16:53.154 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T23:16:53.157 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T23:16:53.168 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-02-16T23:16:53.170 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-02-16T23:16:53.173 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T23:16:53.173 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-02-16T23:16:53.173 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 -1234us[-1234us] +/- 89ms 2024-02-16T23:16:53.174 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -727us[ -730us] +/- 66ms 2024-02-16T23:16:53.174 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +2296us[+2293us] +/- 91ms 2024-02-16T23:16:53.174 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T23:16:53.175 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T23:16:53.175 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-02-16T23:16:53.175 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 -1249us[-1243us] +/- 89ms 2024-02-16T23:16:53.175 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -735us[ -729us] +/- 66ms 2024-02-16T23:16:53.175 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +2262us[+2267us] +/- 91ms 2024-02-16T23:16:53.175 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T23:16:53.175 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-16T23:16:53.186 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-16T23:16:53.186 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-16T23:16:53.234 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-16T23:16:53.320 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-16T23:16:53.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T23:16:53.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T23:16:53.429 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T23:16:53.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T23:16:53.460 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-16T23:16:53.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-16T23:16:53.517 INFO:teuthology.task.internal:Duration was 2243.109562 seconds 2024-02-16T23:16:53.517 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-16T23:16:53.567 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-16T23:16:53.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-16T23:16:53.569 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-16T23:16:53.610 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T23:16:53.612 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T23:16:54.073 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-16T23:16:54.073 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-02-16T23:16:54.073 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-16T23:16:54.094 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-02-16T23:16:54.094 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-16T23:16:54.142 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-16T23:16:54.142 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-16T23:16:54.145 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-16T23:16:54.211 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-16T23:16:54.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-16T23:16:54.214 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-16T23:16:54.866 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-16T23:16:55.169 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-16T23:16:55.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-16T23:16:55.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-16T23:16:55.227 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-16T23:16:55.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-16T23:16:55.242 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-16T23:16:55.265 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-02-16T23:16:55.294 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-02-16T23:16:55.510 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-16T23:16:55.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T23:16:55.524 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-16T23:16:55.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T23:16:55.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-16T23:16:55.578 INFO:teuthology.task.internal:Transferring archived files... 2024-02-16T23:16:55.578 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738/remote/smithi064 2024-02-16T23:16:55.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-16T23:16:55.629 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738/remote/smithi192 2024-02-16T23:16:55.629 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-16T23:16:55.669 INFO:teuthology.task.internal:Removing archive directory... 2024-02-16T23:16:55.669 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-16T23:16:55.671 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-16T23:16:55.718 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-16T23:16:55.731 INFO:teuthology.task.internal:Not uploading archives. 2024-02-16T23:16:55.731 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-16T23:16:55.759 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-16T23:16:55.759 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-16T23:16:55.761 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-16T23:16:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: 265295 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 16 23:16 /home/ubuntu/cephtest 2024-02-16T23:16:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: 556309 4 d--------- 2 ubuntu ubuntu 4096 Feb 16 22:54 /home/ubuntu/cephtest/mnt.0 2024-02-16T23:16:55.773 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-02-16T23:16:55.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T23:16:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-02-16T23:16:55.774 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi064 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-02-16T23:16:55.775 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-16T23:16:55.810 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-16T23:16:55.861 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T23:16:55.861 INFO:teuthology.nuke:Checking targets against current locks 2024-02-16T23:16:55.887 DEBUG:teuthology.nuke:shortname: smithi064 2024-02-16T23:16:55.887 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T23:16:55.910 DEBUG:teuthology.nuke:shortname: smithi192 2024-02-16T23:16:55.910 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T23:16:55.913 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 22:31:26.667808', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6QadFqDbWyDlJmW1q6PrCeWaGARL4XxNcaOu2UAL0qUc2ZoHq+JQYu5jBxZDSH9UlNBy1VtA6EfirNQ3lp4Dg='} 2024-02-16T23:16:55.934 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 22:31:26.666021', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYPHynn3iYPQupgar2Al02NX2cBEApdPgoCudCKznhRvaxCqANchDMEOC0mVh749UR7au1Gi4hVElHqEh4lyDE='} 2024-02-16T23:16:55.957 INFO:teuthology.orchestra.console.smithi064:Power off 2024-02-16T23:16:55.957 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T23:16:55.982 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2024-02-16T23:16:55.982 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T23:16:55.998 INFO:teuthology.orchestra.console.smithi192:Power off 2024-02-16T23:16:55.998 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T23:16:56.006 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-02-16T23:16:56.019 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2024-02-16T23:16:56.019 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T23:16:56.039 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-02-16T23:17:00.007 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T23:17:00.040 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T23:17:00.118 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-02-16T23:17:00.150 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-02-16T23:17:04.124 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T23:17:04.151 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T23:17:04.233 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-02-16T23:17:04.261 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-02-16T23:17:08.238 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T23:17:08.262 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T23:17:08.349 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2024-02-16T23:17:08.370 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2024-02-16T23:17:08.450 INFO:teuthology.orchestra.console.smithi064:Power off completed 2024-02-16T23:17:08.471 INFO:teuthology.orchestra.console.smithi192:Power off completed 2024-02-16T23:17:08.575 INFO:teuthology.run:Summary data: description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2243.1095621585846 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-02-16T23:17:08.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T23:17:08.653 INFO:teuthology.run:FAIL